repositories / overrides

[Otto Behrens]

Hi,

I'm trying to get metacello to load packages from local repositories. I have various directory and filetree repositories. I have a BaselineOfWonkaRuntime class (a subclass of BaselineOf), with the following:

baseline: spec
<baseline>

spec

package: 'Sport';
configuration: 'XMLSupport' with: [ spec loads: #('Core') ];
configuration: 'SoapOpera' with: [ spec loads: #('Client') ];
baseline: 'Seaside30'

with: [

spec

repository: self class localThirdPartySeasideRepository;

loads: #('Base' 'Scriptaculous-Components' 'JQuery-UI') ];

project: 'SeasideDevelopment' copyFrom: 'Seaside30' with: [ spec loads: #('Seaside-Development') ];
configuration: 'Magritte3 Base' with: [ spec loads: #('Magritte-Seaside' 'Deprecated') ];

etc.

I would like to load for example Sport, XMLSupport and SoapOpera from my local 3rd party directory Seaside30 from a cloned filetree repository, etc. 

I don't want any of the configurations / baselines I depend upon to go over the internet and fetch packages because I want them loaded from my local machine.

I would like the dependencies specified of for example the Magritte3 configuration to load its packages and configs from a local repository.

Do I have to specify repository: in each configuration, baseline or package? Or should I be using overrideRepositories: with a list of all the repositories I need?

I am out of ideas. I try using the different repository: repositories: overrideRepositories: methods in different places, but battle to get my stuff loaded.

Any ideas welcome.

Thanks

Otto

[Dale K. Henrichs]

Otto,

This is where you need my sample project ... override repositories is not going to be used and you shouldn't have to modify any exsiting configurations ...  My current sample project does not use a configuration and I think that a configuration (or two) is needed to make the sample complete, but I am bearing down a bit on the internal gemstone work as I am getting close to the deadline ...

So if you could hold off until I can get the sample completed, you'll be able to see how I think that the projects can be loaded and then work out any differences between how I think they should be handled and what you are running into in the wild ...

Dale

---

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to metacello+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Otto Behrens]

Thanks Dale, will be interesting. In the meantime, I'm trying to get it going with what I've got.

[Johan Brichau]

Dale,

Just to understand your reply: does repositoryOverrides only work for monticello repositories?

Johan

On 21 Nov 2013, at 18:04, Otto Behrens <ot...@finworks.biz> wrote:

> Thanks Dale, will be interesting. In the meantime, I'm trying to get it going with what I've got.
>
> On Thu, Nov 21, 2013 at 5:30 PM, Dale K. Henrichs <dale.h...@gemtalksystems.com> wrote:
> Otto,
>
> This is where you need my sample project ... override repositories is not going to be used and you shouldn't have to modify any exsiting configurations ... My current sample project does not use a configuration and I think that a configuration (or two) is needed to make the sample complete, but I am bearing down a bit on the internal gemstone work as I am getting close to the deadline ...
>
> So if you could hold off until I can get the sample completed, you'll be able to see how I think that the projects can be loaded and then work out any differences between how I think they should be handled and what you are running into in the wild ...
>
> Dale

[Dale K. Henrichs]

Sorry about that ... repository overrides works, but the technique of using project locks should make it unnecessary to use repository overrides ...

Moving forward I don't think that repository overrides (making a giant repository where all of your packages for all external projects) is the right answer ... Otto ranted about having to manage the repository overrides, so I would like to show him how to "do it the right way" ...

I also think that there are other issues that Otto is running into that aren't necessarily related to repository overrides, but given the limited amount of time I have right now, I would rather invest in getting Otto moved to using project locks than solve his current problems doing things the way he is doing it now, then follow that on with moving to using repository locks, where we are likely to cover the same ground doing things a different way ...

I assume that the fundamental issue that Otto is having is that we are mixing configurations for GLASS and baselines for GLASS1 and I think that to untangle that particular problem with or without project locks will require changing some configuration/baselines ... if we change the configurations/baselines to support project locking then those edits will be meaningful going forward ... if we make the edits with an eye towards hacking our way around the immediate problems ... those edits won't be compatible with anything going forward and will ahve to managed privately ... and avoiding that mess is precisely why I invented project locks ...

Dale

[Dale K. Henrichs]

An alternate answer to your question is that repository overrides are not compatible with git-based packages ... storing packages into a git repository means that you only have one version of the package "visible" at any time ... the repository override technique relies upon the fact that a single repository holds many versions of the same package which reuquires that you use a monticello directory or http repository ...

Dale

----- Original Message -----
| From: "Johan Brichau" <jo...@yesplan.be>

| To: meta...@googlegroups.com
| Sent: Thursday, November 21, 2013 11:45:36 AM
| Subject: Re: [Metacello] repositories / overrides
|

[Dale K. Henrichs]

Otto,

I should be able to find time over the weekend to finish up the sample project ... hopefully it will be in the neighborhood of what you are looking for ...

Dale

---

From: "Otto Behrens" <ot...@finworks.biz>
To: meta...@googlegroups.com

Sent: Thursday, November 21, 2013 9:04:36 AM
Subject: Re: [Metacello] repositories / overrides

[Dale K. Henrichs]

Otto,

Well, I've got the Example project defined and I've got a real good start documenting my thought on using the `lock` command[1]. Unfortunately, I've run out of time for completing the writeup and won't have too much time in the next couple of days to finish it.

I've pretty much tested the scripts in the writeup and I figure that there is enough information for you to get a pretty good idea of what I am thinking. Soo, I'd really appreciate it if you took a critical look at what I am proposing and let me know if it the ideas can be applied to your situation or not. If not, I'd like to make tweaks to the implementation to fit your needs ...

I did have to fix some bugs and add a new feature or two[2], so if you want to run the scripts, you should load the issue_215 branch of Metacello to pick up the new features and bugfixes:

  Metacello new
    baseline: 'MetacelloPreview';
    repository: 'github://dalehenrich/metacello-work:issue_215/repository';
    load.

BTW, I forked the issue_215 branch before you made your recent changes (another casualty or my lack of time), but that shouldn't affect the functionality.

Dale

[1] https://github.com/dalehenrich/metacello-work/blob/issue_215/docs/LockCommandReference.md
[2] https://github.com/dalehenrich/metacello-work/issues?direction=desc&milestone=17&sort=updated&state=open

---

From: "Otto Behrens" <ot...@finworks.biz>
To: meta...@googlegroups.com

Sent: Thursday, November 21, 2013 9:04:36 AM
Subject: Re: [Metacello] repositories / overrides

[Otto Behrens]

Thanks, Dale.

> I've pretty much tested the scripts in the writeup and I figure that there is enough information for you to get a pretty good idea of what I am thinking. Soo, I'd really appreciate it if you took a critical look at what I am proposing and let me know if it the ideas can be applied to your situation or not. If not, I'd like to make tweaks to the implementation to fit your needs ...

Some ideas:

1. from [1]: "When you lock a baseline-based project: you are telling Metacello to always load the baseline from the given repository and, given the way that baseline-based projects work, to always load the packages referenced by the baseline specification from the given repository."

Yes, agree.

2. from [1]: "When you lock a configuration-based project: you are telling Metacello to always use the given version and to always load the configuration from the given repository"

This to me is semantically different. In 1, I load whatever version I've got checked out. in 2, I only load a specific version. To me, this does not make sense.

I would like the behaviour of 1, always. I don't care what was loaded previously, where it came from or that it is in the registry. I actually want a "Metacello new removeTheRegistry" function. I just want to load what is in the repository that I specify. In the order of dependency specified in the baseline. If I load a new ConfigurationOfGrease or whatever, that defines a new stable that I want to load, I just want it to load that.

3. I'm concerned about the propagation of the lock. If the repository contains various BaselineOf<Project>s, are they also locked? If I lock BaselineOfSeaside30, what happens to ConfigurationOfGrease, currently specified as versionString: '1.0.7.2'.

4. I want to get all my stuff from local repositories. If I load BaselineOfSeaside30, how do I say that Zinc must be loaded from a locally cloned repo? I've now changed our project configurations over to baselines. This is what I've done:

baseline: spec

<baseline>

spec

for: #common

do: [ 

spec

baseline: 'Seaside30'

with: [ 

spec

repository: self class localThirdPartySeasideRepository;

loads: #('Base' 'Scriptaculous-Components' 'JQuery-UI') ]];

for: #gemstone

do: [ 

spec

baseline: 'Zinc' with: [ spec repository: self class localThirdPartyZincRepository ]]

I would like to do this in principle with all the configurations / baselines and I want to be sure that it honors that I've overridden its repository and that it does not hunt for packages / configs elsewhere.

> I did have to fix some bugs and add a new feature or two[2], so if you want to run the scripts, you should load the issue_215 branch of Metacello to pick up the new features and bugfixes:

>
>   Metacello new
>     baseline: 'MetacelloPreview';
>     repository: 'github://dalehenrich/metacello-work:issue_215/repository';
>     load.
>
> BTW, I forked the issue_215 branch before you made your recent changes (another casualty or my lack of time), but that shouldn't affect the functionality.

I see that issue_215 breaks some tests. Don't think it affects the versions we're using. Will try. I've created another pull request. When those tests pass, I'll comment. Not sure if it is all useful.

I got it all loaded eventually with repositoryOverrides:, after unsuccessfully trying locks.

[Otto Behrens]

Does this all apply to upgrading a seaside.extent0.dbf? Or should I start with extent0.dbf?

If I have an existing db, how to I bootstrap Metacello?

www.FinWorks.biz
+27 82 809 2375

[Dale K. Henrichs]

Thanks for the quick response ... your feedback is much appreciated ...

---

From: "Otto Behrens" <ot...@finworks.biz>
To: meta...@googlegroups.com
Sent: Monday, November 25, 2013 8:57:33 AM
Subject: Re: [Metacello] repositories / overrides

Thanks, Dale.

> I've pretty much tested the scripts in the writeup and I figure that there is enough information for you to get a pretty good idea of what I am thinking. Soo, I'd really appreciate it if you took a critical look at what I am proposing and let me know if it the ideas can be applied to your situation or not. If not, I'd like to make tweaks to the implementation to fit your needs ...

Some ideas:

1. from [1]: "When you lock a baseline-based project: you are telling Metacello to always load the baseline from the given repository and, given the way that baseline-based projects work, to always load the packages referenced by the baseline specification from the given repository."

Yes, agree.

2. from [1]: "When you lock a configuration-based project: you are telling Metacello to always use the given version and to always load the configuration from the given repository"

This to me is semantically different. In 1, I load whatever version I've got checked out. in 2, I only load a specific version. To me, this does not make sense.

Good point. The original use case for `lock` was targeted at locking the version only[1]. When I applied the `lock` command to the baseline, I had to extend the lock to cover the repository, since baselines don't have an explicit version. When a baseline-based project is locked, you get the packages "for free" thus expanding the potential range of use cases for the `lock` command into the area of managing external projects ...

When I wrote the above statement in the writeup, I knew that I was introducing an inconsistency, however, I am not convinced that it is possible/desirable to make the same guarantees for a configuration-based project that are possible for a baseline-based project.

[1] https://github.com/dalehenrich/metacello-work/blob/master/docs/MetacelloUserGuide.md#locking

I would like the behaviour of 1, always. I don't care what was loaded previously, where it came from or that it is in the registry. I actually want a "Metacello new removeTheRegistry" function. I just want to load what is in the repository that I specify. In the order of dependency specified in the baseline. If I load a new ConfigurationOfGrease or whatever, that defines a new stable that I want to load, I just want it to load that.

Just to be clear, the registry should be giving you the behavior that you are claiming you want. The registry is simply a record of what project versions are loaded in the image (so we can deprecate the currentVersion nonsense). The registry is used to recognize when there is a request to upgrade or downgrade a project and the scripting api provides you with explicit control over what you want to have happen:

  1. When you load a specific version of Grease, that fact is recorded in the registry
  2. After that point any time if you explicitly load a different version of Grease,
      the new version is loaded and the new version (upgrade or downgrade) is recorded
      in the registry.
  3. When you load a different project that __references__ Grease, you have the option of specifying
      `onUpgrade:` and `onDowngrade:` methods where you can decide whether or not (#allow/#disallow)
      you want the upgrade/downgrade performed.

In your case it sounds to me that you want to consistently #disallow any and all upgrades and downgrades to Grease (except when you explicitly load a different version) and this is completely supported by the Metacello Scripting API ...

Soooo, the fact that you want to "Metacello new removeTheRegistry" means that I haven't done a good enough job of communicating this information to you or you have run into bugs ...

The Metacello Scripting API is intended to support exactly the behavior you are asking for so we need to work out specifically where it is failing your...

3. I'm concerned about the propagation of the lock. If the repository contains various BaselineOf<Project>s, are they also locked? If I lock BaselineOfSeaside30, what happens to ConfigurationOfGrease, currently specified as versionString: '1.0.7.2'.

As the implementation stands today, locks are not propagated. Each project must be explicitly locked or not locked independently.

My intention is to support your use case, so I am willing to work with you to provide tools for doing what you need done ...

A `lock` is project specific, but a tool to transitively `lock` all projects reachable from a particular project may be required ...

4. I want to get all my stuff from local repositories. If I load BaselineOfSeaside30, how do I say that Zinc must be loaded from a locally cloned repo? I've now changed our project configurations over to baselines. This is what I've done:

baseline: spec

<baseline>

spec

for: #common

do: [ 

spec

baseline: 'Seaside30'

with: [ 

spec

repository: self class localThirdPartySeasideRepository;

loads: #('Base' 'Scriptaculous-Components' 'JQuery-UI') ]];

for: #gemstone

do: [ 

spec

baseline: 'Zinc' with: [ spec repository: self class localThirdPartyZincRepository ]]

I agree that the best solution to this problem is to convert the *external projects* into baselines.

In order to "say that Zinc must be loaded from a locally cloned repo", you would execute the following expression:

  Metacello
    baseline: 'Zinc';
    repository: <someclass> localThirdPartyZincRepository;
    lock.

After setting the lock, you would use the following pattern for doing loads to ensure that the Zinc locks are `honored`:

  Metacello ...
...
    onConflict: [ :ex :existing |
      existing locked
        ifTrue: [ ex useIncoming ].
      ex pass ];
    onLock: [ :ex | ex honor ];
...

With the locks set, no matter where the project reference comes from when loaded, Metacello will make sure that the local Zinc repository is always used with one exception. If you do an explicit load of Zinc, the lock will be automatically broken:

  Metacello new
    configuraton: 'Zinc';
    version: '1.x';
    repostority: 'http:anotherZincREpo';
    load.

I would like to do this in principle with all the configurations / baselines and I want to be sure that it honors that I've overridden its repository and that it does not hunt for packages / configs elsewhere.

The `lock` for Zinc applies to both baseline-based or configuration-based references anywhere in the loaded project stack ...

> I did have to fix some bugs and add a new feature or two[2], so if you want to run the scripts, you should load the issue_215 branch of Metacello to pick up the new features and bugfixes:

>
>   Metacello new
>     baseline: 'MetacelloPreview';
>     repository: 'github://dalehenrich/metacello-work:issue_215/repository';
>     load.
>
> BTW, I forked the issue_215 branch before you made your recent changes (another casualty or my lack of time), but that shouldn't affect the functionality.

I see that issue_215 breaks some tests. Don't think it affects the versions we're using.

The tests that failed were not failing consistently across all versions/platforms, so I'm suspicious that network/server availability problems are involved. Nonetheless I will check into the failures tonight.

Will try. I've created another pull request. When those tests pass, I'll comment. Not sure if it is all useful.

I got it all loaded eventually with repositoryOverrides:, after unsuccessfully trying locks.

I really think that locks are the going to be the correct answer ... we need to explore the configuration-based issues ... if you are comfortable converting all of your projects to baselines, then perhaps it isn't important to come up with an equivalent solution for configuration-based projects ...

Frankly I think that if one is going to "take ownership" of an external project (i.e., copy the configuration and packages to a local repository) the only option that makes sense is to use a git-based repository. Firstly, when it comes time to update to a newer version of the project, you must be able to assess all of the new changes that were made and the only way to really make sense of that is ot use git ... Secondly, if you ever find it necessary to make project bugfixes in your local copy, it doesn't make sense to have to update the configuration with a new project version, and new package versions, when a baseline-based project doesn't require all of that extra overheade ... you just make a new git commit ...

Thanks again for your feedback ... I really don't think we are too far apart in what you want and what Metacello provides, but we do have get ourselves on the same page .....

Dale
Dale

[Dale K. Henrichs]

starting with an extent0.seaside.dbf, here's the whole recommended install sequence as a topaz script:

!
! Upgrade to GLASS 1.0-beta.9.1
!
run
ConfigurationOfGLASS project updateProject.
GsDeployer deploy: [ (ConfigurationOfGLASS project version: '1.0-beta.9.1') load ].
%

commit
errorCount

!
! Install most recent Metacello
!
run
[
Metacello new
  baseline: 'Metacello';

  repository: 'github://dalehenrich/metacello-work:issue_215/repository';
  load.

  ] on: Warning
    do: [:ex |
      Transcript cr; show: ex description.
      ex resume].
%
commit
errorCount

[Dale K. Henrichs]

For an existing db, if you are at GLASS 1.0-beta.9.1 or greater (like GLASS1) then you should only need to do the following:

  Metacello new
    baseline: 'MetacelloPreview';
    repository: 'github://dalehenrich/metacello-work:issue_215/repository';
    load.

[Otto Behrens]

> In your case it sounds to me that you want to consistently #disallow any and all upgrades and downgrades to Grease (except when you explicitly load a different version) and this is completely supported by the Metacello Scripting API ...

No, not really. I want to use the same command to load our project. It
must pick up from disk that there's another version of Grease that I
want to deploy (i.e. the ConfigurationOfGrease package changed the
default / core version). And then it must load the new package
versions.

> Soooo, the fact that you want to "Metacello new removeTheRegistry" means that I haven't done a good enough job of communicating this information to you or you have run into bugs ...
>
> The Metacello Scripting API is intended to support exactly the behavior you are asking for so we need to work out specifically where it is failing your...

Thanks for the patience, I really battle to understand why I need
this. I have the code in the DB, on the classes. I don't really care
about package versions. I want to make sure that the code I'm loading
is loaded when I'm done. Even after loading with metacello, we
currently have to reload all the code again by going through the
MCWorkingCopy registry and loading the packages using Gofer. This is
because if there are extensions in a package and the package changes
underneath it, the overrides are lost.

I am considering this:
1. remove all the methods on the classes we work with
2. remove classes that are not referenced by the incoming packages
3. load all the code
4. migrate

This simple process was way faster on previous projects than what we
currently have. It was also much more reliable because we were sure
that we got the methods recompiled and that nothing was missed.

You see, the thing is we don't develop on GS. We do a complete deploy
of our root package every time we update code. Even with developing on
Pharo, we use rebuilt images every day. It may even be easier to load
a 'patch' in pharo generated with a git diff for interim updates. Or
use git tools in the image.

But let's say we still use monticello / metacello. I really don't care
about config versions loaded / not. I just need the package
dependencies and load order from metacello. And then compare the
packages with the versions loaded in the image using Monticello. I
really don't get why we need more than this if we really have to use
Monticello.

> A `lock` is project specific, but a tool to transitively `lock` all projects reachable from a particular project may be required ...

Ok, but please not if ConfigurationOf projects are referenced, because
locking a version is not the behaviour that I want.

> I agree that the best solution to this problem is to convert the *external projects* into baselines.

But we will probably have a mix for a while.

> In order to "say that Zinc must be loaded from a locally cloned repo", you would execute the following expression:
>
> Metacello
> baseline: 'Zinc';
> repository: <someclass> localThirdPartyZincRepository;
> lock.

So this means that I can't just load my root BaselineOfWonkaRuntime
and expect it to know where to find its stuff. If I have to do this,
it means that I should ditch our config that says it depends on Zinc.
I can then just as well write a long script that does this for each of
the configs / baselines we use and load it explicitly.

I thought the idea was that I could just do this:

Metacello new
baseline: 'MyProject';
repository: <someclass> localThirdPartyZincRepository;
load.

and in BaselineOfMyProject there should be enough info to know where
to get all the code and what versions to use.

> I really think that locks are the going to be the correct answer ... we need to explore the configuration-based issues ... if you are comfortable converting all of your projects to baselines, then perhaps it isn't important to come up with an equivalent solution for configuration-based projects ...

I've already converted to baselines. If there can be a solution for
configuration based issues, then I don't need locks. I need a mode
where I just say 'load all the code locally from the local repos'.

> Frankly I think that if one is going to "take ownership" of an external project (i.e., copy the configuration and packages to a local repository) the only option that makes sense is to use a git-based repository. Firstly, when it comes time to update to a newer version of the project, you must be able to assess all of the new changes that were made and the only way to really make sense of that is ot use git ... Secondly, if you ever find it necessary to make project bugfixes in your local copy, it doesn't make sense to have to update the configuration with a new project version, and new package versions, when a baseline-based project doesn't require all of that extra overheade ... you just make a new git commit ...

I think that using git is great. Some of the community seem to like
it, but adoption will possibly not come instantly.

[Dale K. Henrichs]

----- Original Message -----
| From: "Otto Behrens" <ot...@finworks.biz>
| To: meta...@googlegroups.com
| Sent: Monday, November 25, 2013 10:00:27 PM
| Subject: Re: [Metacello] repositories / overrides
|

| > In your case it sounds to me that you want to consistently
| > #disallow any and all upgrades and downgrades to Grease (except
| > when you explicitly load a different version) and this is
| > completely supported by the Metacello Scripting API ...
|
| No, not really. I want to use the same command to load our project.

Not sure what you mean here ...

| It
| must pick up from disk that there's another version of Grease that I
| want to deploy (i.e. the ConfigurationOfGrease package changed the
| default / core version). And then it must load the new package
| versions.

As far as I can tell you have just described what Metacello does, so I'm not sure what your point is ...

|
| > Soooo, the fact that you want to "Metacello new removeTheRegistry"
| > means that I haven't done a good enough job of communicating this
| > information to you or you have run into bugs ...
| >
| > The Metacello Scripting API is intended to support exactly the
| > behavior you are asking for so we need to work out specifically
| > where it is failing your...
|
| Thanks for the patience, I really battle to understand why I need
| this. I have the code in the DB, on the classes. I don't really care
| about package versions.

In the previous section you claimed that "it must load the new package versions". If you don't care about package versions then how does "it" know which ones are new?

| I want to make sure that the code I'm loading
| is loaded when I'm done. Even after loading with metacello, we
| currently have to reload all the code again by going through the
| MCWorkingCopy registry and loading the packages using Gofer. This is
| because if there are extensions in a package and the package changes
| underneath it, the overrides are lost.

This sounds like a different problem, I use packages with extensions all of the time and I don't have to reload packages usig Gofer, so there must be something different in the way that you do your load or how you define your packages or ??? but I don't have enough detail here to know where the issue might lie. I need specifics, not generalities.

|
| I am considering this:
| 1. remove all the methods on the classes we work with
| 2. remove classes that are not referenced by the incoming packages
| 3. load all the code
| 4. migrate

I see that we have moved on from the discussion of how to manage external projects into something completely different. I don't think this has anything to do with Metacello.

I would like to see a step by step description of what you are doing and exactly what problems you are experiencing. I don't think you should have to do this, but I clearly don't understand the exact problems that you are facing so I can't judge whether the steps you are threatening to take are necessary or not ...

|
| This simple process was way faster on previous projects than what we
| currently have. It was also much more reliable because we were sure
| that we got the methods recompiled and that nothing was missed.
|
| You see, the thing is we don't develop on GS. We do a complete deploy
| of our root package every time we update code. Even with developing
| on
| Pharo, we use rebuilt images every day. It may even be easier to load
| a 'patch' in pharo generated with a git diff for interim updates. Or
| use git tools in the image.

.....I don't see a problem being described here ... is speed of updating the only problem? There are techniques for speeding up migrations by handling the migrations in bulk, but It's not clear to me exactly what your problem is...

|
| But let's say we still use monticello / metacello. I really don't
| care
| about config versions loaded / not. I just need the package
| dependencies and load order from metacello. And then compare the
| packages with the versions loaded in the image using Monticello. I
| really don't get why we need more than this if we really have to use
| Monticello.

This is exactly what a Metacello load does. What is the "more than this" that you are referring to?

|
| > A `lock` is project specific, but a tool to transitively `lock` all
| > projects reachable from a particular project may be required ...
|
| Ok, but please not if ConfigurationOf projects are referenced,
| because
| locking a version is not the behaviour that I want.

Telling me that something is "not the behavior that I want" provides no useful information ... Let's try to use small, concrete examples of what you do want ... have you run the example code that I created for you? Have you looked at the scripts that I wrote? I think that you should be able to demonstrate your problems by modifying the example projects to illustrate the problems that you are having ...

You seem to be ovewhelmed with a bunch of different problems, but to make any sense, we have to start at the beginning and work our way through the various problem areas ... one step at a time .... I know that if we were in the same room we'd have an understanding in a couple of hours, but we do not have that luxury ... so we must make do ...

So I propose that we start with the set of projects that I have created and attempt to duplicate ONE of the problems that you are having. So you have to pick ONE of your problems and describe it enough detail that I can understand what the problem might be ... then I will go about tryig to duplicate the problem with the small set of projects and packages ...

|
| > I agree that the best solution to this problem is to convert the
| > *external projects* into baselines.
|
| But we will probably have a mix for a while.
|
| > In order to "say that Zinc must be loaded from a locally cloned
| > repo", you would execute the following expression:
| >
| > Metacello
| > baseline: 'Zinc';
| > repository: <someclass> localThirdPartyZincRepository;
| > lock.
|
| So this means that I can't just load my root BaselineOfWonkaRuntime
| and expect it to know where to find its stuff.

The point of the `lock` command is to allow you to change/override the specification of the Zinc project that is referenced from configurations/baslines that you DON'T OWN ... the `lock` command is supposed to save you the effort of changing every external configuration/baseline that references the Zinc project ...

The `lock` command is simply a means of specifying an alternate location for project artifacts

| If I have to do this,
| it means that I should ditch our config that says it depends on Zinc.
| I can then just as well write a long script that does this for each
| of
| the configs / baselines we use and load it explicitly.
|

Nope ... wrong conclusion...

| I thought the idea was that I could just do this:
|
| Metacello new
| baseline: 'MyProject';
| repository: <someclass> localThirdPartyZincRepository;
| load.
|
| and in BaselineOfMyProject there should be enough info to know where
| to get all the code and what versions to use.

You certainly can do "this", but the problem that `locks` are designed to address comes about when you load an external project that also has a reference to 'MyProject' and that reference specifies a non-local repository location...rerouting the project reference to point to your local repository could have been accomplished by adding some behavior to the onConflict:/onUpgrade:/onDowngrade: messages, but I thought this "re-routing" deserved an explicit specification at the image level rather than forcing you to add the repository mapping logic to every load command ...

with all of that said ... I am not convinced that I have "hit the nail on the head" with the `lock` command, but so far I don't have a clear enough picture of your exact use case to know... I have a feeling that you don't quite fully understand the `lock` command either ...or you need to be more specific in your objections:)

|
| > I really think that locks are the going to be the correct answer
| > ... we need to explore the configuration-based issues ... if you
| > are comfortable converting all of your projects to baselines, then
| > perhaps it isn't important to come up with an equivalent solution
| > for configuration-based projects ...
|
| I've already converted to baselines. If there can be a solution for
| configuration based issues, then I don't need locks. I need a mode
| where I just say 'load all the code locally from the local repos'.

It sounds like you've already done some of the things that I've suggested in the writeup, but it also sounds like you still have some outstanding issues. Why don't you describe the configuration-based issues in detail from your perspective and perhaps we can come up with a solution that works for you...

Dale

[Dale K. Henrichs]

Otto,

Upon reading this mail in the "light of day" I hope I wasn't too harsh ... build and configuration problems are difficult beasts to tame, the only way to tame the beast is to tackle one small problem at a time ... the devil is in the details ...

The primary motivation for the Metacello Scripting API is to improve the build process. In order to truly improve the process, I need to have feedback from other folks and that means we have to open wounds and poke around in them ... a painful process ...

It takes patience on both sides, but it is also "legal" to do a bit of ranting along the way as it reduces stress in the end ...

I am optimistic that we can address the problems that you are having, but in order to do so we have to break the problem down into smaller pieces and build a solution one piece at a time...

Dale

[Otto Behrens]

Ok, don't worry. Yes, certainly we need some patience. I'm also under pressure. Will respond soon. Almost there

--

[Otto Behrens]

| > In your case it sounds to me that you want to consistently
| > #disallow any and all upgrades and downgrades to Grease (except
| > when you explicitly load a different version) and this is
| > completely supported by the Metacello Scripting API ...
|
| No, not really. I want to use the same command to load our project.

Not sure what you mean here ...

I meant to say that I want to do:

Metacello new

baseline: 'WonkaRuntime';

repository: 'filetree:///somewhere';

load

and that is it. I want the BaselineOfWonkaRuntime to do the rest. Inside BaselineOfWonkaRuntime (attached) 

My entry point is for example:

BaselineOfWonkaRuntime 

loadWonkaRuntimeWithProjectsGroup: 'Core' 

fromWonkaRoot: '/home/wonka/development/wonka' 

platform: 'GemStone64Bit2.4.4.4-x86_64.Linux'.

Groups - 'Core' or 'Tests'

fromWonkaRoot - the root of our locally cloned git repo. For this load, the subdirectory structure looks like this:

<wonka root>

- 3rdparty

- glass   (clone of https://github.com/glassdb/glass)

- zinc   (clone of https://github.com/glassdb/zinc)

- Seaside30    (clone of https://github.com/glassdb/Seaside30)

- metacello   (clone of https://github.com/finworks/metacello-work)

- shared_mcz   (contains ConfigurationOf(GsMonticello|XMLWriter|OSProcess|Gofer|...)<version>.mcz packages. Most recent downloaded from repos. Some modified). Also a number of other mcz's used to bootstrap and those packages not in the cloned repos)

- pharo, GemStone64Bit<version>     (contains platform specific mcz packages)

- monticello   (contains all our code as a filetree repo)

[Otto Behrens]

PS.

The method Metacello | handleWonkaLoadConflict (extension in the BaselineOfWonkaRuntime package) tries to state how I want to control what's loaded. It actually just prefers BaselineOf over ConfigurationOf.

[Otto Behrens]

| It
| must pick up from disk that there's another version of Grease that I
| want to deploy (i.e. the ConfigurationOfGrease package changed the
| default / core version). And then it must load the new package
| versions.

As far as I can tell you have just described what Metacello does, so I'm not sure what your point is ...

I wanted to say that if I download and a new ConfigurationOfGrease on my machine, my scripts pick it up and load it. I want the new config to be used. If the default / stable version is bumped in it, I want that to be loaded. From disk, on my machine.

| > Soooo, the fact that you want to "Metacello new removeTheRegistry"
| > means that I haven't done a good enough job of communicating this
| > information to you or you have run into bugs ...
| >
| > The Metacello Scripting API is intended to support exactly the
| > behavior you are asking for so we need to work out specifically
| > where it is failing your...
|
| Thanks for the patience, I really battle to understand why I need
| this. I have the code in the DB, on the classes. I don't really care
| about package versions.

In the previous section you claimed that "it must load the new package versions". If you don't care about package versions then how does "it" know which ones are new?

The newly loaded configuration file as explained in the previous point. Read this in the context of locks. If I locked ConfigurationOfGrease, get a new ConfigurationOfGrease package and load it, my understanding is that it will not load the new config & version (if I use my top level loading script). My conclusion is that I should not lock ConfigurationOf<anything>.

[Otto Behrens]

| I want to make sure that the code I'm loading
| is loaded when I'm done. Even after loading with metacello, we
| currently have to reload all the code again by going through the
| MCWorkingCopy registry and loading the packages using Gofer. This is
| because if there are extensions in a package and the package changes
| underneath it, the overrides are lost.

This sounds like a different problem, I use packages with extensions all of the time and I don't have to reload packages usig Gofer, so there must be something different in the way that you do your load or how you define your packages or ??? but I don't have enough detail here to know where the issue might lie. I need specifics, not generalities.

Sorry, it is not extensions that is the problem. This is the scenario.

Package A defines class Foo.

Package A requires: Package B

load this into GS

Move Foo to Package B (edit in Pharo, save packages).

when loading this in GS, B is required by A, thus loaded first. Because A still defines Foo, the loading of B and then A does not work.

[Otto Behrens]

| I am considering this:
| 1. remove all the methods on the classes we work with
| 2. remove classes that are not referenced by the incoming packages
| 3. load all the code
| 4. migrate

I see that we have moved on from the discussion of how to manage external projects into something completely different. I don't think this has anything to do with Metacello.

I would like to see a step by step description of what you are doing and exactly what problems you are experiencing. I don't think you should have to do this, but I clearly don't understand the exact problems that you are facing so I can't judge whether the steps you are threatening to take are necessary or not ...

I was trying to describe a process that Metacello may choose to support or not. In some ways, the combination of Metacello and Monticello does way more than what I need. I understand that. That is OK. But it makes things complex.

I have chosen to use git as a version control system. I want to choose in git what version of what I want to load. 

I also need a mechanism to specify dependencies between packages. This is so that I can load stuff in the right order. I think that these are separate concerns.

When we develop, we also use git to merge code between developers. A second version control system (i.e. Monticello + ConfigurationOf classes) on top of the one I use gets in the way. It feels as if the stuff that I am struggling with has to do with that. But this is perhaps too vague. So please ignore this if it does not make sense.

[Dale K. Henrichs]

Otto,

After taking some time off, I'm ready to get busy with this again:)

I don't understand your whole build process, but I do see the general approach that you are taking ... I can see some limitations to your approach, but I assume that this is working for you or are there still some outstanding problems?

Dale

---

From: "Otto Behrens" <ot...@finworks.biz>
To: meta...@googlegroups.com
Sent: Tuesday, November 26, 2013 11:34:20 PM
Subject: Re: [Metacello] repositories / overrides

[Dale K. Henrichs]

From: "Otto Behrens" <ot...@finworks.biz>
To: meta...@googlegroups.com
Sent: Wednesday, November 27, 2013 6:19:38 AM
Subject: Re: [Metacello] repositories / overrides

The newly loaded configuration file as explained in the previous point. Read this in the context of locks. If I locked ConfigurationOfGrease, get a new ConfigurationOfGrease package and load it, my understanding is that it will not load the new config & version (if I use my top level loading script). My conclusion is that I should not lock ConfigurationOf.

 I will agree, that as things stand right now. You should not lock `ConfigurationOf` ...

However, it is possible to `lock` a #stable version, so the reason for you to not lock ConfigurationOf is not because "it will not load the new config & version", but because locking a ConfigurationOf does not (currently) affect which repository the packages are loaded from (for a ConfigurationOf) and with the current implementation you would still have to use #repositoryOverrides ...

As I think about this, I think we need a locallized version of  #repositoryOverrides: ... as #repositoryOverrides stands, the repository override applies to all projects and we'd like a per project repository override... Like the `lock` command you should be able to make a declaration that "in this image, for all Metacllo load operations, the packages for the project should be loaded from that repository"...

Does this make sense?

[Dale K. Henrichs]

From: "Otto Behrens" <ot...@finworks.biz>
To: meta...@googlegroups.com
Sent: Wednesday, November 27, 2013 6:34:51 AM
Subject: Re: [Metacello] repositories / overrides

Ah, this is why the #atomic load was invented ... in Monticello if you load each package individually moving a definetion (method or class) from one package to another, Monticello will remove the definition from the image if package A is loaded after package B ... on the other hand if you can arrange to do a multi-package load and load package A and package B in the same load, then Monticello will recognize that the definition was removed and "do the right thing"...

In Metacello you set the #loadType: to #atomic to load all of the packages for a project at the same time ... if package A and package B are in separate projects, things can get a bit sticky, but Metacello honors the #atomic load type across project boundaries accumulating packages for one monster load until the end or until a project with a #linear #loadType: is encountered...

Dale

[Dale K. Henrichs]

From: "Otto Behrens" <ot...@finworks.biz>
To: meta...@googlegroups.com
Sent: Wednesday, November 27, 2013 7:08:22 AM
Subject: Re: [Metacello] repositories / overrides

The BaselinOf was invented to support development that is based on Git. With the BaselineOf you only specify your package load order and dependencies.

The rest of the cruft in a ConfigurationOf is there to fill the gap that exists between Monticello and Git and is not needed when using Git ... Actually the when using Git it is still convenient to tag certain commits with a version name and ConfigurationOf is still useful for tagging..

Dale

[Dale K. Henrichs]

Otto,

How does something like the following sound to you:

  Metacello new
    configuration: 'Grease';
    repository: 'filetree:///home/wonka/development/3rdparty/grease/repository`;
    overrideRepository.

  ".... full set of project to local override repository mappings"

  Metacello new

baseline: 'WonkaRuntime';

repository: 'filetree:///somewhere';

load

You'd be able to get rid of a most if not all of the special class-side load-related methods ... I think that I can arrange to implement the `overrideRepository` command and it would work for both baselines and configurations ...

Dale

---

From: "Dale K. Henrichs" <dale.h...@gemtalksystems.com>
To: meta...@googlegroups.com

[Otto Behrens]

Hi, I'm on leave so my responses may be slow. Should I then call this overrideRepository for all the sub-projects that I use? So should my script read something like:

Metacello new

configuration: 'Grease';

overrideRepository.

....

Metacello new

configuration: 'Magritte3';

overrideRepository.

.... (And then all the others listed in my baseline)

Metacello new

baseline: 'WonkaRuntime'

load

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to metacello+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Dale K. Henrichs]

Turn about's fair play:)

That's basically what I'm thinking, but the trick is that you define a overrideRepository for each project:

  Metacello new

    configuration: 'Grease';
    repository: 'filetree:///home/wonka/development/3rdparty/grease/repository`;

    overrideRepository.

....

  Metacello new

    configuration: 'Magritte3';
    repository: 'filetree:///home/wonka/development/3rdparty/magritte3/repository`;

    overrideRepository.

.... (And then all the others listed in my baseline)

  Metacello new

    baseline: 'WonkaRuntime'

    load

The #repository: message is important because you are associating an override repository with each project ... and the overrideRepo will be used whenever the project is referenced ... Grease has the potential to be referenced from multiple projects ....

Dale

[Dale K. Henrichs]

Otto,

Just checking in to see if you are back and want to pick up where we left off?

Dale

---

From: "Otto Behrens" <ot...@finworks.biz>
To: meta...@googlegroups.com

Sent: Monday, December 2, 2013 9:42:26 PM
Subject: [Metacello] Re: repositories / overrides

[Otto Behrens]

Not yet. Will let you know

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to metacello+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Dale K. Henrichs]

Cool!

[Otto Behrens]

> In Metacello you set the #loadType: to #atomic to load all of the packages
> for a project at the same time ... if package A and package B are in
> separate projects, things can get a bit sticky, but Metacello honors the
> #atomic load type across project boundaries accumulating packages for one
> monster load until the end or until a project with a #linear #loadType: is
> encountered...

I've been searching for the right place to do this, but could not find
this on Metacello. MetacelloProject defines it, and it defaults to
atomic (MetacelloProject >> loadType), so I am assuming I'm already
doing an atomic load if I use Metacello. Is this correct? If not, how
would I do this and verify which load type I'm using?

Thanks
Otto

[Otto Behrens]

> How does something like the following sound to you:
>
> Metacello new
> configuration: 'Grease';
> repository:
> 'filetree:///home/wonka/development/3rdparty/grease/repository`;
> overrideRepository.
>
> ".... full set of project to local override repository mappings"
>
>
> Metacello new
>
> baseline: 'WonkaRuntime';
>
> repository: 'filetree:///somewhere';
> load

I've been thinking about this one for a while. And it is difficult,
I'm not sure how to do this nicely. Here's an attempt to write my
script along these lines.

| metacello base |
metacello := Metacello new.
base := '/home/wonka/development/wonka/'.
#(#CommandShell #Gofer #GoferProjectLoader #Grease #GsCore #GsFastCGI
#GsMisc #GsMonticello #GsOB #GsSqueakCommon #HelpSystem #Magritte3
#OmniBrowser #OSProcess #ProfStef #RefactoringBrowser #SeasideTesting
#SoapOpera #SPort2 #Swazoo2 #XMLSupport #XMLWriter
#ZincHTTPComponents)
do: [ :configName |
metacello
configuration: configName;
overrideRepository: base , '3rdparty/shared_mcz' ].
#(#Zinc #GLASS1 #Seaside30)
do: [ :baselineName |
metacello
baseline: baselineName;
overrideRepository: 'filetree://' , base , '3rdparty/' ,
baselineName , '/repository' ].
metacello
baseline: 'WonkaRuntime';
repository: 'filetree://' , base , 'monticello';
load


Something like this? More information that what you need, I understand.

What bothers me about this is that we have to keep these lists of
packages up to date in more than one place.

Currently this is working, more or less:

Metacello new
base := '/home/wonka/development/wonka/'.
metacello
baseline: 'WonkaRuntime';
repository: 'filetree://' , base , 'monticello';
repositoryOverrides:
{base , '3rdparty/shared_mcz'},
(#(#Zinc #GLASS1 #Seaside30)
collect: [ :baselineName | 'filetree://' , base , '3rdparty/'
, baselineName , '/repository' ]);
load

What's ugly about this is that there's a list of repository overrides
and this just appears to throw all the packages in one big pot.

> You'd be able to get rid of a most if not all of the special class-side
> load-related methods ... I think that I can arrange to implement the
> `overrideRepository` command and it would work for both baselines and
> configurations ...

I don't see how this would get rid of it because it just changes the
API of the load script. These class side load related methods are our
'scripts' to load specifically our code. I think I still need them; if
so, should I put them elsewhere? Or should they not exist - do the
appear to do things that don't make sense?

Explosion of threads here...

[Dale K. Henrichs]

----- Original Message -----
| From: "Otto Behrens" <ot...@finworks.biz>
| To: meta...@googlegroups.com
| Sent: Monday, December 30, 2013 1:10:05 AM
| Subject: Re: [Metacello] repositories / overrides
|

Look at the #project method in your configuration. According to MetacelloConfigTemplate>>project the default #loadType is #linear, so if you haven't explicitly made changes, it's likely that you are using #linear.

The original default was #atomic, but I think I got push back on that (probably indirectly) and I changed the deafult to #linear...
|
| Thanks
| Otto

[Dale K. Henrichs]

----- Original Message -----
| From: "Otto Behrens" <ot...@finworks.biz>
| To: meta...@googlegroups.com
| Sent: Monday, December 30, 2013 2:22:15 AM
| Subject: Re: [Metacello] repositories / overrides
|

Yeah, I think that this is pretty much what I was thinking...

|
| What bothers me about this is that we have to keep these lists of
| packages up to date in more than one place.

I only see one place where they are used, What other places are these lists used?

I've been thinking along the lines of being able to keep a list of Metacello instances (one for each of the projects) that is fully specified for each of the projects ... the instances could be stored somewhere and used as needed, or the isntances could be created on demand.

Ideally you could use the same set of Metacello instances for all of your workflow steps:

BaselineOfWonka configurationProjectSpecs do: [:spec |
"download packages to local disk (see [1] for possible sematics)"
spec fetch ].

BaselineOfWonka projectSpecs do: [:spec |
"download packages to local disk (see [1] for possible sematics)"
spec load ].

If this is the type of thing that you'd like to do, then let's work backwards from here to figure out how we can make that happen...

[1] https://github.com/dalehenrich/metacello-work/blob/master/docs/MetacelloScriptingAPI.md#fetching

[Otto Behrens]

> | #(#CommandShell #Gofer #GoferProjectLoader #Grease #GsCore #GsFastCGI
> | #GsMisc #GsMonticello #GsOB #GsSqueakCommon #HelpSystem #Magritte3
> | #OmniBrowser #OSProcess #ProfStef #RefactoringBrowser #SeasideTesting
> | #SoapOpera #SPort2 #Swazoo2 #XMLSupport #XMLWriter
> | #ZincHTTPComponents)

> | #(#Zinc #GLASS1 #Seaside30)

> | What bothers me about this is that we have to keep these lists of
> | packages up to date in more than one place.
>
> I only see one place where they are used, What other places are these lists used?

My issue is that these lists are embedded in the spec(s) on the
various configurations. If I miss one in this list (that a spec
defines), then I will have trouble finding it.

I still prefer just using repositoryOverrides: because it's less maintenance.

> I've been thinking along the lines of being able to keep a list of Metacello instances (one for each of the projects) that is fully specified for each of the projects ... the instances could be stored somewhere and used as needed, or the isntances could be created on demand.

You've already got an object in the MetacelloProjectRegistration, so I
think that it would be redundant information, which may be
inconsistent and lead to trouble. It may be better to extend the
project spec object to do more.

> Ideally you could use the same set of Metacello instances for all of your workflow steps:
>
> BaselineOfWonka configurationProjectSpecs do: [:spec |
> "download packages to local disk (see [1] for possible sematics)"
> spec fetch ].

Sort of. I want to basically avoid caching, especially the
package-cache because it slows the loading down (If I'm loading stuff
from a repo on the local disk, git or otherwise, it writes a copy of
the package into the package cache. This is redundant as I have the
files on my local disk already.) and I've had situations where there
was inconsistency. This is mainly because we're still manually merging
the stupid version files...

> If this is the type of thing that you'd like to do, then let's work backwards from here to figure out how we can make that happen...

What I would like is to use my original script with my spec (not the
stored / cached spec objects) and ask metacello to copy the mcz
packages into my override repositories. That is because I would like
to update the spec first and then ask it to find what it needs, but
put it in my local repo. (For mcz's and configs).

[Dale Henrichs]

On Mon, Jan 6, 2014 at 11:10 PM, Otto Behrens <ot...@finworks.biz> wrote:

> | #(#CommandShell #Gofer #GoferProjectLoader #Grease #GsCore #GsFastCGI
> | #GsMisc #GsMonticello #GsOB #GsSqueakCommon #HelpSystem #Magritte3
> | #OmniBrowser #OSProcess #ProfStef #RefactoringBrowser #SeasideTesting
> | #SoapOpera #SPort2 #Swazoo2 #XMLSupport #XMLWriter
> | #ZincHTTPComponents)

> | #(#Zinc #GLASS1 #Seaside30)

> | What bothers me about this is that we have to keep these lists of
> | packages up to date in more than one place.
>
> I only see one place where they are used, What other places are these lists used?

My issue is that these lists are embedded in the spec(s) on the
various configurations. If I miss one in this list (that a spec
defines), then I will have trouble finding it.

I still prefer just using repositoryOverrides: because it's less maintenance.

> I've been thinking along the lines of being able to keep a list of Metacello instances (one for each of the projects) that is fully specified for each of the projects ...  the instances could be stored somewhere and used as needed, or the isntances could be created on demand.

You've already got an object in the MetacelloProjectRegistration, so I
think that it would be redundant information, which may be
inconsistent and lead to trouble. It may be better to extend the
project spec object to do more.

The registry reflects the state of the image and currently is completely independent of the Metacello instances ... resolving inconsistencies (under developer control) is what the the scripting api is all about.

Right now I am not worried about whether or not the project spec is the right object or not ... right now I am interested in finding out what information we need to keep of track to do the job right ...

> Ideally you could use the same set of Metacello instances for all of your workflow steps:
>
>   BaselineOfWonka configurationProjectSpecs do: [:spec |
>     "download packages to local disk (see [1] for possible sematics)"
>     spec fetch ].

Sort of. I want to basically avoid caching, especially the
package-cache because it slows the loading down (If I'm loading stuff
from a repo on the local disk, git or otherwise, it writes a copy of
the package into the package cache. This is redundant as I have the
files on my local disk already.) and I've had situations where there
was inconsistency. This is mainly because we're still manually merging
the stupid version files...

`fetch` is not part of the load operation ... you didn't put this into your workflow, but at some point in time you have to refresh your local disk repositories with new copies of the configuration and any new mcz files that have been added to SmalltalkHub or SS3 or whatever ... 

`fetch` would do that job for you and I'm postulating that the "project specs" would contain information like source repository and target repository

The fetch operation would `fetch` the mcz files from the various http repos in the universe and consolidate them in the desired local directory ...

This step was missing from your workflow, but presumably a process needs to be run that polls and downloads fresh versions of the dependent projects at controlled intervals ...

> If this is the type of thing that you'd like to do, then let's work backwards from here to figure out how we can make that happen...

What I would like is to use my original script with my spec (not the
stored / cached spec objects) and ask metacello to copy the mcz
packages into my override repositories.

That is what I imagined the `fetch` command doing ...

 

That is because I would like
to update the spec first and then ask it to find what it needs, but
put it in my local repo. (For mcz's and configs).

So I postulate that there are three fundamental operations that are done: 

  1. copy mczs and configurations to the local override repositories

  2. pull new commits from remote git repos to local repos

  3. load from local (overrides and git) repositories into image 

These three do: loops represent those three operations:

  

 BaselineOfWonka configurationProjectSpecs do: [:spec |
    "copy mcz packages to my override repositories"
    spec copy ].

 

BaselineOfWonka baselineProjectSpecs do: [:spec |

    "pull new git commits to local git repos"
    spec pull ].

  BaselineOfWonka configurationProjectSpecs, 

    BaselineOfWonka baselineProjectSpecs do: [:spec |

      "load from override repositories into image"
      spec load ].

Working backwards from here we need to identify the information that is required to do each of the above steps ... 

I think that if we do this we'll be able to have one method/list/script/? where all of the basic information is stored. 

I want to solve this set of problems independent of the current scripting API. 

My expectation is that we will fold the functionality into the scripting API, but I am willing to make radical changes to the scripting api to get there...

It may make sense to have something like the Metacello configuration (executable specification) for this although for these purposes a method pr two that simple creates instances on demand would be a pretty good soltuion ...

Of course right now the interesting thing to figure out is what information needs to be glommed together for each of the two different project types (mcz vs git) ...

Dale