Strange packaging errors

[Richard Sargent]

CgImageFileFormat class>>#allFormats - Reference to excluded class #GbtPNGFileFormat
CgImageFileFormat class>>#allFormats - Reference (by name) to excluded class #GbtPNGFileFormat

The method in question?

allFormats
"Answer a collection containing all concrete CgImageFileFormat classes
 which are subclasses of the receiver."

^self allSubclasses

The first one is bad enough, but the second one is beyond belief.

[Marcus Wagner]

Courious. There seems to exist a special packager logic (!) dealing with adjustments of class references making use of this protocol. 

I never had heard about that before, so I have no idea about the underlying problem and the presented solution, where allFormats is called.

I hesitate here as I almost never used private classes and I did not ever package something containing private classes. 

Might be related to this.

However it matches my experiences about mixing "meta" information (like allFormats) with active (to be packaged) runtime code.

Can it be, that allFormats (intended to deliver metainformation) gets accidentially (somehow) in scope of to be packaged material?

It remembers me to (ancient) situations where references to development code were left accidentially in the to be packed code (like calls to inspect s.th.) causing the compiler environment to be included in the package which has to fail, as a contraction of the goal of packaging. 

[Louis LaBrunda]

Hi Guys,

I apologize for hijacking this thread (a little).  I want to put my 2 cents for options in the packager that are less aggressive at removing things.  Sure, certain stuff shouldn't get packaged and that's fine.  But the packager doesn't need to remove every last byte.  I have two programs that run on a Raspberry Pi.  It is a tiny little computer the size of a deck of playing cards.  You can get one with a power supply and a memory card for about $100.  The memory cards are plenty big enough to run a GUI version of Linux and my 11MB image and the maybe 50MB of other stuff needed to run a VA Smalltalk program.  It even runs the development environment.  A bit slow but it all fits, no problem.  If the packager ran faster but I ended up with a 20MB image, that would be fine with me.

My point isn't that the packager need to run faster on the Raspberry Pi, there aren't enough of us using it for that to matter.  My point is, memory is cheap but our time debugging oddball packager problems to save a few bytes of image size is not worth it.

Lou

[Richard Sargent]

I agree.

On Tue, Sep 23, 2025 at 9:32 AM Louis LaBrunda <loulab...@gmail.com> wrote:

Hi Guys,

I apologize for hijacking this thread (a little).  I want to put my 2 cents for options in the packager that are less aggressive at removing things.  Sure, certain stuff shouldn't get packaged and that's fine.  But the packager doesn't need to remove every last byte.  I have two programs that run on a Raspberry Pi.  It is a tiny little computer the size of a deck of playing cards.  You can get one with a power supply and a memory card for about $100.  The memory cards are plenty big enough to run a GUI version of Linux and my 11MB image and the maybe 50MB of other stuff needed to run a VA Smalltalk program.  It even runs the development environment.  A bit slow but it all fits, no problem.  If the packager ran faster but I ended up with a 20MB image, that would be fine with me.

My point isn't that the packager need to run faster on the Raspberry Pi, there aren't enough of us using it for that to matter.  My point is, memory is cheap but our time debugging oddball packager problems to save a few bytes of image size is not worth it.

Design choices made when we had 486 and Pentium processes and a few dozen megabytes of memory required aggressive tactics.

With computers O(2) faster than then and O(3) more memory, the forces are different.

Mind you, it is probably harder to dial back the aggression in an established product than one might think.

Lou

On Tuesday, September 23, 2025 at 11:46:21 AM UTC-4 Marcus Wagner wrote:

Courious. There seems to exist a special packager logic (!) dealing with adjustments of class references making use of this protocol. 

I never had heard about that before, so I have no idea about the underlying problem and the presented solution, where allFormats is called.

I hesitate here as I almost never used private classes and I did not ever package something containing private classes. 

Might be related to this.

However it matches my experiences about mixing "meta" information (like allFormats) with active (to be packaged) runtime code.

Can it be, that allFormats (intended to deliver metainformation) gets accidentially (somehow) in scope of to be packaged material?

It remembers me to (ancient) situations where references to development code were left accidentially in the to be packed code (like calls to inspect s.th.) causing the compiler environment to be included in the package which has to fail, as a contraction of the goal of packaging. 

Richard Sargent schrieb am Dienstag, 23. September 2025 um 03:49:15 UTC+2:

CgImageFileFormat class>>#allFormats - Reference to excluded class #GbtPNGFileFormat
CgImageFileFormat class>>#allFormats - Reference (by name) to excluded class #GbtPNGFileFormat

The method in question?

allFormats
"Answer a collection containing all concrete CgImageFileFormat classes
 which are subclasses of the receiver."

^self allSubclasses

The first one is bad enough, but the second one is beyond belief.

--
You received this message because you are subscribed to a topic in the Google Groups "VAST Community Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/va-smalltalk/B0YqzqCZn-w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to va-smalltalk...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/va-smalltalk/423a43cf-88de-4c39-9fc5-022429c1ae00n%40googlegroups.com.

[Richard Sargent]

See below.

On Tue, Sep 23, 2025 at 8:46 AM 'Marcus Wagner' via VAST Community Forum <va-sma...@googlegroups.com> wrote:

Courious. There seems to exist a special packager logic (!) dealing with adjustments of class references making use of this protocol. 

I never had heard about that before, so I have no idea about the underlying problem and the presented solution, where allFormats is called.

I hesitate here as I almost never used private classes and I did not ever package something containing private classes. 

Might be related to this.

I can't imaging how private vs public classes would actually be packaged differently.

However, the point about this method being included at all is valid and that is something I will investigate separately.

The fact remains that these errors are bogus. I admit the possibility that they are trying to point out something important or potentially so. But, if that's the case, the error message is way off base.

 

However it matches my experiences about mixing "meta" information (like allFormats) with active (to be packaged) runtime code.

Can it be, that allFormats (intended to deliver metainformation) gets accidentially (somehow) in scope of to be packaged material?

It remembers me to (ancient) situations where references to development code were left accidentially in the to be packed code (like calls to inspect s.th.) causing the compiler environment to be included in the package which has to fail, as a contraction of the goal of packaging. 

Richard Sargent schrieb am Dienstag, 23. September 2025 um 03:49:15 UTC+2:

CgImageFileFormat class>>#allFormats - Reference to excluded class #GbtPNGFileFormat
CgImageFileFormat class>>#allFormats - Reference (by name) to excluded class #GbtPNGFileFormat

The method in question?

allFormats
"Answer a collection containing all concrete CgImageFileFormat classes
 which are subclasses of the receiver."

^self allSubclasses

The first one is bad enough, but the second one is beyond belief.

--

You received this message because you are subscribed to a topic in the Google Groups "VAST Community Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/va-smalltalk/B0YqzqCZn-w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to va-smalltalk...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/va-smalltalk/6a044265-7f51-4b23-9df8-142752f377f2n%40googlegroups.com.

[Richard Sargent]

For diagnostic purposes, that was encountered in an 11.0.1 image.

I doubt the packager has seen significant changes between then and now.

--

You received this message because you are subscribed to a topic in the Google Groups "VAST Community Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/va-smalltalk/B0YqzqCZn-w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to va-smalltalk...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/va-smalltalk/7c841af7-2057-4be3-aeb1-b04ed91f4510n%40googlegroups.com.

[Richard Sargent]

On Tuesday, September 23, 2025 at 9:32:40 AM UTC-7 Louis LaBrunda wrote:

Hi Guys,

I apologize for hijacking this thread (a little).  I want to put my 2 cents for options in the packager that are less aggressive at removing things.  Sure, certain stuff shouldn't get packaged and that's fine.  But the packager doesn't need to remove every last byte.  I have two programs that run on a Raspberry Pi.  It is a tiny little computer the size of a deck of playing cards.  You can get one with a power supply and a memory card for about $100.  The memory cards are plenty big enough to run a GUI version of Linux and my 11MB image and the maybe 50MB of other stuff needed to run a VA Smalltalk program.  It even runs the development environment.  A bit slow but it all fits, no problem.  If the packager ran faster but I ended up with a 20MB image, that would be fine with me.

My point isn't that the packager need to run faster on the Raspberry Pi, there aren't enough of us using it for that to matter.  My point is, memory is cheap but our time debugging oddball packager problems to save a few bytes of image size is not worth it.

I think there is a defect in CgImageSupport class>>#epClassReferenceAdjustmentsFor:in:.
It uses CgFileFormat subclasses collect: [:aClass | aClass name] to get the subclasses, even those that shouldn't be packaged. And later it gets the AllSubclasses of these abstract subclasses, regardless of whether they should be packaged.

[Marcus Wagner]

Not to escalate this discussion further (having a long tradition) - it is your choice either to run (more or less) the development image or to clearly separate everything which is not to be included, hm, hence to carry the burden, to package. 

Packaging in its nature is aggressive. It is a final decision if something is to be included or not. 

Alpha or Beta errors (in the statistical science sense) have to be avoided at all circumstances because of the dangeous consequenses.

In some situations, you are forced to package and then you depend on its rigourous stability. 

I recently was in need to create and run a (tiny) Smalltalk program without GUI (aka headless, an equivalent to old C style Unix programs, command with arguments, pipes, redirections etc, under Windows). 

The only way to produce such a Smalltalk program is XP packaging. This adanced critical form of packaging even does not allow you to load certain things of the development in the image to be packaged, despite of even being able to execute s.th. in it. The material to be packaged must be clean in a clinical sense.

This is a counter example against the argument of having plenty memory abound: if your material has to cross OS boundaries, you must package.

Currently almost everybody is forced to do so when producing s.th. for another target platform, as the well established conveniences of the current development environment do not exist elsewhere than under Windows. 

May be that this will become enhanced in the near future after modernizing the more or less antique original XMotif infrastructure (under Linux) I occasionally use. Developement here is announced and underway, I hope.

I definitely cannot imagine to produce my stuff (running 2-3 high resolution screens) on a Raspberry Pi, because of complexity (of the required resources, not because of their size). 

Personally, I depend also on the ability to see as many things as possible, hence my visual resolution is currently a triple of monitors, having 2 * 4K plus a 2.5K on the laptop, my personal wide screen development cinema.

Another essential argument is operational cost, which is significantly reduced as implication of having packaged. 

I remember one migration project where the daily production workload was processed using 20 servers a 250MB, double CPU, under the rule of a mainframe.

The migrated 1-1 Java replacement finally required 30 servers a 750MB, and the host has to be upgraded.

The operation cost initially grew tenfold.  Initially that turned out as an economic desaster.

The reason of that was the lack of packaging, as the replacement could not extract the essence. 

However, the hope was to make use of the benefits of the (then) new Java environments and bypassing, the painstaking packaging process, so to reduce significantly the development costs.

Another argument pro packaging: grown vulnerability of the result because of not having removed unnessary things (e.g. all apps run at least their initialisation code when being loaded, no matter if they are actually required to run or not).

In general it is your choice to package s.th. or not, but when providing a framework application as deliverable, you have no such choice:

to be useful, it must be able to be packaged.

I am going to inspect the CGImageSupport application, to understand its underlying problem it had to tackle. 

Obviously the current solution is not stable in the sense of the observed packager complaints. 

As I already mentioned, I found some special support in the packager which had to assist in here, which I do not understand yet.

That gives me an initial hint: an OS specific class hierarchy providing the functionality, which ought to be totally different in other platforms, but has to be packaged equal. The protocol is targetting class references and providing replacments. This an atypical package problem, I personally never had before - I usually hide such problems behind factory patterns.

Finally, this is not be misunderstood as my critic to current XMotif implementation or making use of Raspberry or even Java. 

Sometimes architectural decisions have widespread consequences: 

Smalltalk (or any other integrated development) has to bear the burdon of packaging (or similar production steps) because of the benefit of that integrated development.

Whenever I work in other non-integrated languages, latest when debugging, I occasionally desiderate to run some code snippet integrated, to inspect s.th.

[Marcus Wagner]

In such a case allSubclasses usually is to be replaced by Application defined or similar. However, such would not scope any extensions of Application (or subapplications). An extension then has to care about its own packaging rules covering its extension. Hm. As I wrote elsewhere I do not yet understand the underneath CGImageSupport problem, causing all of this. Subapplications are usually covering platform specifics.

[Marcus Wagner]

Ok. now I understand the code in question.

First a hierarchy of classes implement marshalling of file formats in both directions (read & write) from / to two different external representations (in memory = byte arrays or as file = CfsFileDescriptor). This hierarchy contains abstract and concrete elements.

Abstract members are CgIconFileFormat and CgImageFileFormat.

Abstract members cannot be read or written.

Concrete members can be read or written.

Reading a file poses a factory problem (a hen - egg problem). You have to read s.th. to be able to decide which kind of format you have to apply.

This may fail, if the format is not known as no concrete class is provided for a given indicator.

The protocol allFormats delivers only concrete elements.

The code assumes, that concrete classes inherit from an abstract class / and abstract classes inherit from CgFileFormat.

That means, e.g. a concrete class may not be subclassed by an abstract class. So far this mimics concreteness which has no direct syntax notation in Smalltalk. Class abstraction in Smalltalk can be implemented by individual conventions, but is not directly supported by the languuage syntax. This allows for individual subtle nuances in the implementation -  e.g. a Java abstract class implies further rules which usually do not hold in a Smalltalk world. E.g. there is no Smalltalk restriction about instantiating an abstract class (violating the concept).

The protocol allFormats however is also used in packaging instructions, the  packager epClassReferenceAdjustmentsFor:in: Protocol.

This concept is new for me, but understandable. It creates virtual references of a class to be obeyed when packaging.

Consider a method contains Abstract doSomething. Instead of including Abstract, all its concrete subclasses have to be included in the package, but Abstract itself not. This could have been achieved by using other, "normally used" packaging instructions, but the  epClassReferenceAdjustmentsFor:in: was chosen as shortcut for framework applications.

It is used (as far as I saw) in the framework EwNotebooks, EwList and CgImageSupport.

This protocol was in use in Visual Age 4, 4.01 and 4.02 and then was replaced by a variation of it.

https://www.instantiations.com/vast-support/documentation/1410/mi/migrat46.html#

So far the story seems clear: a special marshalling problem of framework is handled mimicking the abstract class concept (missing in Smalltalk) at runtime, providing the packager information about potential target instances which are created at runtime based on provided input values, so that the need for them is not foreseeable by static analysis at package time.

And now I turn to the initiating complaint:  GbtPNGFileFormat.

This class is not part of the framework. 

However it seems to be involved, e.g. it subclasses some of the two abstract classes  CgIconFileFormat and CgImageFileFormat.

The application defining  GbtPNGFileFormat is the culprit. 

Either it extends the intended package providing an additional format to be used,

then this defining application has to be part of the package (to avoid the mentioned complaint).

Or this defining application is not to be used at all in this package, then it must be unloaded before packaging.

Two critics here: 

1 The (packaging) shorthand  epClassReferenceAdjustmentsFor:in: to be applied in frameworks is risking problems when being derivated.

[Seems that this subsequently led to newer form after version 4.02, unclear if a modernisation here helps].

2 Subclassing such a construct implies further support which is missing here. Warning to all implemtors of applications of conceptual abstract classes of the applications  EwNotebooks, EwList and CgImageSupport.

Conclusion: that all is related to the nitfy definition of abstract class, which was and is excluded from the syntax of the language.

I guess this is intentional: it would pose severe problems handling the class instance of an abstract class. See Java e.g.

Richard Sargent schrieb am Dienstag, 23. September 2025 um 03:49:15 UTC+2:

[Marcus Wagner]

At the second glance, this protocol allFormats, returns only concrete classes and excludes self = CgImageFileFormat. 

The messages (complaints) the first one is related to the Visual Age 4, 4.01 and 4.02, which became obsolete later. 

The second message is related to newer form Visual Age 5 and higher.

The problem I see is that the new form in some form is also serving the old style, but improper concerning the error messages, duplicating and irritating.

As I said, the source is the provider of the class GbtPNGFileFormat.

[Richard Sargent]

On Wednesday, September 24, 2025 at 5:39:59 AM UTC-7 Marcus Wagner wrote:

At the second glance, this protocol allFormats, returns only concrete classes and excludes self = CgImageFileFormat. 

The messages (complaints) the first one is related to the Visual Age 4, 4.01 and 4.02, which became obsolete later. 

The second message is related to newer form Visual Age 5 and higher.

The problem I see is that the new form in some form is also serving the old style, but improper concerning the error messages, duplicating and irritating.

As I said, the source is the provider of the class GbtPNGFileFormat.

Except it is present in an excluded application and so should not be considered. As I said, the method is greedy and ignores the packaging instructions. I would say the original developer expected there to only be classes that should be packaged. i.e. possibly only VA methods; no added classes; no development-only classes, etc.

[Marcus Wagner]

First, a method can never ignore packaging instructions.

Second, the historic decision was that the problem (of marshalling external data) was implemented as a class hierarchy, without use of a registry.

Third, the method allFormats is not greedy. The method allFormats delivers all concrete children, for two purposes: a) at runtime, for the decision which concrete class processes a given input and b) at package time, which classes are not referenced directly but are to be included by the packager to realize a). So greedy is the unreflected use of the method as input for the packaging rule.

Short excursus 1: at the time of this coding of  CgImageSupport  (1995-96) 

* only a few of us heard about "design patterns" and this is an example of a factory.

* and packaging needed some support to treat such particular factory problems. 

This initially lead to providing the epClassReferenceAdjustmentsFor:in:  protocol, which a little bit later (Envy 5.0) became obsolete. 

Here we see an example, why. There still exist three area where it is still in use (see elsewhere in my response)

One correct way to handle this packaging problem is

a) every packaging instruction in an application has to be restricted to its own classes to avoid cross impacts. That means, a protocol like

epClassReferenceAdjustmentsFor:in: in an application should not treat foreign classes, the result of allFormats has to be filtered out.

b) if an application extends the hierarchy, it must react similar, e.g. do the same thing, contain another, its own packaging instruction dealing its own extension, in the same way, e.g. filtering out the resuls of allFormats for its own classes. 

Short excursus 2: a critic of using a hierarchy to implement a factory, to make use of polymorphism: naturally this leads to a pattern (or rule) how to package it, but we see here such a common rule has to be restricted to the scope of the package (and there is no way to express this in general, such a rule has to be added to the global package instruction, not per application).

End of excursus,

Finally, an important question is the purpose of the Class GbtPNGFileFormat, the cause of this dilemma?

a) if it is not needed, it should not be loaded (so to bypass the faulty parametrisation of the call of epClassReferenceAdjustmentsFor:in:).

b) if it is needed, then it must be listed as a prerequisite or included to be part of the package.

My suspicion is (let me know if I am incorrect): 

this class  GbtPNGFileFormat provides a method with a (temporary) fix of a code (in CgImageSupport) to be packaged. 

If this is the case, then it is needed (see b) before).

Now repairing a brittle factory construct in frameworks is an art, as we can see here. 

It is not sufficient to subclass a defect. 

It requires also to care about collatoral impacts, the packaging rule of CgImageSupport demonstrates this.

Excursus 3 about the aspect of privacy of classes: the file format classes are candidates. If they would have been declared private classes, the whole problem would not exist, as the given package rule would not silently extend to other foreign classes, as those could not subclass a private class.

But I guess the design of the hierarchy was made with the idea of extensibily in mind: what if there is another new format to be supported?

So private class was not an option. Instead that was vaguely left open to be able to later add subclasses on demand. 

However, that does not solve the (initial) factory decision, which class is to be used for a given input. Eg. the decision has also to be made extendable, allFormats alone won't do. The construct is conceptually incomplete. A reason of the (packaging) instability.

Conclusion: a old proverb saying "never touch a running system" led to a bunch of problems - even the existance of an initially unsuspicious class is abe to crash the packaging process. 

Did you ever thing about directly replacing the faulty method?

A) directly edit code at the source, make a scratch edition of CgImageSupport and use your own fix when packaging, as long as a fix is officially released).

B) instead of manipulation foreign source, just swap the faulty method, replace the compiled method of CgImageSupport, e.g. when you are loading it in your image.

Depending on other implications, decide what and how to patch. Subclassing is not an ideal way to do this.

Let me know if there is something left to be discussed about this incidence.

[Richard Sargent]

A bit of background. The GbtPNGFileFormat class is used by the GBS development image test suite to screen capture failing GUI tests. One of the tests is packaging a simple application to ensure GBS changes have not broken that ability.
The packaging instructions for that sample application do not include the "test" stuff. The Gbt prefix distinguishes this from private code Gbx prefix and public code Gbs prefix in the GBS product.

I said greedy because #allSubclasses is inherently greedy. It takes everything without regard to any other rules.

And yes, that class is a stop gap to work around a defect that has been fixed but not yet publicly released. And of course, since we support GBS against a variety of VAST versions, we have a problem relying on the fixed version alone. And yes, I could have the loading of the GBS development configuration map hack the method in question, but then tests for accidentally polluting VA code are more complicated. Fortunately, I don't have to provide that fix to GemTalk customers, since the PNG stuff isn't needed (by GBS) in a customer's image.

Given that # epClassReferenceAdjustmentsFor:in: is an obsolete protocol, and that VAST should explicitly include the file format classes via packaging instructions, we wouldn't be having this conversation if that had happened some years ago. And you are correct that if I wanted my class included in the packaged image, it would be my responsibility to provide packaging instructions to do so.

As an aside Marcus, you know a *lot* about VA's history. What is your history / bio / CV in this regard?