Strange packaging errors

[Richard Sargent]

CgImageFileFormat class>>#allFormats - Reference to excluded class #GbtPNGFileFormat
CgImageFileFormat class>>#allFormats - Reference (by name) to excluded class #GbtPNGFileFormat

The method in question?

allFormats
"Answer a collection containing all concrete CgImageFileFormat classes
 which are subclasses of the receiver."

^self allSubclasses

The first one is bad enough, but the second one is beyond belief.

[Marcus Wagner]

Courious. There seems to exist a special packager logic (!) dealing with adjustments of class references making use of this protocol. 

I never had heard about that before, so I have no idea about the underlying problem and the presented solution, where allFormats is called.

I hesitate here as I almost never used private classes and I did not ever package something containing private classes. 

Might be related to this.

However it matches my experiences about mixing "meta" information (like allFormats) with active (to be packaged) runtime code.

Can it be, that allFormats (intended to deliver metainformation) gets accidentially (somehow) in scope of to be packaged material?

It remembers me to (ancient) situations where references to development code were left accidentially in the to be packed code (like calls to inspect s.th.) causing the compiler environment to be included in the package which has to fail, as a contraction of the goal of packaging. 

[Louis LaBrunda]

Hi Guys,

I apologize for hijacking this thread (a little).  I want to put my 2 cents for options in the packager that are less aggressive at removing things.  Sure, certain stuff shouldn't get packaged and that's fine.  But the packager doesn't need to remove every last byte.  I have two programs that run on a Raspberry Pi.  It is a tiny little computer the size of a deck of playing cards.  You can get one with a power supply and a memory card for about $100.  The memory cards are plenty big enough to run a GUI version of Linux and my 11MB image and the maybe 50MB of other stuff needed to run a VA Smalltalk program.  It even runs the development environment.  A bit slow but it all fits, no problem.  If the packager ran faster but I ended up with a 20MB image, that would be fine with me.

My point isn't that the packager need to run faster on the Raspberry Pi, there aren't enough of us using it for that to matter.  My point is, memory is cheap but our time debugging oddball packager problems to save a few bytes of image size is not worth it.

Lou

[Richard Sargent]

I agree.

On Tue, Sep 23, 2025 at 9:32 AM Louis LaBrunda <loulab...@gmail.com> wrote:

Hi Guys,

I apologize for hijacking this thread (a little).  I want to put my 2 cents for options in the packager that are less aggressive at removing things.  Sure, certain stuff shouldn't get packaged and that's fine.  But the packager doesn't need to remove every last byte.  I have two programs that run on a Raspberry Pi.  It is a tiny little computer the size of a deck of playing cards.  You can get one with a power supply and a memory card for about $100.  The memory cards are plenty big enough to run a GUI version of Linux and my 11MB image and the maybe 50MB of other stuff needed to run a VA Smalltalk program.  It even runs the development environment.  A bit slow but it all fits, no problem.  If the packager ran faster but I ended up with a 20MB image, that would be fine with me.

My point isn't that the packager need to run faster on the Raspberry Pi, there aren't enough of us using it for that to matter.  My point is, memory is cheap but our time debugging oddball packager problems to save a few bytes of image size is not worth it.

Design choices made when we had 486 and Pentium processes and a few dozen megabytes of memory required aggressive tactics.

With computers O(2) faster than then and O(3) more memory, the forces are different.

Mind you, it is probably harder to dial back the aggression in an established product than one might think.

Lou

On Tuesday, September 23, 2025 at 11:46:21 AM UTC-4 Marcus Wagner wrote:

Courious. There seems to exist a special packager logic (!) dealing with adjustments of class references making use of this protocol. 

I never had heard about that before, so I have no idea about the underlying problem and the presented solution, where allFormats is called.

I hesitate here as I almost never used private classes and I did not ever package something containing private classes. 

Might be related to this.

However it matches my experiences about mixing "meta" information (like allFormats) with active (to be packaged) runtime code.

Can it be, that allFormats (intended to deliver metainformation) gets accidentially (somehow) in scope of to be packaged material?

It remembers me to (ancient) situations where references to development code were left accidentially in the to be packed code (like calls to inspect s.th.) causing the compiler environment to be included in the package which has to fail, as a contraction of the goal of packaging. 

Richard Sargent schrieb am Dienstag, 23. September 2025 um 03:49:15 UTC+2:

CgImageFileFormat class>>#allFormats - Reference to excluded class #GbtPNGFileFormat
CgImageFileFormat class>>#allFormats - Reference (by name) to excluded class #GbtPNGFileFormat

The method in question?

allFormats
"Answer a collection containing all concrete CgImageFileFormat classes
 which are subclasses of the receiver."

^self allSubclasses

The first one is bad enough, but the second one is beyond belief.

--
You received this message because you are subscribed to a topic in the Google Groups "VAST Community Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/va-smalltalk/B0YqzqCZn-w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to va-smalltalk...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/va-smalltalk/423a43cf-88de-4c39-9fc5-022429c1ae00n%40googlegroups.com.

[Richard Sargent]

See below.

On Tue, Sep 23, 2025 at 8:46 AM 'Marcus Wagner' via VAST Community Forum <va-sma...@googlegroups.com> wrote:

Courious. There seems to exist a special packager logic (!) dealing with adjustments of class references making use of this protocol. 

I never had heard about that before, so I have no idea about the underlying problem and the presented solution, where allFormats is called.

I hesitate here as I almost never used private classes and I did not ever package something containing private classes. 

Might be related to this.

I can't imaging how private vs public classes would actually be packaged differently.

However, the point about this method being included at all is valid and that is something I will investigate separately.

The fact remains that these errors are bogus. I admit the possibility that they are trying to point out something important or potentially so. But, if that's the case, the error message is way off base.

 

However it matches my experiences about mixing "meta" information (like allFormats) with active (to be packaged) runtime code.

Can it be, that allFormats (intended to deliver metainformation) gets accidentially (somehow) in scope of to be packaged material?

It remembers me to (ancient) situations where references to development code were left accidentially in the to be packed code (like calls to inspect s.th.) causing the compiler environment to be included in the package which has to fail, as a contraction of the goal of packaging. 

Richard Sargent schrieb am Dienstag, 23. September 2025 um 03:49:15 UTC+2:

CgImageFileFormat class>>#allFormats - Reference to excluded class #GbtPNGFileFormat
CgImageFileFormat class>>#allFormats - Reference (by name) to excluded class #GbtPNGFileFormat

The method in question?

allFormats
"Answer a collection containing all concrete CgImageFileFormat classes
 which are subclasses of the receiver."

^self allSubclasses

The first one is bad enough, but the second one is beyond belief.

--

You received this message because you are subscribed to a topic in the Google Groups "VAST Community Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/va-smalltalk/B0YqzqCZn-w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to va-smalltalk...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/va-smalltalk/6a044265-7f51-4b23-9df8-142752f377f2n%40googlegroups.com.

[Richard Sargent]

For diagnostic purposes, that was encountered in an 11.0.1 image.

I doubt the packager has seen significant changes between then and now.

--

You received this message because you are subscribed to a topic in the Google Groups "VAST Community Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/va-smalltalk/B0YqzqCZn-w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to va-smalltalk...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/va-smalltalk/7c841af7-2057-4be3-aeb1-b04ed91f4510n%40googlegroups.com.

[Richard Sargent]

On Tuesday, September 23, 2025 at 9:32:40 AM UTC-7 Louis LaBrunda wrote:

Hi Guys,

I apologize for hijacking this thread (a little).  I want to put my 2 cents for options in the packager that are less aggressive at removing things.  Sure, certain stuff shouldn't get packaged and that's fine.  But the packager doesn't need to remove every last byte.  I have two programs that run on a Raspberry Pi.  It is a tiny little computer the size of a deck of playing cards.  You can get one with a power supply and a memory card for about $100.  The memory cards are plenty big enough to run a GUI version of Linux and my 11MB image and the maybe 50MB of other stuff needed to run a VA Smalltalk program.  It even runs the development environment.  A bit slow but it all fits, no problem.  If the packager ran faster but I ended up with a 20MB image, that would be fine with me.

My point isn't that the packager need to run faster on the Raspberry Pi, there aren't enough of us using it for that to matter.  My point is, memory is cheap but our time debugging oddball packager problems to save a few bytes of image size is not worth it.

I think there is a defect in CgImageSupport class>>#epClassReferenceAdjustmentsFor:in:.
It uses CgFileFormat subclasses collect: [:aClass | aClass name] to get the subclasses, even those that shouldn't be packaged. And later it gets the AllSubclasses of these abstract subclasses, regardless of whether they should be packaged.

[Marcus Wagner]

Not to escalate this discussion further (having a long tradition) - it is your choice either to run (more or less) the development image or to clearly separate everything which is not to be included, hm, hence to carry the burden, to package. 

Packaging in its nature is aggressive. It is a final decision if something is to be included or not. 

Alpha or Beta errors (in the statistical science sense) have to be avoided at all circumstances because of the dangeous consequenses.

In some situations, you are forced to package and then you depend on its rigourous stability. 

I recently was in need to create and run a (tiny) Smalltalk program without GUI (aka headless, an equivalent to old C style Unix programs, command with arguments, pipes, redirections etc, under Windows). 

The only way to produce such a Smalltalk program is XP packaging. This adanced critical form of packaging even does not allow you to load certain things of the development in the image to be packaged, despite of even being able to execute s.th. in it. The material to be packaged must be clean in a clinical sense.

This is a counter example against the argument of having plenty memory abound: if your material has to cross OS boundaries, you must package.

Currently almost everybody is forced to do so when producing s.th. for another target platform, as the well established conveniences of the current development environment do not exist elsewhere than under Windows. 

May be that this will become enhanced in the near future after modernizing the more or less antique original XMotif infrastructure (under Linux) I occasionally use. Developement here is announced and underway, I hope.

I definitely cannot imagine to produce my stuff (running 2-3 high resolution screens) on a Raspberry Pi, because of complexity (of the required resources, not because of their size). 

Personally, I depend also on the ability to see as many things as possible, hence my visual resolution is currently a triple of monitors, having 2 * 4K plus a 2.5K on the laptop, my personal wide screen development cinema.

Another essential argument is operational cost, which is significantly reduced as implication of having packaged. 

I remember one migration project where the daily production workload was processed using 20 servers a 250MB, double CPU, under the rule of a mainframe.

The migrated 1-1 Java replacement finally required 30 servers a 750MB, and the host has to be upgraded.

The operation cost initially grew tenfold.  Initially that turned out as an economic desaster.

The reason of that was the lack of packaging, as the replacement could not extract the essence. 

However, the hope was to make use of the benefits of the (then) new Java environments and bypassing, the painstaking packaging process, so to reduce significantly the development costs.

Another argument pro packaging: grown vulnerability of the result because of not having removed unnessary things (e.g. all apps run at least their initialisation code when being loaded, no matter if they are actually required to run or not).

In general it is your choice to package s.th. or not, but when providing a framework application as deliverable, you have no such choice:

to be useful, it must be able to be packaged.

I am going to inspect the CGImageSupport application, to understand its underlying problem it had to tackle. 

Obviously the current solution is not stable in the sense of the observed packager complaints. 

As I already mentioned, I found some special support in the packager which had to assist in here, which I do not understand yet.

That gives me an initial hint: an OS specific class hierarchy providing the functionality, which ought to be totally different in other platforms, but has to be packaged equal. The protocol is targetting class references and providing replacments. This an atypical package problem, I personally never had before - I usually hide such problems behind factory patterns.

Finally, this is not be misunderstood as my critic to current XMotif implementation or making use of Raspberry or even Java. 

Sometimes architectural decisions have widespread consequences: 

Smalltalk (or any other integrated development) has to bear the burdon of packaging (or similar production steps) because of the benefit of that integrated development.

Whenever I work in other non-integrated languages, latest when debugging, I occasionally desiderate to run some code snippet integrated, to inspect s.th.

[Marcus Wagner]

In such a case allSubclasses usually is to be replaced by Application defined or similar. However, such would not scope any extensions of Application (or subapplications). An extension then has to care about its own packaging rules covering its extension. Hm. As I wrote elsewhere I do not yet understand the underneath CGImageSupport problem, causing all of this. Subapplications are usually covering platform specifics.

[Marcus Wagner]

Ok. now I understand the code in question.

First a hierarchy of classes implement marshalling of file formats in both directions (read & write) from / to two different external representations (in memory = byte arrays or as file = CfsFileDescriptor). This hierarchy contains abstract and concrete elements.

Abstract members are CgIconFileFormat and CgImageFileFormat.

Abstract members cannot be read or written.

Concrete members can be read or written.

Reading a file poses a factory problem (a hen - egg problem). You have to read s.th. to be able to decide which kind of format you have to apply.

This may fail, if the format is not known as no concrete class is provided for a given indicator.

The protocol allFormats delivers only concrete elements.

The code assumes, that concrete classes inherit from an abstract class / and abstract classes inherit from CgFileFormat.

That means, e.g. a concrete class may not be subclassed by an abstract class. So far this mimics concreteness which has no direct syntax notation in Smalltalk. Class abstraction in Smalltalk can be implemented by individual conventions, but is not directly supported by the languuage syntax. This allows for individual subtle nuances in the implementation -  e.g. a Java abstract class implies further rules which usually do not hold in a Smalltalk world. E.g. there is no Smalltalk restriction about instantiating an abstract class (violating the concept).

The protocol allFormats however is also used in packaging instructions, the  packager epClassReferenceAdjustmentsFor:in: Protocol.

This concept is new for me, but understandable. It creates virtual references of a class to be obeyed when packaging.

Consider a method contains Abstract doSomething. Instead of including Abstract, all its concrete subclasses have to be included in the package, but Abstract itself not. This could have been achieved by using other, "normally used" packaging instructions, but the  epClassReferenceAdjustmentsFor:in: was chosen as shortcut for framework applications.

It is used (as far as I saw) in the framework EwNotebooks, EwList and CgImageSupport.

This protocol was in use in Visual Age 4, 4.01 and 4.02 and then was replaced by a variation of it.

https://www.instantiations.com/vast-support/documentation/1410/mi/migrat46.html#

So far the story seems clear: a special marshalling problem of framework is handled mimicking the abstract class concept (missing in Smalltalk) at runtime, providing the packager information about potential target instances which are created at runtime based on provided input values, so that the need for them is not foreseeable by static analysis at package time.

And now I turn to the initiating complaint:  GbtPNGFileFormat.

This class is not part of the framework. 

However it seems to be involved, e.g. it subclasses some of the two abstract classes  CgIconFileFormat and CgImageFileFormat.

The application defining  GbtPNGFileFormat is the culprit. 

Either it extends the intended package providing an additional format to be used,

then this defining application has to be part of the package (to avoid the mentioned complaint).

Or this defining application is not to be used at all in this package, then it must be unloaded before packaging.

Two critics here: 

1 The (packaging) shorthand  epClassReferenceAdjustmentsFor:in: to be applied in frameworks is risking problems when being derivated.

[Seems that this subsequently led to newer form after version 4.02, unclear if a modernisation here helps].

2 Subclassing such a construct implies further support which is missing here. Warning to all implemtors of applications of conceptual abstract classes of the applications  EwNotebooks, EwList and CgImageSupport.

Conclusion: that all is related to the nitfy definition of abstract class, which was and is excluded from the syntax of the language.

I guess this is intentional: it would pose severe problems handling the class instance of an abstract class. See Java e.g.

Richard Sargent schrieb am Dienstag, 23. September 2025 um 03:49:15 UTC+2:

[Marcus Wagner]

At the second glance, this protocol allFormats, returns only concrete classes and excludes self = CgImageFileFormat. 

The messages (complaints) the first one is related to the Visual Age 4, 4.01 and 4.02, which became obsolete later. 

The second message is related to newer form Visual Age 5 and higher.

The problem I see is that the new form in some form is also serving the old style, but improper concerning the error messages, duplicating and irritating.

As I said, the source is the provider of the class GbtPNGFileFormat.