Hi,
Having kept extending this post so much, I should say at the beginning and then at the end again: "Thanks and sorry for this long on context and questions, but light of facts post" :)
My intention is to experiment with, and write up something about the lifecycle of packaging, distributing, deploying and installing applications and libraries in Newspeak.
I realize much of this has been discussed, and described in an earlier Gilad's post here:
My main reason for a follow up is to clarify the broader context for myself - which will surely reveal a level of continued ignorance :) . Partly my internal need for clarificaions is maybe due to some workflow and terminology tension or overlap in 'mainstream' software distribution compared to Newspeak.
There are two sections here, A) and B). My purpose of including section A) is ONLY to create context of how today's 'mainstream' software deals with packaging, distributing, deploying and installing applications and libraries, and examine differences of the (to be better understood by me) Newspeak lifecycle in section B).
A note - I already went over parts of such lifecycle in Newspeak during the steps of building and local-server deployment of CounterApp.vfuel described here:
https://mzimmerm.github.io/2021/05/newspeak-a-few-notes#org2fffd3b. But I have not actually dug into details of packaging and building methods there - hence this long post.
A) The lifecycle of develop/build/submit/install/run an application, described for the current "mainstream" way of software distribution using package managers and package repositories
---------------------------------------
Feel free to ignore this section A. It's for context only and it grew out of proportion :) Although there is a reference to the lifecycle in section B)'s questions.
Let me start with my understanding of the current "mainstream" way of software distribution using package repositories. I'd describe such lifecycle as follows:
A1) 'develop' : 'developer' 'develops' the 'program' or 'app'.
A2) 'build' : 'developer' 'builds a 'program package' from the 'program' and (potentially) it's 'dependencies', using a 'build tool'. Such 'build tool' is generally part of a higher level 'package manager' - the 'package manager' may act in 2 steps, in this build step, and in the 'install' step 4. The build tool may be gradle, rpm builder, docker image builder, snapcraft, flatpak builder etc. The resulting 'program package' may be a war file, a vfuel file, an rpm file, etc. The 'program package' may or may not be 'self-contained' (contain all needed components - dependencies - to run the program). Either way, 'dependencies' (libraries needed to run the program) must be resolved either at this build step (if the 'program package' is self-contained) or at the 'target system' 'install' step. As an example, the war or snap file would contain all dependencies, while the rpm file would rely on dependencies to be resolved and installed during the 'install' step.
A3) 'submit': 'developer' 'submits' the 'program package' to a 'package repository'. On the web, developer copies the jar or vfuel to a webserver (so the repository is skipped? unless the browser extensions store is used)
A4) 'install': 'end-user' 'installs' the 'program' on the 'target system' from the 'program package' located in the 'package repository', using a 'package manager'. As discussed in step 2, the 'package manager' of the 'non-self-contained' program packages must calculate and install 'dependencies'.
A5) 'run': 'end-user' 'runs' the 'program' installed in step 4.
Notes:
- steps A1, A2 and A3 (develop, build, submit) generally are executed - or at least can be - on the 'source system' (where the software is developed).
- step A3 (submit to a repository) copies the 'program package' to a publicly available repository (available to both developer and end-user).
- steps A4 and A5 (install, run) are executed on the 'target system' - where the end-user installs and runs the program. Well, unless we are on the web where the 'end-user' points browser to a URL.
- I tried not to use the verb "to package" here (only "to build"). Unless I am missing something, "to package a program" is pretty much equivalent of "to build a package from a program" - so "to build" == "to package". These actions happen on the 'source system'.
- The difference between 'self-contained' program packages and the not-self-contained ones, and how they are handled by their 'package manager' programs, is similar to early and late binding, perhaps with somewhat similar pros and cons.
B) What are roles of the methods (verbs) 'build' and 'package' in the Newspeak lifecycle of develop/build/submit/install/run an application?
---------------------------------------
Grepping the Newspeak codebase for:
find . -type f -exec grep -H "package.*Using.*:" {} \;
Yields the following methods signatures (on code from like a month ago when I started writing this):
- SomeClass.ns: #packageUsing: manifest = ..
- SomeTestsConfiguration.ns: #packageTestsUsing: manifest = .. (or ideNamespace but that's just a name?)
In addition, in the post
Gilad defines
- #packageLibraryUsingManifest: manifest
but this is not used yet anywhere in code.
So we have 3 "#package: manifest" type methods.
I think my first few questions would be:
- Is there a convention around the #packageTestsUsing:, in terms of which class it should be defined in, and invoked (sent) in? I see it generally defined in names like SomeTestsConfiguration.ns. I think the IDE uses this method's presence to show the [runTests] button(?). Are there othe objects invoking this method , e.g. if developers want to include tests in the exported package?
- What is the reason for the 3 method signatures, would perhaps 2 signatures , "#packageUsing: manifest" and "#packageTestsUsing: manifest" suffice? I am guessing there is a role of #packageTestsUsing: if the packager want to include tests. But what different roles do #packageLibraryUsingManifest: vs. #packageUsing: support? If we want to package an app (which I guess must contain a method #main: ) vs. a library, would the presence of #main: suffice to distinguish?
Rephrasing that post slightly:
```
Imagine a convention whereby *every library intended for distribution is sent out as a class which:*
- has a factory (or in general, class method) method ~#packageLibraryUsingManifest: manifest~ -
- has a ~#build: platform~ method - this method, given a platform object, produces a working instance of the module we actually want to distribute
```
(NOTE: Below, I am using abbreviated method name "#package:" instead of "#packageLibraryUsingManifest:" or "#packageUsing:" )
My main reason for including the long section A) above (the section tried to describe the current 'mainstream' package lifecycle and package managers) was to elaborate on the roles and differences between the "#build: platform" and the "#package: manifest" methods.
I realize I could and should build better understanding by studying the code, but wanted to start with context.
So I have some initial questions about the Newspeak lifecycle of develop/build/submit/install/run an application. I realize the questions might not always make sense or out of context, but here they are:
- "#package: manifest" questions:
- At which step of the lifecycle would the "#package: manifest" method be used (invoked)? I think it is on the 'source platform', and so invoking this method is equivalent to the 'build' step in item A2 section A?
- What is the result of invocation of "#package: manifest"? Is it an object that is intended to be possibly streamed in a file for distribution - thus equivalent to a 'program package'. If or how does invoking this method result in the 'vfuel' file? (this is expanded in the next question below)
- Is the general intent that the tools (the IDE) invokes this method upon clicking the [deploy] -> #asVictoryFuelWithMirrors?
- If true, what are other high level steps for the .vfuel file to be produced (which I know is produced, I went through it in the CounterApp, just not quite grasping the steps even on a high level :)
- I think we can consider the vfuel file the 'program package' discussed in A2, A3, A4, but am I right?
- Can the 'program package' in Newspeak be something else than a vfuel file? I can place .vfuel under a webserver and run the CounterApp, but can I, for example, export some other artifact (zip file with binary or source?) that is later loaded into another persons Newspeak IDE?
- "#build: platform" questions (I think I sort of grasp the #package but not the #build):
- At which step of the lifecycle would the "#build: platform" method be invoked? Would it be on the 'target system', making "#build: platform" equivalent to the 'package manager' doing the 'installation step' equivalent [item A4 in section A]?
- If so, how and in which format would the result of the invocation of "#package: manifest" be "shipped" to the 'target platform'?
- Is the general intent that the tools (the IDE) invokes this method? If so (as we do not yet have an example), could you briefly describe how that would work, for example "after downloading the 'program package', in the IDE on top, we will have a button 'install package'" - well, I am being presumptious :)
A bonus question : )
What makes a Newspeak class to show the following buttons: [deploy] [configuration] [run] [debug]? I am making an assumption that:
- [deploy] - shows if the class has the "#package: manifest" type method?
- [configuration] - not sure what makes this show, perhaps class name ending with Configuration?
- [run] and [debug] - shows if the class has "#main: platform args: args" method?
- [runTests] - shows if the class has " #packageTestsUsing: manifest" method?
Can we please correct this and or elaborate?
Thanks and sorry for this long on questions and context, but light of facts post
Milan