Counter proposal: introduce the component concept

[Ingo Oeser]

Summary: Go now requires at a package aggregation level usually called "component". Single package can be a component too,

because the component is small, the differences are currently blurred in Go. Thanks to Robert Griesemer for pointing me there!

Note: If wanted, I will provide this in a Google docs document, too.

Idea has been stimulated by the current proposal to add a way to define a set of packages as internal just to state: "this package has no stable API".

The current Go solution to declare API stability is to use an upper case letter for exported and thus stable APIs 

and a lower case letter for unexported identifiers which can have unstable APIs. 

But the solution is limited to the current package.

Current best practise of handling the issue of defining stability and maturity (as declared by the developers) 

in all other major (language) ecosystems is defined at semver.org.

See how clause 5 ("use 1.0.0 for the [first] public API") answers the current situation much better. 

If there is no stable API, the version is just 0.1.0.

Using semantic versioning is not required at the package level actually, as packages can just use unexported identifiers and thus don't need it.

Components in contrast to packages are quite different with respect to API changes.

 * intra-component changes are rare but happen independently

 * usually a team of developers works within one component during a sprint, project milestone, etc.

 * components might have tooling to pre-process externally provided data (e.g. see the unicode, cldr or timezone tooling)

 * changes within a component are properly orchestrated and lead to few changes outside of the component

 * a component might receive security updates

 * a component has a common set of authors of its packages

 * a component has a common license of its packages

 * a component is a good granularity for system builders to build a single RPM or DEB package

 * a component often is a natural boundary of certain expertise (e.g. unicode, timezone, handling, compiler building, integration of a specific SaaS, IaaS)

 * a component often is a natural boundary of responsibility (e.g. form natural boundaries within companies, project groups)

 * a component often is an extra SCM repository and thus are naturally versioned independent of other components

 * due to components often being an extra repository, several version pinning solutions for components in SCM repositories exist for Go already

 * a component wants a stable API on other components it uses

 * a component doesn't really benefit from stable apis within the packages it consists of

 * a component regularly even suffers in terms of development speed would the apis of it's packages be required to stay the same

Please not that it is actually NOT necessary to manage components and component membership at the import statement level,

but only at build time and during documentation. This includes documentation of responsibility, API changes, licensing.

The only common set between build and documentation is the information on dependencies not provided as a package in the component itself. 

To keep a stable and repeatable build only those dependencies need to be kept at the same version.

Thus I hereby propose to the introduction of a component as a concept of aggregating packages in Go.

It minimally consists of 

* a license statement

* a non-empty list of authors

* a non-empty list of maintainers, if different from the authors

* a semantic version (TODO: format?)

* a list of direct dependencies and their semantic versions (including the Go compiler itself) with which it can be successfully build (TODO: format?)

It optionally contains

* a list of apis, derived from the packages it consists of, which limits the API supposed be covered by the semantic versioning rules to the identifiers listed in it.

The list of apis could be generated by a similar tool like the one used to ensure the Go1 api compatibility

Only the packages listed in the import statement of a file "api.go" are considered.

The api.go file looks like this:

// +build ignore

package api

import (

 _ "component/package_one"

 _ "component/package_two"

)

// eof

It notes that only the packages  "component/package_one" and  "component/package_two" are considered api, 

but NOT the package  "component/package_three" which is not listed in that import statement.

The proposed solution

* solves the immediate problem without a language change

* uses minimal adaption of an existing tool (the api generator)

* allows time to finalize the remaining discussions how dependency lists should be built

* solves a wider problem indicated

* has no impact on current package imports

* is an optional concept for use in larger projects/teams.

* works towards the problem space "large scale programming"

* helps the packaging discussions at go-packaging

* doesn't block further or different aggregation concepts

* needs no changes to the go tool

* allows implementation in an extra tool to be used for those who need it

* doesn't (seem to) block flows like putting everything in one repository

* respects and encourages current Gophers natural workflow of putting components in extra SCM repositories

* respects and encourages loosely coupled components while still allowing tightly coupled packages

* allows fast moving development teams, while still allowing large maintenance windows of their results.

* simplifies system building, e.g as done in Linux distributions.

Thanks for your careful consideration

Ingo Oeser

[Andrew Gerrand]

The biggest point of contention over the "internal" proposal is whether the go tool should enforce the convention.

This proposal adds a lot of optional mechanism that is not enforced. That means you can go ahead and build it if you want it. There's no reason it should be in the go core.

Andrew