Brython Development Model

128 views
Skip to first unread message

anp...@anpylar.com

unread,
Jan 22, 2018, 3:17:01 PM1/22/18
to brython
Hi Pierre et al.,

As you probably know from the description of things in the *AnPyLar* project, the goal of Brython to become widespread is shared. That's why I wanted to open an "open" discussion about the development model.

This is not a critic, rather a view. For example, see below a snaphot of the commits that have taken place in the last days
  • Bug Fixes
  • Website color changes
  • Documentation updates
  • Experiment with async imports (including temporary files)
All has taken place on the *master* branch, on which seems to be a on-the-spot approach (this may be simply the perception from an external user).

One of the things I do believe would benefit the adoption of *Brython* would be to separate some of the things that make up the repository and add some tweaks to the development model. Some proposals:
  • Separate the website into a separate repository
  • Separate the documentation into a separate repository
  • Separate individual files that make up "brython.js" from the actual target location of "brython.js"/"brython_stdlib.js"/"py_VFS.js"/"brython_dist.js"
  • Create branches for the development of features (like async imports), which are then merged when the feature is deemed fit for purpose it was intended.
  • Merge pull-requests first into the "development" branch
  • Keep the master branch as a sort of "master", which is a working branch (may and will have bugs of course)
And also
  • Rework command line utilities to be a lot more like the known standard cli widespread tools (at the moment they feel like specific hacks which do a task but cannot be used generically)

By no means it is implied that this something that is just said and done. Things (if taken into consideration) will take time and effort and coordination amongst the parties.


Last but not least, a long shot (and personally I have no clue as to how those things work)

  •  The creation of a Brython foundation to in charge of development and setting the direction of Brython

As pointed out above I have no clue how those work and it may be really naive thinking, but it seems to have served well the development of things like Mozilla, Python itself and some others. I believe there could be an entire ecosystem around Brython.


Best regards



Auto Generated Inline Image 1

Jonathan Verner

unread,
Jan 23, 2018, 6:04:20 AM1/23/18
to brython
Hi Anpylar,

Thanks for your input. Here are just a few thoughts from a contributor. Most
of your suggestions seem reasonable, though only some of them sound practical
to me for a project of this kind :-)

Re: Branch strategy

I am not sure we should aim towards a very complicated branching model, but I
do think that, perhaps, having a master branch and separate release branches
would be nice. Though to make sense it would need somebody to backport fixes to
the release branches and I am not sure we have enough manpower for that.

Re: Repo organization

Again I think having a lot of separate repositories here would just fragment
things and not be really helpful. I do agree that some sort of separation
would make sense, though:

-- perhaps separating the website into separate top-level folder (ditto for
the docs)
-- creating a dist folder for the brython.js & co. files

Re: CLI utils

Yes, I too think that having a common cli (say manage.py or just brython) to
have a uniform access to different tasks (e.g. preparing a release, running
tests, running a local webserver, ...) would be nice; its just that somebody
needs to step up and write it (I've been thinking about this for some time
now, but so far there were other things I needed/wanted to work on [and Real
Life (TM)]).

Re: Foundation

This is, I think, completely out of scope. Brython is just a hobby project
which people work on in their free time. Creating a foundation would be too
much tedious, un-fun, administrative work for which there are no resources.
Maybe, if Brython takes over the world :-) But right now, I don't think this
is a good idea. On the other hand, having a common place where development
direction is discussed is certainly needed. Here I think that the mailing list
works fine, so far...

Anyway, these are just my thoughts...

Cheers,

Jonathan


--
Jonathan Verner, PhD
Department of Logic, Charles University
Palachovo nám. 2, 116 38 Praha 1
Czech Republic

anp...@anpylar.com

unread,
Jan 24, 2018, 12:04:01 AM1/24/18
to brython
Branch model: the goal is not a complicated branch model, but rather to have mostly *pristine* master branch, which has a known behavior and to which merge/pull-requests are done when things have been properly implemented/tested/verified in for example the *development* branch.

Let me put an example on the table: Pierre is working on `async` imports and has added a new file named `py_import_nevez.py` which for those who are non-native speakers of Breton is simply: `py_import_new.py` (No, I am not a native speaker) If this happens in a *development* branch, there is no real need to have *new* files in place.

Of course, Pierre is the driving force behind *Brython* and this is not about imposing this or that other methodology. It also took me a while to figure the benefits of (the very cheap) branching powers of *git*. I submitted patches before for other topics (alias `mementum`, for example metaclasses) and Pierre asked: *Why do you do that in a separate branch?*. My answer: *To provide the cleanest and more straightforward possible patch and hence merge experience*. It's not only that: by having a patch in a separate branch I can easily rebase it on the current master (i.e: recreate the patch branch taking into account any changes in master) and always have a very clean patch.

Repo organization


I do concur that moving things to different directories (probably top-level) would already be a good change. The idea behind different repos is to decouple things which are not tightly coupled. The best example here would be: changes to the website (like update of colors) have no actual relationship to `brython` as a framework itself. One could actually revamp the website several times without changing a single comma of the `brython` framework. But as long as the website is part of the repo, changes to the webside imply changes to the repository. That's the idea behind the proposal.

CLI Utils

Let's discuss more about this. I can for sure offer some improvements.

Foundation

Or call it working group or something else. It could be in charge for example of a *Patreon* or *Collective* account which manages funds to foster things. These can be as easy as having a gathering in Paris between key core members of Brython to discuss the next steps and for example decide what, how and in which timeframe the CLI utils have to move (or the website for the sake of it) See for example Issue #772 which I submitted for the inclusion of new features.

You have actually nailed it. Let me quote you: "Brython is just a hobby project which people work on in their free time". That's either a virtue or a huge dead weight for the project. It's not because I think there are $$ waiting around the corner, but I do believe that even if not for direct development, to start with, things like the aforementioned gatherings could be a major turnaround for Brython pushing it forward. Else, many things may sit around waiting for someone to find them (See for example pull-requet. #769 to fix relative impots, which as mentioned by Pierre, may have been sitting there for ages)

In any case, the intention here is to foster discussion about the model and not criticize this awesome project and the contributors to it.

Thanks for your answer

anp...@anpylar.com

unread,
Jan 24, 2018, 12:05:50 AM1/24/18
to brython
One thing that I actually forgot:

  • Wouldn't it be nice if Brython discussions happened in a *Brython forum* rather than in a Google Groups list?

It would be even nicer if such a Brython Forum had been developed using *Brython* ...



On Monday, January 22, 2018 at 9:17:01 PM UTC+1, anp...@anpylar.com wrote:

Pierre Quentel

unread,
Jan 25, 2018, 4:53:25 AM1/25/18
to brython
Many thanks for the proposals. I am currently spending a few days in Paris, I will have more time to give my opinion this week-end.

Pierre Quentel

unread,
Jan 27, 2018, 4:26:05 AM1/27/18
to brython
Once again thanks for all these proposals. Here are my reactions below ; don't take them as definitive decisions.

1. Splitting the repository into several parts (core Brython, web site, documentation, tools...)

I don't agree with this suggestion.

First because most of the commits are related to the core Brython scripts : the changes to the other parts are comparatively very rare.

And more important, because these parts are tightly connected : for instance, the test suite, updated for each bug fixed in the core Brython scripts, is a part of the web site. Another example : the CPython "brython" package includes brython.js and brython_dist.js which would obviously be in the core Brython repo. I don't see any advantage of splitting into several repos that would justify the complexity it would bring (all contributors would have to get all the sub-repos synched).

2. Having a branch for development and keep only the release versions in master

One of the rules we apply when updating the master (with very rare exceptions) is that we leave the code in a stable state, that is, at least all the tests in the builtin test suite pass. In most cases, the reason to publish a release is that many improvements and fixes have been added for a few weeks or months, not because a major new feature has been added ; the release is tested more carefully (all the demos in the gallery are tested, most of those in the documentation, a few tests in the console, etc.), which means that it is supposed to be more stable than after other commits, but it's not different in nature.

That being said, I have no strong opinion about it. After the next release, I will try this method and open a development branch. We'll see the pros and cons then.

3. Opening branches for experimental features

I agree with you here. I should have created a branch for asynchronous imports and cache with indexedDB, instead of adding temporary files. I have done it, and will try to learn a little more of the Git features to make merging easier ;-)

4. Creating a repository for generated scripts (brython.js, brython_stdlib.js, etc.)

I agree with this suggestion too.

5. Command line tools

Do you mean those in the folder "scripts" ? If so, I don't understand the sentence "cannot be used generically", can you explain ?

You also mention "standard CLI widespread tools", can you point us to some of them ?

6. Setting up a foundation

Wow, this scares me... I dedicate all the time I can to writing code, documentation, demos, reading and sending posts on the Google group or the Github site, and I enjoy every minute of it. Raising funds, organising meetings or conferences, this would bore me to death. And anyway, I don't think that a project such a Brython requires this kind of infrastructure : it's an implementation of Python, so we can use the various Python conferences if we want. I don't think PyPy, which is a much more ambitious, complex and popular project, organises specific meetings. And I don't think either that a physical meeting would help finding more bugs than what we discover by just writing web applications and finding that they don't work as expected.


Olemis Lang

unread,
Jan 27, 2018, 10:34:51 AM1/27/18
to bry...@googlegroups.com
I will follow after Pierre's comments.

On 1/27/18, Pierre Quentel <pierre....@gmail.com> wrote:
> Once again thanks for all these proposals. Here are my reactions below ;
> don't take them as definitive decisions.
>
> *1. Splitting the repository into several parts* (core Brython, web site,
> documentation, tools...)
>
> I don't agree with this suggestion.
>

Instead of separating , IMO we should have a way of selectively
building a distribution of Brython optimized in size . A tool for
choosing what needs to be included in and leave the rest out .

> First because most of the commits are related to the core Brython scripts :
> the changes to the other parts are comparatively very rare.
>
[...]
>
> *2. Having a branch for development and keep only the release versions in
> master*
>
[...]
>
> That being said, I have no strong opinion about it. After the next release,
>

I'm in favour of this , but only for having develop and master and ,
on exceptional situations, a next branch . The later only needed in
case a new version including a substantial number of incompatible
changes is under development . Process would consist in committing
often in develop , merge with master at release time , and working on
next version at a different pace, Disclaimer : I'm not saying we are
thinking of a next version , it's a hypothetical situation I mention
based on situations I've found in other projects .

> I will try this method and open a development branch. We'll see the pros
> and cons then.
>

\o/

> *3. Opening branches for experimental features*
>
> I agree with you here. I should have created a branch for asynchronous
> imports and cache with indexedDB, instead of adding temporary files. I have
> done it, and will try to learn a little more of the Git features to make
> merging easier ;-)
>

I would be strongly against this . If there is an experimental feature
(that is not considered a next version feature) it should be developed
in another repository until it gets to development branch . For
visibility in brython-dev/brython a PR should be created and that
branch would show up as yet another issue thus not polluting the
branches namespace . IMO official repositories should have the minimal
set of branches (i.e. develop <what we are doing now>, master <what
should be used in production>, probably next <our legacy to our
children>)

> *4. Creating a repository for generated scripts (brython.js,
> brython_stdlib.js, etc.)*
>
> I agree with this suggestion too.

+1

>
> *5. Command line tools*
>
[...]
> can you explain ?
>

/me waiting

> *6. Setting up a foundation*
>
> Wow, this scares me...

I am running a foundation myself and I tell you that , in this case ,
it would be an unnecessary burden and maybe a distraction (learning
corporate law, jurisdictions, staffing, meetings, resolutions,
policies ...) that would interfere with development

[...]
> Raising funds, organising
> meetings or conferences, this would bore me to death.

Why bother creating yet another corporation for this ?

> And anyway, I don't
> think that a project such a Brython requires this kind of infrastructure :
> it's an implementation of Python, so we can use the various Python
> conferences if we want.
>
[...]

... and there are a lot of other well-established foundations, should
we need anything close to this .

--
Regards,

Olemis - @olemislc

Apache™ Bloodhound contributor
http://issues.apache.org/bloodhound
http://blood-hound.net

Brython committer
http://brython.info
http://github.com/brython-dev/brython

SciPy Latin America - Ambassador

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

Olemis Lang

unread,
Jan 27, 2018, 10:39:22 AM1/27/18
to bry...@googlegroups.com
On 1/27/18, Olemis Lang <ole...@gmail.com> wrote:
>
[...]
>
> On 1/27/18, Pierre Quentel <pierre....@gmail.com> wrote:
>
[...]
>
>> *3. Opening branches for experimental features*
>>
>> I agree with you here. I should have created a branch for asynchronous
>> imports and cache with indexedDB, instead of adding temporary files. I
>> have
>> done it, and will try to learn a little more of the Git features to make
>> merging easier ;-)
>>
>
> I would be strongly against this . If there is an experimental feature
> (that is not considered a next version feature) it should be developed
> in another repository until it gets to development branch . For
> visibility in brython-dev/brython a PR should be created and that
> branch would show up as yet another issue thus not polluting the
> branches namespace . IMO official repositories should have the minimal
> set of branches (i.e. develop <what we are doing now>, master <what
> should be used in production>, probably next <our legacy to our
> children>)
>
[...]

Clarification : another repository = a fork of brython-dev/brython

anp...@anpylar.com

unread,
Jan 27, 2018, 2:34:27 PM1/27/18
to brython
Thanks for the answers Pierre. Let me concentrate on the topics for which I believe I could still have something to say or where an answer is due.
 
4. Creating a repository for generated scripts (brython.js, brython_stdlib.js, etc.)

I agree with this suggestion too.

To avoid a misunderstanding, the proposal was to have them in a separate directory, probably at the root of the repository in something like /dist/brython.js or /release/brython.js.

A separate repo would also work.


6. Setting up a foundation

I do understand your concerns and see where you go with PyPy. But see for example this: https://vuejs.org/

In my view that's a project which is by no means as complex as Brython, yet it receives funding both for the main developer (through Patreon) and for the project itself through OpenCollective. Whether you or someone is open for funding for development is a matter of personal decisions, dedication and other factors. But the second part is for sure something beneficial.

It doesn't have to be you the one organizing things. A project like Brython can have people contributing to the core code, to the docs, to some Python libraries and for sure someone could dedicate some hours to some organization.

Don't take this as me saying: "It has to be done". Just wanted to give a counter example which I think is valid.


5. Command line tools

Do you mean those in the folder "scripts" ? If so, I don't understand the sentence "cannot be used generically", can you explain ?

You also mention "standard CLI widespread tools", can you point us to some of them ?

See. I have made a fresh clone of brython. I enter the directory and see *scripts*. One of the 1st temptation is to use the bash shell to see what's executable by pressing TAB twice. The only things shown is

./scripts/ensure_phantom.js

I am brave and by using ls I find there are several others with the extension `.py`. One with the name `make_dist.py` (which puzzles me, because there is another named make_full_dist.py). The lion in me goes for
$ python scripts/make_dist.py
Traceback (most recent call last):
 
File "scripts/make_dist.py", line 207, in <module>
    run
()
 
File "scripts/make_dist.py", line 34, in run
   
with open(package_file, encoding="utf-8") as fobj:
FileNotFoundError: [Errno 2] No such file or directory: '......package.json'

I have edited the path details for privacy.

Because I am not ready to give up, I enter the scripts directory and try again

cd scripts
python make_dist
.py

Traceback (most recent call last):
 
File "scripts/make_dist.py", line 207, in <module>
    run
()
 
File "scripts/make_dist.py", line 34, in run
   
with open(package_file, encoding="utf-8") as fobj:
FileNotFoundError: [Errno 2] No such file or directory: '......brython\\www\\static_doc\\en\\stdlib.html'

The final part of the error simply shows a slightly modified path. `npm.json` has been found (or not) because something else is not found now. The lion turns to cat but keeps on trying by going above and entering *www*.
cd ../www
python ../scripts/make_dist.py

And the same error repeats. The cat has turned into mouse but keeps on trying goes to src and tries once again.
cd src
python ../../scripts/make_dist.py
Traceback (most recent call last):
  File "../../scripts/make_dist.py", line 207, in <module>
    run()
  File "../../scripts/make_dist.py", line 34, in run
    with open(package_file, encoding="utf-8") as fobj:
FileNotFoundError: [Errno 2] No such file or directory: '.......... package.json'


And we have gone full circle to not finding `package.json`.

This is only a story to try to show how the experience feels ... for a newcomer who wants to play with the brython sources and may be contribute
 
Of course I know how to solve the above problems and for reference in case someone would ever look into this the solution is (there may be others)
cd www
python ../scripts/make_docs.py
python ../scripts/make_dist.py

You also mention "standard CLI widespread tools", can you point us to some of them ?

This was meant as to have only one script which is in charge of delegating the work to internal code (or even other scripts). For a generic example take `git` which is the center of the operation and offers several commands.

The same git approach was for example chosen for `anpylar`. It's of course a chicken-egg problem at the beginning, but once the scripts can bootstrap themselves, working in the project (for anyone) becomes a lot smoother.

In that sense one would have central script named for example *brythondev* which would accept commands like *make dist* or *make fulldist* ...

Best regards

Pierre Quentel

unread,
Jan 29, 2018, 2:12:56 AM1/29/18
to brython


Le samedi 27 janvier 2018 20:34:27 UTC+1, anp...@anpylar.com a écrit :
Thanks for the answers Pierre. Let me concentrate on the topics for which I believe I could still have something to say or where an answer is due.
 
4. Creating a repository for generated scripts (brython.js, brython_stdlib.js, etc.)

I agree with this suggestion too.

To avoid a misunderstanding, the proposal was to have them in a separate directory, probably at the root of the repository in something like /dist/brython.js or /release/brython.js.


/release/ is ok for me

6. Setting up a foundation

I do understand your concerns and see where you go with PyPy. But see for example this: https://vuejs.org/

In my view that's a project which is by no means as complex as Brython, yet it receives funding both for the main developer (through Patreon) and for the project itself through OpenCollective. Whether you or someone is open for funding for development is a matter of personal decisions, dedication and other factors. But the second part is for sure something beneficial.

It doesn't have to be you the one organizing things. A project like Brython can have people contributing to the core code, to the docs, to some Python libraries and for sure someone could dedicate some hours to some organization.

Don't take this as me saying: "It has to be done". Just wanted to give a counter example which I think is valid.

If the point is to raise funds, this is out of Brython scope, managing funds is a part of the boring things I want to avoid. Plus, it would be a potential source of pressure from those who fund.

Likewise, it's true that someone else could lead a foundation for Brython and manage the boring stuff, but this would also be a source of potential conflicts. What if the foundation accepts help from a company I or the other core developers don't like ? or spends money for something we don't approve ? or insists on working on something we don't think relevant ? or promote Brython with wrong arguments ?

Don't take this as a criticism of projects such as vue.js which adopt other models. It's just that I do everything I can to avoid unnecessary conflicts.
Your approach when discovering a folder /scripts is interesting, but I'm not sure all developers are as curious as you are ;-)

Those scripts are mostly undocumented ; there is only a mention of make_dist.py in the Wiki page "How Brython works", I think. It's the only script that is supposed to be used by contributors to the project (along with make_doc to generate the static documentation pages ; make_full_dist.py is an old version, I should remove it).

The error messages you get are strange : if you have cloned the Brython repository, there should be a file /npm/package.json, I don't know why the script doesn't find it ; "python make_dist.py" in the folder /scripts works fine for me (on Windows 10). The second error is even more strange, line 34 searches "package.json" and the traceback says that it's "../stdlib.html" which is missing... If there is a bug on make_dist.py, maybe only for some OS, can you open an issue in the tracker ?
 

Pierre Quentel

unread,
Jan 29, 2018, 2:15:46 AM1/29/18
to brython


Le mercredi 24 janvier 2018 06:05:50 UTC+1, anp...@anpylar.com a écrit :
One thing that I actually forgot:

  • Wouldn't it be nice if Brython discussions happened in a *Brython forum* rather than in a Google Groups list?

It would be even nicer if such a Brython Forum had been developed using *Brython* ...


I really would like that, if I started Brython now I would probably not use Google Groups. But I'm afraid it's too late to change, there are nearly 400 members in the group.

Nicolas Pinault

unread,
Jan 29, 2018, 12:18:02 PM1/29/18
to bry...@googlegroups.com
Hi Pierre,
I don't think this a problem.
On another project I use, the communication channel was an e-mail list. The devs decided to switch to discourse. They advertised of the change on the website. The mailing list is still open but when someone posts on it, the answer is to use the discourse site. The switch happened few months ago.  Now, there is rarely a post on the mailing list.

Nicolas
--
You received this message because you are subscribed to the Google Groups "brython" group.
To unsubscribe from this group and stop receiving emails from it, send an email to brython+u...@googlegroups.com.
To post to this group, send email to bry...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/brython/386d6918-25cc-48ab-921a-e04cff33ccba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Kiko

unread,
Jan 30, 2018, 2:05:05 AM1/30/18
to bry...@googlegroups.com
2018-01-29 18:17 GMT+01:00 Nicolas Pinault <d...@famillepinault.fr>:
Hi Pierre,

Le 29/01/2018 à 08:15, Pierre Quentel a écrit :


Le mercredi 24 janvier 2018 06:05:50 UTC+1, anp...@anpylar.com a écrit :
One thing that I actually forgot:

  • Wouldn't it be nice if Brython discussions happened in a *Brython forum* rather than in a Google Groups list?

It would be even nicer if such a Brython Forum had been developed using *Brython* ...


I really would like that, if I started Brython now I would probably not use Google Groups. But I'm afraid it's too late to change, there are nearly 400 members in the group.
I don't think this a problem.
On another project I use, the communication channel was an e-mail list. The devs decided to switch to discourse. They advertised of the change on the website. The mailing list is still open but when someone posts on it, the answer is to use the discourse site. The switch happened few months ago.  Now, there is rarely a post on the mailing list.

If you avoid realtime stuff like slack/telegram/... or derivatives it is ok for me and I think the change wouldn't be a big problem.
 

Nicolas
--
You received this message because you are subscribed to the Google Groups "brython" group.
To unsubscribe from this group and stop receiving emails from it, send an email to brython+unsubscribe@googlegroups.com.

To post to this group, send email to bry...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/brython/386d6918-25cc-48ab-921a-e04cff33ccba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "brython" group.
To unsubscribe from this group and stop receiving emails from it, send an email to brython+unsubscribe@googlegroups.com.

To post to this group, send email to bry...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages