So, basically, what was causing me to hesitate and reconsider the whole project was just a set of small little things at the base of the engine, like the INTEGER/NUMERIC type duality, or some naming conventions, which deeply bothered me. Some decision were taken in the effort of making other things work and some other were simply a relic of the past, when they used to have a precise and valid (at the time) reason to be set up a certain way. Some where simply due to the need of having some things completed a working for existing project based on this software to progress.
After analyzing the codebase we have now for more than a month, I am convinced that we need to restart from the ground up, writing the basics from scratch and building the rest of the code around it.
This doesn't mean throwing everything away. The vast majority of the code, about 95%, isn't directly involved with the most intimate part of the engine. Even functions directly talking with the virtual machine are mostly logic (and good logic), and the interface points are few and well controlled. Module functions are mostly detached from the inner workings of Falcon, and even radical changes, as i.e. rewriting the ITEM API, would involve a quite limited set of (and in most cases a code refactoring might do the trick).
However, this will also provide the occasion to move the engine into C++17 and to uniform both the SDK and the language with the latest standards in terms of algorithm representation (i.e. find_if and lower_bound), without dropping any of the interesting features that were being included in the upcoming 1.0 version.
The operating steps are the following:
- Redesigning the ITEM structure
- Redesigning the Handler interface (what was called Class)
- Redesigning the PStep and CodeStep structures -- so that they can better represent both polymorphic code and code coming from written sources.
- Review the garbage collector (actually, it was one of the things I liked the most, especially when compared against the ones of other generic untyped languages in our same class)
- Redesigning the VM context (or, the operational virtual machine stacks), with particular attention to the process of value resolution.
- Refactoring the VM to serve the changed entities
- Writing tools to manipulate the VM directly (memory dumper, assembler, debugger).
- Adapting (refactoring) the falcon source compiler
- Porting all the modules to the new architecture.
This will be done through "incremental moves". We will start a new repository on github, where we'll work at the new components and we'll move in the code from the pre-existing falcon projects.
Other than an operating roadmap, I am also planning action to attract developers to help out in the process.
But first of all, I will publish a simple documentation on github about the features we had ready in 1.0 -- which are still accessible in the new_engine branch: basically, it will be an expanded version of the cheatsheet we were preparing (the document was never officially published: it was just a note we had circulating around developers and interested people i.e. in the mailing list).