ENB: Hypothesis might make #1269 feasible
Edward K. Ream
#1269 suggests a radical simplification of Leo's key handling code. Python's Hypothesis unit testing framework gives me more confidence that the project could be completed without introducing an unending number of subtle bugs. For this reason, I have reopened #1269.
There are myriad paths through the existing key-handling code. The new code would be radically different. We need some way of testing all paths through both the old and new code. Hypothesis provides a way of thinking about the testing problem. We generate "random" tests from a universe of tests whose union covers all cases. In practice, testing will find erroneous edge cases without testing all test cases.
"Random" testing might not be required. For sure, we will also need explicit tests. Hypothesis can handle such cases, and will likely suggest where random testing is, in fact, necessary.
The general plan
This section will be pre-writing for an expanded first comment of #1269.
The new code will compute a tree of binding dictionaries during startup. Afterward, Leo will handle incoming keys by "traversing" this tree using binding selectors. The selectors will include the keystrokes themselves, states, modes, and widgets. Leo will perform dictionary lookups based on selectors, in some not-yet-defined order. The result of each lookup will be a flat key handling method.
Each flat key handling method will do something simple with the incoming key. In other words, these methods will replace all (or almost all) Leo's existing key handling code, including k.masterKeyHandler and k.masterCommand, and all their helpers. The result should be a significant speedup in key handling speed.
Summary
A complete testing plan is essential for #1269. The problem is to ensure that the new code is exactly equivalent to the old. Hypothesis helps assure us that a complete testing plan is possible.
A tree of binding dictionaries, created at startup time, should save significant time processing keystrokes. Up to five lookups, based on binding selectors, will yield a specific flat key handling method.
The testing challenge is to prove that the new flat key handling methods do the same as the existing code. This testing constraint will likely drive significant parts of the design.
This is still a risky project. I make no promises.
Edward
Edward K. Ream
> #1269 suggests a radical simplification of Leo's key handling code.
...
> The new code will compute...binding dictionaries during startup.
Here I'll start making the plan more specific. Let's start with the following pseudo code, for a new KeyHandlerClass method:
def handle_binding(self, event):
"""Handle the given key event."""
# Use per-state bindings if they exists.
binding, func = event.binding, None
if state:
d = self.state_dict.get(self.state)
func = d.get(binding) or d.get('default')
# Otherwise, use per-window bindings.
if not func:
d = self.window_dict.get(event.window)
func = d.get(binding) or d.get('default')
return func(binding, char)
In this new world, a python dict will define every state. Keys will be binding strings, a canonical representation of the incoming keystroke. A binding string corresponds to what is now called a "stroke".
The new states will be fine-grained. Some examples:
1. vim mode will consist of many sub-states. This will eliminate the "dispatch" code in leoVim.py. For example, the letter 'a' will be handled one way in vim-insert state, another way in vim-colon state, and a third way in vim-normal state. In vim-normal state, the 'd' character will switch to vim-d1 mode, which will handle the expected character of the d command.
2. Alt-x will enter "full-command" state, as at present. In the new code, k.fullCommand will simply init the minibuffer, as in the "state 0" case. There will be no need for a switch on later characters. Instead, k.state_dict('full-command') will return an inner state dict for the "full-command" state, which will dispatch later events to helpers based on binding strings.
State dicts don't simplify k.fullCommand all that much, but the general idea will be useful for user modes.
The big payoff lies in the code:
if not func:
d = self.window_dict.get(event.window)
func = d.get(binding) or d.get('default')This code handles per-pane bindings, which at present takes a huge amount of work at run time. All that work will be done (once per outline) at startup time. k.handle_binding will eliminate k.masterKeyHandler and k.masterCommand. Many helpers will still be needed, but they will be much simpler because they will be much more specific.
In particular, k.handleDefaultChar, k.doUnboundPlainKey and especially c.editCommands.selfInsertCommand will disappear. They will be replaced by simple, special-purpose methods.
Special-case tests now contained in k.masterKeyHandler and its helpers will be refactored into various sub-dictionaries contained in k.window_dict. Yes, there will still be many special cases, but they will appear explicitly in dictionaries.
Finally, all dictionaries should have an entry for the Ctrl-g binding string. The code that computes the dictionaries will likely add that binding string automatically. Similarly, there is a weird special case for the demo.py plugin. Instead, this plugin will call a new (startup) method that will place the required bindings in all (or almost all) dicts.
Testing
I could not have contemplated such a major refactoring without a testing plan. I'll create two new decorators, one for new code and one for the old. The decorators will indicate (somehow!) to the test code that a specific key handler has been called. The new unit tests will then verify that corresponding decorators are called in the old and new code. I am fairly confident that such tests are feasible.
Summary
Leo will create binding dictionaries at startup, once per commander. Creating these dicts is itself a major project.
The new k.handle_binding method will replace horrendous code in k.masterKeyHandler and its many complex helpers. k.handle_binding method will dispatch key events to specific key handlers, based on the binding string of the event, the present key-handling state, and the presently selected window.
The new code will clarify the distinction between binding strings and the characters actually typed. Binding strings should serve only one purpose: to dispatch characters to commands. For most commands, the character actually typed should be ignored. Conversely, for unbound characters, the character itself is all that matters.
Special cases will always be necessary to maintain the illusion of simplicity. The new scheme will embed all special cases in binding dictionaries.
This project requires full unit tests. New testing decorators will help unit tests verify that the old and new code is equivalent, that is, that the new code calls key handlers corresponding to the old key handlers.
Edward
Brian Theado
Here I'll start making the plan more specific. Let's start with the following pseudo code, for a new KeyHandlerClass method:
"""Handle the given key event."""
# Use per-state bindings if they exists.
binding, func = event.binding, None
if state:
d = self.state_dict.get(self.state)
func = d.get(binding) or d.get('default')
# Otherwise, use per-window bindings.
if not func:
d = self.window_dict.get(event.window)
func = d.get(binding) or d.get('default')
return func(binding, char)
Edward K. Ream
Edward K. Ream
> #1269 suggests a radical simplification of Leo's key handling code.
The more I study Leo's key-handling code, the more problematic it becomes.
My goal is a grand simplification of all aspects of key-handling. However, rewriting the code from scratch would be a great mistake. Instead, this project will require a long series of refactorings.
Refactorings alone will not suffice. An overall strategy is required. Ironically, a grand plan must account for all the existing details of key handling. Many cff searches will be essential.