Simplify bpython architecture
anatoly techtonik
Bob Farrell
We've actually got bpython working under Windows with bpython-gtk. It
needs some polishing, but as a proof of concept it's definitely
working. Any help you'd like to offer would be more than welcome. I
haven't done any work on it personally but I think both Simon and
Andreas have, so hopefully they will respond with more info. :-)
> --
> You received this message because you are subscribed to the Google Groups
> "bpython" group.
> To post to this group, send email to bpy...@googlegroups.com.
> To unsubscribe from this group, send email to
> bpython+u...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/bpython?hl=en.
>
anatoly techtonik
Seems like guys are pretty busy. =)
Still the code seems too interwined to me. Is it possible to run
bpython without any extra UI stuff? I want to start bpython in
standard console/terminal first and then see what functions are
missing to make its core features work. If there are some essential
upfront features that should be available in console/terminal just to
start bpython REPL, I'd like to know about it too.
I'd like to see UI configurable so that there is a single entrypoint
in bpython that gives all necessary information about detected UI,
chosen UI, features enabled, features disabled and the reasons why
these features disables (TODO or UI limitations). I this could speed
up porting bpython to other UIs.
--
anatoly t.
Bob Farrell
stuff out for yourself. If you want to contribute, that's great, but
you'll have to spend a lot of time reading code and playing with it,
as with any other project. If somebody else has enough time to hold
your hand and walk you through the whole process then you're in luck,
but I'm afraid that person isn't me. :-)
If you have specific questions about e.g. a particular block of code
or a particular function, feel free to ask, but if you're looking for
a complete explanation of all of the inner workings of bpython then I
definitely can't help you. We know bpython isn't a great piece of
code, but we're all very happy with the end result, and so are all of
our users. If you want to improve on it then please do feel free, but
understand that your vision of the software may not match ours.
Best of luck,
Andreas Stührk
I think the real question is how exactly do you plan to implement the
UI under windows. Basically, bpython needs to be able to jump anywhere
on the screen. At least it needs to be able to reprint older lines
(think of paren highlighting) and it needs some area where it can
display the suggestion window (where the suggestions, the function
signature and the docstring are displayed).
The basic idea of porting new UIs to bpython is to inherit from
"bpython.repl.Repl" and implement some methods (like "current_line"
that returns the current input line, setting the "cpos" attribute that
indicates where the cursor is positioned etc.). That process is not
really documented, please feel free to document it. Also keep in mind
that the API is really rather line-based and not block-based, ie there
is the current line of input and previous lines never change again.
> I'd like to see UI configurable so that there is a single entrypoint
> in bpython that gives all necessary information about detected UI,
> chosen UI, features enabled, features disabled and the reasons why
> these features disables (TODO or UI limitations). I this could speed
> up porting bpython to other UIs.
I don't really understand that part. What exactly do you have in mind
and how does it help to speed up porting bpython to other UIs? Also,
please understand that we are a little bit sceptical about adding new
UIs to bpython. You're not done with adding the basic functionality,
someone also need to be willing to maintain it. Obviously, we (the
current maintainers, that is) don't have the resources to maintain yet
another UI thingie.
Regards,
Andreas
anatoly techtonik
<and...@bpython-interpreter.org> wrote:
>
> I think the real question is how exactly do you plan to implement the
> UI under windows. Basically, bpython needs to be able to jump anywhere
> on the screen. At least it needs to be able to reprint older lines
> (think of paren highlighting) and it needs some area where it can
> display the suggestion window (where the suggestions, the function
> signature and the docstring are displayed).
My idea was to start with user input and see what happens when a key
is pressed. Then describe the loop of handling the key, creating a
list of features (drawing a chain of key processing handlers) on the
way. Then estimate limitation of windows console, data that bpython
features (handlers) require. Then sort the list of features by
complexity and start working on them one by one.
So I don't know how to do this yet, but thanks to your feedback I am a
few steps closer to my goal. Now I know there is:
1. internal buffer for input (multiline; used for paren highlighting)
2. buffer window
3. suggestion window
Is the command prompt a part of current buffer (for code highlighting)
or not? I.e. do previous lines participate in rendering line in
command prompt?
> The basic idea of porting new UIs to bpython is to inherit from
> "bpython.repl.Repl" and implement some methods (like "current_line"
> that returns the current input line, setting the "cpos" attribute that
> indicates where the cursor is positioned etc.).
Got it. Seems pretty low level architecture.
> That process is not
> really documented, please feel free to document it.
Who are the right people to annoy with merge requests?
bobf
federico
ikanobori
marienz
Trundle
> Also keep in mind
> that the API is really rather line-based and not block-based, ie there
> is the current line of input and previous lines never change again.
But while previous lines can't change, their visual representation can
- is that right?
>> I'd like to see UI configurable so that there is a single entrypoint
>> in bpython that gives all necessary information about detected UI,
>> chosen UI, features enabled, features disabled and the reasons why
>> these features disables (TODO or UI limitations). I this could speed
>> up porting bpython to other UIs.
>
> I don't really understand that part. What exactly do you have in mind
> and how does it help to speed up porting bpython to other UIs?
Sorry if it sounds too abstract. Let me explain why adding new UI to
bpython is hard. To add new UI you expect to study how the basic
loop/cycle works using minimal example. My case is even worse, because
even full example doesn't work failing with with "No module named:
_curses" on Windows. To find an entrypoint for further analysis I need
to study how curses works, or how GTK works or how urwid+twisted work
together. All in theory, because neither UI works on this machine. But
there are some things that could help.
To implement new UI with zero knowledge of bpython I need to know if
the main loop is synchronous - i.e. if after pressing a key the
interface doesn't process next key until current one is fully
processed. In this case you can expect input lags if processing is too
heavy. And if some event handler hangs - the whole application hangs.
If the input is asynchronous, then the loop becomes more complicated,
but more interactive as you can give priority to keyboard events and
resume highlighting when there are no events in stdin queue.
For making any changes, it is good to know application components.
While there could be separate docs, code is the best documentation,
unless it is a dead code. The best way to see if the code is alive is
to execute it and follow through the entrypoint. When I executed
bpython, I assumed it is a big application and expected it to do
several things prior to initialization of UI (yes, I am a naive
idealist). Namely detecting available frontends and choosing the
default one. But this doesn't work - you have to explicitly run script
for your UI. More than that - to find these entrypoints - you need to
look through all files in bpython package (urwid interface doesn't
have a command line script, for example). However, these scripts don't
work from checkout without setting PYTHONPATH properly. For fix this
problem I've patched data/* scripts to execute bpython directly from
checkout, so you may pull these changes from
https://bitbucket.org/techtonik/bpython
Ideally I see curses frontend script renamed to `bpython-curses`, and
`bpython` becoming a script that runs autodetection steps, lists
frontends and can be used to query enabled/disabled (i.e. implemented)
features for these frontends. I still don't know if it's possible.
> Also,
> please understand that we are a little bit sceptical about adding new
> UIs to bpython. You're not done with adding the basic functionality,
> someone also need to be willing to maintain it. Obviously, we (the
> current maintainers, that is) don't have the resources to maintain yet
> another UI thingie.
No problem. Let's hope it is at least possible to lower the barrier
for potential maintainers, and identify main problems that make
maintenance hard. If people know what to fix, how to fix and where are
major pitfalls, they can contribute more, and you have to spend less.
Thanks for the feedback.
--
anatoly t.
Andreas Stührk
On Sat, Apr 9, 2011 at 4:25 PM, anatoly techtonik <tech...@gmail.com> wrote:
> Is the command prompt a part of current buffer (for code highlighting)
> or not? I.e. do previous lines participate in rendering line in
> command prompt?
I'm not sure I really understand that part. All lines in the current
block ("current block" being all the lines up to when the prompt turns
from "... " back to ">>> " again) take part in highlighting, yes
(needed for example when highlighting multiline strings: previous
lines influence how the current line is being highlighted). Most of
the hard work in respect to highlighting is done by Pygments. We also
use it for tokenising Python code.
>> The basic idea of porting new UIs to bpython is to inherit from
>> "bpython.repl.Repl" and implement some methods (like "current_line"
>> that returns the current input line, setting the "cpos" attribute that
>> indicates where the cursor is positioned etc.).
>
> Got it. Seems pretty low level architecture.
Yes. It was sufficient so far and nobody came up with some better design yet.
>> That process is not
>> really documented, please feel free to document it.
>
> Who are the right people to annoy with merge requests?
>
> bobf
> federico
> ikanobori
> marienz
> Trundle
bobf and I (Trundle) are the official maintainers, but a lot of the
work is done by ikanobori. For documentation changes, you should bug
ikanobori.
> But while previous lines can't change, their visual representation can
> - is that right?
That is correct, yes.
> Sorry if it sounds too abstract. Let me explain why adding new UI to
> bpython is hard. To add new UI you expect to study how the basic
> loop/cycle works using minimal example. My case is even worse, because
> even full example doesn't work failing with with "No module named:
> _curses" on Windows. To find an entrypoint for further analysis I need
> to study how curses works, or how GTK works or how urwid+twisted work
> together. All in theory, because neither UI works on this machine. But
> there are some things that could help.
I really recommend looking at urwid and gtk instead of curses, it's a
bit less low-level. The usual assumption that worked well so far is
that you know how handling input and outputting stuff works in the UI
toolkit of your choice works and then you try to teach bpython about
it. Also, virtual machines are easy to set up nowadays.
> For making any changes, it is good to know application components.
> While there could be separate docs, code is the best documentation,
> unless it is a dead code. The best way to see if the code is alive is
> to execute it and follow through the entrypoint. When I executed
> bpython, I assumed it is a big application and expected it to do
> several things prior to initialization of UI (yes, I am a naive
> idealist). Namely detecting available frontends and choosing the
> default one. But this doesn't work - you have to explicitly run script
> for your UI. More than that - to find these entrypoints - you need to
> look through all files in bpython package (urwid interface doesn't
> have a command line script, for example).
> However, these scripts don't
> work from checkout without setting PYTHONPATH properly. For fix this
> problem I've patched data/* scripts to execute bpython directly from
> checkout, so you may pull these changes from
> https://bitbucket.org/techtonik/bpython
Please don't try to mess around with sys.path. FWIW, I will never pull
any change that does something like that. If you want to run bpython
from a checkout, either set PYTHONPATH or use "python -m bpython.cli"
(replace "cli" with "gtk_" or "urwid" if you want another UI).
> Ideally I see curses frontend script renamed to `bpython-curses`, and
> `bpython` becoming a script that runs autodetection steps, lists
> frontends and can be used to query enabled/disabled (i.e. implemented)
> features for these frontends. I still don't know if it's possible.
I don't see the advantages of that approach. I neither expect that
bpython has a billion different frontends somewhere near in the future
nor that users switch frontends all the time. I also think that
different frontends serve different needs. For example the bpython-gtk
can be embedded in other X11 aplications or (more or less easy) used
directly as gtk widget.
Regards,
Andreas
anatoly techtonik
<and...@bpython-interpreter.org> wrote:
>
> I really recommend looking at urwid and gtk instead of curses, it's a
> bit less low-level. The usual assumption that worked well so far is
> that you know how handling input and outputting stuff works in the UI
> toolkit of your choice works and then you try to teach bpython about
> it. Also, virtual machines are easy to set up nowadays.
Thanks. This will save me some time, but I am unlikely to mess with VMs.
Mostly because PXE server in Python is still in on TODO list, and
doing it any other way is not pythonic and will make this TODO less
important. =)
http://code.google.com/p/rainforce/issues/detail?id=11
> I don't see the advantages of that approach. I neither expect that
> bpython has a billion different frontends somewhere near in the future
Billion is not real, indeed, but a dozen..
I'll start with some clean up refactorings if you don't mind.
https://bitbucket.org/techtonik/bpython-cleanup/overview
--
anatoly t.
Andreas Stührk
On Sun, Apr 10, 2011 at 3:31 PM, anatoly techtonik <tech...@gmail.com> wrote:
> I'll start with some clean up refactorings if you don't mind.
> https://bitbucket.org/techtonik/bpython-cleanup/overview
Thanks, I merged in your changes, except for the last one: Those
imports are intentional. Otherwise, "import bpython.cli" inside
bpython imports a new module instead of the one bpython uses.
Regards,
Andreas
anatoly techtonik
Where is this can be useful?
--
anatoly t.
Bob Farrell
You're not allowed to play any more.
Bye bye. x