Code completion on TextIOWrapper after "open"

[pete]

Using PyScripter 2.0. In Python 2.6, opening a file in the
"disapproved-by-documentation" way, i.e.

f = file('xyz.txt')

has the big advantage that code completion works after qualifying f.

Alas, now that the file class is no more in Python 3, I must resort to
using "open".

In Python 3, open returns an object of type TextIOWrapper.
Code completion works for TextIOWrapper, but not for object f of type
TextIOWrapper.

I'd like to understand why we don't/can't get the completion on the
object f.
And whether there is anything I can do (e.g. I've tried importing
IO.TextIOWrapper, etc.) that would turn the code completion on for me.

Thanks

[PyScripter]

Given the questions and issues created about code (and parameter)
completion I thought it might help to explain here how it works in
PyScripter and its limitations.

a) Code Completion in the interpreter

This is rather easy to explain. In the interpreter we have live
Python objects, so code completion should be near perfect. PyScipter
uses the inspect module to find the members and doc strings of the
Python objects.

b) Code completion in the editor

The difference here is that most of the code completion is based on
analyzing the source code. So it will never be perfect. The steps
PyScripter is following are:

- PyScripter constantly scans the edited file and extracts
information about variable, function and class definitions. The
result of the scanning by the way is also used by the Code Explorer.

- When the user invokes code completion e.g. by typing "somename.",
PyScipter tries to locate the definition of somename first in the
current scope and then working upwards up to the module level.

- If the variable comes from an imported module (from modulex import
somename or from modulex import *) then PyScripter searches for
modulex as explained below.

- If the modulename is listed in the "Special Packages" IDE option,
then it is imported into the Internal Interpreter and examined using
the inspect module. Otherwise PyScripter tries to locate the source
code of the module and scans it as it does with edited files. This
process may recursively extend to modules imported by the imported
module etc. until the definition of somename is found. If the module
source is not found the PyScripter tries to see whether the module is
already imported, which it the case with builtin modules, and, if it
is, the live Python object is used.

- Once the definition is found PyScripter can provide completion only
if it knows the type of the object. This is easy when somename is a
live Python object (defined in Special Packages or builtin modules),
but difficult otherwise.

- When it scans Python modules PyScripter analyzes assignment
statements to try and determine the assigned variable object type. It
deals with some of the following cases.
somename = "string"
somename = {...}
somename = [..]
somename = 1.2
somename = ClassName(...)

but it does not currently handle
a = functioncall(...)

If functioncall is a built-in function or function from an extension
module it is not possible to get the type of the result, since this is
not known. If the source code of functioncall is available then
PyScripter could try to guess the type of the result by analyzing the
return statement(s), but this is not currently done.

All in all, you cannot expect code completion accuracy that is
possible with strongly typed languages. Of course the accuracy of
code completion can be improved. Two improvements I am planning to
implement are:

1) Cache scanning results to improve speed when working with complex
projects
2) Analyze return statements to determine the type of the return value
of functions

One other idea is to use a third party tool like rope (rope.sf.net).

Any suggestions of further improvements will be welcome.

[pete]

I made another suggestion elsewhere to provide some way that we can
"load our own" overriding metadata about modules, say from an external
pickle or an xml file. So if PyScripter is to cache code completion
info, I'd like to expose your cache to Python, which would allow other
modules to put whatever useful metatdata they'd like to have into your
structures.

Potentially decoupling the code completion metadata from the code
could give good results: I'd probably be willing to work on providing
"good" metatdata, say for a module like pygame.

I understand that lack of explicit declarations and strong typing
really limits effectiveness Another idea might be to parse some
kind of 'annotation' mechanism (e.g. like embedded docstrings).

''' <argTypeInfo x=pygame.Surface>'''

pete


On Aug 30, 1:12 am, PyScripter <pyscrip...@gmail.com> wrote:
...

>
> All in all, you cannot expect code completion accuracy that is
> possible with strongly typed languages.  Of course the accuracy of
> code completion can be improved.  Two improvements I am planning to
> implement are:
>
> 1) Cache scanning results to improve speed when working with complex
> projects
> 2) Analyze return statements to determine the type of the return value
> of functions
>
> One other idea is to use a third party tool like rope (rope.sf.net).
>
> Any suggestions of further improvements will be welcome.
>
> On Aug 24, 12:03 pm, pete <p.wentwo...@ru.ac.za> wrote:
>
>
>
> > Using PyScripter 2.0.  In Python 2.6, opening a file in the
> > "disapproved-by-documentation" way, i.e.
>
> > f = file('xyz.txt')
>
> > has the big advantage that code completion works after qualifying f.
>
> > Alas, now that the file class is no more in Python 3, I must resort to
> > using "open".
>
> > In Python 3, open returns an object of type TextIOWrapper.
> > Code completion works for TextIOWrapper,  but not for object f of type
> > TextIOWrapper.
>
> > I'd like to understand why we don't/can't get the completion on the
> > object f.
> > And whether there is anything I can do (e.g. I've tried importing
> > IO.TextIOWrapper, etc.) that would turn the code completion on for me.
>

> > Thanks- Hide quoted text -
>
> - Show quoted text -

[Eric S. Johansson]

On 8/30/2010 1:10 AM, pete wrote:
> I made another suggestion elsewhere to provide some way that we can
> "load our own" overriding metadata about modules, say from an external
> pickle or an xml file. So if PyScripter is to cache code completion
> info, I'd like to expose your cache to Python, which would allow other
> modules to put whatever useful metatdata they'd like to have into your
> structures.
>
> Potentially decoupling the code completion metadata from the code
> could give good results: I'd probably be willing to work on providing
> "good" metatdata, say for a module like pygame.
>
> I understand that lack of explicit declarations and strong typing
> really limits effectiveness Another idea might be to parse some
> kind of 'annotation' mechanism (e.g. like embedded docstrings).
>
> '''<argTypeInfo x=pygame.Surface>'''

I'm also interested in this because I'm trying to build a new method of
programming using speech recognition. It's real important for people like me
whose hands broke and don't have enough sense to leave the field that damaged a
so badly. I'm trying to build a system which lets me say something like

Automotive supply.shipping.priority is DHL

And have it generate:

autoSuply.shipping.prio("DHL")

it would be even better if I said

Automotive supply.stripping.priority as DHL

And have it say "stripping" not known method for automotive supply and throw the
line back into an editor (but not necessarily pyscripter).

Other things I want to do are things like

"Show methods automotive supply" and have it list all the methods.

But at first, I'm hoping to simply speech enable pyscripter. as I said to
another mailing list, I was looking into how to support "Select-and-Say" and
someone reminded me of a presentation by Joel Gould from Dragon Systems back in
2000. here's some excerpts from a message telling me where to look.

"""

Joel Gould does this in his example windict.py (SampleMacros subdirectory of
the NatLink release) (which I never got working I think).

He clarifies things in the talk about VoiceCode
(www.synapseadaptive.com/joel/VoiceCoders.ppt ).

...

Basically there are 2 strategies:

1. Intercept VDct, (natlink.DictObj) and ensure you can synchronise your way
with the control in question,
2. Use DictGramBase and SelectGramBase of NatLink to get even more basic
control. This is used in VoiceCode.

"""

if you have an interest in working with this, let me know. I got not terribly
useful hands but I have knowledge.

[PyScripter]

Python version 3.x introduces function annotations (look it up in the
help file) so there is no need to invent a new type of metadata.
However the problem is that they are very rarely used yet. The
problem of autocompletion is the following:

When you require autocompletion of a variable say
somename.

you need to first determine the type of somename and then the members
of the type. Determining the type of somename is the difficult part
and no amount of metadata can help with that except when you work with
a very narrow domain of names. Not to mention the fact that in
execution time the same variable may refer to objects of different
type, so the type of a variable cannot be uniquely determined.

[PyScripter]

> 1) Cache scanning results to improve speed when working with complex
> projects
> 2) Analyze return statements to determine the type of the return value
> of functions

1 and 2 have now been implemented in version control.

Here is as an example of what can now be autocompleted

class A():
def a(self):
return "abc"

def g():
x = A()
return x.a()


a = g()

a. <--