Random tracebacks in 2.1

[Carlos Córdoba]

I'm getting this traceback from time to time, in a kind of random,
unpredictable fashion (at least for me):

Traceback (most recent call last):
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/widgets/shell.py",
line 297, in keyPressEvent
self.postprocess_keyevent(event)
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/widgets/shell.py",
line 777, in postprocess_keyevent
ShellBaseWidget.postprocess_keyevent(self, event)
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/widgets/shell.py",
line 352, in postprocess_keyevent
self._key_tab()
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/widgets/shell.py",
line 809, in _key_tab
self.show_code_completion(automatic=False)
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/widgets/shell.py",
line 941, in show_code_completion
obj_list = self.get_module_completion(text)
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/widgets/externalshell/pythonshell.py",
line 121, in get_module_completion
return self.ask_monitor("getmodcomplist('%s')" % objtxt)
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/widgets/externalshell/pythonshell.py",
line 72, in ask_monitor
return communicate(sock, command, settings=settings)
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/utils/bsdsocket.py",
line 56, in communicate
return read_packet(sock)
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/utils/bsdsocket.py",
line 31, in read_packet
dlen, = struct.unpack("l", datalen)
struct.error: unpack requires a string argument of length 8

Sometimes this is accompanied of this other traceback:

Traceback (most recent call last):
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/plugins/inspector.py",
line 366, in refresh_plugin
self.set_object_text(None, force_refresh=False)
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/plugins/inspector.py",
line 529, in set_object_text
found = self.show_help(text, ignore_unknown=ignore_unknown)
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/plugins/inspector.py",
line 698, in show_help
if self.shell.is_defined(obj_text):
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/widgets/externalshell/pythonshell.py",
line 114, in is_defined
% (objtxt, force_import))
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/widgets/externalshell/pythonshell.py",
line 73, in ask_monitor
return communicate(sock, command, settings=settings)
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/utils/bsdsocket.py",
line 56, in communicate
return read_packet(sock)
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/utils/bsdsocket.py",
line 34, in read_packet
data += sock.recv(dlen)
OverflowError: signed integer is greater than maximum

and other times it only appears this one:

Traceback (most recent call last):
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/widgets/shell.py",
line 297, in keyPressEvent
self.postprocess_keyevent(event)
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/widgets/shell.py",
line 777, in postprocess_keyevent
ShellBaseWidget.postprocess_keyevent(self, event)
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/widgets/shell.py",
line 352, in postprocess_keyevent
self._key_tab()
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/widgets/shell.py",
line 809, in _key_tab
self.show_code_completion(automatic=False)
File
"/home/carlos/.virtualenvs/spyder_trunk/lib/python2.7/site-packages/spyderlib/widgets/shell.py",
line 986, in show_code_completion
b_k_g = dir(__builtin__)+self.get_globals_keys()+keyword.kwlist
TypeError: can only concatenate list (not "NoneType") to list

This is happening to me from some time now, I don't remember exactly
since when.

All these tracebacks seem to imply that monitor communication channels
are somehow shutdown, which breaks code completion. But how this can
happen and be fixed?

Cheers,
Carlos

[Carlos Córdoba]

I think I've found one reason for these tracebacks. They seem to happen
during the initialization of plugins and they're generated by trying to
refresh the plugins too early during startup, which is done here:

diff -r 4b43639efb69 spyderlib/plugins/__init__.py
--- a/spyderlib/plugins/__init__.py dom oct 16 15:06:11 2011 +0200
+++ b/spyderlib/plugins/__init__.py dom oct 16 11:59:05 2011 -0500
@@ -195,7 +195,8 @@ class SpyderPluginMixin(object):
if self.DISABLE_ACTIONS_WHEN_HIDDEN:
toggle_actions(self.plugin_actions, visible)
if visible:
- self.refresh_plugin() #XXX Is it a good idea?
+ #self.refresh_plugin() #XXX Is it a good idea?
+ pass
self.isvisible = enable and visible

def set_option(self, option, value):

If I comment that line, then the tracebacks doesn't happen anymore (at
least I didn't see any after several hours of work)

Cheers,
Carlos

El 11/09/11 11:57, Carlos Córdoba escribió:

[Pierre Raybaut]

Ok Carlos, this does not surprise me as I have encountered similar issues:
http://code.google.com/p/spyderlib/source/detail?r=0282cbd8f5f036bfb6254b235a5f9d9594128129
Socket communication fails during startup on "slow" machines (at least
I have never seen this bug on my different machines at work, and I see
it on a quite slow machine at home). And with these bugs, this refresh
plugin line was part of the traceback too.

You certainly have noticed the "#XXX Is it a good idea?" comment that
I wrote a long time ago when building the plugin interface. Honestly
I'm not sure to remember why exactly this could be a bad idea but I
think it had something to do with to many 'refresh_plugin' calls
leading to an inefficient implementation.

Now, commenting this line could be the solution. But the best solution
may be to disable this refresh_plugin method during Spyder's startup
-- with a timer or something like that.

-Pierre

2011/10/16 Carlos Córdoba <ccord...@gmail.com>:

> --
> You received this message because you are subscribed to the Google Groups
> "spyder" group.
> To post to this group, send email to spyd...@googlegroups.com.
> To unsubscribe from this group, send email to
> spyderlib+...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/spyderlib?hl=en.
>
>

[Carlos Córdoba]

Yep, I noticed the comment but since I'm not really sure what
refresh_plugin does, I just commented it. I agree with you that the best
solution would be to disable it during startup.

This bug really bothers me because it shuts down the object inspector
and sometimes messes the code completion in the console. Besides, I
would like to add that my machine is an i7 quad core running Linux, so
this not only happens on slow machines.

Cheers,
Carlos

El 16/10/11 15:20, Pierre Raybaut escribió:

[anatoly techtonik]

It seems my mind is obsessed with another crazy idea. I hope you won't mind if I record it here. =)

I look at recent problems mostly caused by 'too early' or 'late' execution of certain parts of our code. Currently, I don't have even a slightest idea of how Spyder initialization should work, which components should be initialized in which order, and - more importantly - how does it currently work in reality. I feel like I need a lot of time for static analysis to understand if it is an asynchronous run-time problem - or problem with the way our code is statically woven.
(that's the reason I can't reply Carlos to commit review at http://code.google.com/p/spyderlib/source/detail?r=fc5092657bc3cbfd86967de55bed688892b0d0b1 as I have more questions pop up in my head than I can possibly squeeze into a small letter given the time constraints we are all in)

So, I wonder if there is time to invent something that will make analysis of code execution order convenient? 'Something' that is able to dispatch asynchronous tasks, control them and provide visual tool for run-time and post-mortem analysis? The use cases:

1. wait until some task finishes processing before continuing (another approach - wait for data in channel - Go-like)
2. profile bottlenecks and time spent waiting for components
3. debug multithread locks, inspect current Spyder state in real-time
4. visual representation of time spent in threads (bars from UML sequence diagram)
5. provide high-level overview of Spyder initialization sequence using actual real-time data (like profiling, but with high-level components or blocks that are convenient for the given context - i.e. trace initialization and wait time of various Spyder parts)