Problem with Pyramid 1.8 and pdb session

[Lele Gaifax]

Hi all,

I've spent a couple of hours trying to understand an issue I'm facing with
Pyramid 1.8.3 when I enter into an interactive pdb session. It's not clear
what happens, and even less what's causing it: I tried hard to distill a
reproducible recipe, or at least to narrow the context, but without luck.

I will try to summarize, sorry for the length and bear with me please...

It all started with the goal of "modernizing" the dependencies of an
application[1], in particular upgrading it from Pyramid 1.6 to 1.8 and from
SQLAlchemy 1.0 to 1.1. This took very little effort[4], all tests pass, and
everything seems working as expected.

Given that yesterday I noticed a minor i18n issue on the production instance,
and verified that it's still present in my unreleased/modernized version, I
tried to spot the glitch using current sources, adding a pdb.set_trace()
call at the right point and started my local instance as usual. When the
execution enters into pdb however it is immediately clear that something
strange is happening with the in/out streams: the usual pdb interactive
session begins with something like (“@” is the cursor position)

> /path/to/file.py(123)name_of_function()
-> statement_at_line_123_of_file.py
(Pdb) @

where you can enter pdb commands. In particular, I can enter into a full
interactive interpreter, something like

(Pdb) interact
*interactive*
>>> import sys
>>> print(sys.version_info)
sys.version_info(major=3, minor=6, micro=1, releaselevel='candidate', serial=1)
>>> @

With Pyramid 1.8.3 instead the “(Pdb)” prompt is not at the right place and I
may[2] see the following

> /home/lele/wip/sol3/src/sol/models/competitor.py(184)caption()
-> if self.player1 is None:
(Pdb) > /home/lele/wip/sol3/src/sol/models/competitor.py(184)caption()
-> if self.player1 is None:
@

Effectively, it looks like *two* different interleaved sessions are going on
together, see:

interact
*interactive*
(Pdb) l
179 if player_caption is None:
180 player_caption = lambda player, h, l, cc: player.caption(html=h, localized=l,
181 css_class=cc)
>>> 182
183 import pdb; pdb.set_trace()
184 -> if self.player1 is None:
185 return _('Player NOT assigned yet!', just_subst=not localized)
186 else:
187 captions = [player_caption(self.player1, html, localized, css_class)]
188 if self.player2 is not None:
189 captions.append(player_caption(self.player2, html, localized, css_class))
@

At that point I hit the Enter key once and I get the (Pdb) prompt, another
Enter repeats the "l" command I entered above and the I get the usual ">>>"
Python prompt:

(Pdb)
190 if self.player3 is not None:
191 captions.append(player_caption(self.player3, html, localized, css_class))
192 if self.player4 is not None:
193 captions.append(player_caption(self.player4, html, localized, css_class))
194 return njoin(captions, localized=localized)
195
196 description = property(caption)
197
198 @property
199 def player1FullName(self):
200 "Full name of the first player"
>>> @

The "interact" here is just an example to show a different prompt, but the
very same confusion happens trying normal interactive pdb commands. Again, the
feeling is that I have *two* interactive sessions running at the same time,
contending my inputs and interleaving their output.

If I switch back to the master branch[3], reinstall its requirements.txt (thus
running with Pyramid 1.6.1, waitress 0.9.0 and no hupper) and repeat the
session, none of the above non-sense happens!

I then created a "starter" Pyramid project from scratch, but could not
reproduce the issue in any way...

I have obviously read the changelogs of Pyramid and waitress, but did not
spot anything that seems related.

Any hint on how to dig further, or shedding some light, will be highly
appreciated!

Thanks in advance,
ciao, lele.

[1] https://bitbucket.org/lele/sol
[2] “may”, because it's non-deterministic
[3] almost the same sources, different pinned requirements
[4] the only change was related to the dropped logging_file_config() function
--
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
le...@metapensiero.it | -- Fortunato Depero, 1929.

[Michael Merickel]

My first impression is that I don't think this has anything to do with upgrading. If you're using a browser that makes 2 parallel requests to the endpoint that triggers pdb then you will run into really weird issues where you have two pdbs running at once. However, this has pretty much always been the case so you're probably aware of it.

Are you using pserve with or without `--reload`?

--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/87efxyigvr.fsf%40metapensiero.it.
For more options, visit https://groups.google.com/d/optout.

[Lele Gaifax]

Lele Gaifax <le...@metapensiero.it> writes:

> If I switch back to the master branch[3], reinstall its requirements.txt (thus
> running with Pyramid 1.6.1, waitress 0.9.0 and no hupper) and repeat the
> session, none of the above non-sense happens!

Uhm, now I'm really confused: going on with the investigation, I was
eventually able to obtain the same "multiple sessions" even with Pyramid 1.6,
and effectively a "ps H" shows that pserve starts four threads... I'm ashamed
I never noticed this! :-\

Anyway, for some strange reason, the interactive pdb session I get with 1.6 is
/almost/ usable (the fact that I never noticed the issue before supports
that), while under 1.8 it's always impeded...

Well, I won't give up the investigation :-)

ciao, lele.

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/15/2017 02:34 PM, Lele Gaifax wrote:
> Lele Gaifax <le...@metapensiero.it> writes:
>
>> If I switch back to the master branch[3], reinstall its
>> requirements.txt (thus running with Pyramid 1.6.1, waitress 0.9.0
>> and no hupper) and repeat the session, none of the above non-sense
>> happens!
>
> Uhm, now I'm really confused: going on with the investigation, I was
> eventually able to obtain the same "multiple sessions" even with
> Pyramid 1.6, and effectively a "ps H" shows that pserve starts four
> threads... I'm ashamed I never noticed this! :-\
>
> Anyway, for some strange reason, the interactive pdb session I get
> with 1.6 is /almost/ usable (the fact that I never noticed the issue
> before supports that), while under 1.8 it's always impeded...
>
> Well, I won't give up the investigation :-)

Maybe you can work around it using a temporary module-scope variable and
an if statement wrapping the 'import pdb; pdb.set_trace'? Or acquire a
lock before doing the 'pdb.set_trace()'?

- ------------------------- %< ---------------------------------
from threading import RLock # reentrant for just-in-case

pdb_guard = RLock()

def view_to_be_debugged(request):
...
with pbd_guard:
import pdb; pdb.set_trace()
...
- ------------------------- %< ---------------------------------


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 tse...@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJYyZTjAAoJEPKpaDSJE9HYJwMP+wdg9K2BsKdymMm40x98miFA
0Rpc8Nnjaf831W2XObqCkxwm3GvPUxHM6jLiebfBZ9juWRBseExGiGMWPxjSTfgy
g8L+GDfPOtonKy0RkN8raqfcgSK8ZBZTzs+IEaVWZGobFFfiYOr/sFmd/zA5MQrc
Xn/oPEk1z7QbM7vEO8+aZxz/TzijdcFZ8Y0SMu8hjyuPq5OgMISFikssCZaIKuft
pniqxOTClEM02DpWV3lKHjcdBYj2kGYN2L31uXW0gdLSbLqdW/WnFLWFnNvuD7mX
VQ0T5SY1hHMdcvaZaib76qDZ4oKf5bmoQOhSUZcUSVDQEcFg59NAkDQP4i7HgED/
sPiJT4tp4gpOLcRinvPUxgehjV3pjNt0OfNascV7U/IH02ul/dlJdVM/eQj3h5b1
aJnI8pQA4m0CviOfhqyE3Pio5tQq8PCORtQ6wOSS2EFmePZiNhK4RmRI2Y49cOrm
qxgJMZ0LP/H02wINmznGFNhwSN/uvNeABQb4LjJxXFh0wFXR8JGmlHB58FxQZn1e
eaq8NXm7h5z8LPFVLCyPgYxfD5oZDSQqKl3BwMUTAnyfS6X8u5iHizgpksQfElDs
J1iYH40liX2eqgpLcehdaMguZiMrvMMNBAdeLZtxyf7v26CXELNoi6SnBKorHKHF
TTqEcb/vHADzFLAAAfdt
=86VW
-----END PGP SIGNATURE-----

[Jonathan Vanasco]

I've run into variations of this before under a few scenarios:

- a new version of pyramid or a plugin may handle static/etc resources slightly differently

- a browser may emit a secondary request for the 'view source' window if you have it open for debugging html

[Lele Gaifax]

I'm still surprised though: in that application almost all views are called
thru AJAX, they are bridges between a client Javascript app and the DB.