stdio

[Joshua Bronson]

I replaced subprocess.PIPE with None everywhere it occurred in gaedriver.py because I wanted to be able to use pdb and see my app's logging output while running tests locally with the dev_appserver. Is there any interest in adding support for this?

Thanks for releasing gaedriver, it's great.

[Robert Schuppenies]

This means stdout and stderr are not created for ClientThreadBase_ instances, right?

I'm not terribly familiar with pdb, could you explain why this was necessary? Are you familiar with other tools and how they handle such a use case?

cheers,

robert

[Joshua Bronson]

On Wed, Apr 18, 2012 at 2:26 PM, Robert Schuppenies <sch...@google.com> wrote:

This means stdout and stderr are not created for ClientThreadBase_ instances, right?

I grepped for ClientThreadBase_ in the gaedriver code and couldn't find it, so I'm not sure what you're referring to.

But http://docs.python.org/library/subprocess.html#frequently-used-arguments says:

stdin, stdout and stderr specify the executed program’s standard input, standard output and standard error file handles, respectively. Valid values are PIPE, an existing file descriptor (a positive integer), an existing file object, and None. PIPE indicates that a new pipe to the child should be created. With the default settings of None, no redirection will occur; the child’s file handles will be inherited from the parent. 

I'm not terribly familiar with pdb, could you explain why this was necessary?

Without this, the stdio of the subprocess gaedriver spawns with your app in it gets swallowed up. So nothing your app writes to stdout is displayed, including any logging it does to stdout, which is helpful to see while debugging failing tests.

pdb is like other command line debuggers (gdb, jdb, etc). Calling pdb.set_trace() from in your app halts execution, prints a debugging environment to stdout, and waits for input from stdin so you can interact with the state of your program while it's running. But since your program's stdio is not connected to your terminal's when spawned by gaedriver, putting a breakpoint in your app makes it silently stuck forever.

Are you familiar with other tools and how they handle such a use case?

Nose's --pdb-failures option comes to mind, but nose is for unit tests, not end-to-end tests, so doing exactly what nose does wouldn't be possible / appropriate. I'm not sure about other tools, but I agree checking out some other test drivers to see what they do is a good idea.

[Robert Schuppenies]

On Wed Apr 18 13:59:45 GMT-700 2012, Joshua Bronson <jabr...@gmail.com> wrote:

On Wed, Apr 18, 2012 at 2:26 PM, Robert Schuppenies <sch...@google.com> wrote:

This means stdout and stderr are not created for ClientThreadBase_ instances, right?

I grepped for ClientThreadBase_ in the gaedriver code and couldn't find it, so I'm not sure what you're referring to.

Ah, sry, I was referring to http://code.google.com/p/gaedriver/source/browse/py/src/gaedriver/gaedriver.py#364 and #365.

Those variables are assigned with the output of subprocess call and then available to callers for further processing.

 

But http://docs.python.org/library/subprocess.html#frequently-used-arguments says:

stdin, stdout and stderr specify the executed program’s standard input, standard output and standard error file handles, respectively. Valid values are PIPE, an existing file descriptor (a positive integer), an existing file object, and None. PIPE indicates that a new pipe to the child should be created. With the default settings of None, no redirection will occur; the child’s file handles will be inherited from the parent. 

I'm not terribly familiar with pdb, could you explain why this was necessary?

Without this, the stdio of the subprocess gaedriver spawns with your app in it gets swallowed up. So nothing your app writes to stdout is displayed, including any logging it does to stdout, which is helpful to see while debugging failing tests.

That was done to be able to see if appcfg spits out an erro gaedriver can handle for the user (the rollback suggestion):

http://code.google.com/p/gaedriver/source/browse/py/src/gaedriver/gaedriver.py#562

 

pdb is like other command line debuggers (gdb, jdb, etc). Calling pdb.set_trace() from in your app halts execution, prints a debugging environment to stdout, and waits for input from stdin so you can interact with the state of your program while it's running. But since your program's stdio is not connected to your terminal's when spawned by gaedriver, putting a breakpoint in your app makes it silently stuck forever.

Please help me to understand: With "from in your app" you mean your App Engine application or the code that runs gaedriver? If it's the former, does that mean you can you pdb to control the app which is started as a separate process by the gaedriver code?

As an alternative, would it help you if gaedriver would support logging and when enabled you see the output of whatever gaedriver called? It would only show it after it is done, but that might be better than what you have to deal with now. Hacking into the live stream of a subprocess is more involved[1] and something I didn't want to burden gaedriver with

 

Are you familiar with other tools and how they handle such a use case?

Nose's --pdb-failures option comes to mind, but nose is for unit tests, not end-to-end tests, so doing exactly what nose does wouldn't be possible / appropriate. I'm not sure about other tools, but I agree checking out some other test drivers to see what they do is a good idea.

[1] http://hg.python.org/releasing/2.7.3/file/7bb96963d067/Lib/subprocess.py#l723