Running python from pty without prompt

[space.ship...@gmail.com]

Hello.

I'm working on a script runner for Atom.

https://github.com/ioquatix/script-runner

We are trying to understand how to make python work well. I'll use a comparison to the ruby executable because it's convenient to explain the problem.

When you invoke `ruby` from a pty, you get no output (as opposed to `irb`, interactive ruby [shell]). You can write a script to stdin, and send Ctrl-D (EOT / 0x04). Then, ruby will execute the script. stdin is not closed so programs that expect interactive input will work correctly.

When we run python in the same way, we get an output prompt. As soon as a function like `input` is called, the program is interrupted.

I'm happy to hear feedback about how this should work. Perhaps our expectations are wrong, or what we are doing is wrong.

One option which we've been considering is saving the input as a file and executing that. But, it's not as clean compared to simply writing a string to stdin of a running interpreter and then 0x04.

Thanks for any and all help.

Kind regards,
Samuel

[Michael Torrie]

On 12/09/2016 04:11 PM, space.ship...@gmail.com wrote:
> Hello.
>
> I'm working on a script runner for Atom.
>
> https://github.com/ioquatix/script-runner
>
> We are trying to understand how to make python work well. I'll use a
> comparison to the ruby executable because it's convenient to explain
> the problem.
>
> When you invoke `ruby` from a pty, you get no output (as opposed to
> `irb`, interactive ruby [shell]). You can write a script to stdin,
> and send Ctrl-D (EOT / 0x04). Then, ruby will execute the script.
> stdin is not closed so programs that expect interactive input will
> work correctly.
>
> When we run python in the same way, we get an output prompt. As soon
> as a function like `input` is called, the program is interrupted.
>
> I'm happy to hear feedback about how this should work. Perhaps our
> expectations are wrong, or what we are doing is wrong.

Not sure I understand the issue here. You can pipe a script to Python
and it runs it without any immediate-mode prompt. I think python only
shows the REPL prompt if you are attached to a pty.

But if the script is piped into Python or ruby, I don't know how we
could expect raw_input() or input() to function.

> One option which we've been considering is saving the input as a file
> and executing that. But, it's not as clean compared to simply writing
> a string to stdin of a running interpreter and then 0x04.

An intermediate file is unnecessary as you can pipe a script into Python.

[Michael Torrie]

On 12/09/2016 04:11 PM, space.ship...@gmail.com wrote:

> When you invoke `ruby` from a pty, you get no output (as opposed to
> `irb`, interactive ruby [shell]). You can write a script to stdin,
> and send Ctrl-D (EOT / 0x04). Then, ruby will execute the script.
> stdin is not closed so programs that expect interactive input will
> work correctly.
>
> When we run python in the same way, we get an output prompt. As soon
> as a function like `input` is called, the program is interrupted.

Nevermind my other post. I understand what you are saying.

[Samuel Williams]

Just in case it's not clear, this is running on a (virtual) PTY.

[Steve D'Aprano]

On Sat, 10 Dec 2016 10:11 am, space.ship...@gmail.com wrote:

> Hello.
>
> I'm working on a script runner for Atom.
>
> https://github.com/ioquatix/script-runner
>
> We are trying to understand how to make python work well. I'll use a
> comparison to the ruby executable because it's convenient to explain the
> problem.

Could you show a small, simple demonstration of both the Ruby code that
works they way you want, and the Python code that behaves differently?

Preferably one that runs straight from vanilla Python without any
third-party libraries, including your script-runner.



--
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

[Michael Torrie]

On 12/09/2016 06:43 PM, Steve D'Aprano wrote:
> On Sat, 10 Dec 2016 10:11 am, space.ship...@gmail.com wrote:
>
>> Hello.
>>
>> I'm working on a script runner for Atom.
>>
>> https://github.com/ioquatix/script-runner
>>
>> We are trying to understand how to make python work well. I'll use a
>> comparison to the ruby executable because it's convenient to explain the
>> problem.
>
> Could you show a small, simple demonstration of both the Ruby code that
> works they way you want, and the Python code that behaves differently?
>
> Preferably one that runs straight from vanilla Python without any
> third-party libraries, including your script-runner.

I imagine they want to feed Python a combination of a script and also
standard-in input in the same stream. Something like:

$ python << EOF
a = input("Enter your name: ")
print a
^D
bob
EOF

Where ^D is a literal control character (ctrl-v, control-d on the
terminal, but does not actually close the stream or signify the end).
This would be piped into Python's standard-in where it would feed python
both the script to run, and also input to feed the script. Apparently
ruby can do this.

Did I understand this correctly, space.ship.traveller?

[Samuel Williams]

Michael, yes.

FYI, I found out why this works. Pressing Ctrl-D flushes the input
buffer. If you do this on an empty line, it causes read(...) to return
0 which Ruby considers end of input for the script, but the pipe is
not closed.

[Michael Torrie]

Currently Python does not appear to support this behavior. Possibly it
could be patched to support something similar, though.

[Michael Torrie]

On 12/13/2016 09:01 AM, Michael Torrie wrote:
> On 12/13/2016 05:39 AM, Samuel Williams wrote:

> Currently Python does not appear to support this behavior. Possibly it
> could be patched to support something similar, though.

I wonder if you could write a python wrapper that would read the input
file from standard in until you get a ctrl-d, then exec() that input.

[Random832]

On Tue, Dec 13, 2016, at 11:01, Michael Torrie wrote:
> On 12/13/2016 05:39 AM, Samuel Williams wrote:

> Currently Python does not appear to support this behavior. Possibly it
> could be patched to support something similar, though.

The problem is there's currently no way to differentiate "interactive
mode" from "script run on a tty".

You can get similar behavior with python -c "import
sys;exec(sys.stdin.read())"

[Michael Torrie]

On 12/13/2016 10:48 AM, Random832 wrote:
> The problem is there's currently no way to differentiate "interactive
> mode" from "script run on a tty".
>
> You can get similar behavior with python -c "import
> sys;exec(sys.stdin.read())"

Are you sure? I can pipe scripts into Python and they run fine and
Python is not in interactive mode.

python < script.py

The behavior the OP is looking for of course is a way of demarcating the
end of the script and the beginning of data to feed the script.

[Random832]

On Tue, Dec 13, 2016, at 17:09, Michael Torrie wrote:
> On 12/13/2016 10:48 AM, Random832 wrote:
> > The problem is there's currently no way to differentiate "interactive
> > mode" from "script run on a tty".
> >
> > You can get similar behavior with python -c "import
> > sys;exec(sys.stdin.read())"
>
> Are you sure? I can pipe scripts into Python and they run fine and
> Python is not in interactive mode.

Yes, a pipe and a tty are two different things.

> python < script.py
>
> The behavior the OP is looking for of course is a way of demarcating the
> end of the script and the beginning of data to feed the script.

It's more than just that - with a tty you can call sys.stdin.read()
multiple times, and each time end it with ctrl-d.

[Steve D'Aprano]

On Wed, 14 Dec 2016 09:24 am, Random832 wrote:

> On Tue, Dec 13, 2016, at 17:09, Michael Torrie wrote:
>> On 12/13/2016 10:48 AM, Random832 wrote:
>> > The problem is there's currently no way to differentiate "interactive
>> > mode" from "script run on a tty".
>> >
>> > You can get similar behavior with python -c "import
>> > sys;exec(sys.stdin.read())"
>>
>> Are you sure? I can pipe scripts into Python and they run fine and
>> Python is not in interactive mode.
>
> Yes, a pipe and a tty are two different things.

Can you show a simple demonstration of what you are doing?

I'm having difficulty following this thread because I don't know
what "script run on a tty" means.

I thought that with the exception of scripts run from cron, any time you run
a script *or* in interactive mode, there is an associated tty. Am I wrong?

>> python < script.py
>>
>> The behavior the OP is looking for of course is a way of demarcating the
>> end of the script and the beginning of data to feed the script.
>
> It's more than just that - with a tty you can call sys.stdin.read()
> multiple times, and each time end it with ctrl-d.

[Steve D'Aprano]

On Wed, 14 Dec 2016 04:48 am, Random832 wrote:

> On Tue, Dec 13, 2016, at 11:01, Michael Torrie wrote:
>> On 12/13/2016 05:39 AM, Samuel Williams wrote:
>> > Michael, yes.
>> >
>> > FYI, I found out why this works. Pressing Ctrl-D flushes the input
>> > buffer. If you do this on an empty line, it causes read(...) to return
>> > 0 which Ruby considers end of input for the script, but the pipe is
>> > not closed.
>>
>> Currently Python does not appear to support this behavior. Possibly it
>> could be patched to support something similar, though.
>
> The problem is there's currently no way to differentiate "interactive
> mode" from "script run on a tty".

sys.flags.interactive will tell you whether or not your script was launched
with the -i flag.

hasattr(sys, 'ps1') or hasattr(sys, 'ps2') will tell you if you are running
in the REPL (interactive interpreter). The ps1 and ps2 variables aren't
defined in non-interactive mode.

Does that help?

> You can get similar behavior with python -c "import
> sys;exec(sys.stdin.read())"

[steve@ando ~]$ python -c "import sys; print hasattr(sys, 'ps1')"
False

[steve@ando ~]$ python -c "import sys; exec(sys.stdin.read())"
import sys
print hasattr(sys, 'ps1')
False


It's not obvious, but after I entered the line "print hasattr(...)" I typed
Ctrl-D, ending the stream.

[Michael Torrie]

On 12/13/2016 05:10 PM, Steve D'Aprano wrote:
> Can you show a simple demonstration of what you are doing?

I think they want to run Python, perhaps remotely via ssh, and feed it
both a script and input over standard-in (though a tty comes into this
somehow and I'm not clear on that). Apparently in Ruby you can pass a
script to it via standard-in, then a ctrl-d, and standard-in is kept
open so they can then feed the ruby script input. If Python supported
this, an example would look something like this:

$ python << EOF
a = input("Give me something: ")
print (a)
<^D>
test_input
EOF

Where ^D is a literal control-d character the marks the end of the
script and the beginning of input that will go to the script.

The tty part might come into play when they are using ssh to remotely
run the python process. Standard in, though, is the primary mechanism
they want to use if I understand the OP correctly.

I think a wrapper that feeds exec() would do what he desires.

[Ben Finney]

Steve D'Aprano <steve+...@pearwood.info> writes:

> I thought that with the exception of scripts run from cron, any time
> you run a script *or* in interactive mode, there is an associated tty.
> Am I wrong?

Any daemon will, by definition, have no controlling terminal.

Other processes can choose to detach themselves from their controlling
terminal.

Either of those could invoke Python, and then the Python program would
be running without any controlling terminal.

--
\ “I have always wished for my computer to be as easy to use as |
`\ my telephone; my wish has come true because I can no longer |
_o__) figure out how to use my telephone.” —Bjarne Stroustrup |
Ben Finney

[Random832]

On Tue, Dec 13, 2016, at 19:10, Steve D'Aprano wrote:
> Can you show a simple demonstration of what you are doing?
>
> I'm having difficulty following this thread because I don't know
> what "script run on a tty" means.

The question is literally about the input/script being the tty and not
redirected from any other file, which causes an interactive prompt in
CPython, but does not do so in some other languages. I don't understand
what part of this you're not getting.

[Random832]

On Tue, Dec 13, 2016, at 19:20, Steve D'Aprano wrote:
> sys.flags.interactive will tell you whether or not your script was
> launched
> with the -i flag.
>
> hasattr(sys, 'ps1') or hasattr(sys, 'ps2') will tell you if you are
> running
> in the REPL (interactive interpreter). The ps1 and ps2 variables aren't
> defined in non-interactive mode.

There's no way to *tell python to* run in non-interactive mode without
using a file other than the tty as the script. It's not a matter of
finding out from within python whether it's in interactive note, it's a
matter of python finding out whether the user *wants* it to run in
interactive mode.

[Steve D'Aprano]

On Thu, 15 Dec 2016 01:28 am, Random832 wrote:

> On Tue, Dec 13, 2016, at 19:10, Steve D'Aprano wrote:
>> Can you show a simple demonstration of what you are doing?
>>
>> I'm having difficulty following this thread because I don't know
>> what "script run on a tty" means.
>
> The question is literally about the input/script being the tty and not
> redirected from any other file, which causes an interactive prompt in
> CPython, but does not do so in some other languages. I don't understand
> what part of this you're not getting.

What can I say? Maybe I'm just slow. Or maybe you're falling into the curse
of knowledge:

https://en.wikipedia.org/wiki/Curse_of_knowledge

I'm not the only one having trouble understanding the nature of this
problem -- Michael Torrie has also said "though a tty comes into this
somehow and I'm not clear on that".

What is meant by "the input/script being the tty"? And how does that relate
to the subject line which refers to a pty?

That's why I've asked for a simple example that demonstrates the issue. But
apparently this simple example is so simple that nobody knows how to write
it. I cannot replicate the OP's problem from his description alone, and I
have not seen an example where the Python prompt is shown apart from when
running Python interactively.

So... the input is the tty. I don't know what that means, but I think I know
what it isn't. I'm fairly confident it isn't when you pipe the output of
one process to Python:

# not this
[steve@ando ~]$ echo "import sys; print sys.version" | python
2.7.2 (default, May 18 2012, 18:25:10)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)]


And likewise you said it is not when input is *redirected* from a file, so
it probably isn't this:

[steve@ando ~]$ cat myfile
import sys; print sys.version

[steve@ando ~]$ python < myfile
2.7.2 (default, May 18 2012, 18:25:10)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)]


and surely I can eliminate passing the file name as a command line argument
(python myfile) as well. So what is left?


Michael Torrie suggests something more or less like this, redirecting stdin
to Python with a here-doc:

[steve@ando ~]$ python << .
> import sys
> print sys.version
> .
2.7.2 (default, May 18 2012, 18:25:10)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)]

(Michael's version used EOF rather than a dot.) There's a prompt, but it's
not the Python prompt, it's from the shell. Since you are insisting that
the Python interactive prompt is involved, then surely Michael's example
isn't what you mean either.

So I now have *four* ways of running code in Python that *don't* match the
OP's problem (five if you include the standard REPL) and I'm not closer to
understanding the OP's problem.

[Samuel Williams]

Here are some examples of different languages:

https://github.com/ioquatix/script-runner/blob/master/examples/python-eot.py

[Michael Torrie]

On 12/14/2016 09:29 PM, Samuel Williams wrote:
> Here are some examples of different languages:
>
> https://github.com/ioquatix/script-runner/blob/master/examples/python-eot.py

Okay so it looks like you're just opening a pipe to a subprocess and
feeding it a script and input. So there's no pty involved here. Or am
I wrong?

In any case, I think if you made a python wrapper script that could take
the standard in up until the ctrl-d, and then exec() that, what's left
on standard in should be able to feed the exec'd script any input it needs.