run emacs in batch mode without a tty

[Sam Halliday]

Hi all,

I would like to run my elisp unit tests in a docker container, but my CI doesn't let me enable a tty when I'm doing this.

Given that the tests would only be using a dumb terminal anyway, it really just needs to be mocked out, but I don't have anything under `/dev/tty*`

Is it possible to run emacs in batch mode without access to a tty? Is there a way to mock a tty for these purposes?

No user input is expected throughout the entire run.

Best regards,
Sam

[to...@tuxteam.de]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, Jul 03, 2015 at 02:14:58AM -0700, Sam Halliday wrote:
> Hi all,
>
> I would like to run my elisp unit tests in a docker container, but my CI doesn't let me enable a tty when I'm doing this.
>
> Given that the tests would only be using a dumb terminal anyway, it really just needs to be mocked out, but I don't have anything under `/dev/tty*`

Bah. Those young'uns re-inventing all wheels at once. No /dev/tty* ;-)

Do you see a /dev/pty* or /dev/ptmx?

Note that if there are no ptys probably other things won't work
(ssh comes to mind).

regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlWWY0AACgkQBcgs9XrR2kb1MgCfVkFfsR6naDKG3OSm0+xP3lgg
TikAoIFkfsLqkkvnNHYpN9Uny1/UgMUZ
=/AiX
-----END PGP SIGNATURE-----

[Sam Halliday]

Hi, you're right... it's not possible to ssh in and check this stuff but it is possible to run a script in the environment to check that there are no pseudo terminals either (/dev/pty*)

[to...@tuxteam.de]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, Jul 03, 2015 at 03:28:11AM -0700, Sam Halliday wrote:
> Hi, you're right... it's not possible to ssh in and check this stuff but it is possible to run a script in the environment to check that there are no pseudo terminals either (/dev/pty*)

Note that I do't know very much about docker (nor am I very keen on
learning at the moment!), but this seems to have bitten others:

http://stackoverflow.com/questions/22793387/ssh-pseudo-tty-causes-pty-allocation-request-failed-on-channel-0

If I interpret that correctly, it seems that moving to version "0.9.1"
(whatever in the world that might mean) could help you.

regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlWWZ8IACgkQBcgs9XrR2ka6DwCfQt6UMFlqZDVViV87BtklFXCF
wtIAnjNXhwfl7ezzY2MCP1y4h2cYYZug
=EQCX
-----END PGP SIGNATURE-----

[Eli Zaretskii]

> Date: Fri, 3 Jul 2015 02:14:58 -0700 (PDT)
> From: Sam Halliday <sam.ha...@gmail.com>

>
> Is it possible to run emacs in batch mode without access to a tty? Is there a way to mock a tty for these purposes?
>
> No user input is expected throughout the entire run.

If no input is expected, and you run Emacs in batch mode, what do you
need a tty for?

[Sam Halliday]

On Friday, 3 July 2015 13:01:01 UTC+1, Eli Zaretskii wrote:
> If no input is expected, and you run Emacs in batch mode, what do you
> need a tty for?

Emacs won't run without one. It just crashes.

[Sam Halliday]

Hi Tomas,

I don't need to log in to the box when its running without a tty, thanks. I was just saying that you are correct that it is not possible to log in over ssh, and therefore to get this sort of information one needs to script it.

[Eli Zaretskii]

> Date: Fri, 3 Jul 2015 06:26:37 -0700 (PDT)
> From: Sam Halliday <sam.ha...@gmail.com>
>

> Emacs won't run without one. It just crashes.

It does? It doesn't for me:

emacs -batch --eval "(princ \"Hello, world\!\n\")" < /dev/null > foo

I get no crash and the expected message in the file 'foo'.

Anyway, if you see crashes due to lack of a TTY, please report that as
a bug using "M-x report-emacs-bug".

[Barry Margolin]

In article <mailman.6240.14359314...@gnu.org>,

That process still has a TTY, it's just not connected to stdin or
stdout. But it's accessible as /dev/tty and also connected to stderr
(you forgot to do 2>&1 to disconnect stderr).

I don't know offhand if this makes a difference to emacs, though.

--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[Sam Halliday]

On Friday, 3 July 2015 14:51:05 UTC+1, Eli Zaretskii wrote:
> > Date: Fri, 3 Jul 2015 06:26:37 -0700 (PDT)
> > From: Sam Halliday <sam.ha...@gmail.com>
> >
> > Emacs won't run without one. It just crashes.
>
> It does? It doesn't for me:
>
> emacs -batch --eval "(princ \"Hello, world\!\n\")" < /dev/null > foo

Are you running this command in an environment where there is no /dev/[pt]ty* ?

[Sam Halliday]

BTW, I created a little example of running your script in a docker instance that doesn't have a /dev/[pt]ty*

https://github.com/fommil/drone-tests/blob/master/.drone.yml

and that seems to be working! I'll investigate further why my tests seem to need a tty.

[Sam Halliday]

On Friday, 3 July 2015 14:51:05 UTC+1, Eli Zaretskii wrote:

> > Date: Fri, 3 Jul 2015 06:26:37 -0700 (PDT)
> > From: Sam Halliday <sam.ha...@gmail.com>
> >
> > Emacs won't run without one. It just crashes.
>
> It does? It doesn't for me:
>
> emacs -batch --eval "(princ \"Hello, world\!\n\")" < /dev/null > foo

Aah, the difference is that you're using `-batch` and that does allow me to do *some* things in the docker container, but emacs seems to just exit when I start a `process`. The docs don't mention this as a caveat http://www.emacswiki.org/emacs/BatchMode

Is there anything that you can think of that might cause a -batch emacs to exit early?

[Eli Zaretskii]

> From: Barry Margolin <bar...@alum.mit.edu>
> Date: Fri, 03 Jul 2015 11:07:31 -0400

>
> > emacs -batch --eval "(princ \"Hello, world\!\n\")" < /dev/null > foo
> >
> > I get no crash and the expected message in the file 'foo'.
> >
> > Anyway, if you see crashes due to lack of a TTY, please report that as
> > a bug using "M-x report-emacs-bug".
>
> That process still has a TTY, it's just not connected to stdin or
> stdout.

Then I don't understand what you mean by "has a TTY". Please
elaborate.

> But it's accessible as /dev/tty

That's only an issue if Emacs directly access that device. Does it?

> and also connected to stderr (you forgot to do 2>&1 to disconnect
> stderr).

That doesn't change anything, the same is true if I do.

[Eli Zaretskii]

> Date: Fri, 3 Jul 2015 08:42:38 -0700 (PDT)
> From: Sam Halliday <sam.ha...@gmail.com>
>

> > emacs -batch --eval "(princ \"Hello, world\!\n\")" < /dev/null > foo
>
>
> Are you running this command in an environment where there is no /dev/[pt]ty* ?

I don't understand why is that important.

Anyway, like I said: if you have Emacs crashing, please report that
with the relevant information, as an Emacs bug.

[Eli Zaretskii]

> Date: Fri, 3 Jul 2015 10:07:06 -0700 (PDT)
> From: Sam Halliday <sam.ha...@gmail.com>
>

> > emacs -batch --eval "(princ \"Hello, world\!\n\")" < /dev/null > foo
>
> Aah, the difference is that you're using `-batch`

Excuse me? It was you who asked specifically about the batch mode:

> Is it possible to run emacs in batch mode without access to a tty? Is there a way to mock a tty for these purposes?

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

> and that does allow me to do *some* things in the docker container, but emacs seems to just exit when I start a `process`. The docs don't mention this as a caveat http://www.emacswiki.org/emacs/BatchMode

What do you mean by "start a process"?

> Is there anything that you can think of that might cause a -batch emacs to exit early?

How do you see it exiting early? Please provide the details of what
you see, as I have no way of trying this on a system similar to yours.

[Emanuel Berg]

Barry Margolin <bar...@alum.mit.edu> writes:

> That process still has a TTY, it's just not
> connected to stdin or stdout. But it's accessible as
> /dev/tty and also connected to stderr (you forgot to
> do 2>&1 to disconnect stderr).

Indeed:

$ ps -e | grep tty

--
underground experts united
http://user.it.uu.se/~embe8573

[Sam Halliday]

On Friday, 3 July 2015 21:14:36 UTC+1, Eli Zaretskii wrote:
> > Date: Fri, 3 Jul 2015 10:07:06 -0700 (PDT)
> > From: Sam Halliday <sam.ha...@gmail.com>
> >
> > > emacs -batch --eval "(princ \"Hello, world\!\n\")" < /dev/null > foo
> >
> > Aah, the difference is that you're using `-batch`
>
> Excuse me? It was you who asked specifically about the batch mode:
>
> > Is it possible to run emacs in batch mode without access to a tty? Is there a way to mock a tty for these purposes?
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

I wasn't aware that there was a flag to turn emacs into a batch processor. You can invoke commands and load lisp files for purely automated testing without the -batch flag, and that is exactly what we are doing. I was calling that batch mode, but now I don't know what to call it.

> > and that does allow me to do *some* things in the docker container, but emacs seems to just exit when I start a `process`. The docs don't mention this as a caveat http://www.emacswiki.org/emacs/BatchMode
>
> What do you mean by "start a process"?

I mean start a http://www.gnu.org/software/emacs/manual/html_node/elisp/Processes.html

> > Is there anything that you can think of that might cause a -batch emacs to exit early?
>
> How do you see it exiting early? Please provide the details of what
> you see, as I have no way of trying this on a system similar to yours.

Well, this is literally all the information I have: the stdout

http://fommil.com/github.com/ensime/ensime-emacs/drone/ee99fc5d368cafc2dd583520f9fe8e9e23faadb6

from running this script

https://github.com/fommil/ensime-emacs/blob/drone/test/run_emacs_tests.sh

The last thing we see is the output from this function call (which starts the process) https://github.com/fommil/ensime-emacs/blob/drone/ensime-startup.el#L215

So I'm interested to know what implication the -batch flag has for the functionality of emacs. Specifically: launching/monitoring processes, and navigating buffers (i.e. spoofing user actions).

Best regards,
Sam

[Sam Halliday]

Aha! It would appear that when emacs is started in -batch mode then the program exits when it runs through the initial script.

But my tests are starting up a process, attaching a sentinel and then reacting to some triggers in that file. Under non-batch mode, emacs stays open until I call `kill-emacs`... but *now* it's dying too early.

So my question then becomes: is there a way to stop emacs -batch from terminating when it reaches the end of the script? i.e. require an explicit kill-emacs call.

[Sam Halliday]

On Friday, 3 July 2015 22:25:10 UTC+1, Sam Halliday wrote:
> Aha! It would appear that when emacs is started in -batch mode then the program exits when it runs through the initial script.
>
> But my tests are starting up a process, attaching a sentinel and then reacting to some triggers in that file. Under non-batch mode, emacs stays open until I call `kill-emacs`... but *now* it's dying too early.
>
> So my question then becomes: is there a way to stop emacs -batch from terminating when it reaches the end of the script? i.e. require an explicit kill-emacs call.

Bah! Even if I (sit-for 30) something else is ending the script early, the app only runs for a few seconds. I've commented out all the (kill-emacs) calls in my code but I can't see what's terminating the app. I've also added a kill-emacs-hook to print the backtrace, and this is all I see:


backtrace()
(message "%s" (backtrace))
(lambda nil (message "%s" (backtrace)))()
kill-emacs(t)
command-line()
normal-top-level()

so I have no idea where the normal-top-level / command-line (kill-emacs) call is being called... since my app should be (sit-for ...)-ing.

[Sam Halliday]

> So my question then becomes: is there a way to stop emacs -batch from terminating when it reaches the end of the script?

The docs say:

"In batch mode (see Batch Mode), sit-for cannot be interrupted, even by input from the standard input descriptor. It is thus equivalent to sleep-for, which is described below."

but I am pretty sure (sit-for seconds) *IS* interrupted by something.

So... this

(while t
(sleep-for 30))

Stops the script from ending until I'm ready to explicitly end it.

Muahahaha!

[Barry Margolin]

In article <ac8061d2-9984-4ef1...@googlegroups.com>,

Sam Halliday <sam.ha...@gmail.com> wrote:

> Aha! It would appear that when emacs is started in -batch mode then the
> program exits when it runs through the initial script.
>
> But my tests are starting up a process, attaching a sentinel and then
> reacting to some triggers in that file. Under non-batch mode, emacs stays
> open until I call `kill-emacs`... but *now* it's dying too early.
>
> So my question then becomes: is there a way to stop emacs -batch from
> terminating when it reaches the end of the script? i.e. require an explicit
> kill-emacs call.

I don't think so.

In normal Emacs, when the script finishes, it goes to the main input
loop, which displays the buffer and processes terminal input. That's why
it needs a terminal.

In batch mode, there's no terminal or redisplay, so there's no main
input loop. It just runs the script and exits.

Running another process isn't compatible with this mode, because
processing the process output is also part of that main input loop.

[Sam Halliday]

Hi Barry, you might want to read my other responses. Yes it is possible to do what I need.

[Steinar Bang]

>>>>> Eli Zaretskii <el...@gnu.org>:

> It does? It doesn't for me:

> emacs -batch --eval "(princ \"Hello, world\!\n\")" < /dev/null > foo

> I get no crash and the expected message in the file 'foo'.

>From the example above, how do you know that emacs doesn't require a tty
as seen from the startup shell, just to start up and run in batch?

(It's a real question, not a rethorical one. I'm curious. I guess the
tty requirement might differ, depending on the OS? Different for
GNU/linux (and other unixoid systems) and Win32?)

[Eli Zaretskii]

> From: Steinar Bang <s...@dod.no>
> Date: Sat, 04 Jul 2015 08:56:00 +0200

>
> >>>>> Eli Zaretskii <el...@gnu.org>:
>
> > It does? It doesn't for me:
>
> > emacs -batch --eval "(princ \"Hello, world\!\n\")" < /dev/null > foo
>
> > I get no crash and the expected message in the file 'foo'.
>
> >From the example above, how do you know that emacs doesn't require a tty
> as seen from the startup shell, just to start up and run in batch?

The original message talked about crashes. It's easy to know I don't
get that ;-)

As for what you ask, I don't understand the question. What do you
mean by "a tty as seen from the startup shell", and why is that
relevant? Once Emacs is invoked, why should it care about the shell
that invoked it?

> (It's a real question, not a rethorical one. I'm curious. I guess the
> tty requirement might differ, depending on the OS? Different for
> GNU/linux (and other unixoid systems) and Win32?)

They might, but (a) I tried my example both on GNU/Linux and on
MS-Windows, and (b) I explicitly asked for evidence that a tty device
needs to exist for Emacs to start in batch mode, because if such a
precondition exists, it must be a bug of some kind. Meanwhile, the OP
says that "emacs -batch" works on their system, which apparently lacks
a tty device, and that is yet another evidence that a tty device is
not needed in batch mode, as I'd expect.

[Eli Zaretskii]

> From: Barry Margolin <bar...@alum.mit.edu>
> Date: Fri, 03 Jul 2015 18:55:26 -0400

>
> In batch mode, there's no terminal or redisplay, so there's no main
> input loop. It just runs the script and exits.
>
> Running another process isn't compatible with this mode, because
> processing the process output is also part of that main input loop.

We do run subprocesses, both synchronous and asynchronous, in the
Emacs test suite, which is run entirely in batch mode. So evidently
there's no incompatibility here.

[Eli Zaretskii]

> Date: Fri, 3 Jul 2015 15:01:18 -0700 (PDT)
> From: Sam Halliday <sam.ha...@gmail.com>
>

> The docs say:
>
> "In batch mode (see Batch Mode), sit-for cannot be interrupted, even by input from the standard input descriptor. It is thus equivalent to sleep-for, which is described below."
>
> but I am pretty sure (sit-for seconds) *IS* interrupted by something.

It is interrupted by input from your subprocess.

> So... this
>
> (while t
> (sleep-for 30))
>
> Stops the script from ending until I'm ready to explicitly end it.

You need to wait more intelligently, so that the wait never ends until
you've received all the stuff from the subprocess, or until the
subprocess dies, or whatever else is pertinent for your application.

The fact that sleep-for (NOT sit-for!) is interrupted by subprocess
input is arguably a bug, which was recently fixed in the development
version of Emacs.

[Eli Zaretskii]

> Date: Fri, 3 Jul 2015 14:25:08 -0700 (PDT)
> From: Sam Halliday <sam.ha...@gmail.com>
>

> So my question then becomes: is there a way to stop emacs -batch from terminating when it reaches the end of the script? i.e. require an explicit kill-emacs call.

As you've already figured out, you need to have the top-level form
wait until all your processing is done.

[Eli Zaretskii]

> Date: Fri, 3 Jul 2015 13:43:25 -0700 (PDT)
> From: Sam Halliday <sam.ha...@gmail.com>
>

> So I'm interested to know what implication the -batch flag has for the functionality of emacs.

With the obvious exception of display (and related features, like
windows and frames), none.

> Specifically: launching/monitoring processes, and navigating buffers (i.e. spoofing user actions).

Everything should be working as expected.

[to...@tuxteam.de]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, Jul 03, 2015 at 06:27:53AM -0700, Sam Halliday wrote:
> Hi Tomas,
>
> I don't need to log in to the box when its running without a tty, thanks.
> I was just saying that you are correct that it is not possible to log in
> over ssh, and therefore to get this sort of information one needs to
> script it.

I wasn't thinking that far. I'm just the UNIX traditionalist thinking that
a box *should* have a pty -- guess it'll take some time to get used to
those strange worlds :-)

But I see you are on the right path thanks to Eli's and your efforts:
Emacs can very well survive without pty after all and I learnt quite a
few things by lurking. So thanks for that

regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlWXmb4ACgkQBcgs9XrR2kZPcQCdGt/7DDsuznX1KzHug7QCACO2
PGUAn1gvkaeG6/409OIC3Js5H5IwL18N
=pKIR
-----END PGP SIGNATURE-----