xonsh idioms

[Keith Gunderson]

Hi, 

Its my first week using the xonsh shell and I'm wondering if there's a better way to express the following task:

I have a short list of strings that represent a bash pipeline;  ["grep ... | sed ... | cut ... | sudo -su me rsync ...", ]

I can run these via subprocess just fine, thanks to shell=True.

for x in cmds:

.......................     subprocess.call(x, shell=True)

But, is there a better way to use xonsh operators ?  I've tried the following, but run into command not found issues, removing the loop for simplicity.

@( cmds[0] )

xonsh: subprocess mode: command not found: grep ... | ... |  ...

@( cmds[0].split() )

grep: 2: No such file or directory

My brain is a little bashed to get what's going on here.

Keith

[Anthony Scopatz]

Hi Keith!

Thanks for trying out xonsh! Now as per your question...

You should be able to pipe commands together like you would in bash. Is this not working for you for some reason? Do you have a full and complete command that is breaking that you can share, along with the error message?

Be Well
Anthony

--
You received this message because you are subscribed to the Google Groups "xonsh" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xonsh+un...@googlegroups.com.
To post to this group, send email to xo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xonsh/30123083-b359-4685-a084-d1f8b93e2f84%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Asst. Prof. Anthony Scopatz
Nuclear Engineering Program
Mechanical Engineering Dept.
University of South Carolina
sco...@cec.sc.edu
Office: (803) 777-9334
Cell: (512) 827-8239
Office: Horizon 1, #011 (ground floor, not first)
Book a meeting with me at https://scopatz.youcanbook.me/
Open up an issue: https://github.com/scopatz/me/issues
Check my calendar

[Keith Gunderson]

Hi Anthony.

Here's what I'm seeing: 

>>> cmd_sample

"grep -w root /opt/home.txt | sed -e 's|/home||g' | cut -f 2 | sudo -su root rsync -a --files-from=- /home /mnt/archive/home"

>>> @(cmd_sample)

xonsh: subprocess mode: command not found: grep -w root /opt/home.txt | sed -e 's|/home||g' | cut -f 2 | sudo -su root rsync -a --files-from=- /home /mnt/archive/home

>>> @(cmd_sample.split())

grep: 2: No such file or directory

Thanks so much,

Keith

[Anthony Scopatz]

Hi Keith,

Why not just type in:

>>> grep -w root /opt/home.txt | sed -e 's|/home||g' | cut -f 2 | sudo -su root rsync -a --files-from=- /home /mnt/archive/home

I don't think that you should be using @(cmd) here.  That takes a Python object and converts it to a string for a command line argument.  It is meant for passing Python objects into the command.  The second case with @(cmd.split()) is more correct, and it is weird that it isn't finding grep.  However, you *are* circumventing $PATH lookup.  I bet if you used abspaths for all of the commands it might work.

In any event, you should type in the command normally. Let us know how it goes!

Be Well

Anthony

--
You received this message because you are subscribed to the Google Groups "xonsh" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xonsh+un...@googlegroups.com.
To post to this group, send email to xo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/xonsh/CALD07fx9N3FtuU3chXAG4Jy4gmftfQxUDj_XQ205WU4c%3DPra0Q%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Keith Gunderson]

I'm working with a slightly larger workflow (that I only created in ipython a few weeks ago.)  Any further help would be greatly appreciated.

I have xonsh in a small miniconda venv, by the way.

Here's more context (with full paths):

>>> cmds = ["/usr/bin/grep -w {x} /opt/home.txt  | /usr/bin/sed -e 's|/home||g' | /usr/bin/cut -f 2 | /usr/bin/sudo -su {x} /usr/bin/rsync -a --files-from=-

... /home /mnt/archive/home".format(x=x) for x in names]

>>> for x in cmds:

...     @(x.split())

...

/usr/bin/grep: 2: No such file or directory

/usr/bin/grep: 2: No such file or directory

/usr/bin/grep: 2: No such file or directory

/usr/bin/grep: 2: No such file or directory

/usr/bin/grep: 2: No such file or directory

/usr/bin/grep: 2: No such file or directory

/usr/bin/grep: 2: No such file or directory

/usr/bin/grep: 2: No such file or directory

>>> xonfig

+------------------+----------------------+

| xonsh            | 0.6.0                |

| Git SHA          | c29819c5             |

| Commit Date      | Nov 22 17:51:52 2017 |

| Python           | 3.6.3                |

| PLY              | 3.9                  |

| have readline    | True                 |

| prompt toolkit   | 1.0.15               |

| shell type       | prompt_toolkit       |

| pygments         | 2.2.0                |

| on posix         | True                 |

| on linux         | True                 |

| distro           | centos               |

| on darwin        | False                |

| on windows       | False                |

| on cygwin        | False                |

| is superuser     | False                |

| default encoding | utf-8                |

| xonsh encoding   | utf-8                |

| encoding errors  | surrogateescape      |

+------------------+----------------------+

On Tue, Dec 19, 2017 at 11:54 AM, Anthony Scopatz <sco...@gmail.com> wrote:

Hi Keith,

Why not just type in:

>>> grep -w root /opt/home.txt | sed -e 's|/home||g' | cut -f 2 | sudo -su root rsync -a --files-from=- /home /mnt/archive/home

I don't think that you should be using @(cmd) here.  That takes a Python object and converts it to a string for a command line argument.  It is meant for passing Python objects into the command.  The second case with @(cmd.split()) is more correct, and it is weird that it isn't finding grep.  However, you *are* circumventing $PATH lookup.  I bet if you used abspaths for all of the commands it might work.

In any event, you should type in the command normally. Let us know how it goes!

Be Well

Anthony

On Tue, Dec 19, 2017 at 9:49 AM Keith Gunderson <kegu...@gmail.com> wrote:

Hi Anthony.

Here's what I'm seeing: 

>>> cmd_sample

"grep -w root /opt/home.txt | sed -e 's|/home||g' | cut -f 2 | sudo -su root rsync -a --files-from=- /home /mnt/archive/home"

>>> @(cmd_sample)

xonsh: subprocess mode: command not found: grep -w root /opt/home.txt | sed -e 's|/home||g' | cut -f 2 | sudo -su root rsync -a --files-from=- /home /mnt/archive/home

>>> @(cmd_sample.split())

grep: 2: No such file or directory

Thanks so much,

Keith

--
You received this message because you are subscribed to the Google Groups "xonsh" group.

To unsubscribe from this group and stop receiving emails from it, send an email to xonsh+unsubscribe@googlegroups.com.

To post to this group, send email to xo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/xonsh/CALD07fx9N3FtuU3chXAG4Jy4gmftfQxUDj_XQ205WU4c%3DPra0Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Anthony Scopatz]

Have you tried running the code through evalx() or execx() - ie the xonsh equivalents of Python's exec() and eval()?  If you have arbitrary xonsh code you want to run you should be able to do 

For single expressions:

>>> for cmd in cmds:

...     evalx(cmd)

For multiple statements:

>>> for cmd in cmds:

...     execx(cmd)

To unsubscribe from this group and stop receiving emails from it, send an email to xonsh+un...@googlegroups.com.

To post to this group, send email to xo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/xonsh/CALD07fx9N3FtuU3chXAG4Jy4gmftfQxUDj_XQ205WU4c%3DPra0Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Asst. Prof. Anthony Scopatz
Nuclear Engineering Program
Mechanical Engineering Dept.
University of South Carolina
sco...@cec.sc.edu
Office: (803) 777-9334
Cell: (512) 827-8239
Office: Horizon 1, #011 (ground floor, not first)
Book a meeting with me at https://scopatz.youcanbook.me/
Open up an issue: https://github.com/scopatz/me/issues
Check my calendar

--
You received this message because you are subscribed to the Google Groups "xonsh" group.

To unsubscribe from this group and stop receiving emails from it, send an email to xonsh+un...@googlegroups.com.

To post to this group, send email to xo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/xonsh/CALD07fwNwQS-uMpb%3D59bcbjwC-2VK79FT%3Daq%2BYUhfSDFNWfdbA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Keith Gunderson]

Thank you.  That certainly works.  It this merely syntax around subprocess.call ?

Another idiom question,  when I gather unique names from a text column, I'm still heavily bashy.  Is there a suggestion for this that avoids the untidy business at the end?  (The slice and possible the split)

names = $(cat /opt/home.txt | cut -f1 | uniq | sort | uniq).split('\n')[:-1]

To unsubscribe from this group and stop receiving emails from it, send an email to xonsh+unsubscribe@googlegroups.com.

To post to this group, send email to xo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/xonsh/CALD07fx9N3FtuU3chXAG4Jy4gmftfQxUDj_XQ205WU4c%3DPra0Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Asst. Prof. Anthony Scopatz
Nuclear Engineering Program
Mechanical Engineering Dept.
University of South Carolina
sco...@cec.sc.edu
Office: (803) 777-9334
Cell: (512) 827-8239
Office: Horizon 1, #011 (ground floor, not first)
Book a meeting with me at https://scopatz.youcanbook.me/
Open up an issue: https://github.com/scopatz/me/issues
Check my calendar

--
You received this message because you are subscribed to the Google Groups "xonsh" group.

To unsubscribe from this group and stop receiving emails from it, send an email to xonsh+unsubscribe@googlegroups.com.

To post to this group, send email to xo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/xonsh/CALD07fwNwQS-uMpb%3D59bcbjwC-2VK79FT%3Daq%2BYUhfSDFNWfdbA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Nick Demou]

I've played with pyfunctional for what I think you're after (process text by chaining functions the UNIX-pipe way). It is very capable but a bit too verbose. Here's an example:

seq.open('/tmp/foo').filter(lambda x: 'grep grep' not in x).map(lambda x: [len(x.split('|')), x.split('|')]).filter(lambda x: x[0]>6).map(lambda x: x[1]).drop(25).map( lambda x: seq(x).map(lambda x: x.partition(' ')[0].strip()) ).list()

If you highlight it like this it becomes more readable:

  0 seq.

  1 open('/tmp/foo').

  2 filter(lambda x: 'grep grep' not in x).

  3 map(lambda x: x.split('|')).

  4 map(lambda x: [len(x), x]).

  5 filter(lambda x: x[0]>6).

  6 map(lambda x: x[1]).

  7 map(lambda x: seq(x).map(

            lambda x: x.partition(' ')[0].strip()

                                                          ) ).

  8 flatten().

  9 sorted ().

10 distinct()

Obviously it's way too verbose to compete with bash in regards to clarity and typing but the ability to use python and custom python functions makes it appealing. I have a dream that I'll find the time to create a TUI that will allow me to write something more light (see below) with auto completion and the ability to watching the results change as I edit individual steps:

  1 '/tmp/foo'

  2 filter  'bar' in x

  3 map  x.split('|')

  4 map  [len(x), x]

  5 filter  x[0]>6

  6 map  x[1]

  7 map  seq(x).map(lambda x: x.partition(' ')[0].strip())

  8 flatten

  9 sorted

10 distinct

To unsubscribe from this group and stop receiving emails from it, send an email to xonsh+un...@googlegroups.com.

To post to this group, send email to xo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/xonsh/CALD07fx9N3FtuU3chXAG4Jy4gmftfQxUDj_XQ205WU4c%3DPra0Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Asst. Prof. Anthony Scopatz
Nuclear Engineering Program
Mechanical Engineering Dept.
University of South Carolina
sco...@cec.sc.edu
Office: (803) 777-9334
Cell: (512) 827-8239
Office: Horizon 1, #011 (ground floor, not first)
Book a meeting with me at https://scopatz.youcanbook.me/
Open up an issue: https://github.com/scopatz/me/issues
Check my calendar

--
You received this message because you are subscribed to the Google Groups "xonsh" group.

To unsubscribe from this group and stop receiving emails from it, send an email to xonsh+un...@googlegroups.com.

To post to this group, send email to xo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/xonsh/CALD07fwNwQS-uMpb%3D59bcbjwC-2VK79FT%3Daq%2BYUhfSDFNWfdbA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.