Racket Shell
434 views
Skip to first unread message
William G Hatch
Aug 20, 2016, 3:17:47 PM8/20/16
to racket...@googlegroups.com
Hello everyone,
Being obsessed with shells and wanting very badly to have a racket
shell, I've spent a good chunk of time over the last couple of weeks
working on shell-related stuff for Racket. First is a library for shell
pipelines (that may also contain racket functions). The interface is
roughly like this (assuming `my-grep` is a racket function that
implements grep):
(run-pipeline '(ls -l) (list my-grep "foobar") '(wc -l))
The docs for that library are here [1].
Additionally, I've got a rough sketch of a line-based syntax for writing shell
scripts or using a shell repl with embedded racket code. Right now its syntax
looks like this:
#lang rash
ls -l | @my-grep "foobar" | wc -l
;; A line is not wrapped in a pipeline if it starts with &
;; You can get back into #lang rash syntax with @rash{} or @rash/out{}
&@(define ttys @rash/out{ls /dev | grep tty})
I want to change the line syntax a lot (especially to not need &), but I
decided to put it up (with some very rough documentation) here [2] to
get some feedback on what people think of it.
I'm really interested in people's thoughts about both parts and ways people
think they could be improved.
Thanks,
William Hatch
[1] http://docs.racket-lang.org/shell-pipeline/index.html
[2] http://docs.racket-lang.org/rash/index.html
Being obsessed with shells and wanting very badly to have a racket
shell, I've spent a good chunk of time over the last couple of weeks
working on shell-related stuff for Racket. First is a library for shell
pipelines (that may also contain racket functions). The interface is
roughly like this (assuming `my-grep` is a racket function that
implements grep):
(run-pipeline '(ls -l) (list my-grep "foobar") '(wc -l))
The docs for that library are here [1].
Additionally, I've got a rough sketch of a line-based syntax for writing shell
scripts or using a shell repl with embedded racket code. Right now its syntax
looks like this:
#lang rash
ls -l | @my-grep "foobar" | wc -l
;; A line is not wrapped in a pipeline if it starts with &
;; You can get back into #lang rash syntax with @rash{} or @rash/out{}
&@(define ttys @rash/out{ls /dev | grep tty})
I want to change the line syntax a lot (especially to not need &), but I
decided to put it up (with some very rough documentation) here [2] to
get some feedback on what people think of it.
I'm really interested in people's thoughts about both parts and ways people
think they could be improved.
Thanks,
William Hatch
[1] http://docs.racket-lang.org/shell-pipeline/index.html
[2] http://docs.racket-lang.org/rash/index.html
Stephen Chang
Aug 20, 2016, 3:29:14 PM8/20/16
to William G Hatch, Racket-Users List
Cool! Will definitely check it out.
Is this at all related to Vincent's work? [1]
[1]: https://github.com/stamourv/rash
> --
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-users...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
Is this at all related to Vincent's work? [1]
[1]: https://github.com/stamourv/rash
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-users...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
William G Hatch
Aug 20, 2016, 3:45:28 PM8/20/16
to Stephen Chang, Racket-Users List
I had no idea that existed. (in my defense, I did google "racket
shell", "racket rash", etc before starting this)
I'll have to look at it as well.
shell", "racket rash", etc before starting this)
I'll have to look at it as well.
Lehi Toskin
Aug 20, 2016, 6:05:06 PM8/20/16
to Racket Users
This is super neat. I'd be interested in using a Racket-based shell once it gets to be more polished.
Matthias Felleisen
Aug 20, 2016, 6:24:06 PM8/20/16
to William G Hatch, racket...@googlegroups.com
I’d love to use a Racket shell and script Unix in a ‘natural’ way.
You may wish to read up on scsh. While it was way ahead of its
time, Olin Shivers made the syntax as natural as possible so it
would be quickly useful to people used to basic shell scripting
syntax. After all, this syntax has a 40+ history and shouldn’t be
thrown out.
[1] https://scsh.net
Norman Gray
Aug 21, 2016, 4:46:38 PM8/21/16
to William G Hatch, racket...@googlegroups.com, Matthias Felleisen
William, hello.
On 20 Aug 2016, at 23:23, Matthias Felleisen wrote:
> You may wish to read up on scsh. While it was way ahead of its
> time, Olin Shivers made the syntax as natural as possible so it
> would be quickly useful to people used to basic shell scripting
> syntax. After all, this syntax has a 40+ history and shouldn’t be
> thrown out.
I think that Olin Shivers made a lot of excellent syntactical decisions
when designing scsh -- in particular his syntax for regular expressions
and for extended process forms -- to the extent that if one were simply
to reimplement the scsh syntax in Racket, that would be an excellent
departure point. This is not to dispute your own syntactical choices,
but in an enterprise like this, scsh would be the elephant in the room.
There have been a couple of attempts to support the scsh syntax in
Racket, and the scsh code is still under active, though rather
low-level, maintenance <https://github.com/scheme/scsh>. Incidentally,
scsh does start up very quickly, which is an attractive feature.
And this enthusiasm isn't just because scsh was the first Scheme
implementation I spent serious time with (*wipes wistful tear*).
Best wishes,
Norman
--
Norman Gray : https://nxg.me.uk
SUPA School of Physics and Astronomy, University of Glasgow, UK
Matthias Felleisen
Aug 21, 2016, 5:07:43 PM8/21/16
to Norman Gray, William G Hatch, Racket Users
William & Norman,
as much as I give credit to scsh, I just think that taking clues from scsh is enough. One could expand surface syntax into William’s pipe library for example.
The irony is of course that scsh’s start-up time was so bad back then, that we couldn’t use it for anything real.
— Matthias
Matthias Felleisen
Aug 21, 2016, 10:47:33 PM8/21/16
to Norman Gray, William G Hatch, Racket Users, Tony Garnock-Jones
I just ran across this little thing:
https://asciinema.org/a/0utgivr7glk3ssn01fn5uwtey
People who want more shell ought to watch. It’s 2mins and cute — Matthias
William G Hatch
Aug 22, 2016, 12:59:53 AM8/22/16
to Matthias Felleisen, Norman Gray, Racket Users, Tony Garnock-Jones
I've looked over the scsh docs at various times, though I've never
actually used it -- it's never been pre-packaged for distros I've used,
and every time I've tried to build it I've run into errors. Perhaps I
should try again. As far as the process syntax goes, a little bit of
macros over my pipeline library should be able to get you something
pretty close. I'll probably do that later.
The parts of scsh aside from the process forms, like the regex stuff, I
think are mostly orthogonal to the process stuff, so that could probably
just be a different library, then they could be used together.
A lot of that does look pretty good. I think he's probably right that
the |> piped output should be the last argument (I've had my `shellify`
wrapper making it the first).
>I’d love to use a Racket shell and script Unix in a ‘natural’ way.
Not just Unix -- it works on Windows too ;)
So is scsh what you consider natural?
I definitely want a nice s-expression syntax for it, but also a
convenient line-based syntax for interactive use. I like the idea that
you can more or less turn a transcript of such commands into a script --
one of the reasons I always encourage people to get used to using a
shell is that it makes you think a lot more about automating tasks when
you type the same command sequence often. It's something you just can't
do with GUI interfaces, and automating boring stuff is one of the joys
of programming. Having a nice upgrade path (just gradually put more of
it in regular racket until it turns into a "real" program) for those
scripts that seem to grow over time is also really appealing to me.
Thanks for the feedback.
William
actually used it -- it's never been pre-packaged for distros I've used,
and every time I've tried to build it I've run into errors. Perhaps I
should try again. As far as the process syntax goes, a little bit of
macros over my pipeline library should be able to get you something
pretty close. I'll probably do that later.
The parts of scsh aside from the process forms, like the regex stuff, I
think are mostly orthogonal to the process stuff, so that could probably
just be a different library, then they could be used together.
A lot of that does look pretty good. I think he's probably right that
the |> piped output should be the last argument (I've had my `shellify`
wrapper making it the first).
>I’d love to use a Racket shell and script Unix in a ‘natural’ way.
So is scsh what you consider natural?
I definitely want a nice s-expression syntax for it, but also a
convenient line-based syntax for interactive use. I like the idea that
you can more or less turn a transcript of such commands into a script --
one of the reasons I always encourage people to get used to using a
shell is that it makes you think a lot more about automating tasks when
you type the same command sequence often. It's something you just can't
do with GUI interfaces, and automating boring stuff is one of the joys
of programming. Having a nice upgrade path (just gradually put more of
it in regular racket until it turns into a "real" program) for those
scripts that seem to grow over time is also really appealing to me.
Thanks for the feedback.
William
Matthias Felleisen
Aug 22, 2016, 9:33:47 AM8/22/16
to William G Hatch, Norman Gray, Racket Users, Tony Garnock-Jones
> On Aug 22, 2016, at 12:59 AM, William G Hatch <wil...@hatch.uno> wrote:
>
> is scsh what you consider natural?
— one for people who program in zsh, bash, tcsh, csh, sh, and perhaps even more primitive things (if they exist); scsh syntax appeals to those
— one for people like me who write Racket programs for their shell-like needs (HtDP/2e has several large script like that)
And of course, the ideal world has a path to go back and forth from these points. — Matthias, who always dreams of every thing
Vincent St-Amour
Aug 22, 2016, 12:17:34 PM8/22/16
to William G Hatch, Stephen Chang, Racket-Users List
your Racket namespace. That's one of the aspects of bash that I think
are really interesting and useful, and was the motivation for the project.
That it's also a nice showcase for `#%app` is just gravy. :)
The other aspect of bash that I find interesting, and was hoping to
capture in `rash` was easy port redirection, pipelines, etc. bash makes
simple patterns really easy. Racket is more flexible, but requires more
scaffolding. I didn't find a design that would combine the simplicity of
bash with Racket's syntax.
Then there's also the implicit coercion story. Unix utilities basically
all consume newline-separated strings. Racket functions typically need
more specific kinds of inputs. So mixing the two together (which is
really the point of having bash as a #lang) ended up requiring a lot of
adapters. It's not clear to me how to get that without making the result
as "ad-hoc" as bash itself, though.
Anyway, just a few thoughts on the topic.
Vincent
Hendrik Boom
Aug 22, 2016, 12:38:05 PM8/22/16
to Matthias Felleisen, Racket Users
Scheme, and one that's just ordinary text with a bit of markup..
-- hendrik
Andrew Gwozdziewycz
Aug 22, 2016, 12:38:41 PM8/22/16
to Vincent St-Amour, William G Hatch, Stephen Chang, Racket-Users List
This discussion has reminded of SHILL
(https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-moore.pdf)
which is less user focused, and more security / capabilities focused,
but the prototype (in Racket) utilizes contracts to enforce
capabilities checks.
I'm really into the idea of a nice Racket based shell that provides
more benefits than simply pipelines, though, `something/shell` did
look pretty neat by itself...
http://www.apgwoz.com
(https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-moore.pdf)
which is less user focused, and more security / capabilities focused,
but the prototype (in Racket) utilizes contracts to enforce
capabilities checks.
I'm really into the idea of a nice Racket based shell that provides
more benefits than simply pipelines, though, `something/shell` did
look pretty neat by itself...
> --
> You received this message because you are subscribed to the Google Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
--
> You received this message because you are subscribed to the Google Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
http://www.apgwoz.com
Vincent St-Amour
Aug 22, 2016, 4:18:18 PM8/22/16
to Andrew Gwozdziewycz, Vincent St-Amour, William G Hatch, Stephen Chang, Racket-Users List
On Mon, 22 Aug 2016 11:38:37 -0500,
Andrew Gwozdziewycz wrote:
>
> This discussion has reminded of SHILL
> (https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-moore.pdf)
> which is less user focused, and more security / capabilities focused,
> but the prototype (in Racket) utilizes contracts to enforce
> capabilities checks.
Yes, that's really cool work!
Andrew Gwozdziewycz wrote:
>
> This discussion has reminded of SHILL
> (https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-moore.pdf)
> which is less user focused, and more security / capabilities focused,
> but the prototype (in Racket) utilizes contracts to enforce
> capabilities checks.
For those interested, Scott will be speaking about Shill at RacketCon[1]
next month.
Vincent
[1] con.racket-lang.org
Andrew Gwozdziewycz
Aug 23, 2016, 2:02:05 AM8/23/16
to Vincent St-Amour, William G Hatch, Stephen Chang, Racket-Users List
Daniel Bastos
Aug 24, 2016, 8:55:57 AM8/24/16
to William G Hatch, Racket Users
On Sat, Aug 20, 2016 at 4:17 PM, William G Hatch <wil...@hatch.uno> wrote:
> I'm really interested in people's thoughts about both parts and ways people
> think they could be improved.
Hi, William. Have you looked at ESHELL to see if it has anything of interest?
> I'm really interested in people's thoughts about both parts and ways people
> think they could be improved.
%(format-time-string "%D %H:%M:%S" (seconds-to-time 1471456370))
08/17/16 14:52:50
%
%for i in (list 'a 'b 'c) {echo $i}
a
b
c
%
%for i in (list 'a 'b 'c) {echo x = $i}
("x" "=" a)
("x" "=" b)
("x" "=" c)
%
This echo is actually a Lisp function.
%which echo
eshell/echo is a compiled Lisp function in `em-basic.el'
%
%which *echo
c:/emacs/git/usr/bin/echo.exe
%
To invoke the echo program, use an asterisk:
%for i in (list 'a 'b 'c) {*echo x = $i}
x = a
x = b
x = c
%
The manual:
https://www.gnu.org/software/emacs/manual/html_node/eshell/index.html
Eli Barzilay
Aug 25, 2016, 9:18:15 PM8/25/16
to William G Hatch, Matthias Felleisen, Norman Gray, Racket Users, Tony Garnock-Jones
On Mon, Aug 22, 2016 at 12:59 AM, William G Hatch <wil...@hatch.uno> wrote:
> I've looked over the scsh docs at various times, though I've never
> actually used it -- it's never been pre-packaged for distros I've
> used, and every time I've tried to build it I've run into errors.
> Perhaps I should try again. As far as the process syntax goes, a
> little bit of macros over my pipeline library should be able to get
> you something pretty close. I'll probably do that later.
There were some problematic points with the scsh macro -- something
> I've looked over the scsh docs at various times, though I've never
> actually used it -- it's never been pre-packaged for distros I've
> used, and every time I've tried to build it I've run into errors.
> Perhaps I should try again. As far as the process syntax goes, a
> little bit of macros over my pipeline library should be able to get
> you something pretty close. I'll probably do that later.
around the implicit backquoting that made things not compose nicely.
I think that this was what Mike Sperber told me at some point.
--
((x=>x(x))(x=>x(x))) Eli Barzilay:
http://barzilay.org/ Maze is Life!
Kieron Hardy
Sep 21, 2016, 2:17:09 AM9/21/16
to Racket Users, William G Hatch
I've been experimenting with 'shell/pipeline' on windows and post some tests and example results here in case they are of use to others:
- basic examples demonstrating windows external commands,
- basic examples demonstrating windows internal commands with standard cmd.shell,
- basic examples demonstrating windows internal commands with 'new' powershell.cmd shell,
- a more complex example showing a command pipeline using powershell commands to do sed-like and grep-like operations.
Cheers,
Kieron.
****
Examples using Windows external commands:
#lang racket/base
; windows-pipeline-1.rkt - basic windows commands with output to stdout and stderr
(require shell/pipeline)
; run racket, do arithmetic
(run-pipeline '(racket -e "(+ 41 1)") )
; run java, get java version
(run-pipeline '(java -version) )
; run where, find location of windows where.exe on path
(run-pipeline '(where where) )
; run where, find location of windows cmd.exe on path
(run-pipeline '(where cmd) )
Example execution from Windows command prompt:
C:\home\racket\shell>racket windows-pipeline-1.rkt > windows-pipeline-1.rkt.out 2>&1
C:\home\racket\shell>type windows-pipeline-1.rkt.out
42
java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)
C:\Windows\System32\where.exe
C:\Windows\System32\cmd.exe
0
0
0
0
Examples using Windows internal commands, i.e. commands built-in to the cmd.exe shell:
#lang racket/base
; windows-pipeline-2.rkt - basic windows internal (cmd.exe) commands with output to stdout and stderr
(require shell/pipeline)
; note: run cmd.exe with /c switch to terminate cmd.exe after executing the given command, i.e. terminate without needing to execute the 'exit' command
; run windows cmd.exe shell, run internal 'dir' command to get directory listing of the root of C: drive
(run-pipeline '(cmd /c dir C:\\) )
; run windows cmd.exe shell, run internal 'set' command to the get a complete listing of the environment variables and their values
(run-pipeline '(cmd /c set) )
; run windows cmd.exe shell, run internal 'echo' command to send arguments to standard out
(run-pipeline '(cmd /c echo hello world) )
; run windows cmd.exe shell, run internal 'path' command to get the value of the PATH environment variable
(run-pipeline '(cmd /c path) )
; run windows cmd.exe shell, run internal 'echo' command to get the value of the PATH environment variable
(run-pipeline '(cmd /c echo %PATH%) )
Example execution from Windows command prompt:
C:\home\racket\shell>racket windows-pipeline-2.rkt > windows-pipeline-2.rkt.out 2>&1
C:\home\racket\shell>type windows-pipeline-2.rkt.out
Volume in drive C is Windows
Volume Serial Number is 4A0E-59A3
Directory of C:\
05/06/2016 10:53 PM <DIR> Analytics
07/06/2015 09:57 AM <DIR> Brother
09/14/2015 07:49 AM <DIR> cygwin64
01/08/2016 05:00 PM <DIR> data
11/04/2015 02:18 PM 321,531 DUMP38fa.tmp
09/20/2016 02:51 PM <DIR> home
12/22/2014 07:04 PM <DIR> Intel
07/02/2015 01:19 PM <DIR> OSGeo4W64
07/13/2009 09:20 PM <DIR> PerfLogs
08/28/2016 08:48 PM <DIR> Program Files
09/20/2016 09:21 AM <DIR> Program Files (x86)
09/09/2015 05:50 PM <DIR> Python27
11/12/2015 01:16 PM <DIR> Python27_64
08/29/2016 02:47 PM <DIR> SWSETUP
09/20/2016 09:02 AM <DIR> tools
06/19/2015 02:41 PM <DIR> Users
08/02/2015 02:12 PM <DIR> wamp
09/14/2016 11:48 AM <DIR> Windows
09/07/2016 07:35 PM 42 windows-version.txt
2 File(s) 321,573 bytes
17 Dir(s) 366,015,729,664 bytes free
ALLUSERSPROFILE=C:\ProgramData
APPDATA=C:\Users\Kieron\AppData\Roaming
asl.log=Destination=file
CLASSPATH=.;C:\tools\apple\QuickTime\QTSystem\QTJava.zip
CommonProgramFiles=C:\Program Files (x86)\Common Files
CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files
CommonProgramW6432=C:\Program Files\Common Files
COMPUTERNAME=NIETZSCHE
...
hello world
PATH=C:\ProgramData\Oracle\Java\javapath;C:\tools\racket\racket32_6.5\Racket;C:\Program Files (x86)\...
C:\ProgramData\Oracle\Java\javapath;C:\tools\racket\racket32_6.5\Racket;C:\Program Files (x86)\...
0
0
0
0
0
Examples using Windows powershell shell:
#lang racket/base
; windows-pipeline-3.rkt - basic windows internal (powershell.exe) commands with output to stdout and stderr
(require shell/pipeline)
; note: run powershell.exe with /Command switch to terminate powershell.exe after executing the given command, i.e. terminate without needing to execute the 'exit' command
; run windows external command 'where.exe', find location of windows powershell.exe on path
(run-pipeline '(where powershell) )
; run windows powershell.exe shell, run internal 'echo' command to get the value of the arguments to standard out
(run-pipeline '(powershell /Command "& {echo Hello World}") )
; run windows powershell.exe shell, run internal 'write-output' command
(run-pipeline '(powershell /Command "& {write-output "test output"}") )
; run windows powershell.exe shell, run internal 'write-output' command to get the list of running processes - similar to unix 'ps'
(run-pipeline '(powershell /Command "& {$p = get-process; write-output $p;}") )
Example execution from Windows command prompt:
C:\home\racket\shell>racket windows-pipeline-3.rkt > windows-pipeline-3.rkt.out 2>&1
C:\home\racket\shell>type windows-pipeline-3.rkt.out
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
Hello
World
test
output
Handles NPM(K) PM(K) WS(K) VM(M) CPU(s) Id ProcessName
------- ------ ----- ----- ----- ------ -- -----------
78 9 2800 3052 86 2.57 1076 AccelerometerSt
163 10 7324 8220 574 0.50 1200 bash
192 23 42140 48632 228 525.05 3184 chrome
181 22 35228 27380 216 2.20 3944 chrome
168 9 1496 3276 66 0.12 4608 chrome
187 22 36600 28572 214 1.37 5304 chrome
267 27 35896 30700 306 1.72 5708 chrome
284 30 62344 56520 334 103.52 5740 chrome
1508 101 69124 86340 408 136.67 2768 explorer
1156 110 335276 389796 936 1,722.89 2124 firefox
0 0 0 24 0 0 Idle
384 27 8976 11068 91 1940 spoolsv
797 35 31096 23580 135 580 svchost
80 10 1672 2524 48 668 wininit
118 9 3196 5396 57 960 winlogon
...
0
0
0
0
Examples using Windows command pipelines that include the powershell shell to mimic some basic *nix utilities:
#lang racket/base
; windows-pipeline-4.rkt - examples of command pipelines that include powershell.exe commands
(require shell/pipeline)
; note: run cmd.exe with /C switch to terminate cmd.exe after executing the given command, i.e. terminate without needing to execute the 'exit' command
; note: run powershell.exe with /Command switch to terminate powershell.exe after executing the given command, i.e. terminate without needing to execute the 'exit' command
; a 3-component chain:
; run windows internal cmd.exe command 'path' to get the contents of the environment variable PATH, send to a powershell to split on ';', send to another powershell to look for the case insensitive string 'system'
(run-pipeline '(cmd /c path) '(powershell /Command "$Input -split ';'") '(powershell /Command "$Input | select-string -pattern 'system'") )
; a 3-component chain:
; run windows internal cmd.exe command 'path' to get the contents of the environment variable PATH, send to a powershell to split on ';', send to another powershell to look for the case sensitive string 'system'
(run-pipeline '(cmd /c path) '(powershell /Command "$Input -split ';'") '(powershell /Command "$Input | select-string -pattern 'system' -casesensitive") )
; for completeness the same command in one powershell step:
; run windows internal powershell.exe command to get the 'path' environment variable, split on ';', select only those with case sensitive string 'system'
(run-pipeline '(powershell /Command "$Env:path | %{$_.split(';')} | select-string -pattern 'system' -casesensitive") )
Example execution from Windows command prompt:
C:\home\racket\shell>racket windows-pipeline-4.rkt > windows-pipeline-4.rkt.out 2>&1
C:\home\racket\shell>type windows-pipeline-4.rkt.out
C:\windows\system32
C:\windows\System32\Wbem
C:\windows\System32\WindowsPowerShell\v1.0\
C:\tools\apple\QuickTime\QTSystem\
C:\windows\system32
C:\windows\system32
Reply all
Reply to author
Forward
0 new messages