Gear VR

[John Carmack]

The intersection of Gear VR owners and Racket users may consist of just me at the moment, but if anyone else here is interested in discussing the work I am doing with driving VR by Racket/Scheme, I would welcome the input.  Would that be an appropriate topic on the list?

 

[Matthias Felleisen]

We welcome all posts connected to Racket. Post away. -- Matthias

p.s. The question is whether you get suitable feedback. My life is so abstract, I sometimes wonder what reality is :-) 

On Jun 23, 2015, at 7:36 PM, John Carmack wrote:

The intersection of Gear VR owners and Racket users may consist of just me at the moment, but if anyone else here is interested in discussing the work I am doing with driving VR by Racket/Scheme, I would welcome the input.  Would that be an appropriate topic on the list?

 

-- 
You received this message because you are subscribed to the Google Groups "Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Neil Toronto]

Very yes. Also, I'm definitely in the intersection: I've just changed
research topics to functional 3D, and I've written a rendering engine
and modeling API in Racket for it.

What part do you have Racket playing in driving VR?

Neil ⊥

[John Carmack]

Here is the overview of the work that I recently sent out internally.

I started out with a pure-functional scripting interface, but it very rapidly became imperative, and it is in heavy flux. The sample scripts do capture the general "flavor" that I want this to have -- I think it will be easy enough to get interesting things happening that it will still tempt people who might be a little hostile to working in a Lisp.

------------------------------------------

Here is the first preview of the general purpose VR scripting environment I have been working on. There isn’t anything exciting to look at, but if you squint real hard you can see where it might be going. After I get a bit farther along, we can build a PC version as well for cross platform support.

Motivation:

Doing VR GUI development in the native apps is unpleasant – you can gut it out in C++ with our current interfaces, but the iteration times and code structure don’t make it a lot of fun.

In-editor prototyping is nice in Unity, but actual on-device testing cycle times are terrible, and you wind up with all the bulk of Unity and poor access to the native VR features like TimeWarp layers.

With app signing, we have no way for developers to casually share work with each other or limited communities.

My solution is to support development of VR applications in a scripting language with several modes:

“Web like”, where the script is downloaded from the net for each execution and run by a single signed app that we provide (NetHMD). Fully specified by an app-scheme URI, this allows VR experiences to be launched directly from web pages or app-links on Facebook, which I think is VERY powerful – read about how great something is while browsing on your phone, then just click the link and get the “insert into HMD” dialog and jump right there. VR scripts can link to other VR scripts without leaving VR. There is no direct monetization here, but IAP can eventually be supported.

“App like”, where the script is bundled into a conventional apk to be distributed through the store, and the scripting is simply used as a more effective development environment for some classes of applications.

“Remote development”, where the script is actually executed in an IDE on a PC, communicating with NetHMD over a TCP connection. This allows sub-second code-change to VR change iteration cycles, and the use of a debugger. There might also be a use for a related mode where a central server directly drives one or more clients.

Where it will win first:

My “reference application” is the linking of multiple panoramic photos with text and audio annotations. This is a completely obvious application that is not performance sensitive, and even a non-programmer could copy-paste-modify the simple script for this to make a high quality experience.

Once I hook up an ExoPlayer interface, this will completely subsume what I wanted to accomplish with “experience files” – the addition of static prop models to panoramic video, linking between different videos, merging of models or panoramas with videos that aren’t fully panoramic (GoPro, etc), and so on.

Once I hook up the Social interfaces, this will make it much easier to build trivial “activities” for multi-user experiences – board games, card games, etc.

It is already a big help for basic graphics development work – being able to just position a few quads with a custom shader and iterate on the shader programming remotely without rebuilding is a huge win.

Implementation:

This will be controversial, and I will have a longer discussion about it, but I chose Scheme (a Lisp) for the scripting language. If this all works out, we can eventually make C# and Javascript versions, but I have a bunch of reasons for going this way first. I am using Racket http://racket-lang.org/ for the PC development environment, and Chibi Scheme for the embedded interpreter on Android. Download Racket and walk through their quick-intro-with-pictures to get a feel for the language and you should be able to hack on the sample scripts I have been writing pretty quickly. Note that Racket is a very extended Scheme targeted at desktop apps, the embedded version is just the basic standard.

All texture / model / sound resources are loaded and cached from network URI. For testing, you can just paste in any URI from a Google image search, which is convenient. For an actual project development, dropbox should work fine.

I am favoring ease of development over performance, and it definitely won’t be suitable for a lot of tasks, but everything we have done with first party apps so far should be at least a candidate for development in this style once it is mature.

Everything is super-rough right now, and I keep thinking “one more day of work will clean it up better for people to look at”, but it is past time I at least get this on everyone’s radar.

<actual Android apk redacted -- it is too crappy to show the broader world right now>

Running it will load a default home script, which has links to a few test programs I have made. The back button returns to the home page.

The sample scripts are at:

http://s3.amazonaws.com/o.oculuscdn.com/netasset/home.rkt
http://s3.amazonaws.com/o.oculuscdn.com/netasset/office.rkt
http://s3.amazonaws.com/o.oculuscdn.com/netasset/shader.rkt
http://s3.amazonaws.com/o.oculuscdn.com/netasset/spin.rkt
http://s3.amazonaws.com/o.oculuscdn.com/netasset/text-panels.rkt
http://s3.amazonaws.com/o.oculuscdn.com/netasset/reversi.rkt

You could edit them dawn-of-the-web style by downloading, editing, uploading someplace else, and sending a new intent to nethmd:

adb shell am start -d nethmd://s3.amazonaws.com/o.oculuscdn.com/netasset/shader.rkt

However, the preferred development strategy is to use the DrRacket IDE. In addition to the scripts you are working on, download the following two files into the same directory:

https://s3.amazonaws.com/o.oculuscdn.com/netasset/vr.rkt
https://s3.amazonaws.com/o.oculuscdn.com/netasset/remote.rkt

Make sure you have your phone and PC on the same WiFi network, and they can ping each other. Run the NetHmd app (which reports the IP address on the home page for convenience). Open the script you want to work with in DrRacket and put the ip address in the last line, which looks something like this:
(remote "172.22.52.94" #f tic)

Hit ctrl-R or select the Racket->Run menu option to start execution. NetHMD on the phone should change from the home screen to the test scene. You can load and run other scripts, or restart the current one, without restarting NetHmd on the phone. You can move around with the joypad during development. Button Y / 4-dot resets the view to the home position.

I am still figuring out exactly what the “standard library” should look like, and I’m sure it will go through incompatible changes, but I would be happy to support anyone interested in doing bleeding-edge experimentation with this.

[Neil Toronto]

On 06/23/2015 08:32 PM, John Carmack wrote:
> Here is the overview of the work that I recently sent out internally.
>
> I started out with a pure-functional scripting interface, but it very rapidly became imperative, and it is in heavy flux. The sample scripts do capture the general "flavor" that I want this to have -- I think it will be easy enough to get interesting things happening that it will still tempt people who might be a little hostile to working in a Lisp.

Cool. Maybe keeping things more familiarly imperative will help reduce
grumbling at first.

What are the motivating factors besides sending commands to the hardware?

An imperative API makes functional abstraction harder. What are the main
selling points for Scheme/Racket now?

> [...]

>
> “Remote development”, where the script is actually executed in an IDE on a PC, communicating with NetHMD over a TCP connection. This allows sub-second code-change to VR change iteration cycles, and the use of a debugger. There might also be a use for a related mode where a central server directly drives one or more clients.

Remote development mode sounds very cool. (I didn't even have to squint
hard to see that.) Is the latency low enough?

> [...]

>
> The sample scripts are at:
>
> http://s3.amazonaws.com/o.oculuscdn.com/netasset/home.rkt
> http://s3.amazonaws.com/o.oculuscdn.com/netasset/office.rkt
> http://s3.amazonaws.com/o.oculuscdn.com/netasset/shader.rkt
> http://s3.amazonaws.com/o.oculuscdn.com/netasset/spin.rkt
> http://s3.amazonaws.com/o.oculuscdn.com/netasset/text-panels.rkt
> http://s3.amazonaws.com/o.oculuscdn.com/netasset/reversi.rkt
>

>[...]
>
> https://s3.amazonaws.com/o.oculuscdn.com/netasset/vr.rkt
> https://s3.amazonaws.com/o.oculuscdn.com/netasset/remote.rkt

Starting to look over these now. Some random comments follow.

Initial impression: 7 years after starting to program in Racket, it
still surprises me how easy it is to do something useful in just a few
lines of code.

Seems like the first thing anyone does when using the OpenGL API is
write something like `cmd-quad!`.

The `opengl` package defines all the OpenGL constants. I know you can't
use it directly, but cut-and-paste should give you everything you'd ever
need. Also, looking at the way it generates definitions from the OpenGL
spec could be useful.

In case the problems importing SRFI 1 were just finding the proper
incantation, it should be this:

(import (srfi :1))

If you're at all interested in a matrix/vector library whose operations
have < 4 ulps error (provided floating-point arithmetic is correctly
rounded) and transforming normals even when the affine matrix is
noninvertible, check out

https://github.com/ntoronto/pict3d/tree/master/pict3d/private/math

particularly the files "fl3.rkt", "fl4.rkt" and "flt3-unboxed-ops.rkt".
(Warnings: it's rather undocumented, there's a prevalent
`call/data-type-values` pattern that I haven't taken the time to get rid
of yet, and it relies on some (easily copied) math/flonum macros.)

This is really nifty. Thanks for sharing.

Neil ⊥

[John Carmack]

>An imperative API makes functional abstraction harder. What are the main selling points for Scheme/Racket now?

I am a big believer in functional programming (and static types) for large projects, but there is an undeniable bit of awkwardness compared to just imperatively poking things for small projects. That is one of the wins for Scheme -- I can make it super-easy to get easy things working, but it isn't just a "scripting language" unsuitable for large scale development. I am going to have to sort out my Racket / Chibi module strategy sometime soon, though.

As far as language choice goes, I don't claim to have broadly evaluated every possibility and chosen the optimal one.

Java or C# would have been more familiar to a broader base of game industry developers, but I really didn't want to drag in all the bulk of a JVM / .NET system, and a class focused world view seems less suited for the smaller scripting tasks.

Javascript would have been more familiar to a broader base of web industry developers, but I have basically no experience with javascript, and little desire to explore it (which is admittedly a fault of mine).

S-expression reading and writing is a strong factor for network communication, and I like the fact that there are available options for Scheme to interpret / compile / compile-to-C. I can see valid use cases for all of them, and I'm not sure how important each will be.

The bottom line is that I have been enjoying myself working with Racket / Scheme this year, and I have evidence that it has been objectively productive for me, so I'm going out on a bit of a limb and placing a bet on it.

>Initial impression: 7 years after starting to program in Racket, it still surprises me how easy it is to do something
>useful in just a few lines of code.

A big goal here is to make the authored code very clear and simple. I may yet get to a single #lang vr <ip address> declaration that gets rid of the boilerplate at the top and bottom of the file and flips the right switches to make Racket closer to R7RS.

There are sets of creative-types that are learning how to put things together in Unity to accomplish fairly simple media related tasks. I think I can make it easier for them with very domain specific helper libraries and a little bit of scheme.

>Seems like the first thing anyone does when using the OpenGL API is write something like `cmd-quad!`.

I am probably also going to make something similar to the old immediate mode APIs for building up static geometry meshes at init time. The design usage is to reference pre-processed models loaded from the net, but there will still be cases where you want to algorithmically build something like a camera-specific projection screen.

>In case the problems importing SRFI 1 were just finding the proper incantation, it should be this:
>
> (import (srfi :1))

I was having pair / mpair issues with the r6rs.

>If you're at all interested in a matrix/vector library whose operations have < 4 ulps error (provided floating-point arithmetic is correctly
>rounded) and transforming normals even when the affine matrix is noninvertible, check out
>
> https://github.com/ntoronto/pict3d/tree/master/pict3d/private/math
>
>particularly the files "fl3.rkt", "fl4.rkt" and "flt3-unboxed-ops.rkt".
>(Warnings: it's rather undocumented, there's a prevalent `call/data-type-values` pattern that I haven't taken the time to get rid of yet, and it >relies on some (easily copied) math/flonum macros.)

I'll take a look. I am mostly mimicking the math operations we have in our C++ codebase now, and I expect that I will make C versions of at least the expensive matrix multiply / matrix invert operations to accelerate the embedded Scheme code.

[AJ Campbell]

On Tuesday, June 23, 2015 at 4:36:12 PM UTC-7, John Carmack wrote:
> The intersection of Gear VR owners and Racket users may consist of just me at the moment, but if anyone else here is interested in discussing the work I am doing with driving VR by Racket/Scheme, I would welcome the input.  Would that be
> an appropriate topic on the list?
>
>  

I don't mind digging into Racket, but I would like to advocate for Javascript compatibility as early as possible. The devs building multi-server-multi-client VR apps are mostly on Node.js and/or Three.js. JSON data is super nimble. Objects sent from over the air are currently popping into our scenes in a very jarring fashion, but eventually we'll have new file types that re-package each poly with its surface data so that we can all figure out what progressive rendering should look like. (I'm guessing it will look something like people and scenery getting beamed up right before our eyes.)

JSON is probably going to be the go-to format to send/receive renderable 3D packets. The thought of doing it with XML makes me feel ill. I'm sure Racket can handle JSON data (it very well might already for all I know), but Javascript found its way across the whole stack (among other reasons) because we love the idea of having a universal language to eliminate the serialization between client and server, plus it knocks down communication barriers between front-end and back-end team members.

[Erik Post]

On Wednesday, 24 June 2015 01:36:12 UTC+2, John Carmack wrote:
> The intersection of Gear VR owners and Racket users may consist of just me at the moment, but if anyone else here is interested in discussing the work I am doing with driving VR by Racket/Scheme, I would welcome the input.  Would that be
> an appropriate topic on the list?

At the risk of being somewhat OT, I'd like to point out LambdaCube3D, a purely functional DSL that can be used instead of OpenGL API + GLSL.

http://lambdacube3d.com/editor.html
http://lambdacube3d.com/

The guys that created it gave a demo in Amsterdam two weeks ago. I was very impressed and thought it might be of interest here.

Cheers,
Erik

[Ray Alez]

On Tuesday, June 23, 2015 at 7:36:12 PM UTC-4, John Carmack wrote:
> The intersection of Gear VR owners and Racket users may consist of just me at the moment, but if anyone else here is interested in discussing the work I am doing with driving VR by Racket/Scheme, I would welcome the input.  Would that be
> an appropriate topic on the list?
>
>  

Oh my goodness this sounds absolutely amazing. I really-really hope that you can make it happen. Working on VR in lisp would be the most fun thing in the world ever.

[Gabriel Laddel]

*Sigh*

While I'm happy to see John Carmack using a Lisp, you're missing the point. If you offer Lisp as a "scripting language" you'll fail to show off why *exactly* it's better. S-expressions shine because they save you from the entirely mechanical task of parsing. There are emergent properties of this notational scheme that are also lovely in their own right e.g., macros & extensibility via incremental compilation, but all of this originates in the notation.

If you build up a scripting language in which you pass around shader strings and whatever random formats happen into your program, you're going to destroy the underlying "lispyness" because anyone who wants to make a change to the implementation will have to understand the format in its entirety, or be forced to guess at the semantics implied by so-and-so syntax - at which point, what has Lisp gotten me? Sure, you'll have macros for the "high level language", but they're not magic. The reason that e.g., the Symbolics Lisp machine is so fondly remembered was because of the "crystalline pyramid of comprehensible mutually-interlocking concepts" (http://www.loper-os.org/?p=42) which allowed a user not intimately familiar with the machine or its architecture to make meaningful changes.

"Lisping" without addressing the underling problems of OpenGL etc. is an absurd exercise. Even if this stunt were to make you personally a billon dollars, you'll end up unable to buy anything interesting because everyone's time is being spent ferrying around magic strings (where we are today).

[Matthias Felleisen]

On Jul 1, 2015, at 7:07 AM, AJ Campbell <a...@rocketsail.com> wrote:

> JSON is probably going to be the go-to format to send/receive renderable 3D packets. The thought of doing it with XML makes me feel ill. I'm sure Racket can handle JSON data (it very well might already for all I know),

It does of course.

> but Javascript found its way across the whole stack (among other reasons) because we love the idea of having a universal language to eliminate the serialization between client and server, plus it knocks down communication barriers between front-end and back-end team members.

That's actually a problem. When we launched Racket, Matthew insisted from the get-go that office partners do not communicate orally about things. Pop quiz: why was he right?


In my experience teaching courses where I give free choice of PLs for a distributed system, few languages make it easy for seniors to implement cross-process/network communication right. I always implement the systems by myself and let students vote to which format they want to stick: S-expressions (1958, but still better than what people invent now), XML, or JSON. For me, it's a two-line switch. For those on the losing end, it's a catastrophe. Pop quiz: why?

[John Carmack]

While I got tagged as a "technical idealist" for a long time, in reality I am deeply pragmatic. I fantasize about building a "crystalline pyramid of comprehensible mutually-interlocking concepts", but I know the folly of it. I have a not-so-large number of hours that I can possibly devote to this before it needs to stand on its own and provide real value to other developers, which means that it needs to be built on, and leverage, existing systems, warts and all.

S-expressions actually are one of the core wins from my use of lisp so far -- embracing read/write (and the associated bandwidth cost) as a wire protocol over yet another hand crafted binary format has been a significant win (however, the flexibility of the win seems to fight with static typing, perhaps at a fundamental level, in my limited typed-racket experience so far -- a good topic for discussion?).

Most users of this particular system will not interact with OpenGL at all, they will just move pre-built models around the world, play sound effects, start movie clips, etc. The fact that I could easily add the ability to hack on shader code, and soon geometry generation, is a huge pragmatic win.

I really do want to hear suggestions, do you have some concrete directions that you think would be useful?

I'm only three weeks into this project. Give me a little more time to change the world. :-)

-----Original Message-----
From: Gabriel Laddel [mailto:gabrielva...@gmail.com]
Sent: Wednesday, July 01, 2015 8:18 AM
To: racket...@googlegroups.com
Cc: John Carmack; us...@racket-lang.org
Subject: Re: [racket-users] Gear VR

*Sigh*

While I'm happy to see John Carmack using a Lisp, you're missing the point. If you offer Lisp as a "scripting language" you'll fail to show off why *exactly* it's better. S-expressions shine because they save you from the entirely mechanical task of parsing. There are emergent properties of this notational scheme that are also lovely in their own right e.g., macros & extensibility via incremental compilation, but all of this originates in the notation.

If you build up a scripting language in which you pass around shader strings and whatever random formats happen into your program, you're going to destroy the underlying "lispyness" because anyone who wants to make a change to the implementation will have to understand the format in its entirety, or be forced to guess at the semantics implied by so-and-so syntax - at which point, what has Lisp gotten me? Sure, you'll have macros for the "high level language", but they're not magic. The reason that e.g., the Symbolics Lisp machine is so fondly remembered was because of the "crystalline pyramid of comprehensible mutually-interlocking concepts" (https://urldefense.proofpoint.com/v1/url?u=http://www.loper-os.org/?p%3D42&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=Kjg6LltY9QjkipKooaVldA%3D%3D%0A&m=CwjU98OYOwj9cQ%2BK9b9GLWnR0Hnu8XBOyQG4zDiRCZw%3D%0A&s=5510a782e7801c2475be1390700d6531f0ffc308991efe43038561bb13d9c163) which allowed a user not intimately familiar with the machine or its architecture to make meaningful changes.

[Henri Tuhola]

keskiviikko 24. kesäkuuta 2015 2.36.12 UTC+3 John Carmack kirjoitti:
> The intersection of Gear VR owners and Racket users may consist of just me at the moment, but if anyone else here is interested in discussing the work I am doing with driving VR by Racket/Scheme, I would welcome the input.  Would that be
> an appropriate topic on the list?

I don't know why you stick to Racket/Scheme, but you might find libmarpa to be a game changer there as much as it was for me.

Here's something I dare to do because I've got grammar driven parsing: https://www.youtube.com/watch?v=3Sx-eVRwDmY&list=PLhFvCwO-WKvsRJglGrJ5_r2vPAs1E704D&index=5
You may note up few things:
* I don't have syntax for closures or setting a variable prior the session. I wrote the grammar rules and traversing functions matching to the semantics, slightly before I actually implemented them.
* The grammar is immediately used after the changes. It throws out errors that point out what's missing. I got the advantages of rich syntax, but I don't need hand written parsers or LALR to get it.

The parser that I've written in python is slow though. It takes a second on my computer to parse 1000 lines long file.

[Alexander McLin]

On Wednesday, July 1, 2015 at 7:07:03 AM UTC-4, AJ Campbell wrote:

> JSON is probably going to be the go-to format to send/receive renderable 3D packets. The thought of doing it with XML makes me feel ill. I'm sure Racket can handle JSON data (it very well might already for all I know), but Javascript found its way across the whole stack (among other reasons) because we love the idea of having a universal language to eliminate the serialization between client and server, plus it knocks down communication barriers between front-end and back-end team members.

I'm one of those who really wish Javascript hasn't spread across multiple stacks. I also question the effectiveness of having an universal language represented by JavaScript used for front-end and back-end. As a person who regularly juggles both modes in his daily job, the APIs and execution models in two cases are very different and result in mental context-switch costs regardless of whether they use the same language or not.

Personally I think team members would be better off communicating via design planning and documentation rather than relying on whether languages they use for respective project areas happen to coincide.

I also have serious questions about long-term viability of JavaScript now that WebAssembly has been announced. Once it's mature, I suspect JS will be abandoned or at least usage severely reduced with corresponding drop in support in other stacks.

The irony is WebAssembly is designed to be a compressed abstract syntax format! Essentially S-Expressions in a different disguise, so you can argue that Racket/Lisp is already well positioned to target WebAssembly directly for web apps.

I figure that Racket is probably better suited as a "universal language" since it's possible to implement languages in it to fit whatever demands diverse developers or problem domains have, including creating non-S-expression-oriented languages.

I do realize business considerations have an important impact on what tools are selected, and John Carmack needs to be able to reach a large audience of regular and technical users. I hope he will be able to leverage Racket or preferred Scheme variant to accomplish that.

[David Tin Nyo]

In case you are not aware of Arcadia, you should check out this project: http://arcadia-unity.tumblr.com/

It uses a (hacked) CLR fork of Clojure to connect to Unity. It might help to inform you what directions to take.

I can tell you connecting a REPL to a live Unity game is a godsend. Being able to do this with GearVR will be amazing. Now we gotta figure out how to edit / REPL without taking the hmd off...

I'm not associated with Arcadia, just a fan. Although, I have hooked up Chicken Scheme to OpenFrameworks with partial success.

[Neil Toronto]

On 07/01/2015 10:47 AM, John Carmack wrote:
>
> S-expressions actually are one of the core wins from my use of lisp so far -- embracing read/write (and the associated bandwidth cost) as a wire protocol over yet another hand crafted binary format has been a significant win (however, the flexibility of the win seems to fight with static typing, perhaps at a fundamental level, in my limited typed-racket experience so far -- a good topic for discussion?).

Definitely a good topic for discussion.

Where do you predict fights with static typing? Or, if it's just a
general feeling based on past experience with static typing, how do you
expect to use it?

There's every chance that Typed Racket's union types, singleton symbol
types, and occurrence typing will make it fairly easy to type an
s-expression-based wire protocol; e.g.

(define-type Command (List '<command-name> <arg-types> ...)
(define-type Commands (Pair 'begin (Listof Command)))
(define-predicate valid-commands? Commands)

At a rough count, I've written more than 100,000 lines of Typed Racket
code, much of which has had to be available to call in untyped Racket
without significant performance loss. I admit that Typed Racket is the
only Racket language that has made me want to throw things, but that's
increasingly rare, it's been much more pleasant than other type systems
I've used, and I'm pleased with the guarantees and increased performance
I've gotten from it.

Neil ⊥

[Tony Garnock-Jones]

On 07/01/2015 01:13 PM, Neil Toronto wrote:
> There's every chance that Typed Racket's union types, singleton symbol
> types, and occurrence typing will make it fairly easy to type an
> s-expression-based wire protocol; e.g.
>
> (define-type Command (List '<command-name> <arg-types> ...)
> (define-type Commands (Pair 'begin (Listof Command)))
> (define-predicate valid-commands? Commands)

This is a really neat feature of TR: it can automatically generate a
validator for a type you give it.

Most other languages require you to write parsers and validators by
hand, or using some tool not connected directly to the type system. In
TR, you get to use read for the parsing part, and the
automatically-generated validators to be sure the results inhabit the
type you need.

I only know of one other typed system that does something similar: Alice
ML [1], where pickled values coming off the wire are checked and dropped
into the required type. (If my recollection is correct.)

Tony

[1] https://www.ps.uni-saarland.de/alice/

[Dan Liebgold]

On Tuesday, June 23, 2015 at 4:36:12 PM UTC-7, John Carmack wrote:

> The intersection of Gear VR owners and Racket users may consist of just me at the moment, but if anyone else here is interested in discussing the work I am doing with driving VR by Racket/Scheme, I would welcome the input.  Would that be
> an appropriate topic on the list?
>
>  

Hey John -

I'm glad to see another game industry pro discover the value of Racket. We use it extensively here at Naughty Dog for tools and scripting (since the days of MzSchem back in 2005), and have great interest in the production aspects of the Racket ecosystem.

Dan

[Max Gonzih]

On Wednesday, June 24, 2015 at 1:36:12 AM UTC+2, John Carmack wrote:
> The intersection of Gear VR owners and Racket users may consist of just me at the moment, but if anyone else here is interested in discussing the work I am doing with driving VR by Racket/Scheme, I would welcome the input.  Would that be
> an appropriate topic on the list?
>
>  

This is super cool! Now I have reason to buy VR gear :) Good job, looking forward to see the future progress!

[Christian Schafmeister]

I've implemented a Common Lisp that interoperates with C++ and uses LLVM as the backend because I was motivated by the same shortcomings in traditional software development languages and tools (https://github.com/drmeister/clasp). I choose Common Lisp rather than Scheme because the killer feature of Lisp is Macros and they are easier to use in Common Lisp than they are in Scheme. Also, Slime (Emacs interface to Common Lisp) is one of the most powerful software development environments available IMHO.

[Max Gonzih]

For Scheme/Racket there are Geiser/Quack. They are not as good as SLIME,
but pretty close. In my opinion Scheme is much cleaner language. Yes,
macrosystem is different from one in common lips, but you just need to
adjust your habbits IMHO to make them work for you. Also Racket in
particular is full just bundle full of amazing things :)

[Gabriel Laddel]

> (however, the flexibility of the win seems to fight with static typing, perhaps at a fundamental level, in my limited typed-racket experience so far -- a good topic for discussion?).

Details please? Yes, this is a wonderful topic for discussion.

> Most users of this particular system will not interact with OpenGL at all, they will just move pre-built models around the world, play sound effects, start movie clips, etc. The fact that I could easily add the ability to hack on shader code, and soon geometry generation, is a huge pragmatic win.

A user need not see an s-expression to feel the difference between something that works and something that doesn't. Lisp makes developers happy because it saves them a *lot* of purely mechanical labor (parsing). This, in turn, makes business types (the good ones anyway) happy because of the tight feedback loop between their requirements and the implementation. The client/customer is happy because the organization in question is receptive to their needs. For example, at Amazon:

"The original brilliant guys and gals here only allowed two languages in Amazon's hallowed source repository: C and Lisp.

[...]

Shel, Eric, Greg, and others like them that I wasn't fortunate enough to work with directly: they didn't allow C++ here, and they didn't allow Perl. (Or Java, for that matter). They knew better.

Now C++, Java and Perl are all we write in. The elders have moved on to greener pastures too.

Shel wrote Mailman in C, and Customer Service wrapped it in Lisp. Emacs-Lisp. You don't know what Mailman is. Not unless you're a longtime Amazon employee, probably non-technical, and you've had to make our customers happy. Not indirectly, because some bullshit feature you wrote broke (because it was in C++) and pissed off our customers, so you had to go and fix it to restore happiness. No, I mean directly; i.e., you had to talk to them. Our lovely, illiterate, eloquent, well-meaning, hopeful, confused, helpful, angry, happy customers, the real ones, the ones buying stuff from us, our customers. Then you know Mailman.

Mailman was the Customer Service customer-email processing application for ... four, five years? A long time, anyway. It was written in Emacs. Everyone loved it.

People still love it. To this very day, I still have to listen to long stories from our non-technical folks about how much they miss Mailman. I'm not shitting you. Last Christmas I was at an Amazon party, some party I have no idea how I got invited to, filled with business people, all of them much prettier and more charming than me and the folks I work with here in the Furnace, the Boiler Room of Amazon. Four young women found out I was in Customer Service, cornered me, and talked for fifteen minutes about how much they missed Mailman and Emacs, and how Arizona (the JSP replacement we'd spent years developing) still just wasn't doing it for them.

It was truly surreal. I think they may have spiked the eggnog.

-- Steve Yegge (https://sites.google.com/site/steveyegge2/tour-de-babel)"

"Abstractions push up" (http://log.bitcoin-assets.com//?date=10-03-2015#1047626) and a small investment in fixing some of the issues with OpenGL will pay (large) dividends in the years ahead.

> I really do want to hear suggestions, do you have some concrete directions that you think would be useful?

Yes.

At some point you'll have to address the hostility to Lisp. It comes in three forms:

1. Jaded experts. They're doing all the hard labor (C++) and really are the ones who pull the organization forwards. They've invested a lot of time and effort into becoming good. If a change is going to be made it had better be a sure thing and radically simplify their lives.

2. Bureaucrats. Some of these may actually understand Lisp, but are more than happy to have an easy job in which they can get by doing purely repetitive labor. A meatspace acquaintance of mine, and a prototypical bureaucrat, got his PHD by translating things that worked in Lisp into other languages, creating problems in the process, requiring he write papers on parsing logic...

3. Populists. "Javscript has momentum, hence all of our efforts should be spent there".

You want to convince 1s. 2s and 3s don't have the prerequisite mental machinery to hold convictions and are thus irrelevant to this discussion. If the "real" devs start using Lisp in their day-to-day at Oculus (Facebook? Sony?) killing Lisp becomes tantamount to killing the organization itself. I don't know anything about the politics involved, but the following scheme should suffice:

Choose a stack for the initial "win" and DO NOT DEVIATE FROM IT. Select specific versions of android, linux, OpenGL, racket, editor + required Unity/C++/etc you will support internally. When your #1s decide that they'd like to try out "that Lisp stuff" they'll be able to do so without any hiccups. Have a Q&A person run through the process a few times just to make sure it all works swimmingly. You could even have them setup some end-to-end systems in a public space near the #1s, with comfortable chairs and *slightly* more space than normal devs get. Someone will play with it eventually, and who knows, maybe they'll do something neat.

Make a point to sanitize the development experience. Most of the problems, the "unlispy" stuff, originates in OpenGL. This doesn't matter for those who've already invested the time into learning shaders, the quirks, the sequences of commands that crash the system etc. but for those of us who just want to draw something on the screen it's a serious(ly painful) time investment that shouldn't be necessary. Even if your position is that the "user won't see it" you've still got to hire people to hack OpenGL, and they'll appreciate it.

First, fork the OpenGL documentation. They've had 22 years to do things correctly and have utterly and conclusively failed. Since you've got a single version of OpenGL you're supporting for the time being, it is eminently possible to make the spec / docs (http://docs.gl/) machine readable (i.e., sexprs) for tooling. With this in place you can (programmatically) ask questions such as:

- Which OpenGL constructs do not have a corresponding racket construct?
- Which OpenGL constructs do not have docstrings?
- Which OpenGL consturcts are missing type signatures or return types?

This can then be used to annotate the existing OpenGL racket library (walk the program, find the CFFI stub corresponding to each construct and spit out a new stub with its docstring). It is impossible to play with an idea when you've got to bounce between the editor, browser, pdf documentation & shell to express something simple as "draw a colored sphere".

You're currently passing around GLSL strings for shaders. Cbaggers (https://github.com/cbaggers) and |3b| (http://3bb.cc/) have independently developed sexpr to GLSL compilers (video: https://www.youtube.com/watch?v=DS_cwQJc8HY). They're working in CL, but the code they've got so far could be translate to scheme in ~a day's work. They've shown themselves to be motivated, capable and to have good taste. I suggest you hire both and stick them into a room with a signing bonus, dead-tree copies of the ADA reference manual, CLtL2 and the instructions: "combine your shaders systems into a sane racket library. If the project is a success you can each have an intern for the next project - removing the GLSL IR". Cbaggers has already gone through the trouble of generating a machine-readable GLSL spec (http://techsnuffle.com/2015/06/22/machine-readable-glsl-spec/). Sanity is two hires away.

So far I've seen two people go through the mental change from "programming is the process of constructing an insanely complicated machine, munging text and making wild guesses" to grokking Lisp ("all the parsing I've done over the years was fundamentally braindamaged, incremental compilation is the only way to program"). The implications of not parsing take a while to sink in, especially when the person in question has to bounce between two or more languages in their day-to-day.

With this in mind, I wouldn't push the "functional" angle as it will only serve to distract. You've seen for yourself that programs intended to be "purely functional" end up having to deal with the underlying machine eventually. "Lisp" is an ill-defined term anyways and is indifferent to functional/imperative programming. My experience suggests when people ask for "functional", "declarative" programming or "DSLs" what they're really after is a comprehensible design, which Lisp delivers in spades.

> Doing VR GUI development in the native apps is unpleasant – you can gut it out in C++ with our current interfaces, but the iteration times and code structure don’t make it a lot of fun.

> In-editor prototyping is nice in Unity, but actual on-device testing cycle times are terrible, and you wind up with all the bulk of Unity and poor access to the native VR features like TimeWarp layers.

> There are sets of creative-types that are learning how to put things together in Unity to accomplish fairly simple media related tasks. I think I can make it easier for them with very domain specific helper libraries and a little bit of scheme.

All of the problems with the current paradigm originate in not having incremental compilation, which is due to the lack of s-expressions. I've used the oculus v1. It was lovely, but the software toolchain was shit. We already know what the end game of VR computing ("the general purpose VR scripting environment") is: https://www.youtube.com/watch?v=nbY-meOL57I

> With app signing, we have no way for developers to casually share work with each other or limited communities.

This sounds like a political problem rather than a technical one.

BTW: Since a javascript interface will need to be developed at some point, I recommend you check out https://github.com/marijnh/parse-js which will allow you to deal with S-expressions rather than JS for all of your tooling. The FORTRAN/C/ALGOL tooling for Symbolics Lisp machines rocked because they had a clue about compilers.

[Neil Toronto]

On 07/02/2015 08:49 AM, Gabriel Laddel wrote:
>
> [classification of Lisp hostiles]

>
> Choose a stack for the initial "win" and DO NOT DEVIATE FROM IT. Select specific versions of android, linux, OpenGL, racket, editor + required Unity/C++/etc you will support internally. When your #1s decide that they'd like to try out "that Lisp stuff" they'll be able to do so without any hiccups. Have a Q&A person run through the process a few times just to make sure it all works swimmingly. You could even have them setup some end-to-end systems in a public space near the #1s, with comfortable chairs and *slightly* more space than normal devs get. Someone will play with it eventually, and who knows, maybe they'll do something neat.
>

> Make a point to sanitize the development experience...

Regardless of whether one agrees with your classifications, this seems
like a good idea, at least eventually.

> First, fork the OpenGL documentation. They've had 22 years to do things correctly and have utterly and conclusively failed. Since you've got a single version of OpenGL you're supporting for the time being, it is eminently possible to make the spec / docs (http://docs.gl/) machine readable (i.e., sexprs) for tooling. With this in place you can (programmatically) ask questions such as:
>
> - Which OpenGL constructs do not have a corresponding racket construct?
> - Which OpenGL constructs do not have docstrings?
> - Which OpenGL consturcts are missing type signatures or return types?
>
> This can then be used to annotate the existing OpenGL racket library (walk the program, find the CFFI stub corresponding to each construct and spit out a new stub with its docstring). It is impossible to play with an idea when you've got to bounce between the editor, browser, pdf documentation & shell to express something simple as "draw a colored sphere".

It's probably a little more complicated than that because commands are
sent to an Android device running Chibi Scheme.

Related: There's a Racket "opengl" package (indexed by the main package
server; GitHub project is at https://github.com/stephanh42/RacketGL)
that generates Racket FFI bindings from a machine-readable OpenGL spec.
For Pict3D, I created a typed fork that I'll eventually split off into a
separate "typed-opengl" package.

> You're currently passing around GLSL strings for shaders. Cbaggers (https://github.com/cbaggers) and |3b| (http://3bb.cc/) have independently developed sexpr to GLSL compilers (video: https://www.youtube.com/watch?v=DS_cwQJc8HY).

I've been considering doing something similar, so thanks for the pointers!

Neil ⊥

[Matthias Felleisen]

Let me add one thought here. One of the thoughts I have played with in this context is to have a dissertation on XML-style type extensions for Typed Racket (CDuce, XDuce). I am not interested in XML (or JSON) per se but in type systems that can say more about S-expression-like data. Of course, as in the past, this would have to be balanced between theoretical expressive power and practical expressiveness/impact. (We tend to err in the direction of the former, because we finance Racket via research grants.)

In a way, TR's union-type system ought to make an integration of these ideas simpler. You could then successively refine types of S-expressions and verify more claims about them if this played a role in your program.

But as Neil and Tony said, the type system gets in your way with such things, but no longer enough to be annoying.

-- Matthias

[Konrad Hinsen]

On 02/07/2015 19:50, Matthias Felleisen wrote:

> Let me add one thought here. One of the thoughts I have played with in this context is to have a dissertation on XML-style type extensions for Typed Racket (CDuce, XDuce). I am not interested in XML (or JSON) per se but in type systems that can say more about S-expression-like data. Of course, as in the past, this would have to be balanced between theoretical expressive power and practical expressiveness/impact. (We tend to err in the direction of the former, because we finance Racket via research grants.)

That reminds me of an approach that is more than 20 years old:

http://www.sciencedirect.com/science/article/pii/030439759290302V

That kind of type system should fit many kinds of s-expressions nicely.

Konrad.

[jaden horst]

Hey John it looks like you're busy with Minecraft. I am interested in bleeding edge. I have no complaints; I just want to get started. A lot of people in Oculus Social want to play cards.

Thanks,

Jaden...@gmail.com