> (however, the flexibility of the win seems to fight with static typing, perhaps at a fundamental level, in my limited typed-racket experience so far -- a good topic for discussion?).
Details please? Yes, this is a wonderful topic for discussion.
> Most users of this particular system will not interact with OpenGL at all, they will just move pre-built models around the world, play sound effects, start movie clips, etc. The fact that I could easily add the ability to hack on shader code, and soon geometry generation, is a huge pragmatic win.
A user need not see an s-expression to feel the difference between something that works and something that doesn't. Lisp makes developers happy because it saves them a *lot* of purely mechanical labor (parsing). This, in turn, makes business types (the good ones anyway) happy because of the tight feedback loop between their requirements and the implementation. The client/customer is happy because the organization in question is receptive to their needs. For example, at Amazon:
"The original brilliant guys and gals here only allowed two languages in Amazon's hallowed source repository: C and Lisp.
[...]
Shel, Eric, Greg, and others like them that I wasn't fortunate enough to work with directly: they didn't allow C++ here, and they didn't allow Perl. (Or Java, for that matter). They knew better.
Now C++, Java and Perl are all we write in. The elders have moved on to greener pastures too.
Shel wrote Mailman in C, and Customer Service wrapped it in Lisp. Emacs-Lisp. You don't know what Mailman is. Not unless you're a longtime Amazon employee, probably non-technical, and you've had to make our customers happy. Not indirectly, because some bullshit feature you wrote broke (because it was in C++) and pissed off our customers, so you had to go and fix it to restore happiness. No, I mean directly; i.e., you had to talk to them. Our lovely, illiterate, eloquent, well-meaning, hopeful, confused, helpful, angry, happy customers, the real ones, the ones buying stuff from us, our customers. Then you know Mailman.
Mailman was the Customer Service customer-email processing application for ... four, five years? A long time, anyway. It was written in Emacs. Everyone loved it.
People still love it. To this very day, I still have to listen to long stories from our non-technical folks about how much they miss Mailman. I'm not shitting you. Last Christmas I was at an Amazon party, some party I have no idea how I got invited to, filled with business people, all of them much prettier and more charming than me and the folks I work with here in the Furnace, the Boiler Room of Amazon. Four young women found out I was in Customer Service, cornered me, and talked for fifteen minutes about how much they missed Mailman and Emacs, and how Arizona (the JSP replacement we'd spent years developing) still just wasn't doing it for them.
It was truly surreal. I think they may have spiked the eggnog.
-- Steve Yegge (
https://sites.google.com/site/steveyegge2/tour-de-babel)"
"Abstractions push up" (
http://log.bitcoin-assets.com//?date=10-03-2015#1047626) and a small investment in fixing some of the issues with OpenGL will pay (large) dividends in the years ahead.
> I really do want to hear suggestions, do you have some concrete directions that you think would be useful?
Yes.
At some point you'll have to address the hostility to Lisp. It comes in three forms:
1. Jaded experts. They're doing all the hard labor (C++) and really are the ones who pull the organization forwards. They've invested a lot of time and effort into becoming good. If a change is going to be made it had better be a sure thing and radically simplify their lives.
2. Bureaucrats. Some of these may actually understand Lisp, but are more than happy to have an easy job in which they can get by doing purely repetitive labor. A meatspace acquaintance of mine, and a prototypical bureaucrat, got his PHD by translating things that worked in Lisp into other languages, creating problems in the process, requiring he write papers on parsing logic...
3. Populists. "Javscript has momentum, hence all of our efforts should be spent there".
You want to convince 1s. 2s and 3s don't have the prerequisite mental machinery to hold convictions and are thus irrelevant to this discussion. If the "real" devs start using Lisp in their day-to-day at Oculus (Facebook? Sony?) killing Lisp becomes tantamount to killing the organization itself. I don't know anything about the politics involved, but the following scheme should suffice:
Choose a stack for the initial "win" and DO NOT DEVIATE FROM IT. Select specific versions of android, linux, OpenGL, racket, editor + required Unity/C++/etc you will support internally. When your #1s decide that they'd like to try out "that Lisp stuff" they'll be able to do so without any hiccups. Have a Q&A person run through the process a few times just to make sure it all works swimmingly. You could even have them setup some end-to-end systems in a public space near the #1s, with comfortable chairs and *slightly* more space than normal devs get. Someone will play with it eventually, and who knows, maybe they'll do something neat.
Make a point to sanitize the development experience. Most of the problems, the "unlispy" stuff, originates in OpenGL. This doesn't matter for those who've already invested the time into learning shaders, the quirks, the sequences of commands that crash the system etc. but for those of us who just want to draw something on the screen it's a serious(ly painful) time investment that shouldn't be necessary. Even if your position is that the "user won't see it" you've still got to hire people to hack OpenGL, and they'll appreciate it.
First, fork the OpenGL documentation. They've had 22 years to do things correctly and have utterly and conclusively failed. Since you've got a single version of OpenGL you're supporting for the time being, it is eminently possible to make the spec / docs (
http://docs.gl/) machine readable (i.e., sexprs) for tooling. With this in place you can (programmatically) ask questions such as:
- Which OpenGL constructs do not have a corresponding racket construct?
- Which OpenGL constructs do not have docstrings?
- Which OpenGL consturcts are missing type signatures or return types?
This can then be used to annotate the existing OpenGL racket library (walk the program, find the CFFI stub corresponding to each construct and spit out a new stub with its docstring). It is impossible to play with an idea when you've got to bounce between the editor, browser, pdf documentation & shell to express something simple as "draw a colored sphere".
You're currently passing around GLSL strings for shaders. Cbaggers (
https://github.com/cbaggers) and |3b| (
http://3bb.cc/) have independently developed sexpr to GLSL compilers (video:
https://www.youtube.com/watch?v=DS_cwQJc8HY). They're working in CL, but the code they've got so far could be translate to scheme in ~a day's work. They've shown themselves to be motivated, capable and to have good taste. I suggest you hire both and stick them into a room with a signing bonus, dead-tree copies of the ADA reference manual, CLtL2 and the instructions: "combine your shaders systems into a sane racket library. If the project is a success you can each have an intern for the next project - removing the GLSL IR". Cbaggers has already gone through the trouble of generating a machine-readable GLSL spec (
http://techsnuffle.com/2015/06/22/machine-readable-glsl-spec/). Sanity is two hires away.
So far I've seen two people go through the mental change from "programming is the process of constructing an insanely complicated machine, munging text and making wild guesses" to grokking Lisp ("all the parsing I've done over the years was fundamentally braindamaged, incremental compilation is the only way to program"). The implications of not parsing take a while to sink in, especially when the person in question has to bounce between two or more languages in their day-to-day.
With this in mind, I wouldn't push the "functional" angle as it will only serve to distract. You've seen for yourself that programs intended to be "purely functional" end up having to deal with the underlying machine eventually. "Lisp" is an ill-defined term anyways and is indifferent to functional/imperative programming. My experience suggests when people ask for "functional", "declarative" programming or "DSLs" what they're really after is a comprehensible design, which Lisp delivers in spades.
> Doing VR GUI development in the native apps is unpleasant – you can gut it out in C++ with our current interfaces, but the iteration times and code structure don’t make it a lot of fun.
> In-editor prototyping is nice in Unity, but actual on-device testing cycle times are terrible, and you wind up with all the bulk of Unity and poor access to the native VR features like TimeWarp layers.
> There are sets of creative-types that are learning how to put things together in Unity to accomplish fairly simple media related tasks. I think I can make it easier for them with very domain specific helper libraries and a little bit of scheme.
All of the problems with the current paradigm originate in not having incremental compilation, which is due to the lack of s-expressions. I've used the oculus v1. It was lovely, but the software toolchain was shit. We already know what the end game of VR computing ("the general purpose VR scripting environment") is:
https://www.youtube.com/watch?v=nbY-meOL57I
> With app signing, we have no way for developers to casually share work with each other or limited communities.
This sounds like a political problem rather than a technical one.
BTW: Since a javascript interface will need to be developed at some point, I recommend you check out
https://github.com/marijnh/parse-js which will allow you to deal with S-expressions rather than JS for all of your tooling. The FORTRAN/C/ALGOL tooling for Symbolics Lisp machines rocked because they had a clue about compilers.