Bret Victor: Seeing Spaces

[Sean McDirmid]

https://vimeo.com/97903574

 

I’m still honestly inspired, but not as impressed as with his previous work. For one thing, this work resembles much of the output I see most designers produce (amazing what one can do with AfterEffects!). A few related projects:

 

Kato’s Phybots provides an environment for robot programming that relies on a ceiling camera. I’m sure the resemblance is a coincidence.

 

Microsoft has been playing around with war rooms for programming projects. Basically, it’s a conference room with a bunch of projectors set up; the team works closely together using the screens for communicating updates and doing reviews.

 

 

[Jonathan Edwards]

This is verging on TED-quality inspirations.

I actually disagree with his claim that data leads to science. It is the other way around. Science is about questioning your understanding, as embodied in the data you already know to observe, and inventing new instruments that produce new data to test new understandings. What he describes is all engineering.

--
You received this message because you are subscribed to the Google Groups "Augmented Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to augmented-progra...@googlegroups.com.
To post to this group, send email to augmented-...@googlegroups.com.
Visit this group at http://groups.google.com/group/augmented-programming.
For more options, visit https://groups.google.com/d/optout.

[Sean McDirmid]

TED…I thought those were just infomercials?

 

Recording everything and then mining it a posteriori is Google’s style. I like the idea of a data rich space, but data visualization is actually hard and not so easy to conjure up…what he is proposing is CSI for makers, with the same fictional value as the TV show itself J.

[David Barbour]

Very nice. I'd love to tackle that sort of 'seeing room' by leveraging augmented reality.

[Dmitri Shuralyov]

One short term goal that I get out of this (incredibly inspiring, as always) short talk is:

As of the last few years, we have git/other VCS that enable us to "see" into the history of code (think branches/older commits/blame/commit messages). Perhaps 10 years ago that was not the case.

However, today, we have nothing in wide spread use for seeing history of program executions and the output.

For example, if I run my program and it produces some output, then I make some changes to code, run it again, it's pretty much "manual labour" to do a diff between the two outputs. Or see the diff between outputs from yesterday and today. Or a month ago on a specific branch, and today. Or 3 runs ago and 5 runs ago.

That's something I want to try to tackle "soon"...

[J A Burrows]

I just got around to reading this thread. Isn't this exactly what Tony Stark has for making Iron Man? ;-)

On a more serious note, this is akin to having a very modern, augmented reality notebook. The notebook requires the user to hold all the discipline for making observations and using it for either tinkering, engineering, or science. This requires a similar amount of user discipline as to when they are using it for engineering or science. User error will always stymie technological advancement.

Oh, I guess I haven't introduced myself to the list yet. I'm Jill Burrows. I'm just a senior software engineer and college dropout with many varied interests -- including programming languages. When I blog (rarely), it's at http://jburrows.wordpress.com. When the circus that is currently my life turns into something more organized, I hope to blog more frequently and generally be more active in the community.

--

J A Burrows

Programmer / Technologist / Consultant

+1 503 208 5455 | ji...@adaburrows.com

http://adaburrows.com | @jburrows

[Sean McDirmid]

I didn’t get this out of Bret’s talk, but I agree it is a good feature to have. I’ve been thinking about how to add branching time to Glitch, would allow for multiple “timelines” to exist at one time, possibly with different code. Then it is just a matter of visualizing the differences between different points in time and/or different branches at the same time (and of course, this “just” is an incredibly complicated UX problem).

 

From: augmented-...@googlegroups.com [mailto:augmented-...@googlegroups.com] On Behalf Of Dmitri Shuralyov
Sent: Sunday, July 6, 2014 7:18 AM
To: augmented-...@googlegroups.com
Subject: Re: Bret Victor: Seeing Spaces

 

One short term goal that I get out of this (incredibly inspiring, as always) short talk is:

--

[Josh Marinacci]

I’m working on a new IDE for Arduino and I think it would be a good place to try out some of these ideas.

First a bit of background.  Arduino is an open source system of embedded micro controllers. If you want to blink some LEDs on a dress or control servo motors, Arduino is what you’ll probably use. It is not only a series of micro controller boards but also an IDE, a compiler toolchain, and tons of libraries for reading/controlling almost anything you can think of. It was designed for artists and educators, not programmers, so everything is as integrated as possible. This isn’t to say that you don’t program. You are still writing code, but you aren’t wrestling with obscure linker errors, installing toolchains, or working in assembly. And of course, being open source Arduino is extremely hackable and there are many interesting derivative products.

Unfortunately, the Arduino IDE is horrible. It’s an old Java Swing program based on an even older program built by people who don’t write software for a living. It has horrible fonts, no syntax highlighting, broken keybindings, an confusing menu system, and countless other usability problems. To solve this I started writing a new, from scratch, IDE called Electron  (you can check it out at www.electron.pw).  Now that the basics are working I want to building features that allow you to properly reason about what your project is doing, which brings us back to to Bret Victor’s talk.

Creating a tool which lets you visually reason about the entire state of your code, and it’s complete history, is a very tall order. I’m not sure we know how to do that yet.  However, Arduino sketches (as Arduino programs are called) are rather limited in scope. In addition to the limited resources available (< 16Mhz 8bit CPU, < 32KB storage, <2KB RAM) the typical sketch is usually only concerned with a few values: the sensors you are trying to read and the actuators you want to control. Even something as complex as a robot car probably reduces to less than 20 values of interest.  This limited problem size makes Arduino a great test case for tools that reason about program state.

So... what are some ways we could visualize what’s going on inside of an Arduino project?  My first thought is monitoring the pins. A common project is to read the values of a few sensors attached to some CPU pins. It would be easy to have a diagram of the CPU board listing all of the pins. The user could click on the pin to select it for monitoring. This gives you a continuous readout of that pin’s value over time in a zoomable chart. This data can be stored on disk in the same RCS as the code. 

There’s lots of details to be worked out of course, but the scope seems tractable. Given that I’m writing a new IDE from scratch anyway, it gives us a playground for new ideas.

Whaddyathink?

- Josh

[Ian Johnson]

Im really excited to hear about a new IDE for arduino, as a software developer toying with arduino it sounds like you are adressing a lot of my complaints :)

I think the key to implementing Bret Victors ideas curently is to do as you say and choose things that are limited in scope or domain specific. Solvong for manageable cases should give us insight to generalization.

Since our laptops are multiple orders of magnitude more powerful than an arduino, could we just send up all of the sketch's state so we could visualize the value of any variable over time? of course calling out the natural inputs from the board make for a great intuitive start, i think seeing what your calculating from those inputs would be great too!

ill check out the IDE! thanks for sharing.

[Sandy Klausner]

For a glimpse of "seeing spaces" at Internet of Things scale, I suggest that you review a Reference Architecture recently proposed to the Industrial Internet Consortium.

[Sean McDirmid]

I’m working on a new IDE also, but am doing that concurrently with a Python-like language design. It’s completely live, based on a concept known as managed time. I’m innovating at the presentation aspect, though I think many of my choices will be controversial; e.g.

 

 

The editor is free form, not structural, the embellishments come from a smart way of presenting code (it will also inline syntax, type, and even execution errors!). The managed time concept provides for a “steady frame” (see Hancock’s dissertation) that make visualizing state and state changes much easier. The problem with a pure imperative language is that, even if you can record everything, you have no steady frame from which to reason about state changes; “computer time” is entangled with “real time” and transient but benign inconsistencies make the reasoning very noisy.

 

Jun Kato has done a lot of work related to robotics IDEs; see http://junkato.jp/phybots/

 

 

From: augmented-...@googlegroups.com [mailto:augmented-...@googlegroups.com] On Behalf Of Josh Marinacci
Sent: Monday, July 7, 2014 1:11 AM
To: augmented-...@googlegroups.com
Subject: Re: Bret Victor: Seeing Spaces

 

I’m working on a new IDE for Arduino and I think it would be a good place to try out some of these ideas.

[Josh Marinacci]

It turns out there is a protocol called Firmata, which has Arduino support, which is designed just exactly this idea: forwarding the values of pins and state variables to the host computer. Doing something like this seems a lot more tractable than trying to simulate the entire Arduino in software. My plan is to build initial Firmata support after Electron 1.0 is released.