Conference report: FITC Amsterdam 2015

[Chris Mills]

See below for what I’ve bene up to in the last three days. There is a lot here, but I’ve summarised the most interesting parts up the top so you can skip all the raw notes at the bottom.

------------------------------------------
FITC conference, February 23-24, Amsterdam
------------------------------------------

I went to speak at the FITC conference in Amsterdam (http://fitc.ca/event/am15/); this used to be a Flash conference, but these days deals more in a mix of inspirational design talks, and really interesting code art/experimentation (for example, VR, WebGL, Glitch music/art.)

The Parkhaus Zweiger venue in Amsterdam is a really cool creative technology hub, so really the perfect venue for this. The conference was introduced with a great opening title sequence, as always.

MY TALK
-------

I presented my "Web versus Native: Round 1?" talk again, covering a bunch of cool device API and performance optimization stuff we've got going on at the moment, as well as saying a few words about Firefox OS, our dev tools, and more. It seemed to go down well; I think people were quite happy to have a more practical webby talk in the middle of all the arty cool weird stuff ;-)

People came up afterwards with praise and some good questions. One was about when this stuff will be standardised, and it was a shame that I wasn't able to give them concerete answers. The other was about polyfills - I got a good offer for some polyfills to link to on MDN.

I got surprisingly little tweets. Usually you get some, good or bad ;-)

OTHER TALKS
-----------

I'll not go through all details of every talk I saw immediately, as that would be too much information for everyone. I've included my full notes at the bottom, but immediately below you can find summary information of some of the more interesting and useful talks I saw. You can check out my full notes for more detail, in each case.

Signal, noise & machine
So Kanno---------------
--------

So Kanno is a Japanese artist/technologist exploring cool and interesting ways to make music and visual art out of glitches and noise (as in, signal/noise.) He is focused on what can be done with unintended signal, as well as the intended signal that most of us focus on. He showed us some interesting creations, such as devices that pick out waves from electronic devices and make music from them, and graffiti paint bots.

Flash Professional CC: The Future of Animation
Ajay Shukla-----------------------------------
-----------

I mainly went to this talk because I was interested to see how Flash might be relevant in 2015. Adobe have done a good job of keeping it relevant, by making it output to Canvas/WebGL as well as SWF.

I talked to Ajay about asm.js, and the possibility of getting Flash to output it. He liked the idea, as performance is one of their main concerns. I will follow up with him.

Adventures in VR
Carlos Ulloa----
------------

Carlos is an old friend of mine from the Flash community, and I'd not seen him for about 5 years, so it was really cool to catch up with him. He has been doing a lot of WebGL work, and more recently VR, with Oculus Rift. His talk was a well-paced introduction to VR development, and the things you have to be concerned about.

He is really interested in Mozilla's WebVR work, and would be a really good person to get involved with testing, demo writing, etc. (WebGL stuff too).

The Joy of order
Mario Klingemann
----------------

Mario is a well-known German code artist (see https://twitter.com/Lowpolybot for a great example), and his recent work has been concerned with ordering/categorizing a large series of images scanned from books in the British Library (flickr.com/photos/britishlibrary). His talk went through the process, tools and algorithms he used to order all these images into groups of similar type.

Things imperfect
Brendan Dawes---
-------------

Brendan is a legendary British interactive designer/developer guy, and his talks are always really funny and interesting. After the obligatory geeking out about paperclips and pencils, he talked a bit about some of his projects (digital and physical), and how he likes things to be imperfect because it makes them more human, more believable, more emotional.

My favourite parts were about his 3D printing work. He has printed out visualisations of different songs, and he also did an installation for a conference, whereby he used people's twitter data to create little swimming fish-like creatures. Their size was based on how many followers the person has. Their eye size was based on how many people they follow relative to their number of followers, and their colours were based on number of tweets, content, etc. He rendered these models as 3D (WebGL, or perhaps Processing) models and then also printed some of them out as physical toys.











RAW NOTES
---------

Signal, noise & machine
So Kanno---------------
--------

music, glitch and other stuff
ovalprocess
compose music by drawing a pattern in black and white
or by allowing a program to scan people
jamming gear - making music with gears with different numbers of teeth to change pitch of piano
technophone converts electrmagnetic waves into sound and music
xbox kinect scanning
3d printer making art out of noise rather than signal (noise = undesired? not all the time)
drawing bot that draws the path of a double pendulum with sn LED on the end
Kiduit - graffiti with fire extinguisher
They made a robot that does the same thing
installation in subway station automatically draws with pens on a wall but takes input from the noise of the crowds passing by
glitch art, speculative design, digital fabrication
fabric printers - design of clothes in future done with machines
Autodesk 123D catch
take 3D copy of clothes, print it out on fabric, then make exact copy of a garment a perdson is wearing
model justin bieber ;-)
google tango, 3d manipulation


Next: How to jump off a cliff (and live to tell the tale), Bjorn Kummeneje-------------------------------------------------------
---------

Innovative advertising projects. His company is perfect fools. Sharing experiences
**Converse - canvas experimental, allowing expressiveness on a canvas made up of 480 converse shoes.
risk everything - don't be afraid to fail. This is easy to say when you succeed ;-)
always have a backup plan
find good people to work with, especially to bolster the skills you don't have.
**Remote controlled cafe, located in helsinki, that could be controlled via the web
design is important. They decided to show what bad design is.
solve problems, even if they're not yours
using existing technologies
**Swisscom
game - win the Galaxy S4 if you can stare at it for 60 minutes.
Loads of distractions, hotdogs, bikers, tourists, remote controlled cars, other stunts.
In a public place, central station Zurich
be prepared for last minute changes
go the extra mile - not sure I agree with his idea of pulling all-nighters to finish things. Work-life is important
**Casumo - boredom ride
Erase boredom
Make taxi rides to not be boring
give them the part of the journey they were bored back in money off the bill
cameras to analyse their face during the journey and work out how long they were bored.
** SONOS - sounds of NYC
Speaker company, big noble
Showcase music of NYC
A screen again, but this time using speakers as pixels, not trainers!
speakers that light up and move, and make sounds
** Last project - to do with merger between hasan and partners and perfect fools
during radio interview, electrocute the bosses if you think they are talking bullshit
via a web-based interface


Flash Professional CC: The Future of Animation
Ajay Shukla-----------------------------------
-----------

Like WHOA, Flash in 2015??
Looks like it is outputting animations to HTML5/CSS3
Flash handles cross platform/device output
This includes Android Lollipop and iOS 64bit output
Flash + CreateJS (Grant Skinner's tool) to provide canvas output
WebGL available as well
So, hrm, exciting stuff
Not sure I agree with his assertion of "You shouldn't care about the underlying APIs". Also it seems to assume that you want to create black box stuff, like Flash. But I guess this is what canvas is as well.
****My major question - how is the support added to iOS? I think it is done via Adobe AIR (HTML5 output?)
Also talked about Plugins allowing you to output to Unity and other platforms/environments, GAF (Generic Animation Format)
****speed of canvas was brought up. what about asm.js/WebGL output? NO. He did n't know about ASM. Will mail him about it.

Edge animate outputs CSS3/DOM animations
Flash outputs Canvas/WebGL
ajsh...@adobe.com


Colour in the gray areas
Frank Reitburger--------
----------------

Creative coding, WebGL
Cool 3D videos, animations. Shader programming
He also loves the demoscene, and contributes to that when he can
game developer
WebGL for fractals, mandlebrot sets, etc.
Math is a huge inspiration - take existing math and apply it to computer graphics



Hands-on digital immersion
Romain Altain Aldea-------
-------------------

sortieenmer.com - a website to teach you about the dangers of drowning. Immersive experience. It is pretty impressive, and a simple idea. They did it for free, as it was a cool project. The original partner company said no to the idea, as the message was too negative (everybody dies if they don't have a lifejacket.)

The agency was persistent, so found another idea of a way to publish the web site.

Quite a lot of stuff about the interactions and how they were worked out, but overall it was a bit "look at me, I'm impressive".



Adventures in VR
Carlos Ulloa----
------------

Carlos is an old friend of mine - I knew him as a developer of Papervision 3D, but now he does some very interesting things with WebGL and VR/Oculus Rift. And this is what this talk was largely about. He is also very interested in the WebVR stuff Mozilla are doing.

Virtual reality = being in a different place/envinronment
You have to hack your senses to achieve this
Presence is when this is achieved successfully
It is horrible when you don't achieve this. You feel sick!

HMD - head mounted display (OR?) Tracking + visuals
3D stereo visuals - the display needs to move with you, just like in the real world.
Resolution - very hard to deliver HD quality to the Eye
Screen door effect - when there is not enough resolution and the UI looks pixellated
Refresh rate - needs to be fast, to look realistic. much higher than monitor
Persistence - the eye remembers what it just saw, so the monitor keeps flicking on and off rapidly, it doesn't look like a strobe as it is so fast; it relies on the eye remembering
field of view
sensors
head orientation
Gyroscope
Accelerometer
Magnometer

Like on a phone, but much higher refresh rate. The phone OS caps the refresh rate to save on battery consumption

Camera needed for positional tracking

Proper hardware

PCVR 1000hz refresh rate - high quality of tracking

Hardware:
Oculus rift DK2
Oculus rift Crescent bay
Sony Project Morpheus

Mobile VR

100hz

Google Cardboard
Mattel/Google Mobile viewer
Samsung Gear VR - this uses a note 4, but sensors inside the headset


Programming it:

Unity
Unreal engine
60-90 fps needed - a lot of data generated
TWICE - once for each eye
so more like 180 fps

You'll need a dedicated desktop machine to generate all this data

Screen/VR
Different - stuff looks better in VR
VR has lots of depth
Individual perceptions
Depth is real, not simulated
Intense
Fragile - experiences can break really easily, because of user vision

Things that don't work
Lense effects, e.g. flare
Normal mapping looks fake
Depth of field - we can't currently track where the eyes are
Mirrors - standard effect doesn't look good, looks painted. You need to render the view properly for each mirror view/depth

Comfort
VOR - Vestibulo-Ocular reflex. Focus on an object even when your head is moving.
In VR this is annoying, as it can cause Simulator Sickness. Combination of eye and ear information. Visuals and balance

You don't get sick when watching a movie because of field of view. You are not moving. Your eyes are capturing movement as your head moves, but your ear is not capturing changes in balance as your body moves.

In real life when you move both will happen, but in VR you are not really moving so your ears don't get the signals when they should be doing logically

To not get simulator sickness:
Don't move
Don't accelerate
Don't rotate

Try to avoid these

Optics

IPD - Inter-Pupillary distance. Needs to be calibrated for each person - this amount can differ a lot. 64mm is the average

chromatic aberration - when the lens separates the colours, like a prism. The software compensates for this automatically

Motion to Photon
Prediction
Time warp - compare the last render position with the next predicted position, and average it out.

VR Audio
Haptics - vibrations when you hear a loud sound
Buttkicker, Kor-FX emulate this effect by creating vibrations along with the sound.

Challenges
How to see yourself inside the VR
Distant objects, how to see them realistically
Text inside the VR - witout positional tracking, you can't get right into the page to see the text
Locomotion
User inputs -joypad is unnatural, leapmotion is really low fps, so not great
Viewing diferent places
New language needs to be defined to deal with applications - how to guide users

Fantasynth project
Based on Hellorun



Understanding your core
Naomi Atkinson---------
--------------

This was a fairly fluffy feel good talk on understanding your skills and how to make the best of them for your career.

If you can ... one thing, you can ... everything.

Replace ... with teach, communicate, engineer, perform. (not sure I agree with this. You need supplementary info)

Realise and capitalise

Her core: designer
web industry
advertising industry
freelance

Saying you are a solo employee often limits opportunity. Saying you have a team immediately opens up more opportunities, and positively affects how clients perceive you.

She told a story about animate, a web animation tool that they worked on together, and wanted to get to market. But then they fealised their heart wasn't in it.

Instagram beta - she was involved in photography for them

Her shop project is really cool - a physical shop that sells independant artwork.



The Joy of order
Mario Klingemann
----------------

Mario is a code artist, and creates all kinds of amazing data visualizations.

LowPolyBot is amazing. Creates low fi polygon versions of pictures sent to it. It's a twitter bot.

He calls himself an obsessive compulsive orderer
He combs the beach to find order in the items in the sand, when on holiday. He gets bored on the beach

Not so much the joy of sorting, more the joy of finding

Along with Cat Ladies, trainspotters, etc.
We are dopamine junkies
It is the brain's natural reward system

bldigital - the british library have scanned in 50,000 old books (from 500 years ago until early 20C) and extracted the images from those books. flickr.com/photos/britishlibrary

These are public domain and free to use, but they are somewhat unsorted. So hard to browse.

Mario has been commissioned to write a program to order all these images

what is order?

Order is a state where every element is as close as possble to its most similar neighbours.

Like ordering dice - why is lowest to highest the most obvious order?

the lower the differences between each dice position, the better the order

But dice number ordering is easy. It won't always be this easy. what about different numbers, sizes, etc.? Features and dimensions. Number, Size, Colour

The problem is normalization. How do you compare a colour to a size, for example? You need to turn all into ranges. A larger dice can be a bigger number. A brighter coloured dice can be a bigger number

Then write feature vectors, e.g.

Dice1(color: 0.1, size: 0.3, value: 0.6)

Then plot these on a map, then use pytthagoras to meaure the distsance between these dice,

Images will just have more dimensions.

To work these out, look up t-SNE clustering algorithm

He used this to categorise the british library portraits, with a homegrown tool. Works out clusters, then puts lines between the nearest neighbours, then clusters the clusters

R language for data vis: r-project.org
Python adapter: scipy.org
Great resources: github.com/karpathy

His findings

Finds unusual objects
old cartoons
geological profiles
rock formations
fossils
pictures of sad girls
pictures of open doors
sea travel
potraits
people who are looking outwards
people who all look 44

He created a Collagenerator (collage generator)
Ernst - creates surrealist art



Things imperfect
Brendan Dawes---
-------------

Design, technology, art
paperclips - very utilitarian, but there are differences in different ones from around the world
Pencils, of course. Pencil geekery
a slide about knobs - doro knobs, etc.
Write your thoughts down. Write them all.

The things you love are often full of imperfections
How can we make use of imperfections in digital ?
Everything starts out ugly
Start with loads of input, then boil, simmer, reduce, down to a beautiful single point


arduino
electric imp

Dawesduino

Kennedy app - remember where you were when...press a button, and it remembers what was going on. Transmits a sentence to twitter or wherever. digital memories.

Give the things you love a name
Put things out into the world, even if you aren't sure what it is.
Cinemaredux - printing out all frames of a movie next to one another
the happiness machine

invent your own tools

Data by itself is not enough - data needs poetry

vogel spiral

3D printing
around since 1985, but only recently tangible to more people
BRendan started printing songs, and twitter streams, data waffles, other random stuff


Chris Mills
Senior tech writer || Mozilla
developer.mozilla.org || MDN
cmi...@mozilla.com || @chrisdavidmills