Sheet Music Editor w/ Canvas and Web Audio API

[Greg]

The new web audio api is pretty awesome and I think it would be fun to
use it to build a music app. How about building a sheet music editor
for musicians to use to write music notation?

I’m thinking that we could build a slimmed down version of NoteFlight
(http://www.noteflight.com) but use html5 instead of Adobe Flash.
Check out the Live Demo on NoteFlight's website to see what it can do.

Vexflow (http://www.vexflow.com/) is a javascript library that does
music notation with the html5 canvas element and could be used by this
project to display sheet music.

This project would involve building a UI for editing sheet music with
Vexflow and also doing audio synthesis so you can listen to the sheet
music. Anyone interested in working on this?

[Colin A. S. Parsons]

This sounds like a great project! I'm very new to web development and my coding skills are almost nonexistent, but I do have plenty of experience with music and audio. If you get a team together for this I'd do my best to contribute somehow.

colin

[Brandon Thies]

I second colin, I'd be really interested in working on a project like this and would like to contribute anyway I can ( I have almost no technical computing knowledge though :(

Best, 

Brandon Thies 

brando...@gmail.com

[Gregory Jopa]

Colin and Brandon,

You guys are welcome to join the team. Scott Brunswig will also be working on this project. Scott specializes in front-end development so he is going to focus on developing the UI for the Vexflow Editor.

There are other things you can contribute to the team besides programming. There’s a need for a project manager, a presenter, and a tester who can put together some use cases. You could also put together a low fidelity prototype for the project which doesn’t involve any programming. This involves sketching the different screens for the app. For this project there is the editor screen when you are editing notes and there is the play screen when you are listening to the audio. You could sketch these screens with pencil and paper and scan them to make a digital copy or you could use Microsoft PowerPoint to design the screens. If you want to start on something before Wed. you can make a prototype. Read more about prototyping here: http://www.usability.gov/methods/design_site/prototyping.html#PrototypeFidelity

I created a Google Docs collection for this project and made it public. We can use this for collaboration. You should be able to access it here: https://docs.google.com/leaf?id=0BxVgr8DOJUHsNjNjZDhjYzgtY2Y0NS00MGY2LWFhNjItMzliMWJjZmYyOGYx&hl=en_US

Inside this Google Docs collection I uploaded some code that roughly demos how to append notes to existing sheet music with Vexflow. We can use this code as a starting point for the project.

This team can still use some more developers. We need someone with experience programming audio and someone with experience with html5 canvas.

--

Greg

[Ruben Oanta]

I would like to join this group! I am a developer. I don't have much
experience with html5 audio or canvas but I'll try to do some research
before Wednesday.

On Aug 1, 6:41 pm, Gregory Jopa <grj...@gmail.com> wrote:
> Colin and Brandon,
>
> You guys are welcome to join the team. Scott Brunswig will also be working
> on this project. Scott specializes in front-end development so he is going
> to focus on developing the UI for the Vexflow Editor.
>
>  There are other things you can contribute to the team besides
> programming. There’s
> a need for a project manager, a presenter, and a tester who can put together
> some use cases. You could also put together a low fidelity prototype for the
> project which doesn’t involve any programming. This involves sketching the
> different screens for the app. For this project there is the editor screen
> when you are editing notes and there is the play screen when you are
> listening to the audio. You could sketch these screens with pencil and paper
> and scan them to make a digital copy or you could use Microsoft PowerPoint
> to design the screens. If you want to start on something before Wed. you can

> make a prototype. Read more about prototyping here:http://www.usability.gov/methods/design_site/prototyping.html#Prototy...
>
> <http://www.usability.gov/methods/design_site/prototyping.html#Prototy...>

>
> I created a Google Docs collection for this project and made it public. We

> can use this for collaboration. You should be able to access it here:https://docs.google.com/leaf?id=0BxVgr8DOJUHsNjNjZDhjYzgtY2Y0NS00MGY2...

>
> Inside this Google Docs collection I uploaded some code that roughly demos
> how to append notes to existing sheet music with Vexflow. We can use this
> code as a starting point for the project.
>
> This team can still use some more developers. We need someone with
> experience programming audio and someone with experience with html5 canvas.
>
> --
>
> Greg
>

> On Mon, Aug 1, 2011 at 2:55 PM, Brandon Thies <brandon.th...@gmail.com>wrote:
>
>
>
>
>
>
>
> > I second colin, I'd be really interested in working on a project like this
> > and would like to contribute anyway I can ( I have almost no technical
> > computing knowledge though :(
>
> > Best,
> > Brandon Thies

> > brandon.th...@gmail.com

[Colin A. S. Parsons]

I'm about to get on a plane, so I'll have time to do some brainstorming (or BS-ing as I like to call it)

To me it makes sense to approach this as a megasimple sketchpad for composers and dilettantes alike, rather than as a full-fledged score editor. I'm imagining something that allows a musician to jot down their ideas and then export them (MIDI?) to use them later in Sibelius or Pro Tools or whatever.

I also think it might be a good idea to start with a mobile version optimized for smaller touch screens, then expand to a desktop version which takes advantage of the larger screen and the keyboard/mouse.

Thoughts?

[Colin A. S. Parsons]

I'm about to get on a plane, so I'll have time to do some brainstorming (or BS-ing as I like to call it).

To me it makes sense to approach this as a sketchpad for composers rather than as a full-fledged score editor. Something that allows a musician to jot down their ideas and then export them (MIDI?) to use them later in Sibelius or Pro Tools or whatever.

Another thought I had was 

On Aug 1, 2011, at 5:41 PM, Gregory Jopa wrote:

[Scott Brunswig]

Checkout maestro on the app market I think it has similarities

[Brandon Thies]

Hi Greg,

Thank you for inviting me to join your team, I'm really excited to be working on a project of great personal interest to me and I am eager to learn all I can. If no one else more qualified steps forward, I'd like to try my hand at project management. Thanks for putting this all together. I look forward to working on Wednesday.  

Best, 

Brandon Thies 

brando...@gmail.com

[Greg]

Right now we have the following team members for this HTML5 Sheet
Music Editor project:

1. Greg Jopa (me)
2. Scott Brunswig
3. Colin Parsons
4. Brandon Thies
5. Ruben Oanta
6. Ji

I like the idea of creating a sketchpad for composers vs. a full-
fledged score editor. This will make the project a bit more manageable
and I think designing for mobile is the way to go. The export feature
sounds like a great way to share music with other score editors and
MIDI or MusicXML would be great formats to use. However, I don’t think
we will have enough time tomorrow to implement any export features so
this can be a “wish list” item for now.

The Maestro app in the Android App market is a great example of a
sketchpad app for composers. I think we should use Maestro’s UI design
as a starting point. Using just the Treble clef for sheet music is a
good idea too to keep the project more manageable. Maestro doesn’t
seem to do chords, only single notes, and I would like to implement
chords in our project.

Has anyone had a chance to play around with the Web Audio API in
chrome? You need to install a dev version of Chrome in order to test
out this web audio api. Download the canary build from the dev channel
(http://www.chromium.org/getting-involved/dev-channel) then launch
Chrome and type “about: flags” into the url and then enable the Web
Audio API feature. Once you get it installed test out the demos here:
http://chromium.googlecode.com/svn/trunk/samples/audio/index.html
My laptop is running Windows Vista and the web audio api demos are
really glitchy for me. The web audio api is experimental and is a work-
in-progress. I think there are issues with how the web audio api runs
on Windows right now but it should run smoothly on a mac so hopefully
we can use a mac for the presentation.

The audiolib.js (https://github.com/jussi-kalliokoski/audiolib.js)
javascript library is a wrapper for the audio apis and we should use
it in the project to make audio synthesis a bit easier.

We have discussed using jQuery Mobile as the framework for this
project. There is also a jQuery Mobile plugin that might be cool to
use called mobiscroll. Check it out here: http://code.google.com/p/mobiscroll/

Its used for dates and times but perhaps we could hack it and use it
for selecting the note name, octave, and duration when adding/editing
a note.

I put together another demo that shows how the user can select a note
by clicking on the canvas. In the demo when you click on the canvas
you should see a white box that displays behind the selected note.
Vexflow stores note positions in a "map" array that we can use to
lookup a specific note by pixel position on the canvas. I have
uploaded this code to the Google doc and you can see the demo here:
http://code.gregjopa.com/html5/vexflow/note_selector/

On Aug 1, 8:58 pm, Brandon Thies <brandon.th...@gmail.com> wrote:
> Hi Greg,
>
> Thank you for inviting me to join your team, I'm really excited to be
> working on a project of great personal interest to me and I am eager to
> learn all I can. If no one else more qualified steps forward, I'd like to
> try my hand at project management. Thanks for putting this all together. I
> look forward to working on Wednesday.
>
> Best,
> Brandon Thies

[Colin A. S. Parsons]

The demo looks good! I'm gonna play with Maestro tonight and think of ways we can add chords and improve on the interface.

See you tomorrow.

colin