Problem with portmidi?

[Paul Fisher]

So I finally updated to .7 (from all the way back at .59!) and I'm all hung up getting portmidi working (or understanding how it is suppose to work perhaps). I had been using rtmidi so a lot of my code needs changing but I'm testing with the portmidi-output example with 4 IAC Buses (OS X 10.11.6) with no joy.

Any suggestions would be greatly appreciated.

Executing this:

;;; portmidi.xtm:examples -- portmidi example

;; Author: Ben Swift

;; Keywords: extempore

;; Required dylibs: libportmidi

;;; Commentary:

;; A simple MIDI output example using portmidi - helpful if you only

;; want to send MIDI messages to other devices (and don't need to

;; recieve any MIDI messages in Extempore)

;;; Code:

(sys:load "libs/external/portmidi.xtm")

(pm_initialize)

;; call this function to print a list of the MIDI devices on your

;; machine

(pm_print_devices)

;; if the midi output you want to use has device id 1, then we use

;; that device id to create an output stream (which we will send our

;; midi output from). If the device id is some number other than 1,

;; change it here:

(define *midi-out* (pm_create_output_stream 4))

Here's what I get:

Starting utility process

Trying to connect to 'localhost' on port 7098

New Client Connection 

Successfully connected to remote process

Starting primary process

Trying to connect to 'localhost' on port 7099

New Client Connection 

Successfully connected to remote process

Loading xtmbase library... done in 2.159456 seconds

New Client Connection 

Loading xtmportmidi library... done in 2.263946 seconds

sys:load notification portmidi already loaded 

Portmidi successfully initialised.

-- MIDI input devices --

  device id 0 : <PmDeviceInfo: interface=CoreMIDI name=IAC Driver IAC Bus 1 I/O?:I>

  device id 1 : <PmDeviceInfo: interface=CoreMIDI name=IAC Driver IAC Bus 2 I/O?:I>

  device id 2 : <PmDeviceInfo: interface=CoreMIDI name=IAC Driver IAC Bus 3 I/O?:I>

  device id 3 : <PmDeviceInfo: interface=CoreMIDI name=IAC Driver IAC Bus 4 I/O?:I>

-- MIDI output devices --

  device id 4 : <PmDeviceInfo: interface=CoreMIDI name=IAC Driver IAC Bus 1 I/O?:O>

  device id 5 : <PmDeviceInfo: interface=CoreMIDI name=IAC Driver IAC Bus 2 I/O?:O>

  device id 6 : <PmDeviceInfo: interface=CoreMIDI name=IAC Driver IAC Bus 3 I/O?:O>

  device id 7 : <PmDeviceInfo: interface=CoreMIDI name=IAC Driver IAC Bus 4 I/O?:O>

PortMIDI error opening output port: 'PortMidi: `Invalid device ID''

[Paul Fisher]

Okay, so I got portmidi to work basically one time for me by shutting everything down and making sure I made no errors. Since the "no error" constraint is not one I can count on, I obviously need to understand this better.

I seem to have seen in an older post there is an issue with portmidi hanging on to port bindings s that is probably part of my problem. How do I clear a binding?

So I have a bunch of questions if anyone can offer some advice:

1) is rtmidi still a supported library? I notice that it is moved to contrib (none of the examples have been modified to reflect that) but I get numerous unbound variable errors when I try to use it.

2) is portmidi the proper replacement for rtmidi? It seems quite different in how it works and there does not seem to be an easy match between rtmidi's devices and ports and portmidi's streams.

3) is there any documentation of portmidi beyond the two examples and the source? I'm at a bit of a loss how to port some moderately complex impromptu projects that use multiple IAC bus devices and multiple channels on each device. 

Basically I was using impromptu as a live coding virtual studio controller for a half dozen virtual instruments and samplers. An example would be Live on IAC bus 1 with several instances of Alchemy on different channels and perhaps a couple of Kontakt instances, Reason on IAC bus 2 with a bunch of different instruments on multiple channels,  a hardware XV-88 in Performance mode and the whole mess listening to incoming MIDI from a GR-33 guitar synth.

I'd love to get it all working again.

Thanks.

(I loved impromptu and I'm seeing the advantages of extempore but I am struggling with the transition a bit)

[Johannes Kroll]

Hey,

On Fri, 25 Nov 2016 16:33:53 -0800 (PST)
Paul Fisher <blackbro...@gmail.com> wrote:

> Okay, so I got portmidi to work basically one time for me by shutting
> everything down and making sure I made no errors. Since the "no error"
> constraint is not one I can count on, I obviously need to understand this
> better.
>
> I seem to have seen in an older post there is an issue with portmidi
> hanging on to port bindings s that is probably part of my problem. How do I
> clear a binding?

I ran into this as well and found it confusing. Use Pm_Close:

(bind-func closemidi
(lambda (h)
(Pm_Close h)))

I use a Scheme function to reopen the MIDI port:

(define *midi-out* (pm_create_output_stream 0))
(define reopen-midi-output
(lambda ()
(closemidi *midi-out*)
(set! *midi-out* (pm_create_output_stream (pm_find_output_stream "Midi Through Port-0")))))
(reopen-midi-output)


And this function to look up a MIDI device by name is more practical
than using cryptic integer IDs which might change on every reboot:

;; returns integer id of first device whose name contains given string
(bind-func pm_find_output_stream
(lambda (name:i8*)
(let ((ndev (Pm_CountDevices))
(devid -1))
(doloop (i ndev)
(if (= (tref (Pm_GetDeviceInfo i) 4) 1)
(if (<> null (strstr (tref (Pm_GetDeviceInfo i) 2) name))
(begin
(printf "Matched device '%s' (id %d) for search string '%s'\n"
(tref (Pm_GetDeviceInfo i) 2) i name)
(set! devid i) ; this will be returned
(set! i ndev))))) ; to break out of the loop o.O
devid)))

> So I have a bunch of questions if anyone can offer some advice:
>
> 1) is rtmidi still a supported library? I notice that it is moved to
> contrib (none of the examples have been modified to reflect that) but I get
> numerous unbound variable errors when I try to use it.
>
> 2) is portmidi the proper replacement for rtmidi? It seems quite different
> in how it works and there does not seem to be an easy match between
> rtmidi's devices and ports and portmidi's streams.

Extempore uses portaudio, so it probably makes sense to use portmidi
instead of a completely different library. The main issue with portmidi
seems that there is no way to get callback-based MIDI input. You have
to poll periodically for input which is inefficient and causes jitter
due to the lack of event timestamping. I don't like it. :/

> 3) is there any documentation of portmidi beyond the two examples and the
> source? I'm at a bit of a loss how to port some moderately complex
> impromptu projects that use multiple IAC bus devices and multiple channels
> on each device.

I haven't found much. I looked through the library source and the
official portmidi documentation and got something working.

[Paul Fisher]

Johannes,

Thanks for the detailed suggestions! Your reply is very helpful.  That will get the basics working. 

I'm obviously going to need to rethink my MIDI management for some of the more complex projects. (sigh)

On Saturday, November 26, 2016 at 8:39:43 AM UTC-5, Johannes Kroll wrote:

I ran into this as well and found it confusing. 

> So I have a bunch of questions if anyone can offer some advice:
>
> 1) is rtmidi still a supported library? 
>

> 2) is portmidi the proper replacement for rtmidi?

Extempore uses portaudio, so it probably makes sense to use portmidi
instead of a completely different library. The main issue with portmidi
seems that there is no way to get callback-based MIDI input. You have
to poll periodically for input which is inefficient and causes jitter
due to the lack of event timestamping. I don't like it. :/


> 3) is there any documentation of portmidi beyond the two examples and the
> source?

[algomusic]

Hi Paul,

While this does not address the closing of MIDI ports (I had similar issues) here is my 'simple' MIDI setup procedure that binds to the ports I want within xtlang world and scheme world (as there are different midi play routines available for both).

(sys:load "libs/external/portmidi.xtm")

(pm_initialize)

(pm_print_devices) ;; check Terminal for printout of MIDI device hardware id numbers

(bind-val  midi_in PmStream* (pm_create_input_stream 0)) ;; change number to suit hardware setup

(bind-func get_midi_in (lambda () midi_in))

(define *scheme-midi-in* (get_midi_out))

(bind-val midi_out PmStream* (pm_create_output_stream 3)) ;; change number to suit hardware setup

(bind-func get_midi_out (lambda () midi_out))

(define *scheme-midi-out* (get_midi_out))

Cheers, Andrew Brown

[Ben Swift]

Hi folks

As you've all discovered, Extempore's MIDI support is missing some
functionality and (especially) documentation.

Re: rtmidi - this was always pretty hacky - Rtmidi is in C++ and lacks a
nice C wrapper, so I had to write/maintain this myself. Portmidi is
pure C, so it's nicer in this regard - plus the fact that we use
portaudio as well.

However, the Extempore portmidi library bindings
(libs/external/portmidi.xtm) are incomplete. Portmidi *does* support
callback-based use, but it isn't in the wrapper (or in the examples).
It's certainly a desirable thing, though - we should implement it.

As far as documentation goes, it would be great to have a guide for
setting up/using MIDI in the Tutorials & Guides section. Writing a guide
is pretty easy - it's just rst files in the "docs/" subdirectory, and
you don't even have to worry about formatting it nicely yourself - if
the content is there I'm happy to do the legwork to get it up on
digego.github.io/extempore

http://digego.github.io/extempore/about-this-documentation.html

So, I'm sorry you're having troubles, and the stuff you want to do is
possible (and not *too* difficult to get set up in most cases) but the
documentation sucks :( I'll have a look at it when I can, but I'm going
on holidays soon and I'm not sure if I'll get to it before then. I'm
certainly happy to provide help if the community wanted to improve the
MIDI situation, though.

Thanks all, and sorry about the frustrations - keep us posted :)

Cheers,
Ben

[Andrew Sorensen]

Also, please make sure you are using native xtlang scheduling (not scheme scheduling) for midi-input polling/processing.  Please see/use/modify libs/external/midi_input.xtm 

Cheers,

Andrew.

--
You received this message because you are subscribed to the Google Groups "Extempore" group.
To unsubscribe from this group and stop receiving emails from it, send an email to extemporelang+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ross Bencina]

On 28/11/2016 8:53 AM, Ben Swift wrote:
> Portmidi is pure C, so it's nicer in this regard -
> plus the fact that we use portaudio as well.

Hey Ben,

To be clear: aside from the name, there is little relationship between
PortMidi and PortAudio. They are developed independently and are
maintained by different teams.

Cheers,

Ross.

[Ben Swift]

Yeah, Ross is right. Sorry I wasn't clearer earlier. It's mainly the
pure C aspect that's the appeal.

Cheers
Ben

[Johannes Kroll]

On Mon, 28 Nov 2016 10:13:45 +1000
Andrew Sorensen <dig...@gmail.com> wrote:

> Also, please make sure you are using native xtlang scheduling (not scheme
> scheduling) for midi-input polling/processing. Please see/use/modify
> libs/external/midi_input.xtm

I was wondering why you suggest that, and looked at the source. In
libs/core/scheduler.xtm:

(bind-func clock_scheduler
(lambda (hz:double)
(let ((evtlist:SchedEvt{double}* null)
(ft:[void,double,i8*]* null)
(scheduler:[i64,double]* null)
(running:i64 1)
(time 0.0)
(err 0)
(i (/ 1.0 hz))
(f (lambda ()
(spawn (lambda ()
(set! scheduler (scheduler_init evtlist ft))
(set! time (clock_clock))
(while (> running 0)
(scheduler time)
(while (< (clock_clock) time) (thread_sleep 0 100000))
(set! time (+ time i))
void)
(println "exiting scheduler")))
void)))
(f)
f)))

Is there a reason for the (while (< (clock_clock) time) (thread_sleep 0
100000)) loop? I looked up the C function corresponding to
thread_sleep, the second parameter is in nanoseconds, so that sleeps in
steps of a hundredth of a microsecond, which seems strange to me. Why
not just sleep for the necessary time once: (thread_sleep (modulo
time_left 1) (modulo (* time_left 1000000000) 1000000000))) or
something similar? I realize that the sleep functions usually only
guarantee to suspend thread execution for "at least" the given time...
So trying to avoid to sleep too long is probably the reason for the
"sleep loop". However, if the OS scheduling interval is the reason for
"oversleeping", like the crude ancient Windows implementation which was
using the PIT timer AFAIK, the loop won't help much anyway, so I'm not
sure it will help in practice. But it does produce overhead.

Anyway, if the event list was sorted by time, (scheduler time) would
just have to look at the first element in the list, not the whole list.

Finally, having a sorted list, we could know the exact time of the next
event by looking at the first element. So we could wait for that time
and do away with the "hz" parameter altogether, somewhat similar to the
NO_HZ linux kernel config. That would make it necessary to make the
sleep interruptible whenever a new event is queued, because the new
event might become the new "next" event. I think a wait on a semaphore
with a timeout would work, aka
http://en.cppreference.com/w/cpp/thread/condition_variable/wait_for .
The "tickless" scheduler could be in a separate closure,
clock_scheduler_nohz or something.

What are your thoughts? If you think that sounds good, I might have a
go at implementing it.

[Paul Fisher]

Ben,

NO need to apologize!

If I wanted safe and easy I'd use Garage Band. :-)

Thanks to all for the various pointers and suggestions. It provides a way forward.

I actually have two purposes. One is to get back into my own musical projects using extempore. The other is to be able to use extempore to teach a computer science class (high school). Several years ago our intro class used a mix of DrScheme/Racket and impromptu for our intro course (we've been "scheme" based for years). The integrated environment and ability to engage with temporal recursion using sound and graphics was awesome. Extempore, up to now and with emacs, has been a pretty heavy lift for complete newbie 14 year olds so I've been using Processing instead. 

Still, I want to bring REPL, temporal recursion and "musical structure as a a path into data structure" back and, with Atom, this is doable once I've updated all my old audio-unit and impromptu OpenGL stuff for extempore. Basically that means getting a utility library for MIDI control of VSTs and audio units in place for the students.

[Ben Swift]

Hi Paul

Thanks mate :)

I actually teach a first-year uni course in Processing as well, and I'd
*love* to be able to teach it in Extempore, so any efforts to bring down the
initial barrier to entry (e.g. binary install *with* Atom ready to go)
are dear to my heart.

Cheers,
Ben

Paul Fisher <blackbro...@gmail.com> writes:

[Andrew Sorensen]

The 'hz' loop is required because (as you mention later in your post) you don't want to sleep through the arrival of new, possibly earlier, events.  I agree that an interrupt version of clock_scheduler would be nice, and if you feel like exploring that option then go for it.  Just remember that it needs to operate accurately on Windows, OSX  and Linux platforms.

[Andrew Sorensen]

I'll just clarify a few things.

1) the problem here is with midi-input, midi-output is no problem.

2) portmidi input using xtlang works fine - well even ;)

3) the problem is that there is no Scheme interface for midi-input (there is for output).

4) rtmidi had exactly the same problem (xtlang only for midi-input), so rtmidi is not a solution.

What is required is a nice Scheme interface for portmidi input - which we currently don't have.  We've now put it on the agenda, and if no one else gets to it before us (strongly encouraged ;) either Ben or I will put something together, when time permits.

Cheers,

Andrew.

[Andrew Sorensen]

I've just pushed a small change to libs/external/midi_input.xtm with rough-and-ready scheme support.  It's not a work of art but will give everyone something to be getting on with.

[Johannes Kroll]

Hi Ben,

On Mon, 28 Nov 2016 08:53:12 +1100
Ben Swift <b...@benswift.me> wrote:

> Hi folks
>
> As you've all discovered, Extempore's MIDI support is missing some
> functionality and (especially) documentation.
>
> Re: rtmidi - this was always pretty hacky - Rtmidi is in C++ and lacks a
> nice C wrapper, so I had to write/maintain this myself. Portmidi is
> pure C, so it's nicer in this regard - plus the fact that we use
> portaudio as well.
>
> However, the Extempore portmidi library bindings
> (libs/external/portmidi.xtm) are incomplete. Portmidi *does* support
> callback-based use, but it isn't in the wrapper (or in the examples).
> It's certainly a desirable thing, though - we should implement it.

does it? I looked through the Portmidi documentation and I didn't find
anything callback-related.

http://portmedia.sourceforge.net/portmidi/doxygen/