DSPatch - C++ real-time, object-oriented, flow-based programming library

1,238 views
Skip to first unread message

Marcus Tomlinson

unread,
Oct 30, 2012, 11:14:18 AM10/30/12
to flow-based-...@googlegroups.com
I've created a C++ object-oriented, flow-based programming library called "DSPatch".

I originally designed DSPatch to be a re-usable, modular process-chain engine I could use when writing audio applications (much like the patching engines found in DAW programs such as Pro Tools, FL Studio, etc.), but it eventually developed into a generic FBP framework, not limited to any particular signal / data type. 

What I think may be of particular interest is how multi-threading is handled: The library user has the ability to select the number of threads in which he/she requires the system to process, rather than the thread count rising as more components are added. So for example, an application running on a quad core could be limited to 4 threads in order to allow each core to handle only one thread. This also means that systems with no threading support whatsoever (E.g. an embedded application), can still benefit from this library by setting the thread count to 0 and ticking the system "manually" from the main application thread.

Please give it a download and let me know what you think is good OR bad about it :)


P.S. Apologies for the lack of comments, I'll get around to fixing that ASAP.

Sam Watkins

unread,
Oct 30, 2012, 1:37:41 PM10/30/12
to flow-based-...@googlegroups.com
looks interesting and I will take a look... triple post for the win! :)

Chris Brody

unread,
Oct 30, 2012, 1:41:46 PM10/30/12
to flow-based-...@googlegroups.com, Sam Watkins
I also like the idea but am not a real fsn of the GPL license. Would you consider something like MIT license?

Francisco León

unread,
Oct 30, 2012, 3:17:09 PM10/30/12
to flow-based-...@googlegroups.com
Yes, about GPL that license is too hostile. Could you consider another license that allows static linking? There are alternative licenses that protect your intellectual work from patent issues and encourage feedback and sharing source modifications from community users, those like Apache-like or Mozilla Public Licenses are good enough.

Marcus Tomlinson

unread,
Oct 30, 2012, 3:48:28 PM10/30/12
to flow-based-...@googlegroups.com
I fully understand where you guys are coming from, the GPL is pretty strict. I will definitely consider a more proprietary friendly license. I haven't done a lot of research into these other licenses, but from what I've read so far, the Mozilla public license looks like quite a nice middle ground. Will do some more research.

Francisco León

unread,
Oct 30, 2012, 4:04:31 PM10/30/12
to flow-based-...@googlegroups.com
:) It's your choice, and would be welcome anyways. GPL3 for open source applications could be fine, however for libraries isn't practical because in most cases users would like to combine that gpl'ed library with other software under strict requirements (opensourced but propietary also), which could carry many legal conflicts with other licenses. Most people would avoid that library.

Marcus Tomlinson

unread,
Oct 30, 2012, 4:06:07 PM10/30/12
to flow-based-...@googlegroups.com
Thanks for the heads up Francisco :)
Message has been deleted

Marcus Tomlinson

unread,
Nov 1, 2012, 3:48:21 AM11/1/12
to flow-based-...@googlegroups.com
I really didn't expect this post to turn into a licencing discussion (I hope that someone will have something to say about the library soon) :p
Anyway, how would you guys (and others in general) feel about LGPL?

Francisco León

unread,
Nov 1, 2012, 2:57:07 PM11/1/12
to flow-based-...@googlegroups.com
LGPL could be fine too. The last version 3 is recommended.

ifknot

unread,
Nov 1, 2012, 3:25:43 PM11/1/12
to flow-based-...@googlegroups.com
Hi Marcus!

I think it would help if you gave an overview of your classes and how they work together and perhaps just a little in file documentation or a class diagram?

I have had a quick look and it seems interesting is it built around your template overloaded smart pointer? 1-10 pointers?

Don't see a buffer per se?

Each component has/its own thread?

How is it Real Time?

MsSleep does nothing under Linux?

Where is scalable scheduler?

I'm sure these questions will seem dumb, but my first glance only and it's difficult without class or interaction diagrams.

It would be good if you spent a little time explaining as not all here are C++ nerds partic not at template meta programming level

I'm sure there would be much more feedback for you if you were willing to invest time on explication

I have my own toy C++ template based FBP implementation using in MSc on dataflow scalability and cache considerate behaviour it's quick and dirty but would you maybe have a look and give your thoughts?

ATB

ifknot

Marcus Tomlinson

unread,
Nov 1, 2012, 4:26:40 PM11/1/12
to flow-based-...@googlegroups.com
Hey ifknot, thanks for the feedback!

In terms of a system overview have you read the wiki page on sourceforge yet? (http://sourceforge.net/p/dspatch/wiki/Home/) currently this is all I have in that respect, I'm working on using doxygen to document the whole library nicely in HTML, hopefully it'll shed more light on the specifics of each class. I'll be working on that this weekend for v1.3 that'll also evidently be re-licensed to LGPL :)

The safepointer is an encapsulation of the c standard pointer, abstracting all standard pointer operations. These safepointer objects greatly simplify the usual pointer responsibilities, they work like c# references in that a reference count is kept for each instance of new()ed memory and is cleaned up when all references are detached. It was designed to be purposely incompatible with standard pointers so that corrupting the pointer is as close to impossible as possible :) In this kind of framework, a corrupted pointer is a BIG problem. The 10 New() method templates your referring is my attempt at abstracting the standard new() function, I stopped at 10 constructor parameters as the max for a safepointer object -I think that should be sufficient.

Data is transferred between components via signals, one variable at a time (a signal holds a single value). Hense, buffers can be transferred either as a stream of single variables (E.g. A stream of bytes -and by stream I mean one after the other) or as vectors/arrays (also technically single variables) on each circuit tick. In the demo components provided, these audio components relay vectors of audio samples between each other.

The multi-threading aspect of DSPatch was designed to allow the user to specify the number of threads in which he/she required the circuit to process rather than the thread count growing as the system did. So for example, an application running on a quad core system could be limited to 4 threads in order to allow each core to handle just one thread. This essentially how it works: The circuit runs through its array of components, calling each components process method in a single thread loop (as I've described on the sourceforge wiki page). As each component is done processing it hands over control to the next waiting circuit thread. Therefore, if you had 5 components in a process chain, and 5 circuit threads, technically at any one point in time you could have one thread per component processing in parallel. I say technically because thats asuming all components take roughly the same amount of time to process. With this in place, you now also have the option to select 0 circuit threads. In this state, the circuit's Tick() and Reset() methods can be called in a while loop, for example, in the main application thread. I've added a null thread class that just implements stub methods so DSPatch can compile on platforms with no native thread support. What I've just described are circuit threads (threads that traverse the whole circuit). Another thread type is the component thread. The component thread simply ticks a single component over and over. As the circuit class inherits from component, we use it's component thread to tick the circuit automatically whilst using the main application thread for control.

The term real-time was kinda inherited from my original audio engine development. Essentially, the framework is fast meaning low latency between input and output.

MsSleep does nothing on Linux? Oops! Will have to look into that when I get back to my PC.

Hope this helps understand my thinking behind all the madness :)

P.S. Yeah, I'd be happy to look at your code. Post a link :)

ifknot

unread,
Nov 1, 2012, 5:32:35 PM11/1/12
to flow-based-...@googlegroups.com
Marcus,

Ah 'circuit' I see and yes indeed checking out the wiki helps :) I like the signal approach it's neat.

I was struggling because you're using a circuit metaphor! Mine is a more literal interpretation from Morrison's book (2nd ed) with scheduler agnostic task based components (currently processed by a tunable threadpool), ips are T* (as you point out anything from byte to BLOB to container) passed along a templated connection that, again, is a literal interpretation of JPM's model using a cache optimal buffer (variable capacity). If a connection (buffer, edge, wire - choose your nomencalture) is full process next component (task, node, erm component), ditto if it's empty - simples!

In it's current incarnation it's a baroque monstrosity that has evolved over a couple of years packed with timing strategies and delay mechanisms (the 'jittery-jiffy' is my favourite) to model different behaviours and different approaches to graceful halting. As well as a couple of implementations of the component as a PDA (as JPM suggests) a nice user friendly map and string based one that is an order of magnitude slower than the vector of int statuses approach - which is one of my central points that a FBP approach is a square peg in the round hole of modern CPU multi-tiered cache architecture. 

The FBP software as an artefact is a means to an end not the end result I also don't know what the IP (capitals ie intellectual property) rights are?! (Partially funded from uni and HEIF and local) It's also on the department of CS internal SVN repository. The thesis (if I ever finish) will be public domain but the software? (I asked my supervisor a while back but he said better not get into it.)

It didn't seem relevant as the software exists only to produce results for me to analyse with R to explore Amdahl, Gustafson & Karp-Flatt evidence for proposing FBP as a solution to Sutter's "end of the free lunch".

Does anybody here know if I rewrite it with just the basic classes and I do it at home does that get me out of the IP issue and then is it "my own" for me to open source it?

btw it's all Linux mix of POSIX and Intel TBB and hand-spun if you want any help with Linux/OS X port I might be able to help or then again I might be teaching you to suck eggs? :)

wrp dspatch I'm not sure you can call it RT if the OS is not RT and you don't schedule RT?

Ultimately, I think (not all agree) & whatever your metaphor, in its abstract form, we're all taking about Hoare's communicating sequential processes and that's very powerful in a massively multicore world, functional programming is not the only answer - I don't need a new language just a new library.

Oh and... C++ FTW!

ifknot

Marcus Tomlinson

unread,
Nov 2, 2012, 1:36:54 AM11/2/12
to flow-based-...@googlegroups.com
Oh my goodness, the description of your implementation reminds me of my first attempt. My thinking was: "whatever can be done is parallel, will be done in parallel". Everything was free-running, each component was active, spawning their own thread(s) on construction and handling all of its own internals. It was a multi-threaded nightmare!! I faced all kinds of really complex issues such as the precise order of events in constructing and connecting a component, safely disconnecting/destroying components, feedback loops, branch synchronization, and basically all synchronization in general! Sure it worked, but it was OVERLY complex and a mission to maintain. A bigger problem than that however is that OS's have limits on how many threads a process can have, which means a limit on the number of components your system can have! When you want to model an additive synthesizer with a bank of say: 1000 sine wave oscillators, you've got a problem. My aim is to solve this problem as neatly as possible, avoiding unnecessary over-complication, to make it easy to understand, and more importantly easy to develop and maintain. Don't let the v1.x fool you, DSPatch took 3 years to become what it is now. It's funny how simple something can look when its done (Isn't it OBVIOUS the world is round??).

The "real-time" term also refers to the fact that operations such as manipulating components, connecting/disconnecting wires, adjusting the thread count, etc. can be done while the circuit is running. Perhaps I need a more fitting term for what I'm trying to say?

Oh yes... C++ FTW!

ifknot

unread,
Nov 2, 2012, 9:21:23 AM11/2/12
to flow-based-...@googlegroups.com
Marcus,

I love the free running! Lets you probe some fascinating CS questions. The FBP software is task based and doesn't care about the underlying scheduler itll run on processes/pthreads/green threads any lib you fancy. I use a work stealing thread pool to experiment with that defaults to cpuid core count but as you would expect is highly configurable in order to experiment with and indeed performance collapse with thread (overhead) overload is one of my key graphs (but I can tell you Linux on an 8 core intel is hard to bring to its knees!)

I like how 2 different metaphors have resulted in 2 different approaches but still FBP with your permission may I mention your work as a contrast?

My work is all about scalability it's only an MSc and is as much about quantitative research in comp Sci and the issues of parallelisation as it is the experimental results.

I can't help noticing an implicit challenge in your reply ;) I think only a few classes are needed for a free running FBP implementation.

Hmmm look out for 'quick & dirty FBP' the baroque monster is going to have a baby...

Lol

Ifknot

ifknot

unread,
Nov 2, 2012, 9:36:18 AM11/2/12
to flow-based-...@googlegroups.com
Marcus,

Oh I forgot...

I think that 'runtime reconfigurable' might describe dispatch?

Seems to me that the thing with FBP is that you can throw just about any buzzword at it and it will stick JPM talks about this in his book and I can't help feel with a bit of tongue in cheek!

ifknot

ifknot

unread,
Nov 2, 2012, 10:17:13 AM11/2/12
to flow-based-...@googlegroups.com

Paul Tarvydas

unread,
Nov 2, 2012, 12:20:17 PM11/2/12
to flow-based-...@googlegroups.com
On 12-11-02 09:21 AM, ifknot wrote:
> Marcus,
>
> I love the free running!
In my opinion, free running is the reason to use FBP.

It makes solving parallel/distributed problems easier - everything is
parallel unless you explicitly design synchronization in.

Just, don't use more than 1 thread per core :-).

> Lets you probe some fascinating CS questions.

A lot of those questions are already asked and answered.

They lay in digital hardware (e.g. TTL) design books.

IC's are "free run" by nature, hence, hardware designers had to work out
synchronization issues (when synchronization was needed).

They got the T-shirts (worn out by now).

Brad Cox coined the phrase "software IC's" when he was trying to borrow
ideas from the hardware world :-).

I smile every time I get to use (dredge up from the past) a daisy-chain
to coordinate concurrent tasks...

Multiplexers are "load sharing" devices.

National Semiconductor TTL manuals are "doxygen" and "specification" and
"TDD" rolled into one.

And so on.

pt

Paul Tarvydas

unread,
Nov 2, 2012, 12:33:05 PM11/2/12
to flow-based-...@googlegroups.com
On 12-11-01 05:32 PM, ifknot wrote:
>
> Does anybody here know if I rewrite it with just the basic classes and
> I do it at home does that get me out of the IP issue and then is it
> "my own" for me to open source it?
>

IP (intellectual property) breaks down into various issues, at minimum:

- trade secrets

- ownership of code (specific implementation)

- compensation (salaries, etc, who paid to have you write it)

- contracts (did you sign away your "shower time"?)

- is anybody going to bother enforcing the issues (aka, how many $'s is
it actually worth)

- legal precedent (have other thesis students worked on open source
projects while also working on their theses?)

FBP isn't a trade secret - there's JPM's book, my papers (which indicate
how to do it without threads), and other implementations.

pt

Raoul Duke

unread,
Nov 2, 2012, 1:56:11 PM11/2/12
to flow-based-...@googlegroups.com
On Thu, Nov 1, 2012 at 10:36 PM, Marcus Tomlinson
<themarcus...@gmail.com> wrote:
> internals. It was a multi-threaded nightmare!! I faced all kinds of really
> complex issues such as the precise order of events in constructing and
> connecting a component, safely disconnecting/destroying components, feedback
> loops, branch synchronization, and basically all synchronization in general!

$0.02, seems like there are some projects that have delved into those
issues? anything that is "functional-reactive" or "dataflow" ish might
have. FrTime, Erlang OTP, Mozart/Oz, or the Esterel type families? in
case any of those can either be informative, or a base to start from.

Marcus Tomlinson

unread,
Nov 2, 2012, 3:04:53 PM11/2/12
to flow-based-...@googlegroups.com
Ifknot,

You are more than welcome to mention my work in your thesis :)

I'm sorry, I re-read my last message to you, and realized how it could have sounded. I wasn't implying that your code is all the things I said! Hahaha! I just get a little emotional when I think about that old code :)

The "real-time" thing was obviously the wrong term to use, I've removed it from the classification.

Looking forward to the FBP baroque monster. Quick I like, but dirty... not so much :)

Marcus Tomlinson

unread,
Nov 2, 2012, 3:11:51 PM11/2/12
to flow-based-...@googlegroups.com
BTW everybody! Just released DSPatch v2.0 on LGPL. There are also a few new improvements, fixes etc. (I've decided that doxygenation will just have to wait till v2.1)

ifknot

unread,
Nov 2, 2012, 4:50:51 PM11/2/12
to flow-based-...@googlegroups.com
Paul,

>A lot of those questions are already asked and answered. 

Ouch man! Yes and my supervisor is letting me rediscover them for myself and yes a lot of this is old, I'm digging around in papers from the mid 70s onwards but do you know what? I'm learning and I'm trying to share my enthusiasm here.

>They got the T-shirts (worn out by now). 

Oh well let's all just go home then.

ifknot

ifknot

unread,
Nov 2, 2012, 4:55:18 PM11/2/12
to flow-based-...@googlegroups.com

That's actually helpful thanks.

I will take steps to clarify.

ifknot

Ged Byrne

unread,
Nov 2, 2012, 5:06:01 PM11/2/12
to flow-based-...@googlegroups.com
The IP for FBP is very firmly in the public domain.  From the book

"Although the concepts are not well known, they have actually been in the public domain for many years. The way this happened is as follows: in late 1970 or early '71 I approached IBM Canada's Intellectual Property department to see if we could take out a patent on the basic idea. Their recommendation, which I feel was prescient, was that this concept seemed to them more like a law of nature, which is not patentable. They did recommend, however, that I write up a Technical Disclosure Bulletin (TDB), which was duly published and distributed to patent offices world-wide (Morrison 1971). A TDB is a sort of inverse patent - while a patent protects the owner but requires him or her to try to predict all possible variations on a concept, a TDB puts a concept into the public domain, and thereby protects the registering body from being restricted or impeded in the future in any use they may wish to make of the concept. In the case of a TDB, it places the onus on someone else who might be trying to patent something based on your concept to prove that their variation was not obvious to someone 'skilled in the art'."

Paul Morrison

unread,
Nov 2, 2012, 6:34:14 PM11/2/12
to flow-based-...@googlegroups.com
Thanks, Ged,

I was going to draw Marcus' attention to this exact paragraph!  Apparently TDBs are now called "IBM TDBs" - see https://en.wikipedia.org/wiki/IBM_Technical_Disclosure_Bulletin .

Al the best,

Paul

Paul Morrison

unread,
Nov 2, 2012, 6:53:34 PM11/2/12
to flow-based-...@googlegroups.com
On 01/11/2012 5:32 PM, ifknot wrote:
> a FBP approach is a square peg in the round hole of modern
> CPU multi-tiered cache architecture.
Can you expand on this? What would fit the round hole better?! Or
maybe we could use Piet Hein's squircle -
http://en.wikipedia.org/wiki/Squircle ...

Regards,

Paul

David Barbour

unread,
Nov 2, 2012, 7:06:10 PM11/2/12
to flow-based-...@googlegroups.com

Really, calling modern CPU architectures a "round hole" - or any simple shape - is ridiculous. But maybe Escher could have described it

ifknot

unread,
Nov 2, 2012, 8:49:06 PM11/2/12
to flow-based-...@googlegroups.com
I certainly feel like I've fallen down the rabbit hole :)

ifknot

unread,
Nov 2, 2012, 9:20:28 PM11/2/12
to flow-based-...@googlegroups.com
Paul, (tugs forelock)

I may have been over-egging the pudding with a 19th century metaphor but then again maybe not?


goto 48:00

'compactness matters'

it's all about locality of reference a FBP graph is throwing data back and forth between multiple cores and not taking the cache(s) into consideration risks orders of magnitude performance hits
IPs in disjoint memory locations are a cache line hostile approach


I don't have one explicitly for FBP yet, but its on my todo list  


"...you see it through the medium of your likings and dislikings, and insist upon forcing a square peg into a round hole, because in a round hole you, being a round peg, feel tight and comfortable. Now I call that irrational."

ifknot

On Friday, 2 November 2012 22:53:34 UTC, Paul Morrison wrote:

ifknot

unread,
Nov 2, 2012, 9:28:42 PM11/2/12
to flow-based-...@googlegroups.com
Erm... it's not the shape it's just a metaphor :s

John Cowan

unread,
Nov 2, 2012, 11:34:03 PM11/2/12
to flow-based-...@googlegroups.com
Ged Byrne scripsit:

> The IP for FBP is very firmly in the public domain. From the book

The patent and trade secret rights don't exist, that's true. There
remains the copyright in specific code. Whether that belongs to the
author's employer or not is a matter of local law: in New York it does,
in California it does not, provided the work was not done on employer
time or equipment. (That is why California has a Silicon Valley and
New York does not.)

--
First known example of political correctness: John Cowan
After Nurhachi had united all the other http://www.ccil.org/~cowan
Jurchen tribes under the leadership of the co...@ccil.org
Manchus, his successor Abahai (1592-1643)
issued an order that the name Jurchen should --S. Robert Ramsey,
be banned, and from then on, they were all The Languages of China
to be called Manchus.

Marcus Tomlinson

unread,
Nov 3, 2012, 2:41:15 AM11/3/12
to flow-based-...@googlegroups.com
Paul M,

TDB's are quite an interesting concept! The "anti-patent". Hmmm... But I was never intending to patent my FBP approach, just wanted to make a cool library :) It was Ifknot that was asking about patenting in relation to his thesis work.

Paul Morrison

unread,
Nov 3, 2012, 10:51:49 AM11/3/12
to flow-based-...@googlegroups.com
Ah, light dawns!  Bjarne is apparently talking about vectors vs. lists...  I'm not sure how this is relevant to FBP as the only place where we would use either one is in the connections, and these contain object references.  In some early implementations I did implement connections by linking the IPs together in FIFO sequence, but for most of them (including the very first, and also the Java and C# ones) I use the array of references approach, which should give good locality for a given connection.  If we in turn make the connection capacities quite large, then a given process should retain control for a longer period, also improving locality.  Of course, if Java (or C#) decides to give control to another process before either the connections are full or empty, I don't know of any way to control that, but I feel I got pretty good performance in my last JavaFBP benchmark.  I freely admit that you can probably get better performance using a language like C++ as you can control the logic down to a finer level, but - as of now - I do find Java a more friendly language than C++ - at least for the application logic.  Maybe I just have to get over all those **** asterisks!

One other factor affecting locality is where storage allocation places new objects - this is surely a problem even in C++ if you use malloc for 'new'ing objects.  Once an IP is allocated, the FBP implementations that I have worked on don't move it until it is discarded.  Again, I believe there have been discussions of implementations where IPs are never modified, but are just copied and the new copy is output, and the old one discarded, but my feeling is that this would have the problems with locality that you are referring to.   Now are you advocating doing your own storage management in C++?      In the very first FBP implementation (AMPS) we did our own storage management, and the literature at the time was full of discussions of different storage management schemes, so we were able to experiment.  If this is of interest, we could discuss some of the techniques we used, and maybe see if they could be applied to C++... 

This conversation is very interesting, and maybe I'm just showing my lack of understanding of recent architecture.  Please educate me, and please no forelock tugging - it sounds uncomfortable!  :-)

Best regards,

Paul M.

Paul Morrison

unread,
Nov 3, 2012, 11:08:22 AM11/3/12
to flow-based-...@googlegroups.com
On 01/11/2012 5:32 PM, ifknot wrote:
> a templated connection
Just curious if you mean the connection can only hold one IP type or if
the template refers to packets which in turn hold the data. The latter
is what I do, as you may need to pass a mixture of different IP types
across a particular connection.

In my book, I describe components that don't care what type of data they
are handling, such as Concatenate, or which get as much type information
as they need from parameters (IIPs), e.g. the Sort component described
in my book. In my view, this type of component is essential, but it
causes problems for network-define-time type checking, e.g. say you have
A->B->C, where A puts out IPs of type X, and B is something like
Concatenate, you'd like to be able to check that C expects X's or a
supertype of X. This is no problem at run-time, and in fact the latest
versions of both JavaFBP and C#FBP support that, but one might feel it's
a bit late! Conversely, you really don't want to have separate
Concatenate modules for every IP type in the application... Or has
someone solved this one?!

Marcus Tomlinson

unread,
Nov 3, 2012, 12:07:13 PM11/3/12
to flow-based-...@googlegroups.com
The way I approach the problem of differing incoming types is just try mulyiple typecasts (E.g. switch statement) until I get a match then act on the data appropriately. Any reason why that's not appropriate?

Paul Morrison

unread,
Nov 3, 2012, 12:15:53 PM11/3/12
to flow-based-...@googlegroups.com
On 03/11/2012 12:07 PM, Marcus Tomlinson wrote:
> The way I approach the problem of differing incoming types is just try mulyiple typecasts (E.g. switch statement) until I get a match then act on the data appropriately. Any reason why that's not appropriate?
>
That's fine - similar to the way I handle the output of Collate in
chapter 9 of my book. I was referring to ifknot's statement about
"templated connections", and wondering if that meant his connections can
only handle one IP type...

Regards,

Paul

Marcus Tomlinson

unread,
Nov 6, 2012, 11:09:52 AM11/6/12
to flow-based-...@googlegroups.com
Hi guys, I've put up a web site for DSPatch at: http://www.adaptaudio.com/DSPatch
Hopefully the documentation here can shine some more light on how the DSPatch internals work.
I've also release version 2.1 with the doxygen comments included, along with a very nice performance improvement (http://sourceforge.net/projects/dspatch/files/DSPatch-2.1.zip/download) -I've removed my smart pointer implementation: "DspSafePointer", completely from the engine and replaced all instances with standard c pointers, as the reference counting was inducing quite a performance hit to the system. I must admit I was a little ambitious and trigger happy with the whole DspSafePointer thing, maybe it'll come in handy in something less time critical. A few simple benchmarks have already proven DSPatch to be around 20% faster with this change in place! (Altogether now... "I could have told you that, Marcus!")

Max Yaffe

unread,
Dec 27, 2012, 10:45:44 AM12/27/12
to flow-based-...@googlegroups.com
Marcus,
I've downloaded DSPatch.  By the way, thanks for the LGPL3 license, otherwise it's untouchable. 

Been looking at it and it looks interesting.  I'm not real comfortable with the external clock for kicking off component execution.   I'd like components to run asynchronously so as soon as there is an incoming message, Process_ happens.  This does raise ugly synchronization/race issues, of course.
 
Have you tried compiling it with VS2010?  I ran into all sorts of linking errors doing so.  I did use the VS2008 to VS2010 upgrade wizard which, I think may have some problems.   It may be that the VS2008 precompiled library, DspDemoComponents.lib is not compatible with the VS2010 compiled demo code, but I doubt it.  It's probably an inadvertent compiler switch.  VS2010 offers some nice dual screen flexibility so this may be worth figuring out.

Max 

Max Yaffe

unread,
Jan 24, 2013, 4:26:06 PM1/24/13
to flow-based-...@googlegroups.com
I'm trying to port the Windows build environment to VS2010 and running into some problems with DLLIMPORT.  I think the biggest problem is the precompiled DspDemoComponents.lib.   Do you have a stub version of this in source I can try compiling in VS 2010?
Max
 
Message has been deleted

Marcus Tomlinson

unread,
Jan 25, 2013, 1:45:22 AM1/25/13
to flow-based-...@googlegroups.com
I'm really sorry about the late reply, I've been scoping out the interest in DSPatch on the Boost mailing list during December, so I've been rather busy.

The reason I packaged these libs statically this way is because the demo components: DspMp3Decoder and DspAudioDevice have 3rd party library dependencies for mp3 decoding and audio streaming that I wanted to abstract from the end user. Having to download these dependencies in order to run the demo project would be quite irritating I think. I may end up going this route as the size the distribution is quite large with these binaries included.

Until I come up with a better idea, I've included the vs2010 libs with the latest DSPatch version: v2.32 (sourceforge.net/projects/dspatch).

All you'll need to do now from your dspdemo vs2010 solution (converted from vs2008) is edit the "Linker\General\Additional Library Directories" property of the dspdemo project from "../dependencies_vs2008/" to "../dependencies_vs2010/".

Hope this helps.

Marcus Tomlinson

unread,
Feb 28, 2013, 12:37:26 AM2/28/13
to flow-based-...@googlegroups.com
Hi Rosen,

hmmm, interesting that I haven't come across this error yet. Basically, when the vector of DspCircuitThreads ("_circuitThreads") in DspCircuit is resized calls are made to the constructor + destructor of DspCircuitThread. When DspCircuitThread destructs it requests the contained DspThread to Stop(), but as the PThread in DspThread at this point has not yet initialised, you get that run-time error.

Could you please try this:
In the method DspCircuitThread::Stop() in "DSPatch\src\DspCircuitThread.cpp" add the following if statement in bold:

void DspCircuitThread::Stop()
{
  if( !_stopped )
  {
    <method code>
  }
}

Let me know if this works. I need to make changes to DspThread to encapsulate this behavior, but this fix is essentially all you should need.
Thanks for the observation! I'll get cracking on getting this fix incorporated in the next version.

On Wednesday, 27 February 2013 21:44:53 UTC+2, Rosen wrote:
Hi Marcus,

I'm trying to run the DSPatch tutorial project under Linux amd64. I manage to compile it with almost no problem but at runtime I'm gatting segmentation fault. Here is the stacktrace:

==2785== Conditional jump or move depends on uninitialised value(s)
==2785==    at 0x5351007: pthread_detach (pthread_detach.c:32)
==2785==    by 0x404289: DspThread::Stop() (DspThreadUnix.h:67)
==2785==    by 0x40689A: DspCircuitThread::Stop() (DspCircuitThread.cpp:80)
==2785==    by 0x4066F1: DspCircuitThread::~DspCircuitThread() (DspCircuitThread.cpp:42)
==2785==    by 0x403022: DspCircuit::SetThreadCount(unsigned short) (DspCircuit.cpp:84)
==2785==    by 0x40194B: main (main.cpp:61)

Do you have idea what might be the problem?

Rosen

Rosen Krastev

unread,
Feb 28, 2013, 1:03:17 AM2/28/13
to flow-based-...@googlegroups.com
now the problem moved to:

==3118== Conditional jump or move depends on uninitialised value(s)
==3118==    at 0x5351007: pthread_detach (pthread_detach.c:32)
==3118==    by 0x4042A9: DspThread::Stop() (DspThreadUnix.h:67)
==3118==    by 0x404208: DspThread::~DspThread() (DspThreadUnix.h:42)
==3118==    by 0x40674F: DspCircuitThread::~DspCircuitThread() (DspCircuitThread.cpp:40)
==3118==    by 0x403042: DspCircuit::SetThreadCount(unsigned short) (DspCircuit.cpp:84)
==3118==    by 0x401976: main (main.cpp:62)
==3118==  Uninitialised value was created by a stack allocation
==3118==    at 0x402F29: DspCircuit::SetThreadCount(unsigned short) (DspCircuit.cpp:72)

btw
void DspCircuitThread::Stop()
{
  if( _stopped )
     return;

  <method code>
}

is slightly better.


--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Marcus Tomlinson

unread,
Feb 28, 2013, 5:40:39 AM2/28/13
to flow-based-...@googlegroups.com
Ok, this is actually still the same problem but instead the phenomena is occurring in the DspThread destructor where Stop() is being called.
As I said earlier the change needs to be done in DspThread. I will make this update to DSPatch v2.42, but here is the idea if you'd like to implement it yourself for now:

DspThread::Stop() should only call pthread_detach if the thread (_thread) had been started prior to this method call (i.e. DspThread::Start() was called at some point). I'll add a boolean class member to keep track of whether or not the thread is active, and call pthread_detach in DspThread::Stop() only if the thread is currently active.
Message has been deleted

Marcus Tomlinson

unread,
Nov 15, 2015, 3:21:09 PM11/15/15
to flow-based-...@googlegroups.com
Hi Hen,

Adding the following to the example/CMakeLists.txt should fix your immediate problem:

target_link_libraries(
    ${PROJECT_NAME}
    pthread
)

But really, this project has been structured to be built from the root CMakeLists.txt, so running cmake+make on just examples folder does not bring in the necessary dependancies (i.e. the pthread linkage defined in the root CMakeLists.txt).

The only step you need execute is ./build.sh. Via that script, the library along with the example and tutorial application are built into a new folder in the root directory called “build”. When ./build.sh completes, you can run the example application from dspatch/build/example/DSPatchExample.

The equivalent manual cmake process is (from root directory):

$ mkdir build
$ cd build
$ cmake ../
$ make

(the ./install.sh script installs DSPatch to /usr/local/include and /usr/local/lib)

On 14 Nov 2015, at 5:16 PM, Hen Rik <henni...@googlemail.com> wrote:

Hi there,

I'm trying to compile the example project on Ubuntu 14.04 amd64 but I'm getting problems with rtaudio.
I've done the following steps (libasound2-dev is installed already):
1. (in root dir) ./build.sh
2. ./install.sh
3. (in example project dir) cmake CMakeLists.txt
4. Changed EXAMPLE_WAV_FILE in main.cpp to "./Sample.wav"
5. Changed EXAMPLE_PLUGIN_FILE in main.cpp to "./oscillator-plugin/libDspOscillator.so"
6. make
With the last make command I get the following error:
/usr/bin/ld: CMakeFiles/DSPatchExample.dir/rtaudio/RtAudio.o: undefined reference to symbol 'pthread_create@@GLIBC_2.2.5'
//lib/x86_64-linux-gnu/libpthread.so.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status

I've already tried to compile the rtaudio main.cpp with g++ and -lpthread flag, but I also get errors.
Can somebody explain me how I can build the project with Linux?
If it works, I want to develop some audio effects on the Raspberry Pi.

Thanks!



--
You received this message because you are subscribed to a topic in the Google Groups "Flow Based Programming" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/flow-based-programming/D_drsRHuKJo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to flow-based-progra...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hen Rik

unread,
Nov 17, 2015, 12:51:21 PM11/17/15
to Flow Based Programming
Hi Marcus,

thanks a lot for you reply!
I successfully built your example project, but now I get some runtime problems.
When I execute your example project, I don't get any sound and also no warnings or errors...
I also updated the rtaudio files with the up-to-date ones from GitHub, but I still don't get any sound.
Can anybody help me?
Hi Hen,
To unsubscribe from this group and all its topics, send an email to flow-based-programming+unsub...@googlegroups.com.

Hen Rik

unread,
Nov 17, 2015, 1:46:48 PM11/17/15
to Flow Based Programming
Ok, i found the problem.
The pulseaudio daemon was running in the background. Moreover a created a .asoundrc file and defined my default ALSA interfaces-.-

Marcus Tomlinson

unread,
Nov 17, 2015, 3:57:51 PM11/17/15
to flow-based-...@googlegroups.com
Glad you found the issue! Hope you joy the library :)
To unsubscribe from this group and all its topics, send an email to flow-based-progra...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages