Audio proposal p1386
146 views
Skip to first unread message
Roger Orr
Jan 26, 2019, 11:36:39 AM1/26/19
to SG13 - HMI
Hello all,
the pre-meeting paper list is already published, and includes a first paper on a proposal for audio:
A Standard Audio API for C++:
Motivation, Scope, and Basic Design
Motivation, Scope, and Basic Design
- Guy Somberg, Guy Davidson, and Timur Doumler
The paper can be viewed using http://wg21.link/p1386
It will be discussed in person in the SG13 session at Kona (Fri pm)
However, I plan on some initial discussion of this during Wednesday's call.
Roger.
vinnie...@gmail.com
Jan 26, 2019, 1:10:34 PM1/26/19
to SG13 - HMI
On Saturday, January 26, 2019 at 8:36:39 AM UTC-8, Roger Orr wrote:
A Standard Audio API for C++:Motivation, Scope, and Basic Design- Guy Somberg, Guy Davidson, and Timur Doumler
This is a cross-posting of my message to LEWG:
---
I would like to see the trend of "paper first, implementation later" reversed, in particular for library-only solutions. std::audio is a perfect example of a library that can be written as either its own independent offering, or part of a collection (including but not limited to Boost). Users get a benefit without the cost of committee time. They also get a faster turnaround, no need to wait around for the entire planet to vote yes on a feature, and then wait for every implementor in the world to duplicate the effort to get it onto their platform.
In the past I have done significant work in the JUCE (<https://juce.com/>) community, publishing a number of open source components for working with audio and mentoring other open source authors using JUCE. I have hands on experience working with digital audio workstations, both as a developer (authoring VST plugins) and as a user (producing recordings and operating virtual instruments).
My DSPFilters library comes with a live demo that uses JUCE:
<https://github.com/vinniefalco/DSPFilters>
I also have an example program which shows how to mix audio in a real-time thread:
<https://github.com/vinniefalco/SimpleDJ>
(These programs have not been touched in a while and may be suffering from uh... some bit rot, caveat emptor)
I have read through p1386r0 and what follows is my feedback. I will proceed from the top of the paper to the bottom, quoting text as needed and responding to it inline.
---
1.1
> "Wouldn’t it be great if the basic functionality of talking to your sound card would come for free with every C++ compiler, as part of the C++ standard library?"
Yes that would be great but nothing of value in the standard (or life in general) comes "for free", it is just that as a user the cost has been shifted onto the vendors instead of you. This can be a good thing since it allows the efforts of a handful of specialists to become leveraged for everyone to use. However it is not without cost.
In addition to the cost paid by implementors, when a proposal is merged there is a hidden tax paid in perpetuity by everyone participating in the committee. That tax is in the form of the additional labor required to make sure that every future change is in harmony with the thing that got merged.
> "Throughout all of this time, PCM is the lingua franca of audio"
This is true but I believe there is a more powerful statement begging to be said:
"Portable interfaces to audio input and output can be made uncontroversial."
Unlike graphics, for which there are radical hardware improvements made every year and new computation models coming out every few years, audio is largely a Solved Problem. With respect to getting audio samples into and out of a program, there just isn't that much API surface area needed. The complications of hardware acceleration simply don't exist in audio: sample rates top out around 192K for high end, and around 48K for consumer with no pressing need to ever improve that due to the limits of the human auditory system.
1.2
> "It [C++] is also a language which lacks library support for many fundamental aspects of client computing. If C++ is to be a language for the client as well as the server, it needs to complete its HCI support."
C++ needs HCI support but it does not follow that such support must exist in the standard library.
1.3
> "Consider that a hypothetical Boost.Audio library existed with some interface not dissimilar to what is described in this paper. This is great for everybody who has access to a platform that Boost supports, and who is on a team that includes the boost libraries."
It seems to me that there is a small but vocal group of companies that are allergic to anything outside of std:: (and allergic to boost:: in particular). This subgroup appears to be holding the entire standard library hostage, demanding additions to it for no other reason than "we can only use what's in std::" or "we can't use boost::."
To put it bluntly, this practice needs to stop; we should not add a std:: component only because a handful of users are averse to using external libraries. Is it really the best use of the committee's time to add something like scoped_allocator_adaptor (receiving only 14,500 hits on Google) to std::? I would prefer to see this energy invested towards eliminating the actual barriers to using external libraries: modules, dependencies, and package management.
> "If, on the other hand, audio capabilities were a part of the standard library provided by the compiler, then the embedded chip or OS vendor could write one single high-quality implementation, ship it with the standard library, and all of their customers will immediately be able to use their existing std::audio code on the new hardware."
That's not how it currently works. Each operating system provides its own non-portable abstraction (examples: DirectSound, CoreAudio). The chip maker writes the driver for each OS. Then a hypothetical std::audio, which is written against the non-portable abstraction (for example, the standard library for Visual C++ would use at least DirectSound) automatically can take advantage of this new hardware without the need for a new standard library.
A Boost.Audio which is written against the API offered by Windows, will work with all new hardware that targets Windows, without modification. If Windows ever rolls out a new audio API, then Boost.Audio can optionally take advantage of that too. If Windows removes the old API, then Boost.Audio will _require_ an update. This is no different than the scenario where the Visual Studio standard library offers a std::audio: it would need an update to 1. optionally take advantage of the new API, or 2. implement the changes caused by the old API disappearing.
4.
> "...there is a category of programmers for whom this library is too low-level. For those people, we hope to include a suite of libraries in succession papers once this paper has been put through its paces."
The implication here is that anything worthwhile *has* to be incorporated into the standard library specification. This is not a sustainable process. I am weakly in favor of std::audio, because having portable abstractions over hardware (or more correctly, operating system-specific APIs in this case) is one of the strong capabilities of C++. I also believe that going all the way low-level is the correct for std::audio.
But I am strongly against an endless parade of new libraries proposed for the standard which build on std::audio and nothing else, since these new libraries being proposed could easily be made completely portable and published as independent libraries.
6.1
> "The API...is a low-level ...API...lays the foundations for higher-level abstractions that can come in the future, or which users can build upon to create their own software and libraries"
I am very much in favor of this model of library componentization. Standardise the lowest layer, and let the free market of external libraries satisfy consumers needs and desires by building on that component. As the external library ecosystem flourishes, only then should we decide on what established practice to further standardise. This worked great for std::shared_ptr, et. al.
Note that I have taken this approach in Boost.Beast (which is similarly low-level).
Conclusion
An advantage of writing an independent library is that it can continue to be maintained even after it is voted into the standard. This is of great benefit to those who are not yet on C++Latest (which is most of the community) as they get access to new C++ library components even while using an older version of C++. For this reason, I feel that it is good engineering practice when authoring a library to avoid needlessly using recent C++ language or library features. In my opinion the benefit of gaining access to the larger audience of C++ developers on older versions is worth the cost of having to write out a bit more cumbersome or clumsy implementation (with exceptions of course, some libraries fundamentally require new C++ stuff). Having a larger audience of users for your library means that you will get more feedback, a better view of how the library is used, and in general this information helps make the library better over time.
While it is true that there are already a number of similar audio libraries out there, all of those libraries were written from the perspective of a user and not from the perspective of a library implementor. In other words, they were not written with the goal of standardisation in mind. Furthermore, the authors of those respective existing libraries did not have the same expertise and experience with writing std-quality-C++.
However, this does raise an important question. If an independent audio:: library cannot become popular and pervasive (i.e. a de-facto standard) then why would that change just because it is prefixed with "std::" (modulo non-reasons such as external-library-phobia).
Note, the repository mentioned in the paper contained no code at the time of this writing:
<https://github.com/stdcpp-audio/libstdaudio/tree/ebdc0fd138d9d8bc1ace62e6422f5adb553ee12a>
Regards
I would like to see the trend of "paper first, implementation later" reversed, in particular for library-only solutions. std::audio is a perfect example of a library that can be written as either its own independent offering, or part of a collection (including but not limited to Boost). Users get a benefit without the cost of committee time. They also get a faster turnaround, no need to wait around for the entire planet to vote yes on a feature, and then wait for every implementor in the world to duplicate the effort to get it onto their platform.
In the past I have done significant work in the JUCE (<https://juce.com/>) community, publishing a number of open source components for working with audio and mentoring other open source authors using JUCE. I have hands on experience working with digital audio workstations, both as a developer (authoring VST plugins) and as a user (producing recordings and operating virtual instruments).
My DSPFilters library comes with a live demo that uses JUCE:
<https://github.com/vinniefalco/DSPFilters>
I also have an example program which shows how to mix audio in a real-time thread:
<https://github.com/vinniefalco/SimpleDJ>
(These programs have not been touched in a while and may be suffering from uh... some bit rot, caveat emptor)
I have read through p1386r0 and what follows is my feedback. I will proceed from the top of the paper to the bottom, quoting text as needed and responding to it inline.
---
1.1
> "Wouldn’t it be great if the basic functionality of talking to your sound card would come for free with every C++ compiler, as part of the C++ standard library?"
Yes that would be great but nothing of value in the standard (or life in general) comes "for free", it is just that as a user the cost has been shifted onto the vendors instead of you. This can be a good thing since it allows the efforts of a handful of specialists to become leveraged for everyone to use. However it is not without cost.
In addition to the cost paid by implementors, when a proposal is merged there is a hidden tax paid in perpetuity by everyone participating in the committee. That tax is in the form of the additional labor required to make sure that every future change is in harmony with the thing that got merged.
> "Throughout all of this time, PCM is the lingua franca of audio"
This is true but I believe there is a more powerful statement begging to be said:
"Portable interfaces to audio input and output can be made uncontroversial."
Unlike graphics, for which there are radical hardware improvements made every year and new computation models coming out every few years, audio is largely a Solved Problem. With respect to getting audio samples into and out of a program, there just isn't that much API surface area needed. The complications of hardware acceleration simply don't exist in audio: sample rates top out around 192K for high end, and around 48K for consumer with no pressing need to ever improve that due to the limits of the human auditory system.
1.2
> "It [C++] is also a language which lacks library support for many fundamental aspects of client computing. If C++ is to be a language for the client as well as the server, it needs to complete its HCI support."
C++ needs HCI support but it does not follow that such support must exist in the standard library.
1.3
> "Consider that a hypothetical Boost.Audio library existed with some interface not dissimilar to what is described in this paper. This is great for everybody who has access to a platform that Boost supports, and who is on a team that includes the boost libraries."
It seems to me that there is a small but vocal group of companies that are allergic to anything outside of std:: (and allergic to boost:: in particular). This subgroup appears to be holding the entire standard library hostage, demanding additions to it for no other reason than "we can only use what's in std::" or "we can't use boost::."
To put it bluntly, this practice needs to stop; we should not add a std:: component only because a handful of users are averse to using external libraries. Is it really the best use of the committee's time to add something like scoped_allocator_adaptor (receiving only 14,500 hits on Google) to std::? I would prefer to see this energy invested towards eliminating the actual barriers to using external libraries: modules, dependencies, and package management.
> "If, on the other hand, audio capabilities were a part of the standard library provided by the compiler, then the embedded chip or OS vendor could write one single high-quality implementation, ship it with the standard library, and all of their customers will immediately be able to use their existing std::audio code on the new hardware."
That's not how it currently works. Each operating system provides its own non-portable abstraction (examples: DirectSound, CoreAudio). The chip maker writes the driver for each OS. Then a hypothetical std::audio, which is written against the non-portable abstraction (for example, the standard library for Visual C++ would use at least DirectSound) automatically can take advantage of this new hardware without the need for a new standard library.
A Boost.Audio which is written against the API offered by Windows, will work with all new hardware that targets Windows, without modification. If Windows ever rolls out a new audio API, then Boost.Audio can optionally take advantage of that too. If Windows removes the old API, then Boost.Audio will _require_ an update. This is no different than the scenario where the Visual Studio standard library offers a std::audio: it would need an update to 1. optionally take advantage of the new API, or 2. implement the changes caused by the old API disappearing.
4.
> "...there is a category of programmers for whom this library is too low-level. For those people, we hope to include a suite of libraries in succession papers once this paper has been put through its paces."
The implication here is that anything worthwhile *has* to be incorporated into the standard library specification. This is not a sustainable process. I am weakly in favor of std::audio, because having portable abstractions over hardware (or more correctly, operating system-specific APIs in this case) is one of the strong capabilities of C++. I also believe that going all the way low-level is the correct for std::audio.
But I am strongly against an endless parade of new libraries proposed for the standard which build on std::audio and nothing else, since these new libraries being proposed could easily be made completely portable and published as independent libraries.
6.1
> "The API...is a low-level ...API...lays the foundations for higher-level abstractions that can come in the future, or which users can build upon to create their own software and libraries"
I am very much in favor of this model of library componentization. Standardise the lowest layer, and let the free market of external libraries satisfy consumers needs and desires by building on that component. As the external library ecosystem flourishes, only then should we decide on what established practice to further standardise. This worked great for std::shared_ptr, et. al.
Note that I have taken this approach in Boost.Beast (which is similarly low-level).
Conclusion
An advantage of writing an independent library is that it can continue to be maintained even after it is voted into the standard. This is of great benefit to those who are not yet on C++Latest (which is most of the community) as they get access to new C++ library components even while using an older version of C++. For this reason, I feel that it is good engineering practice when authoring a library to avoid needlessly using recent C++ language or library features. In my opinion the benefit of gaining access to the larger audience of C++ developers on older versions is worth the cost of having to write out a bit more cumbersome or clumsy implementation (with exceptions of course, some libraries fundamentally require new C++ stuff). Having a larger audience of users for your library means that you will get more feedback, a better view of how the library is used, and in general this information helps make the library better over time.
While it is true that there are already a number of similar audio libraries out there, all of those libraries were written from the perspective of a user and not from the perspective of a library implementor. In other words, they were not written with the goal of standardisation in mind. Furthermore, the authors of those respective existing libraries did not have the same expertise and experience with writing std-quality-C++.
However, this does raise an important question. If an independent audio:: library cannot become popular and pervasive (i.e. a de-facto standard) then why would that change just because it is prefixed with "std::" (modulo non-reasons such as external-library-phobia).
Note, the repository mentioned in the paper contained no code at the time of this writing:
<https://github.com/stdcpp-audio/libstdaudio/tree/ebdc0fd138d9d8bc1ace62e6422f5adb553ee12a>
Regards
Guy Davidson
Jan 27, 2019, 3:51:45 PM1/27/19
to SG13 - HMI, vinnie...@gmail.com
Thanks for the post Vinnie.
I think you may be mischaracterising the "std:: only" camp. The organisation I'm involved with forbids most things from outside the standard for legal and regulatory reasons: modules, dependencies and package management will make no difference at all to the regulatory environment.
For example, I wanted to use libvp8 in our codebase rather than RADtools Bink for video playback. It took me three years to leap the regulatory hurdles, due to the lack of any kind of indemnity licence. There were extensive legal issues to consider.
Boost is forbidden in our codebase because there is no indemnity licence (or am I wrong?) but also there is no expiry or maintenance guarantee. A library can become stale and fail to build with a new compiler. This is not something we can afford to bear.
Most of these issues are to do with software houses which deliver client side software, rather than server side software. There is a conversation that the committee community needs to have about whether C++ should be a language for the client as well as a language for the server: this is surely the raison d'etre of this study group. If it IS to be a language for the client, then it needs to support audio, windowing and keyboard/mouse/arbitrary controller input.
Cheers,
G
G
Vinnie Falco
Jan 27, 2019, 4:51:50 PM1/27/19
to Guy Davidson, SG13 - HMI
On Sun, Jan 27, 2019 at 12:51 PM Guy Davidson <elgu...@gmail.com> wrote:
> I think you may be mischaracterising the "std:: only" camp.
> The organisation I'm involved with forbids most things from outside the
> standard for legal and regulatory reasons: modules, dependencies and
> package management will make no difference at all to the regulatory environment.
Actually this anecdote shows that my characterization is precise. To
> I think you may be mischaracterising the "std:: only" camp.
> The organisation I'm involved with forbids most things from outside the
> standard for legal and regulatory reasons: modules, dependencies and
> package management will make no difference at all to the regulatory environment.
elaborate, I will ask - why should an organization get to externalize
the cost of corporate or regulatory policies? Specifically, why should
N compiler vendors create and support M ongoing implementations of
feature X for no other reason than that a company or regulatory body
has enacted an arbitrary restriction and needs it that way? To make
matters worse, it is incredibly rare (perhaps non-existant) that a
company offers a product which works and is maintained on "literally"
_every platform_ (or more precise, every platform on which C++
exists). Therefore, it is guaranteed that any (hypothetical) feature
merged to std:: only to satisfy a special interest will consume
unnecessary resources. Specifically, the platforms which the entity in
question does not need supported, which will need vendor
implementation to have conforming C++.
>...This is not something we can afford to bear.
But the cost is not going away it is just being externalized (forcing
other people to pay it) by having vendors implement it and maintain it
for you. There's an opportunity cost here, because resources are
finite - what is the community as a whole unable to enjoy because
those resources were diverted to satisfy inefficient policies?
> C++ should...support audio, windowing and keyboard/mouse/arbitrary controller input.
C++ already supports these things, and almost audio processing
software is written using it, and I agree that these things are
important, but I disagree with the implication that they need to be
baked into the standard, as standardized components have very high
costs which are usually invisible or hidden from most people. Since an
approved paper effectively creates a lot of up-front work and ongoing
work for a fair number of vendors, there should be a more thorough
*quantitative* analysis of the benefits of that work - and I believe
that standing behind and supporting an independent implementation of a
to-be-proposed library component first is a great way to support that
analysis. Not just for std::audio but everything.
Regards
David Ludwig
Jan 29, 2019, 11:10:12 AM1/29/19
to sg...@isocpp.org
Is it always the case that 'std' library components must be reimplemented by each compiler vendor?
Could, for example, one implement a component and have it effectively be shared amongst 'std' library implementations, in some part?
-- David L.
> --
> You received this message because you are subscribed to the Google Groups "SG13 - HMI" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sg13+uns...@isocpp.org.
> To post to this group, send email to sg...@isocpp.org.
> Visit this group at https://groups.google.com/a/isocpp.org/group/sg13/.
> To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/sg13/CA%2BEzHGcYsHiAGgoNFwDCRf2iQWkhivQQG%2Bb7cSzq9fXrHW-Vow%40mail.gmail.com.
Could, for example, one implement a component and have it effectively be shared amongst 'std' library implementations, in some part?
-- David L.
> You received this message because you are subscribed to the Google Groups "SG13 - HMI" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sg13+uns...@isocpp.org.
> To post to this group, send email to sg...@isocpp.org.
> Visit this group at https://groups.google.com/a/isocpp.org/group/sg13/.
> To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/sg13/CA%2BEzHGcYsHiAGgoNFwDCRf2iQWkhivQQG%2Bb7cSzq9fXrHW-Vow%40mail.gmail.com.
Thiago Macieira
Jan 29, 2019, 11:14:55 AM1/29/19
to sg...@isocpp.org
On Tuesday, 29 January 2019 08:10:06 PST David Ludwig wrote:
> Is it always the case that 'std' library components must be reimplemented by
> each compiler vendor?
>
> Could, for example, one implement a component and have it effectively be
> shared amongst 'std' library implementations, in some part?
It's possible, but it would need to be suitably licensed in a way that all
> Is it always the case that 'std' library components must be reimplemented by
> each compiler vendor?
>
> Could, for example, one implement a component and have it effectively be
> shared amongst 'std' library implementations, in some part?
vendors sharing the code feel comfortable with, plus is cross-platform to the
platforms they're targetting. It's a possibility for libc++ and libstdc++ to
share some MIT-licensed code, for example, but highly unlikely that Microsoft
could or would adopt such code.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
Vinnie Falco
Jan 29, 2019, 11:15:01 AM1/29/19
to SG13 - HMI
On Tue, Jan 29, 2019 at 8:10 AM David Ludwig <dlu...@pobox.com> wrote:
> Is it always the case that 'std' library components must be reimplemented by each compiler vendor?
>
> Could, for example, one implement a component and have it effectively be shared
> amongst 'std' library implementations, in some part?
Hah! I have wondered that myself. After maintaining Boost.Beast for
> Is it always the case that 'std' library components must be reimplemented by each compiler vendor?
>
> Could, for example, one implement a component and have it effectively be shared
> amongst 'std' library implementations, in some part?
over a year, I now know the answer. Any non-trivial library component
that tries to be portable (i.e. work on all compilers) will inevitably
accumulate an increasing amount of #ifdef and workarounds for missing,
broken, or different features. For example:
<https://github.com/boostorg/beast/blob/5361224a7d4c3b318fc92616eff23442851bdc6a/include/boost/beast/core/detail/get_executor_type.hpp#L20>
Most of the time it will be more efficient and optimal for a
particular compiler if the author can assume it will only be used with
that toolchain. From a workflow perspective, you also get a better
result because you have an "expert" in one toolchain producing the
component for that one toolchain, rather than an engineer who has more
broad knowledge across toolchains but is not as proficient in any
particular one.
Regards
Ville Voutilainen
Jan 29, 2019, 11:30:34 AM1/29/19
to sg...@isocpp.org
On Tue, 29 Jan 2019 at 18:10, David Ludwig <dlu...@pobox.com> wrote:
> Could, for example, one implement a component and have it effectively be shared amongst 'std' library implementations, in some part?
Yes. That's what's happening with the parallel algorithms
> Could, for example, one implement a component and have it effectively be shared amongst 'std' library implementations, in some part?
implementation, the same implementation is contributed to both
libstdc++
and libc++.
Reply all
Reply to author
Forward
0 new messages