220 26168 article Path: news.gmane.org!not-for-mail From: ron novy Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: Any interest to adding audio support to the std library? Date: Fri, 3 Jun 2016 20:51:09 -0700 Lines: 505 Approved: news@gmane.org Message-ID: References: <5751B06A.4080501@mail1.stofanet.dk> <5290948.va25uYNizX@tjmaciei-mobl1> <1580229.cHEgQxCuA1@tjmaciei-mobl1> <62b7649a-5de5-4330-9bac-708375d36e2c@isocpp.org> Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=001a1143e8e001208d05346bc0d3 X-Trace: ger.gmane.org 1465012280 16009 80.91.229.3 (4 Jun 2016 03:51:20 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sat, 4 Jun 2016 03:51:20 +0000 (UTC) To: std-proposals@isocpp.org Original-X-From: std-proposals+bncBDO2TPGE4QNBBL5AZG5AKGQETXNUEIQ@isocpp.org Sat Jun 04 05:51:15 2016 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-yw0-f200.google.com ([209.85.161.200]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1b92cD-00008F-Gv for gclcip-std-proposals@m.gmane.org; Sat, 04 Jun 2016 05:51:13 +0200 Original-Received: by mail-yw0-f200.google.com with SMTP id n63sf34733396ywf.3 for ; Fri, 03 Jun 2016 20:51:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isocpp-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :x-original-sender:x-original-authentication-results:reply-to :precedence:mailing-list:list-id:x-spam-checked-in-group:list-post :list-help:list-archive:list-subscribe:list-unsubscribe; bh=+ImiplCQkFbzU5bBUCylkuLO5J6Gtgc3jV56SCqu95I=; b=J/YTBGyBkamwCutVIGBw+sN1DJqjO/GbCs+3lsnzs/EDONZoIzJx5VDJ44oRAHRxUh gMUqibVJErdt4OhYHW61KkL7Rrcy7QyN6pGC/2Z+LENHL/ZUjNFK8bZuLsoSklJToYYU r6gQ1tvj/FXrUQ74SVXpvDjiAEdAJ3jqCoAPJjjyeUfAOww70iLoevVA0ETV7tgCWpGM 47sGyJ91rdF/t0pDMJLHTWmnqlHYeZ/BrXq/apOgUZ+9AaLKPZoWswzF9pPVMgv52gof Yu0sJDiI/l5oLOeLoeI4DW9mATd7NMiGeJpZD3fEBkSlDeZQNvbL6vG36D8vIkSuIwiK +exw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=+ImiplCQkFbzU5bBUCylkuLO5J6Gtgc3jV56SCqu95I=; b=kod86zclAsEKKIRW/01JnCGciH1rfi02hrowDD4QJSW1AjsFN3jX6FlHa3ze3uAM5L uN11SM51FGAREoDO4fxW1lfHCamonQj8AJHliDKe4mbkdvkMa3sY8CGvFbFNlVlXCRO7 JL/MYZopNBuEWW1/3TwJ2GuoSp4DEUfZUrqvZh9Xt6125RDcOWuMZyCT1uSwe6TjnpLa XxqtI4/zavOjjowwmZ1vYA+pm6s6a3KVW9W5gy+0iCFsWJihQxxCmkV6/QC5huSm11se 1wPl4Mv6W+gSzFOcujrl1OwHv60xycao2+AdiyGwj40REgLT7Bj+/Wv1Qd3xaZbrPL2y m44A== X-Gm-Message-State: ALyK8tLUvHbTk04khlS+bivMFDzHBMl7yRu+i6G1SfcAoBchEbIsAxfQsxskRLNDqMRVPQ== X-Received: by 10.129.51.72 with SMTP id z69mr5044154ywz.33.1465012272268; Fri, 03 Jun 2016 20:51:12 -0700 (PDT) X-BeenThere: std-proposals@isocpp.org Original-Received: by 10.107.128.30 with SMTP id b30ls602486iod.79.gmail; Fri, 03 Jun 2016 20:51:11 -0700 (PDT) X-Received: by 10.107.43.214 with SMTP id r205mr6572849ior.81.1465012271234; Fri, 03 Jun 2016 20:51:11 -0700 (PDT) Original-Received: from mail-io0-x234.google.com (mail-io0-x234.google.com. [2607:f8b0:4001:c06::234]) by mx.google.com with ESMTPS id 68si9749847iot.168.2016.06.03.20.51.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Jun 2016 20:51:11 -0700 (PDT) Received-SPF: pass (google.com: domain of rsn10100@gmail.com designates 2607:f8b0:4001:c06::234 as permitted sender) client-ip=2607:f8b0:4001:c06::234; Original-Received: by mail-io0-x234.google.com with SMTP id t40so98171655ioi.0 for ; Fri, 03 Jun 2016 20:51:11 -0700 (PDT) X-Received: by 10.36.67.196 with SMTP id s187mr3659739itb.78.1465012270995; Fri, 03 Jun 2016 20:51:10 -0700 (PDT) Original-Received: by 10.64.142.113 with HTTP; Fri, 3 Jun 2016 20:51:09 -0700 (PDT) In-Reply-To: X-Original-Sender: rsn10100@gmail.com X-Original-Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com; spf=pass (google.com: domain of rsn10100@gmail.com designates 2607:f8b0:4001:c06::234 as permitted sender) smtp.mailfrom=rsn10100@gmail.com; dmarc=pass (p=NONE dis=NONE) header.from=gmail.com Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Spam-Checked-In-Group: std-proposals@isocpp.org X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:26168 Archived-At: --001a1143e8e001208d05346bc0d3 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Well, it needs to start somewhere. I think having basic buffers using std::array or plain old arrays might do for now and get some basic audio recording and playback into C++. It should be as simple as possible to get to the interfaces on the machine and make them work. The data processing can build onto it with more complex primitives, templates and classes. This way there is at least something semi-usable with code we have today while the rest of the building blocks are being worked on. Just a thought though. On Fri, Jun 3, 2016 at 7:35 PM, wrote: > On Friday, June 3, 2016 at 9:11:26 PM UTC-5, Ron wrote: > > Okay, I've done some thinking here. The simplest solution should be th= e > correct one. I think the standard should stick to something very basic > here, otherwise we get into an area that is too complicated to implement. > All we should have in terms of a standard audio interface for C++ is: > > > > > > 1) A method for enumerating the interfaces on a machine. > > 2) A method for getting each interfaces capabilities (supported rates, > bit depths etc.). > > 3) A method for activating a requested device for sending and receiving > frames in the requested format. > > > > > > Valid types for audio samples should be either a signed integer, a floa= t > or a double. And that's pretty much it. Get audio input, put audio > output. Anything other than that would be a different proposal all > together. Audio, video or DSP processing in general should certainly be = a > different proposal. > > > > > > But I would propose we also add an int24_t class into the standard for > processing 24-bit samples. Something as simple as this would suffice for > now: > > class int24_t > > { > > private: > > int8_t val[3]; > > }; > > > > > > Something more fully featured would be desirable, like this: > https://github.com/RonNovy/CppDSP/blob/master/src/int24_t.h > > But I guess that, would be another proposal. > > > > > > On Fri, Jun 3, 2016 at 6:47 PM, wrote: > > On Friday, June 3, 2016 at 8:30:49 PM UTC-5, Jeffrey Yasskin wrote: > > > > > On Fri, Jun 3, 2016 at 6:09 PM, Thiago Macieira > wrote: > > > > > On s=C3=A1bado, 4 de junho de 2016 02:20:15 BRT Andrey Semashev wrote= : > > > > > > > > > > > What is needed is a standardized interface for these different > modules > > > > > > > > > > > to work with each other. The interface should also allow me, the > > > > > > > > > > > developer, to work with the media (e.g. create my own audio or imag= e > > > > > > > > > > > filter or a new codec or a new device driver). > > > > > > > > > > > > > > > > > > > > That I agree with. > > > > > > > > > > > > > > > > > > > > But should the standard mandate that there should be at least one? Ho= w > does > > > > > > > > > > someone write a plugin to the C++ Standard Library? Will we now > mandate this > > > > > > > > > > kind of ability? > > > > > > > > > > > > > > > > > > > > Yes. We already do this with things like the random engines. > > > > > > > > > > > > > > > If not, then will there be a requirement that the C++ library vendor > provide > > > > > > > > > > it? If so, please think carefully how Apple should code libc++ to wor= k > on iOS. > > > > > > > > > > > > > > > > > > > > WebCrypto has an example of navigating a legal minefield for > implementers. > > > > > > > > > > > > > > > > > > > > Also remember to leave the legal speculation to lawyers. It's good to > have the heads-up that "hey lawyers should look at this", but anything mo= re > is unwise. > > > > > > > > > > > > > > > Again, I think this is not material for the C++ Standard Library. > > > > > > > > > > > > > > > I think it's a great area for the C++ library to expand into. I'm > definitely worried about the expertise of the people writing the proposal > (e.g. if I proposed an audio library for C++, I should be laughed off the > stage), but we can overcome that by asking well-known audio users for the= ir > opinions before standardizing the proposal. > > > > > > > > > > > > > > > Jeffrey > > > > > > > > I agree about the issue of expertise. By no means would I call myself a= n > expert, but I have a degree in audio engineering and a pretty solid grasp > of the c++ language. Are there people out there more skilled than me at > each topic? Of course, but that's why I am here, to get the opinions of > those people if I can an figure out what needs to be done to make this > library a reality. > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > Groups "ISO C++ Standard - Future Proposals" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > an email to std-proposal...@isocpp.org. > > > > To post to this group, send email to std-pr...@isocpp.org. > > > > To view this discussion on the web visit > https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/62b7649a-5de= 5-4330-9bac-708375d36e2c%40isocpp.org > . > > I think that simplicity is a good way to go but I believe that what you > are proposing may be too basic. As it stands none of the std containers > meet the needs for audio. The closest I would argue could be std::array o= f > you are okay with a buffer length fixed at compile time or std::vector bu= t > std::vector could cause a dynamic memory allocation on a real time thread > if the user tries to push_back and the vector needs to expand. This > allocation could lead to an unknown wait time for the audio thread and > possibly result in buffer under flow to the driver. You could use a plain > array but that leaves room for memory leaks and makes copying buffers > difficult. > > A container designed for real-time usage would be needed. > > I don't see how processing would need to be a different proposal, what is > an audio library without a well organized and effiecint way means for > interacting with the audio data? > > If processing does need to be a different proposal, I do believe that the= y > would need to be designed simultaneously so that they might be easily > interfaces with each other, because I feel that they would most likely be > used in conjunction in many cases. > > -- > You received this message because you are subscribed to the Google Groups > "ISO C++ Standard - Future Proposals" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to std-proposals+unsubscribe@isocpp.org. > To post to this group, send email to std-proposals@isocpp.org. > To view this discussion on the web visit > https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/a27c2580-c76= 6-4488-bc26-a7b15783e0fd%40isocpp.org > . > --=20 You received this message because you are subscribed to the Google Groups "= ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. To view this discussion on the web visit https://groups.google.com/a/isocpp= ..org/d/msgid/std-proposals/CAOcFa%3DethRsOWP2QQeTtOdtQngEVdo5e29nX%2BSQAFzy= xTdhJzA%40mail.gmail.com. --001a1143e8e001208d05346bc0d3 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Well, it needs to start somewhere.=C2=A0 I think having ba= sic buffers using std::array or plain old arrays might do for now and get s= ome basic audio recording and playback into C++.=C2=A0 It should be as simp= le as possible to get to the interfaces on the machine and make them work.= =C2=A0 The data processing can build onto it with more complex primitives, = templates and classes.=C2=A0 This way there is at least something semi-usab= le with code we have today while the rest of the building blocks are being = worked on.=C2=A0 Just a thought though.


On Fri, Jun 3, 2016 at 7:35 PM, = <alexander.zywicki@gmail.com> wrote:
On Friday, June 3, 2016 at 9:11:26 P= M UTC-5, Ron wrote:
> Okay, I've done some thinking here.=C2=A0 The simplest solution sh= ould be the correct one.=C2=A0 I think the standard should stick to somethi= ng very basic here, otherwise we get into an area that is too complicated t= o implement.=C2=A0 All we should have in terms of a standard audio interfac= e for C++ is:
>
>
> 1) A method for enumerating the interfaces on a machine.
> 2) A method for getting each interfaces capabilities (supported rates,= bit depths etc.).
> 3) A method for activating a requested device for sending and receivin= g frames in the requested format.
>
>
> Valid types for audio samples should be either a signed integer, a flo= at or a double.=C2=A0 And that's pretty much it.=C2=A0 Get audio input,= put audio output.=C2=A0 Anything other than that would be a different prop= osal all together.=C2=A0 Audio, video or DSP processing in general should c= ertainly be a different proposal.
>
>
> But I would propose we also add an int24_t class into the standard for= processing 24-bit samples.=C2=A0 Something as simple as this would suffice= for now:
> class int24_t
> {
> private:
> =C2=A0 =C2=A0 int8_t val[3];
> };
>
>
> Something more fully featured would be desirable, like this: https://github.com/RonNovy/CppDSP/blob/master/sr= c/int24_t.h
> But I guess that, would be another proposal.
>
>
> On Fri, Jun 3, 2016 at 6:47 PM,=C2=A0 <alexande...@gmail.com> wrote: > On Friday, June 3, 2016 at 8:30:49 PM UTC-5, Jeffrey Yasskin wrote: >
> > On Fri, Jun 3, 2016 at 6:09 PM, Thiago Macieira <thi...@macieira.org> wrote:
>
> > On s=C3=A1bado, 4 de junho de 2016 02:20:15 BRT Andrey Semashev w= rote:
>
> >
>
> > > What is needed is a standardized interface for these differe= nt modules
>
> >
>
> > > to work with each other. The interface should also allow me,= the
>
> >
>
> > > developer, to work with the media (e.g. create my own audio = or image
>
> >
>
> > > filter or a new codec or a new device driver).
>
> >
>
> >
>
> >
>
> > That I agree with.
>
> >
>
> >
>
> >
>
> > But should the standard mandate that there should be at least one= ? How does
>
> >
>
> > someone write a plugin to the C++ Standard Library? Will we now m= andate this
>
> >
>
> > kind of ability?
>
> >
>
> >
>
> >
>
> > Yes. We already do this with things like the random engines.
>
> >
>
> >
>
> > If not, then will there be a requirement that the C++ library ven= dor provide
>
> >
>
> > it? If so, please think carefully how Apple should code libc++ to= work on iOS.
>
> >
>
> >
>
> >
>
> > WebCrypto has an example of=C2=A0navigating a legal minefield for= implementers.
>
> >
>
> >
>
> >
>
> > Also remember to leave the legal speculation to lawyers. It's= good to have the heads-up that "hey lawyers should look at this"= , but anything more is unwise.
>
> >
>
> >
>
> > Again, I think this is not material for the C++ Standard Library.=
>
> >
>
> >
>
> > I think it's a great area for the C++ library to expand into.= I'm definitely worried about the expertise of the people writing the p= roposal (e.g. if I proposed an audio library for C++, I should be laughed o= ff the stage), but we can overcome that by asking well-known audio users fo= r their opinions before standardizing the proposal.
>
> >
>
> >
>
> > Jeffrey
>
>
>
> I agree about the issue of expertise. By no means would I call myself = an expert, but I have a degree in audio engineering and a pretty solid gras= p of the c++ language. Are there people out there more skilled than me at e= ach topic? Of course, but that's why I am here, to get the opinions of = those people if I can an figure out what needs to be done to make this libr= ary a reality.
>
>
>
> --
>
> You received this message because you are subscribed to the Google Gro= ups "ISO C++ Standard - Future Proposals" group.
>
> To unsubscribe from this group and stop receiving emails from i= t, send an email to std-propo= sal...@isocpp.org.
>
> To post to this group, send email to std-pr...@isocpp.org.
>
> To view this discussion on the web visit https://groups.goog= le.com/a/isocpp.org/d/msgid/std-proposals/62b7649a-5de5-4330-9bac-708375d36= e2c%40isocpp.org.

I think that simplicity is a good way to go but I believe that what = you are proposing may be too basic. As it stands none of the std containers= meet the needs for audio. The closest I would argue could be std::array of= you are okay with a buffer length fixed at compile time or std::vector but= std::vector could cause a dynamic memory allocation on a real time thread = if the user tries to push_back and the vector needs to expand. This allocat= ion could lead to an unknown wait time for the audio thread and possibly re= sult in buffer under flow to the driver. You could use a plain array but th= at leaves room for memory leaks and makes copying buffers difficult.

A container designed for real-time usage would be needed.

I don't see how processing would need to be a different proposal, what = is an audio library without a well organized and effiecint way means for in= teracting with the audio data?

If processing does need to be a different proposal, I do believe that they = would need to be designed simultaneously so that they might be easily inter= faces with each other, because I feel that they would most likely be used i= n conjunction in many cases.

--
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-propo= sals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.go= ogle.com/a/isocpp.org/d/msgid/std-proposals/a27c2580-c766-4488-bc26-a7b1578= 3e0fd%40isocpp.org.

--
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposa= ls+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/CAOcFa%3DethR= sOWP2QQeTtOdtQngEVdo5e29nX%2BSQAFzyxTdhJzA%40mail.gmail.com.
--001a1143e8e001208d05346bc0d3-- .