220 26183 <23998f11-3bb2-41cf-a8fa-e0be2d80dfd2@isocpp.org> article Path: news.gmane.org!not-for-mail From: alexander.zywicki@gmail.com Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: Any interest to adding audio support to the std library? Date: Sat, 4 Jun 2016 09:39:49 -0700 (PDT) Lines: 475 Approved: news@gmane.org Message-ID: <23998f11-3bb2-41cf-a8fa-e0be2d80dfd2@isocpp.org> References: <5751B06A.4080501@mail1.stofanet.dk> <62b7649a-5de5-4330-9bac-708375d36e2c@isocpp.org> <4cfc7939-1ebf-415f-8ef4-d51e7cdadef5@isocpp.org> <239cb9ab-c5d5-4ce5-b876-b5220cc1e116@isocpp.org> <134e61f0-6011-47e6-a09f-c03bfecf7303@isocpp.org> <20160604161527.4915281.1654.11659@gmail.com> Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_3103_99253408.1465058390095" X-Trace: ger.gmane.org 1465058534 6480 80.91.229.3 (4 Jun 2016 16:42:14 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sat, 4 Jun 2016 16:42:14 +0000 (UTC) Cc: alexander.zywicki@gmail.com To: ISO C++ Standard - Future Proposals Original-X-From: std-proposals+bncBCS7TY6DTQOBBV4IZS5AKGQEKFTN37Q@isocpp.org Sat Jun 04 18:42:14 2016 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-qg0-f69.google.com ([209.85.192.69]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1b9Ec5-0006YG-KM for gclcip-std-proposals@m.gmane.org; Sat, 04 Jun 2016 18:39:53 +0200 Original-Received: by mail-qg0-f69.google.com with SMTP id z67sf104018646qgz.0 for ; Sat, 04 Jun 2016 09:39:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isocpp-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:message-id:in-reply-to:references:subject :mime-version:x-original-sender:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=vuHYGdE0k7WAVkAvdPsyVuiumATB9MJH0TthLAfDwyU=; b=PNf7dhTgMkmT7eq6b+rxZVzH74Nnb1FXioLGkv70/Hx0ZBJyLd3Yu7h9KtFPfY1TiV Fc24xNcKmhY1GETfM/kxs4I87QRjnjUur+z+Ccru6CTMFvqC4wLo/ZtuuASNfK1dnpTU DbgqjgAvYZBmNHo02IYrXBJPvibKrmmlJJ/PyltXuAQ98PI6+O+fOIwkRaJjbTUjvFtn I6mhZ/v67OHk9haFDWjUlUNUyLxpi5I49gUU1UiVOtY27hlqRUpHDcxa5hp/dTbiumC3 JT8tTuFv2oW6NBh9NKBgSjZxJU/7yOfaqeQfFMeXYOBzL1zmxbEUOhK2tQaLNai/kIvD 3wsA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:message-id:in-reply-to:references:subject :mime-version:x-original-sender:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=vuHYGdE0k7WAVkAvdPsyVuiumATB9MJH0TthLAfDwyU=; b=IfS8ZmFUhngNyUJc9vDe91LnLszFBcT2JXybEf5218jLt1B0GUqBFKK8f1x2RcX+TD suDFMUvcTJczou8WtWl9saRnuMJ/AwPezyHznu3yz1l+IgMZiRBQVEbQA0NfO2XHHQux iXpP4adR7fRLf5voqZ9MrAPDwansN+7/9xs8TQIffibG8m9PIpQfi1lm4MyUQzi4fERM gIlBOf80JsXfRaHmNap1hIBkYT0jjw2+WJSYxnivAIFKlijEyK603+JJarNm3u/UPgZW u8AJtXBKPGpgq+3sCTyZV+jfJUFNvoKuwgODaOirBLvS5A8naLKhNqGLcLkeZWSic252 WyeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:message-id:in-reply-to :references:subject:mime-version:x-original-sender:reply-to :precedence:mailing-list:list-id:x-spam-checked-in-group:list-post :list-help:list-archive:list-subscribe:list-unsubscribe; bh=vuHYGdE0k7WAVkAvdPsyVuiumATB9MJH0TthLAfDwyU=; b=YMnsOXfNdV6VR3JRwexHWyIBq6vJNnyuVwSRCNv+IC0iG4MvP/Nu8IRP5eidhI21zG k1aTkde6sOkRrxeRXfBjbk51OgDK5dUyHrmczgigNPXZ/vsF1YOpEq0ZwhhidooIpGrK 0X3ESkBgAcoVKBjuHzndLdoaB+MrFMHEJeHvx262Ok7UUn/h9b6kGJngq6ljEwYLrJ94 y+2Q4PqqqzVNvV8G3EUNVS+rIGFRdwr0cQqq3E371OCaZwz7Rtj08QUaw3IwJ0i9wOfy IjYx5n4ayT3uDKCMuA31tJxTBvkJzhZNt6C3Aj2bb38MDd6WD91C8gWa70oUWs87vC2L Noww== X-Gm-Message-State: ALyK8tLDlAWZQopsvmxLXtvFYSnodzPc+abkzGEtkrGgPuWGu6rK2kXXDFXg6fc0LBkO0A== X-Received: by 10.13.215.11 with SMTP id z11mr6966209ywd.4.1465058392322; Sat, 04 Jun 2016 09:39:52 -0700 (PDT) X-BeenThere: std-proposals@isocpp.org Original-Received: by 10.36.76.149 with SMTP id a143ls256579itb.17.gmail; Sat, 04 Jun 2016 09:39:51 -0700 (PDT) X-Received: by 10.36.79.141 with SMTP id c135mr136815itb.10.1465058391306; Sat, 04 Jun 2016 09:39:51 -0700 (PDT) In-Reply-To: <20160604161527.4915281.1654.11659@gmail.com> X-Original-Sender: alexander.zywicki@gmail.com Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Spam-Checked-In-Group: std-proposals@isocpp.org X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:26183 Archived-At: ------=_Part_3103_99253408.1465058390095 Content-Type: multipart/alternative; boundary="----=_Part_3104_990868243.1465058390095" ------=_Part_3104_990868243.1465058390095 Content-Type: text/plain; charset=UTF-8 I agree that in place processing is desired, but an issue comes up when you need to preserve that data for some other process to use as well, you would either have to copy before hand and pass a copy to each process or make the buffer a const& and force you to copy it and work in place on the output buffer On Saturday, June 4, 2016 at 11:15:31 AM UTC-5, Tony V E wrote: > > But most of the time you don't want to copy the data. > > Whenever possible you want to process in place. > > Sent from my BlackBerry portable Babbage Device > *From: *alexande...@gmail.com > *Sent: *Saturday, June 4, 2016 10:41 AM > *To: *ISO C++ Standard - Future Proposals > *Reply To: *std-pr...@isocpp.org > *Cc: *alexande...@gmail.com > *Subject: *Re: [std-proposals] Any interest to adding audio support to > the std library? > > Note that the input must be a const& because in aparalell context it > would allow multiple processes to read safely, this has the side effect of > disabling the range for syntax unless you do this: > > //std::audio example 1 "single process" > void example_1(){ > double sample_rate = 44100; > std::size_t frame_size =2; > std::size_t buffer_size=128; > > std::audio_context ctx{sample_rate,buffer_size,frame_size};//contruct > from values > > std::astream_process proc(ctx,[](std::frame_buffer const > & input, std::frame_buffer& output){ > output=intput;//note the copy required > for(auto&& i: output){ > i*=1.0; > } > }); > > proc.start(); > //do other stuff > proc.stop(); > } > > > > On Saturday, June 4, 2016 at 9:38:54 AM UTC-5, alexande...@gmail.com > wrote: >> >> Is this example more agreeable? >> >> //std::audio example 1 "single process" >> void example_1(){ >> double sample_rate = 44100; >> std::size_t frame_size =2; >> std::size_t buffer_size=128; >> >> std::audio_context ctx{sample_rate,buffer_size,frame_size};//contruct >> from values >> >> std::astream_process proc(ctx,[](std::frame_buffer >> const& input, std::frame_buffer& output){ >> for(std::size_t i =0; i > output[i] = input[i] * 1.0; >> } >> }); >> >> proc.start(); >> //do other stuff >> proc.stop(); >> } >> >> >> >> On Saturday, June 4, 2016 at 1:28:01 AM UTC-5, Ron wrote: >>> >>> Some thoughts... >>> >>> 'N' should be runtime for channels in a frame and for a frame buffer. >>> The buffer size might be negotiable with the device. >>> >>> I like a lot of that code in the original post, but I don't like the >>> comparison to iostreams. Its really the '>>' operator that I don't like. >>> When you copy into your frame buffer, an '=' operator should be used since >>> anyone that sees an '=' operator knows its a copy. But really, you >>> shouldn't need to move data around so much. If you stream from input to >>> 'buff', then process and then move from 'buff' to output then you are >>> moving a lot of data unnecessarily. >>> >>> I think what should happen is this; The process chain is triggered when >>> the output signals it is ready for data by calling the first linked >>> process. Starting at the output, each process kernel calls the previous >>> until it reaches the input where the input device/file/etc copies the data >>> to the chains frame buffer. Each process then operates on that single >>> buffer in turn until finally returning to the output, thus altering the >>> buffer instead of moving data from place to place. I like to think of the >>> processing chain as starting from the output or destination. When the >>> output is ready for data it calls to a linked process for information, it >>> calls to the next link and so on until data begins to get processed. So a >>> bottom up approach, where the input doesn't officially start until the >>> output is ready. I know it is opposite of the way audio really flows >>> through a system but it makes things easier then moving data like a bucket >>> brigade. It can also allow easier parallel processing when there are >>> multiple branches on a node. So if you think of an audio mixer, where >>> there are many inputs and a single output, the output calls to many >>> channels/nodes at once so they can all be processed in parallel and mixed >>> together after each of the processes end. >>> >>> Also, I'm not sure the word 'format' should be used to refer to files. >>> I know people like to say "file format" but then, what format is the audio >>> data in that is in the file format? Its just confusing, so I think >>> 'container' is a better word for files. Then that leaves the word format >>> open for use as a way of describing the data format of an audio frame >>> buffer. So an audio_format class would contain information on sample rate, >>> channels, interleaving, channel routing and other information and a >>> container class would deal with files. >>> >>> I personally would rather have all audio with greater than 2 or 3 >>> channels processed in Ambisonic-B format and then have the audio driver >>> figure out how to extract the channels for each speaker, but I know this >>> won't fly with everyone so we would need a class for channel routing >>> information as well. >>> >>> All I can think of at the moment.. >>> >>> >>> >>> On Friday, June 3, 2016 at 9:08:45 PM UTC-7, alexande...@gmail.com >>> wrote: >>>> >>>> I agree, how do you feel about the concept i proposed in my original >>>> about having std audio stream classes similar to std::fstream. The >>>> std::astream/iastream/oastream would represent the default audio device on >>>> the system as determined by some method of polling the system. The user >>>> could either choose to use them as is or manually select a different device? >>>> >>>> On Friday, June 3, 2016 at 10:51:12 PM UTC-5, Ron wrote: >>>>> >>>>> Well, it needs to start somewhere. I think having basic buffers using >>>>> std::array or plain old arrays might do for now and get some basic audio >>>>> recording and playback into C++. It should be as simple as possible to get >>>>> to the interfaces on the machine and make them work. The data processing >>>>> can build onto it with more complex primitives, templates and classes. >>>>> This way there is at least something semi-usable with code we have today >>>>> while the rest of the building blocks are being worked on. Just a thought >>>>> though. >>>>> >>>>> >>>>> >>>>> -- > You received this message because you are subscribed to the Google Groups > "ISO C++ Standard - Future Proposals" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to std-proposal...@isocpp.org . > To post to this group, send email to std-pr...@isocpp.org . > To view this discussion on the web visit > https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/d6c30daa-bab3-454d-899d-2ad906fc55df%40isocpp.org > > . > > -- You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/23998f11-3bb2-41cf-a8fa-e0be2d80dfd2%40isocpp.org. ------=_Part_3104_990868243.1465058390095 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I agree that in place processing is desired, but an issue = comes up when you need to preserve that data for some other process to use = as well, you would either have to copy before hand and pass a copy to each = process or make the buffer a const& and force you to copy it and work i= n place on the output buffer

On Saturday, June 4, 2016 at 11:15:31 A= M UTC-5, Tony V E wrote:
=
But most of the time you don= 9;t want to copy the data.=C2=A0

Whenever possible you want to process in p= lace.
=

= = =
Sent=C2=A0from=C2=A0my=C2= =A0BlackBerry=C2=A0portable=C2=A0Babbage=C2=A0Device
= = =
=
Sent: Saturday, June 4, 2016 10:41 AM
To: ISO C++ Standard - Future Proposals
Subject: Re: [std-proposals] Any interest to adding audio support = to the std library?

Note that the input must be a const& = because in=C2=A0 aparalell context it would allow multiple processes to rea= d safely, this has the side effect of disabling the range for syntax unless= you do this:

//std::audio example 1 "s= ingle process"
void example_1(){
=C2=A0 =C2= =A0 double sample_rate =3D 44100;
=C2=A0 =C2=A0 std<= span style=3D"color:#660">::size_t frame_= size =3D2;
= =C2=A0 =C2=A0 std::size_t buffer_size=3D<= span style=3D"color:#066">128;

=C2=A0 =C2=A0 std::audio_context<float> ctx<= span style=3D"color:#660">{sample_rate,buffer_si= ze,frame_size};//contruct from values

=C2= =A0 =C2=A0 std::astream_process<float> proc(ctx,[](std::= frame_buffer&l= t;float> const& input, std::frame_buffer<float>& outpu= t){
= =C2=A0 =C2=A0 =C2=A0 =C2=A0 output=3Dintput;//note the copy required
=C2=A0 =C2=A0 =C2=A0 =C2=A0 for(auto&& i: output){
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 i*=3D1.0;
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 }=C2=A0 =C2=A0 });

=C2=A0 =C2=A0 proc.start();<= /span>
=C2=A0 =C2=A0 //do other stuff
=C2=A0 =C2= =A0 proc.stop();
}


=
On Saturday, June 4, 2016 at 9:38:54 AM UTC-5, alexande...@gmail.com= wrote:
Is this= example more agreeable?

//std::audio exampl= e 1 "single process"
= void example_1= (){
= =C2=A0 =C2=A0 double sample_rate =3D 44100;
=C2=A0 =C2=A0 s= td::siz= e_t frame_size =3D2;
=C2=A0 =C2=A0 std::size_t buffer_size= =3D128;=

=C2=A0 =C2=A0 std::audio_context<= span style=3D"color:#080"><float> c= tx{samp= le_rate,buffer_size,frame_size};//contruct from values
=C2=A0 =C2=A0 std::astream_process<float= > proc(ctx,= [](std:= :frame_buffer<float> const& input, std::frame_buffer<float= >& output){
=C2=A0 =C2=A0 =C2=A0 =C2=A0 for(std::size_t i <= /span>=3D0; i = <input.size= ();++i<= span style=3D"color:#660">){
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 output= [i] =3D= input[= i] * 1.0;
=C2=A0 =C2=A0 =C2=A0= =C2=A0 }
=C2=A0 =C2=A0 });

=C2=A0 =C2=A0 proc.start= ();
=C2=A0 =C2=A0 //do other stuff
=C2= =A0 =C2=A0 proc.stop();
}
<= br>

On Saturday, June 4, 2016 at 1:28:01 AM UTC-5, Ron wrote:
Some thoughts...

'N' should be runtime for channels in a frame= and for a frame buffer. =C2=A0The buffer size might be negotiable with the= device.

I like a lot of that code in the original= post, but I don't like the comparison to iostreams. =C2=A0Its really t= he '>>' operator that I don't like. =C2=A0When you copy i= nto your frame buffer, an '=3D' operator should be used since anyon= e that sees an '=3D' operator knows its a copy. =C2=A0But really, y= ou shouldn't need to move data around so much. =C2=A0If you stream from= input to 'buff', then process and then move from 'buff' to= output then you are moving a lot of data unnecessarily.

I think what should happen is this; The process chain is triggered w= hen the output signals it is ready for data by calling the first linked pro= cess. =C2=A0Starting at the output, each process kernel calls the previous = until it reaches the input where the input device/file/etc copies the data = to the chains frame buffer. =C2=A0Each process then operates on that single= buffer in turn until finally returning to the output, thus altering the bu= ffer instead of moving data from place to place. =C2=A0I like to think of t= he processing chain as starting from the output or destination. =C2=A0When = the output is ready for data it calls to a linked process for information, = it calls to the next link and so on until data begins to get processed. =C2= =A0So a bottom up approach, where the input doesn't officially start un= til the output is ready. =C2=A0I know it is opposite of the way audio reall= y flows through a system but it makes things easier then moving data like a= bucket brigade. =C2=A0It can also allow easier parallel processing when th= ere are multiple branches on a node. =C2=A0So if you think of an audio mixe= r, where there are many inputs and a single output, the output calls to man= y channels/nodes at once so they can all be processed in parallel and mixed= together after each of the processes end.

Al= so, I'm not sure the word 'format' should be used to refer to f= iles. =C2=A0I know people like to say "file format" but then, wha= t format is the audio data in that is in the file format? =C2=A0Its just co= nfusing, so I think 'container' is a better word for files. =C2=A0T= hen that leaves the word format open for use as a way of describing the dat= a format of an audio frame buffer. =C2=A0So an audio_format class would con= tain information on sample rate, channels, interleaving, channel routing an= d other information and a container class would deal with files.

I personally would rather have all audio with greater = than 2 or 3 channels processed in Ambisonic-B format and then have the audi= o driver figure out how to extract the channels for each speaker, but I kno= w this won't fly with everyone so we would need a class for channel rou= ting information as well.

All I can think of at th= e moment..



On Friday, June= 3, 2016 at 9:08:45 PM UTC-7, alexande...@gmail.com wrote:
I agree, how do you feel abo= ut the concept i proposed in my original about having std audio stream clas= ses similar to std::fstream. The std::astream/iastream/oastream would repre= sent the default audio device on the system as determined by some method of= polling the system. The user could either choose to use them as is or manu= ally select a different device?

On Friday, June 3, 2016 at 10:51:12 = PM UTC-5, Ron wrote:
Well, it needs to start somewhere.=C2=A0 I think having basic buffers usi= ng std::array or plain old arrays might do for now and get some basic audio= recording and playback into C++.=C2=A0 It should be as simple as possible = to get to the interfaces on the machine and make them work.=C2=A0 The data = processing can build onto it with more complex primitives, templates and cl= asses.=C2=A0 This way there is at least something semi-usable with code we = have today while the rest of the building blocks are being worked on.=C2=A0= Just a thought though.



--
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.
To view this discussion on the web visit https://groups.google.com= /a/isocpp.org/d/msgid/std-proposals/d6c30daa-bab3-454d-899d-= 2ad906fc55df%40isocpp.org.

--
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposa= ls+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.= com/a/isocpp.org/d/msgid/std-proposals/23998f11-3bb2-41cf-a8fa-e0be2d80dfd2= %40isocpp.org.
------=_Part_3104_990868243.1465058390095-- ------=_Part_3103_99253408.1465058390095-- .