220 26178 article Path: news.gmane.org!not-for-mail From: alexander.zywicki@gmail.com Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: Any interest to adding audio support to the std library? Date: Sat, 4 Jun 2016 07:41:22 -0700 (PDT) Lines: 400 Approved: news@gmane.org Message-ID: References: <5751B06A.4080501@mail1.stofanet.dk> <5290948.va25uYNizX@tjmaciei-mobl1> <1580229.cHEgQxCuA1@tjmaciei-mobl1> <62b7649a-5de5-4330-9bac-708375d36e2c@isocpp.org> <4cfc7939-1ebf-415f-8ef4-d51e7cdadef5@isocpp.org> <239cb9ab-c5d5-4ce5-b876-b5220cc1e116@isocpp.org> <134e61f0-6011-47e6-a09f-c03bfecf7303@isocpp.org> Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_2808_997316795.1465051282403" X-Trace: ger.gmane.org 1465051289 31001 80.91.229.3 (4 Jun 2016 14:41:29 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sat, 4 Jun 2016 14:41:29 +0000 (UTC) Cc: alexander.zywicki@gmail.com To: ISO C++ Standard - Future Proposals Original-X-From: std-proposals+bncBCS7TY6DTQOBBE6RZO5AKGQESAK3F4A@isocpp.org Sat Jun 04 16:41:28 2016 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-oi0-f72.google.com ([209.85.218.72]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1b9ClR-0000Xf-Tu for gclcip-std-proposals@m.gmane.org; Sat, 04 Jun 2016 16:41:26 +0200 Original-Received: by mail-oi0-f72.google.com with SMTP id w9sf33793573oia.3 for ; Sat, 04 Jun 2016 07:41:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isocpp-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:message-id:in-reply-to:references:subject :mime-version:x-original-sender:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=2V5uVW3g993whcUFZFvT/fep8fM+FNCrW/P8J2UFRNw=; b=pAkljj+WGU32c7kIns8w3aYmRIWP6LZ4/0ywdJeOiysBKZbWJNYzGruyxKSOcZTknV 8/2FHfuWxmVjDLP2ydPIXSqnjC6iHzVCIV8YrDkMiwqxeRIwuh/akl/2pnbKVSKeA/wk nadWs0FgojJk9tg4KiYyRYuaahuz97i919FfUVoDRee5xoy0goAFrgWDcs5NYaGQc6zL Ujh4ue3fwocLbn7BRY0QQMPU6shDyi52Mvj2S5dZgWLaoWF9XriT2IDOaMpZO4AajvWo 6xa0HJe+GrFGER/2RM0s4dkivLuMCxCsW+QYgAA2R4sZb/KhBzN0KJpui217v1hRaWoc SRCA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:message-id:in-reply-to:references:subject :mime-version:x-original-sender:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=2V5uVW3g993whcUFZFvT/fep8fM+FNCrW/P8J2UFRNw=; b=jv3ZhP2OMRmNorKX/L3RcuUY4p0cmo7n68Ak004uXLWVQTlL1H7IN/lSqy88T2Y/4B NYLgdA7vOGO3jWRWlnx6lWh92Ya3AQUdpEJDBGmwTPAQdqIQV9b1IgtwBaRYvWIzSMiu ribyKT4OYAomYlEz9yR4DRydP9MQbQW1bhZ4KmWgCpJJLvkdQBJ/+7RdldBDmaxBAw7M eLmuwqgz63leILumITNHBxqoIw9LBe6XI/96g9w4EVyB1DfegEo080MYEQVQYUxXbkAD IfneiQehdgknocT7AI6nBCExeM7Pb3Esl1xrIC3cK86auqKLfemNGUCWb+MmrLKa5Odl WoqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:message-id:in-reply-to :references:subject:mime-version:x-original-sender:reply-to :precedence:mailing-list:list-id:x-spam-checked-in-group:list-post :list-help:list-archive:list-subscribe:list-unsubscribe; bh=2V5uVW3g993whcUFZFvT/fep8fM+FNCrW/P8J2UFRNw=; b=e0u0AXiywlIq5+fSyqUb/3ci1+zsZSlBUFhg0RgpgA0Y5hUX9lOOdXGj5DhsfNrkue MQkfWMiKlbf5Z2LMqL5Cl9trxMnvxYXeHGQ3xWIHEJYJhDdVCXzVC/M75QJ4odxWIpl9 xtOCqPFiy6/tZChCVA6CdzEewuJMNrdQlENPOWkPuha6fzFH0Zj9jNt7OEtgOQAnJ9/6 0Gag5frO9neX71wUCA1oKBBw6doDcg+LUbqJYgkWO9D7b8RdniZDhugaSfXhhhvXvQM2 7Z04tRdlVzrAwD2kXBc3dOnDmXtsgxp5Bn4/7gFZ92ZlyrZBcU+nUebuhab8hmalkwHy 9tPA== X-Gm-Message-State: ALyK8tJcxnDSZfG5MdRj+kk+U7wjEfh9U6ADC/99NCbLzM341BBFNz5H+TTFWLDbWShpqw== X-Received: by 10.157.5.148 with SMTP id 20mr6407543otd.41.1465051284464; Sat, 04 Jun 2016 07:41:24 -0700 (PDT) X-BeenThere: std-proposals@isocpp.org Original-Received: by 10.36.242.65 with SMTP id j62ls237850ith.23.canary; Sat, 04 Jun 2016 07:41:23 -0700 (PDT) X-Received: by 10.36.76.150 with SMTP id a144mr126915itb.10.1465051283319; Sat, 04 Jun 2016 07:41:23 -0700 (PDT) In-Reply-To: <134e61f0-6011-47e6-a09f-c03bfecf7303@isocpp.org> X-Original-Sender: alexander.zywicki@gmail.com Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Spam-Checked-In-Group: std-proposals@isocpp.org X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:26178 Archived-At: ------=_Part_2808_997316795.1465051282403 Content-Type: multipart/alternative; boundary="----=_Part_2809_1925105233.1465051282404" ------=_Part_2809_1925105233.1465051282404 Content-Type: text/plain; charset=UTF-8 Note that the input must be a const& because in aparalell context it would allow multiple processes to read safely, this has the side effect of disabling the range for syntax unless you do this: //std::audio example 1 "single process" void example_1(){ double sample_rate = 44100; std::size_t frame_size =2; std::size_t buffer_size=128; std::audio_context ctx{sample_rate,buffer_size,frame_size};//contruct from values std::astream_process proc(ctx,[](std::frame_buffer const& input, std::frame_buffer& output){ output=intput;//note the copy required for(auto&& i: output){ i*=1.0; } }); proc.start(); //do other stuff proc.stop(); } On Saturday, June 4, 2016 at 9:38:54 AM UTC-5, alexande...@gmail.com wrote: > > Is this example more agreeable? > > //std::audio example 1 "single process" > void example_1(){ > double sample_rate = 44100; > std::size_t frame_size =2; > std::size_t buffer_size=128; > > std::audio_context ctx{sample_rate,buffer_size,frame_size};//contruct > from values > > std::astream_process proc(ctx,[](std::frame_buffer const > & input, std::frame_buffer& output){ > for(std::size_t i =0; i output[i] = input[i] * 1.0; > } > }); > > proc.start(); > //do other stuff > proc.stop(); > } > > > > On Saturday, June 4, 2016 at 1:28:01 AM UTC-5, Ron wrote: >> >> Some thoughts... >> >> 'N' should be runtime for channels in a frame and for a frame buffer. >> The buffer size might be negotiable with the device. >> >> I like a lot of that code in the original post, but I don't like the >> comparison to iostreams. Its really the '>>' operator that I don't like. >> When you copy into your frame buffer, an '=' operator should be used since >> anyone that sees an '=' operator knows its a copy. But really, you >> shouldn't need to move data around so much. If you stream from input to >> 'buff', then process and then move from 'buff' to output then you are >> moving a lot of data unnecessarily. >> >> I think what should happen is this; The process chain is triggered when >> the output signals it is ready for data by calling the first linked >> process. Starting at the output, each process kernel calls the previous >> until it reaches the input where the input device/file/etc copies the data >> to the chains frame buffer. Each process then operates on that single >> buffer in turn until finally returning to the output, thus altering the >> buffer instead of moving data from place to place. I like to think of the >> processing chain as starting from the output or destination. When the >> output is ready for data it calls to a linked process for information, it >> calls to the next link and so on until data begins to get processed. So a >> bottom up approach, where the input doesn't officially start until the >> output is ready. I know it is opposite of the way audio really flows >> through a system but it makes things easier then moving data like a bucket >> brigade. It can also allow easier parallel processing when there are >> multiple branches on a node. So if you think of an audio mixer, where >> there are many inputs and a single output, the output calls to many >> channels/nodes at once so they can all be processed in parallel and mixed >> together after each of the processes end. >> >> Also, I'm not sure the word 'format' should be used to refer to files. I >> know people like to say "file format" but then, what format is the audio >> data in that is in the file format? Its just confusing, so I think >> 'container' is a better word for files. Then that leaves the word format >> open for use as a way of describing the data format of an audio frame >> buffer. So an audio_format class would contain information on sample rate, >> channels, interleaving, channel routing and other information and a >> container class would deal with files. >> >> I personally would rather have all audio with greater than 2 or 3 >> channels processed in Ambisonic-B format and then have the audio driver >> figure out how to extract the channels for each speaker, but I know this >> won't fly with everyone so we would need a class for channel routing >> information as well. >> >> All I can think of at the moment.. >> >> >> >> On Friday, June 3, 2016 at 9:08:45 PM UTC-7, alexande...@gmail.com wrote: >>> >>> I agree, how do you feel about the concept i proposed in my original >>> about having std audio stream classes similar to std::fstream. The >>> std::astream/iastream/oastream would represent the default audio device on >>> the system as determined by some method of polling the system. The user >>> could either choose to use them as is or manually select a different device? >>> >>> On Friday, June 3, 2016 at 10:51:12 PM UTC-5, Ron wrote: >>>> >>>> Well, it needs to start somewhere. I think having basic buffers using >>>> std::array or plain old arrays might do for now and get some basic audio >>>> recording and playback into C++. It should be as simple as possible to get >>>> to the interfaces on the machine and make them work. The data processing >>>> can build onto it with more complex primitives, templates and classes. >>>> This way there is at least something semi-usable with code we have today >>>> while the rest of the building blocks are being worked on. Just a thought >>>> though. >>>> >>>> >>>> >>>> -- You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/d6c30daa-bab3-454d-899d-2ad906fc55df%40isocpp.org. ------=_Part_2809_1925105233.1465051282404 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Note that the input must be a const& because in=C2=A0 = aparalell context it would allow multiple processes to read safely, this ha= s the side effect of disabling the range for syntax unless you do this:
=
//std::a= udio example 1 "single process"
void example_1(){
=C2=A0 =C2=A0 double sample_rate =3D 44100;<= /span>
=C2=A0 = =C2=A0 std= ::size_t f= rame_size = =3D2;
=C2=A0 =C2=A0 std::size_t buffer_size=3D128;

=C2=A0 =C2=A0 std::audio_context<float> ctx{sample_rate,buffer_size,frame_size};//contruct from values

=C2=A0 =C2=A0 std::astream_process<float> proc(ctx,[](std::frame_buffer<float> c= onst&<= /span> input, std::frame_buffer<float>& output){
=C2=A0 =C2=A0 =C2=A0 =C2=A0 output=3Dintput;//note the copy required
=C2=A0 =C2=A0 =C2= =A0 =C2=A0 for(auto&& i: output){
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 i<= /span>*=3D= 1.0;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 <= /span>}
=C2=A0 =C2=A0 });

=C2=A0 =C2= =A0 proc.<= /span>start();
=C2=A0 =C2=A0 //do other stuf= f
=C2= =A0 =C2=A0 proc.stop= ();=
}



On Saturday, June 4, 2016 at 9:38:54 AM UTC-5, alexande...@= gmail.com wrote:
Is this example more agreeable?

//std::au= dio example 1 "single process"<= br>void= example_1(){
=C2=A0 =C2=A0 double sample_rate =3D 44100;
=C2= =A0 =C2=A0 std::size_t frame_size =3D2;
=C2=A0 =C2=A0 std::= size_t buffer_size=3D128;

=C2=A0 =C2=A0 std::audio_cont= ext<float> ctx{sample_rate,buffer_size,frame_size};= //contruct from values

=C2=A0 =C2=A0 std::astream_process<float> proc(ctx,[](std::frame_buffer<float> const& input, std::frame_buffer<float>& output){
=C2=A0 =C2=A0 =C2=A0 =C2=A0 for(std::size_t i =3D0; i <input.size();++i){<= br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 output[i] = =3D input[i] *= 1.0;
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 }
=C2=A0 =C2=A0 });<= span style=3D"color:#000">

=C2=A0 =C2=A0 proc.start();
=C2=A0 =C2=A0 //do other stuff<= br>=C2=A0 =C2=A0 proc.stop();
}
=



On Saturday, June 4, 2016 at 1:28:01 AM UTC-5, Ron wrote:=
Some thoughts= ....

'N' should be runtime for channels in = a frame and for a frame buffer. =C2=A0The buffer size might be negotiable w= ith the device.

I like a lot of that code in the o= riginal post, but I don't like the comparison to iostreams. =C2=A0Its r= eally the '>>' operator that I don't like. =C2=A0When you= copy into your frame buffer, an '=3D' operator should be used sinc= e anyone that sees an '=3D' operator knows its a copy. =C2=A0But re= ally, you shouldn't need to move data around so much. =C2=A0If you stre= am from input to 'buff', then process and then move from 'buff&= #39; to output then you are moving a lot of data unnecessarily.
<= br>
I think what should happen is this; The process chain is trig= gered when the output signals it is ready for data by calling the first lin= ked process. =C2=A0Starting at the output, each process kernel calls the pr= evious until it reaches the input where the input device/file/etc copies th= e data to the chains frame buffer. =C2=A0Each process then operates on that= single buffer in turn until finally returning to the output, thus altering= the buffer instead of moving data from place to place. =C2=A0I like to thi= nk of the processing chain as starting from the output or destination. =C2= =A0When the output is ready for data it calls to a linked process for infor= mation, it calls to the next link and so on until data begins to get proces= sed. =C2=A0So a bottom up approach, where the input doesn't officially = start until the output is ready. =C2=A0I know it is opposite of the way aud= io really flows through a system but it makes things easier then moving dat= a like a bucket brigade. =C2=A0It can also allow easier parallel processing= when there are multiple branches on a node. =C2=A0So if you think of an au= dio mixer, where there are many inputs and a single output, the output call= s to many channels/nodes at once so they can all be processed in parallel a= nd mixed together after each of the processes end.

Also, I'm not sure the word 'format' should be used to re= fer to files. =C2=A0I know people like to say "file format" but t= hen, what format is the audio data in that is in the file format? =C2=A0Its= just confusing, so I think 'container' is a better word for files.= =C2=A0Then that leaves the word format open for use as a way of describing= the data format of an audio frame buffer. =C2=A0So an audio_format class w= ould contain information on sample rate, channels, interleaving, channel ro= uting and other information and a container class would deal with files.

I personally would rather have all audio with = greater than 2 or 3 channels processed in Ambisonic-B format and then have = the audio driver figure out how to extract the channels for each speaker, b= ut I know this won't fly with everyone so we would need a class for cha= nnel routing information as well.

All I can think = of at the moment..



On Frid= ay, June 3, 2016 at 9:08:45 PM UTC-7, alexande...@gmail.com wrote:
I agree, how do you = feel about the concept i proposed in my original about having std audio str= eam classes similar to std::fstream. The std::astream/iastream/oastream wou= ld represent the default audio device on the system as determined by some m= ethod of polling the system. The user could either choose to use them as is= or manually select a different device?

On Friday, June 3, 2016 at 1= 0:51:12 PM UTC-5, Ron wrote:
Well, it needs to start somewhere.=C2=A0 I think having basic buf= fers using std::array or plain old arrays might do for now and get some bas= ic audio recording and playback into C++.=C2=A0 It should be as simple as p= ossible to get to the interfaces on the machine and make them work.=C2=A0 T= he data processing can build onto it with more complex primitives, template= s and classes.=C2=A0 This way there is at least something semi-usable with = code we have today while the rest of the building blocks are being worked o= n.=C2=A0 Just a thought though.



--
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposa= ls+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.= com/a/isocpp.org/d/msgid/std-proposals/d6c30daa-bab3-454d-899d-2ad906fc55df= %40isocpp.org.
------=_Part_2809_1925105233.1465051282404-- ------=_Part_2808_997316795.1465051282403-- .