220 5269 article Path: news.gmane.org!not-for-mail From: minchul park Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: Re: Some thoughts about coroutines Date: Sat, 29 Jun 2013 10:02:58 +0900 Lines: 295 Approved: news@gmane.org Message-ID: References: <291db835-d42d-40cd-8760-47cdf327680d@isocpp.org> <7d8a2d85-0995-4833-ad0c-75577f293b63@isocpp.org> <2c77506f-433f-43b7-871f-e429c4b295bf@isocpp.org> <3e94ef89-4ea8-4549-827f-94dd7a32b9fa@isocpp.org> <1a1fd94c-eb51-4ff4-8e3d-7781a6c9b925@isocpp.org> Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=089e013c6a726a391c04e0408e39 X-Trace: ger.gmane.org 1372467779 18766 80.91.229.3 (29 Jun 2013 01:02:59 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sat, 29 Jun 2013 01:02:59 +0000 (UTC) To: std-proposals@isocpp.org Original-X-From: std-proposals+bncBDH3HYO5YQIBBQ7EXCHAKGQE3W3X3JI@isocpp.org Sat Jun 29 03:03:01 2013 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-ob0-f197.google.com ([209.85.214.197]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1UsjZ6-0003rw-Nl for gclcip-std-proposals@m.gmane.org; Sat, 29 Jun 2013 03:03:01 +0200 Original-Received: by mail-ob0-f197.google.com with SMTP id v19sf9079447obq.0 for ; Fri, 28 Jun 2013 18:02:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-beenthere:mime-version:in-reply-to:references:date:message-id :subject:from:to:x-original-sender:x-original-authentication-results :reply-to:precedence:mailing-list:list-id:x-google-group-id :list-post:list-help:list-archive:list-subscribe:list-unsubscribe :content-type; bh=/2sFH1JDZpP0nnSxr5FJvSyyUKVLZwJ6vsoKRsuCcRc=; b=lzZZx4pj1eHjW5UnVmJHdDFFSbL1Rq4kmk0dkKipQ5SYCcf9s9jZ/JWkqcbXTQtE2n xxMxBgCauehSLxLeLRBX950QL4puhyGG0fTBUcuGgHixZ2HYbfCmVl7WgDieqpqgfPvQ 8mvMJPfzsdYKxMPk2Dkl3C9ssKFAcJDnPPX7OKzfDH9n/JYnTuBNrDK49kuAlKbtwrp1 XbG4Ak3S0snkBovDv89IMbuZXvj3hUesnaUcjOPBdR5l45Y7rXeb4VyQt2ow42cH0Zc0 /zEwYBCNGEieUiPzhjrhe1+dHbsi7BJpwuLtu/BCCNCVdsQlVrmZsd1TnSE84ygXe3HD 9/fw== X-Received: by 10.182.241.161 with SMTP id wj1mr4432773obc.3.1372467779623; Fri, 28 Jun 2013 18:02:59 -0700 (PDT) X-BeenThere: std-proposals@isocpp.org Original-Received: by 10.182.103.164 with SMTP id fx4ls253288obb.44.gmail; Fri, 28 Jun 2013 18:02:59 -0700 (PDT) X-Received: by 10.182.79.68 with SMTP id h4mr7441589obx.68.1372467779102; Fri, 28 Jun 2013 18:02:59 -0700 (PDT) Original-Received: from mail-ie0-x230.google.com (mail-ie0-x230.google.com [2607:f8b0:4001:c03::230]) by mx.google.com with ESMTPS id kk1si2407842oeb.155.2013.06.28.18.02.59 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 28 Jun 2013 18:02:59 -0700 (PDT) Received-SPF: pass (google.com: domain of summerlight00@gmail.com designates 2607:f8b0:4001:c03::230 as permitted sender) client-ip=2607:f8b0:4001:c03::230; Original-Received: by mail-ie0-f176.google.com with SMTP id ar20so5331637iec.21 for ; Fri, 28 Jun 2013 18:02:58 -0700 (PDT) X-Received: by 10.50.3.42 with SMTP id 10mr6323351igz.39.1372467778695; Fri, 28 Jun 2013 18:02:58 -0700 (PDT) Original-Received: by 10.64.62.200 with HTTP; Fri, 28 Jun 2013 18:02:58 -0700 (PDT) In-Reply-To: X-Original-Sender: summerlight00@gmail.com X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of summerlight00@gmail.com designates 2607:f8b0:4001:c03::230 as permitted sender) smtp.mail=summerlight00@gmail.com; dkim=pass header.i=@gmail.com Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:5269 Archived-At: --089e013c6a726a391c04e0408e39 Content-Type: text/plain; charset=ISO-8859-1 Oops, the last code should be fixed as below. void some_protocol(end_point dest) { session s(dest); // some processing s.send(buffer); s.recv(buffer); // deep yielded inside of network library // use received data } On Sat, Jun 29, 2013 at 9:55 AM, minchul park wrote: > For the sake of clarity, it's worth to note that the second proposal is a > description of generator, rather than coroutine. They should be > distinguished. > > Coroutine is not just about a simple programming pattern like > producer/consumer (which is actually external iterator) - It's more > primitive building block to support every kind of cooperative multitasking, > so it should maintain its own continuation. So if you want to call > something coroutine, it should have its own callstack(which represents > continuation in C++). > > Of course, generator has its own advantage. It has much easier to use, and > maybe more efficient on its primary use cases since it's just simple > syntax-sugared state machine functor. Since iterator pattern is widely > adapted in C++ community, it deserves its own syntax. So I'm not against > about generator proposal, but still, it can't be a replacement of coroutine. > > > And I guess you have doubt about necessity of deep yield. Think about > simple IO scenario. Maybe the most simple pseudo code may looks like below. > > void some_protocol(end_point dest) { > session s(dest); > // some processing > s.send(buffer); > s.recv(buffer); // unbounded blocking > > // use received data > } > > But the code has some performance issue. One recv call can blocks other > computation. It can be resolved by mapping one thread to one session , but > it has its own performance problem. So let's introduce some asynchronous > code. > > end_point dest; > session s(dest); // connected and maintained by some other code - its > lifetime is not bounded to the scope. > > void async_recv_handler(session& s) { > // use received data > } > > void async_protocol(session& s) { > // some processing > s.send(buffer); > s.recv(recv_handler); > } > > It's much more complex due to the nature of asynchronous process. Maybe > CPS with lambda can mitigates it, but it's still ugly. However, if we have > "deep yield" backed up by coroutine-based cooperative task scheduler, then > the code becomes drastically simpler. > > void some_protocol(end_point dest) { > session s(dest); > > s.recv(buffer); // deep yielded inside of network library > > // use received data > } > > > The root cause of the problem is strong coupling between continuation and > thread. We want to map one continuation to one conceptual computation > context, but in C++11, thread is the only way to achieve it since they are > strongly coupled. > > The fundamental solution is to decouple continuation from thread and treat > continuation as first-class object. Though it's very hard (or technically > infeasible) to implement it as a first class object in the context of C++, > but coroutine can be the practically sufficient solution which can be > applied almost every cases. The necessity of "deep yield" means this. > > I also want to note that generator just maintains only its own function > frame, which is not sufficient to represent continuation in general use > case. > > > On Fri, Jun 28, 2013 at 10:10 AM, wrote: > >> Another thing - what should *operator()* return when the coroutine is >> done? We can't know it's done before the internal function actually ends, >> so we can't just check *isdone() *(or, maybe, *operator bool()* ) to >> know if we shouldn't call *operator()*. >> (One solution is to let the coroutine not to return control just when >> yielding, but instead continue running until just before the next *return >> * statement, but I think it's kind of evil) >> >> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "ISO C++ Standard - Future Proposals" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to std-proposals+unsubscribe@isocpp.org. >> To post to this group, send email to std-proposals@isocpp.org. >> Visit this group at >> http://groups.google.com/a/isocpp.org/group/std-proposals/. >> >> >> > > > > -- > Min-chul Park (summerlight00@gmail.com / http://summerlight.tistory.com) > -- Min-chul Park (summerlight00@gmail.com / http://summerlight.tistory.com) -- --- You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. Visit this group at http://groups.google.com/a/isocpp.org/group/std-proposals/. --089e013c6a726a391c04e0408e39 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Oops, the last code should be fixed as below.

void some_protocol(end_point dest) {
=A0 session s(dest);

=A0 // some processing
=A0 s.send(buffer);
=A0 s.recv(buffer); //= deep yielded inside of network library

=A0 //= use received data
}


On Sat, Jun 29, 2013 at 9:55 AM, minchul park <summerlight00@gmail.c= om> wrote:
For the sake of clarity, it's worth to note that the s= econd proposal is a description of generator, rather than coroutine. They s= hould be distinguished.

Coroutine is not just about a si= mple programming pattern like producer/consumer (which is actually external= iterator) - It's more primitive building block to support every kind o= f cooperative multitasking, so it should maintain its own continuation. So = if you want to call something coroutine, it should have its own callstack(w= hich represents continuation in C++).

Of course, generator has its own advantage. It has much= easier to use, and maybe more efficient on its primary use cases since it&= #39;s just simple syntax-sugared state machine functor. Since iterator patt= ern is widely adapted in C++ community, it deserves its own syntax. So I= 9;m not against about generator proposal, but still, it can't be a repl= acement of coroutine.


And I guess you have doubt about necessi= ty of deep yield. Think about simple IO scenario. Maybe the most simple pse= udo code may looks like below.

void some_pro= tocol(end_point dest) {
=A0 session s(dest);
=A0 // some processing
=A0 s.= send(buffer);
=A0 s.recv(buffer); // unbounded blocking
=

=A0 // use received data
}

But the code has some performance issue. On= e recv call can blocks other computation. It can be resolved by mapping one= thread to one session , but it has its own performance problem. So let'= ;s introduce some asynchronous code.

end_point dest;
session s(dest); // connected= and maintained by some other code - its lifetime is not bounded to the sco= pe.

void async_recv_handler(session& s) {
=A0 // use received data
}

vo= id async_protocol(session& s) {
=A0 // some processing
=A0 s.send(buffer);
=A0 s.recv(recv_handler);
}

It's muc= h more complex due to the nature of asynchronous process. Maybe CPS with la= mbda can mitigates it, but it's still ugly. However, if we have "d= eep yield" backed up by coroutine-based cooperative task scheduler, th= en the code becomes drastically simpler.

void some_protocol(end_point dest) {
=A0= session s(dest);

=A0 s.recv(buffer); // deep yiel= ded inside of network library

=A0 // use recei= ved data
}


The root cause of the proble= m is strong coupling between continuation and thread. We want to map one co= ntinuation to one conceptual computation context, but in C++11, thread is t= he only way to achieve it since they are strongly coupled.

The fundamental solution is to decouple continuation fr= om thread and treat continuation as first-class object. Though it's ver= y hard (or technically infeasible) to implement it as a first class object = in the context of C++, but coroutine can be the practically sufficient solu= tion which can be applied almost every cases. The necessity of "deep y= ield" means this.

I also want to note that generator just maintains only = its own function frame, which is not sufficient to represent continuation i= n general use case.=A0


On Fri, Jun 28, 2013 at 10:10 AM, <asae= lr@gmail.com> wrote:
Another thing - what should operator() return when the coroutine is = done? We can't know it's done before the internal function actually= ends, so we can't just check isdone() (or, maybe, operator b= ool() ) to know if we shouldn't call operator().
(One solution is to let the coroutine not to return control just when yield= ing, but instead continue running until just before the next return = statement, but I think it's kind of evil)

--
=A0
---
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
Visit this group at http://groups.google.com/a/isocpp.org/gro= up/std-proposals/.
=A0
=A0



<= /div>--



--

--
 
---
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
Visit this group at http://groups.google.com/a/isocpp.org/group/std-proposals/<= /a>.
 
 
--089e013c6a726a391c04e0408e39-- .