220 40687 article Path: news.gmane.org!.POSTED!not-for-mail From: Ryan McDougall Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: The need for std::unique_function Date: Tue, 23 Oct 2018 09:17:45 -0700 Lines: 1590 Approved: news@gmane.org Message-ID: References: Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="0000000000007e6af00578e7b6b2" X-Trace: blaine.gmane.org 1540311355 29493 195.159.176.226 (23 Oct 2018 16:15:55 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Tue, 23 Oct 2018 16:15:55 +0000 (UTC) Cc: bbudge@fb.com To: std-proposals@isocpp.org Original-X-From: std-proposals+bncBDS4B5X37MCRBN4TXXPAKGQEE5NJUTA@isocpp.org Tue Oct 23 18:15:51 2018 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-wm1-f70.google.com ([209.85.128.70]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gEzLS-0007aI-64 for gclcip-std-proposals@m.gmane.org; Tue, 23 Oct 2018 18:15:50 +0200 Original-Received: by mail-wm1-f70.google.com with SMTP id y185-v6sf1572873wmg.6 for ; Tue, 23 Oct 2018 09:18:01 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1540311481; cv=pass; d=google.com; s=arc-20160816; b=NRg2ABBFOyK9La1NV7Dngs2hF5Edub26cJUzZln671hzJcY4FprHIRVEwjPmwKMszg AiQgC767TQyr4a4r2CQaJyxyeZ9uslFLvRFfp4+5eVayPTs/lu+/ls5TzekxUlmz/Lwu we1xUUHsmtygBy1Vkm6dNsGXK13r+Hydxz21nUaX3LlCLCkgP/8DOeaUVPEZ7fTn3WV0 /+mNNXmRITpak96YwuE81UKk5h05bhWShn86sz0TrWS0/gEjbbzaLS8KaXIQT/k38BMZ DevZvhuUoftM6WmETh7ccIJKsUcES176CIXc6LA63I/6FbUFcCi/vajP6E87zbcG4Nq4 bXEQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:cc:to:subject:message-id :date:from:in-reply-to:references:mime-version:dkim-signature; bh=E1qukJzhESJSO6oVkTYTLrInMYQeh+fwScMFIsOxnbE=; b=SRoGZbluPjsA+0AgxH/NnudF9yr9fLzu8DZdNHtaOVKWXfqSikcOIoZbzR9C024q74 +ok/n97EUJFJ+Rxo5D69lJmIsGowIxhnakhnwF5JexXnjxa6URI0fIaYH7+VERV5aJ0C aq/tqZu8aw32rNAsdy8iaxxMYiBtVHw3X4Vqyv1s//Ye/vEUzPzVC9oLsC7HwLACJhww HI4TxOmZTCdguVNtk1NsKqyHIX4z/2ysx9m1V18oG8wattfSuOJ/IeYy7f0fQ76jUDfP j9oVVE1MySdmMAacj7Ij61HNT+hFy50SWnTBhqnBnjVuedErUpYAF23jLwxBzNlF2GbL U07Q== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=F0brPC7v; spf=pass (google.com: domain of mcdougall.ryan@gmail.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=mcdougall.ryan@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isocpp-org.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:x-original-sender:x-original-authentication-results:reply-to :precedence:mailing-list:list-id:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=E1qukJzhESJSO6oVkTYTLrInMYQeh+fwScMFIsOxnbE=; b=rkTKpVO8Thk2C+kHqvez6FsBcn4lYA6ZCqMHh1vkL8DDLFEyziVx2eybsh/SpiFSJC pwaP+0hNrDO0Iq8+kAlRILndJ4TEQZ5Ux9Io7LnHfpI9AxamOJjTEunwpDaqmFhhectg Yi8oDXM/hfaEgPqAiGgncXoUosRXtgzuxxOonfBGUMOsWQKwA+/HIhDDpeRIzr4aZ2+o i+4O1b4en5dGFtUO7igNnCYPaUdRaIe/qhItc2g8UACTk8G33DeDbj1LVB2IqcRJdwZZ EYRV3VyCi3XtOtH28AG+g/A50gRabPVppNkBfnAWvE0xY8QgLlkkkd2zmpCRzLF34kNU A/Pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=E1qukJzhESJSO6oVkTYTLrInMYQeh+fwScMFIsOxnbE=; b=d8jVbL/eC3nxy/4VB5sLHoiRze3CShorEMe5aA/xKUegh7fuKXo/5D8VOBPqXGqucT RXaCd3UGQA33SgKgO2J8Px0CzDw6mocMvHevQH/f34A+S4ZeiTcbKRnreZefL/MtXptQ fn24xEavl8JQ0dg4WdLvZE9YtBoqYDqYjJ2H0bTK368a8TJbWMPaW+PU1+6ZNtwbXsOX g0DVlYx4HFW8VidA1H1tOwehDUTnuBiLDvlt9k2aOcV0ZQepmDzB2X0Luqo+omewtaNM TT1fEsfyff1HK6SZ1c4iqaPf6cQASJuOWI21oow3PXxsQzDWfifKlHIqLc2/x6YBWnhV Hqpw== X-Gm-Message-State: ABuFfogINNgQgObtxlQtSdwgZ9t7UiFXz+8HCTUgXSxmdUOXMJx20JK6 LoD8VHvjSotzPe0nEC0hPRWyPg== X-Google-Smtp-Source: ACcGV60WvC4zFp2gOGikOuVn8IVgca+EUYdeVmdoILrYEnkMcCJD3kx+3MCG9OYn6W2ykFL4hOMuYg== X-Received: by 2002:a1c:d8:: with SMTP id 207-v6mr2309999wma.12.1540311480826; Tue, 23 Oct 2018 09:18:00 -0700 (PDT) X-BeenThere: std-proposals@isocpp.org Original-Received: by 2002:adf:8b97:: with SMTP id o23-v6ls891318wra.15.gmail; Tue, 23 Oct 2018 09:17:59 -0700 (PDT) X-Received: by 2002:a5d:420d:: with SMTP id n13-v6mr25864560wrq.201.1540311479199; Tue, 23 Oct 2018 09:17:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1540311479; cv=none; d=google.com; s=arc-20160816; b=bVAjtL+Z8BJ46dzSdhARhwozKhX1h1NmN1bX5Vneo96pEtJBvexY4BlyQXcJO7DcVj sbz6/vkmoKQFshXpFPMpU/WHm9EgGCxRu/AJTsXj9N4JD0A7sFy2QhxbbJX4s8mnqGAa lFyDW7juQw2px+iVYRewZE4bbrVayqH5ZUnAHfyiHqmIJi+dUsEQAAxZAvlu/pH8mjaP RBB8Zxhoq1tNMJChOOBXtIHyKXk3f4sTZH0J+kBbwBamygEfat/1iVUms2khPXxu1lNU wMEPkNcdDdb3a3yiw/yzTVJcUjlYZt1kmR0zSn+JhI2v5fxxw7jDpzHYRG4FkooP90fS CAyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=jzXvbV7DkfkiHJ0xNRBrnYKdTt9m9eGtuOHp4CKSFhw=; b=irA/Tu6f8I38v4fakkq65eg9q/CuTPfjtNgIAn1qnD+qUnKFoO/Uu0Sq4SKD4jr5rq D15MaKymmO7iGUu/T3T1SiP7YWSOicVcjzKuYC4fSzvlYhicHKRu2ANO7IOXD/VkQf4D dWzgblHlvEnFp0N/6SB6phZKmfe8HVKy2WNI4NzvFJu/y1IFmsC+J6MIxlFe9j1Kbg6g wCQy/CFJBvPoERiE/CyzSQnHvWM2LXgCIXTWfwrFbgzxbJBE4IrOpV4WFxskUjnoTee/ /hUkV2QXiGIal3N8z3uWyqbVHPsMYKqnfYaPCg87dWcq9Pe/wPkXOr6wov1YVyR5Hq5t IrVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=F0brPC7v; spf=pass (google.com: domain of mcdougall.ryan@gmail.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=mcdougall.ryan@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Original-Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id 6-v6sor1388096wmy.21.2018.10.23.09.17.59 for (Google Transport Security); Tue, 23 Oct 2018 09:17:59 -0700 (PDT) Received-SPF: pass (google.com: domain of mcdougall.ryan@gmail.com designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; X-Received: by 2002:a1c:2dc5:: with SMTP id t188-v6mr19965422wmt.94.1540311478068; Tue, 23 Oct 2018 09:17:58 -0700 (PDT) In-Reply-To: X-Original-Sender: mcdougall.ryan@gmail.com X-Original-Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=F0brPC7v; spf=pass (google.com: domain of mcdougall.ryan@gmail.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=mcdougall.ryan@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Spam-Checked-In-Group: std-proposals@isocpp.org X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:40687 Archived-At: --0000000000007e6af00578e7b6b2 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Yes, I'll be at SD in order to push this paper -- thought I don't know if there's room on anyone schedule. I have hope however since it's relatively uncontroversial it can yet be fast tracked. I would very much like help with wording if you're available. Cheers, On Tue, Oct 23, 2018 at 9:14 AM Lee Howes wrote: > Hi Ryan, > Are you planning to attend the meeting and argue the case for > unique_function? This is something that came up again this week for us at > Facebook as well, and while we work around it with folly::Function as you > noted, having to do so is not optimal. It would certainly be a good thing > to get a unique_function into the standard soonish. > > Lee Howes > > On Mon, 22 Oct 2018 at 11:19, Ryan McDougall > wrote: > >> =E2=80=9CThis is motivated by increasing usage of things like executors = and the >> task >> >> queue where it is useful to embed move-only types like a std::promise >> within >> >> a type erased function. That isn't possible without this version of a ty= pe >> >> erased function.=E2=80=9D -- Chandler Carruth[1] >> Thanks >> >> =E2=80=9Cstd::function and Beyond=E2=80=9C N4159 >> , >> =E2=80=9CQualified std::function Signatures=E2=80=9D P0045R1 >> , >> and =E2=80=9CA polymorphic wrapper for all Callable objects (rev. 3)=E2= =80=9D P0288R1 >> et >> al. for laying the groundwork for this paper. Thank you to David Krauss = and >> Arthur O=E2=80=99Dwyer for your discussion and feedback. Thank you all a= uthors of >> alternative type erased callable containers for your proof and inspirati= on. >> Motivation >> >> std::function models both CopyConstructible and CopyAssignable, and >> requires its erased target type to declare a copy constructor. This mean= s >> std::function is hopelessly incompatible with std::unique_ptr, >> std::promise, or any other move-only type. This is a functional gap felt= by >> C++ users to the degree that there=E2=80=99s some half-dozen popular and >> high-quality implementations available under the name =E2=80=9Cunique_fu= nction=E2=80=9D. >> [2][3][4][5][6] >> >> C++ has many move-only vocabulary types, and when introduced they impose >> tighter constraints on the interface, and can become =E2=80=9Cviral=E2= =80=9D -- causing any >> previously copyable paths to require move or forwarding operations. Cons= ider >> >> class DbResult { >> >> private: >> >> std::unique_ptr data_; // now required! >> >> }; >> >> class Reactor { >> >> private: >> >> std::map> reactions_; >> >> }; >> >> reactor.on_event(=E2=80=9Cdb-result-ready=E2=80=9D, [&] { >> >> reactor.on_event(=E2=80=9Chas-towel=E2=80=9D, [result =3D std::move(db.= result())] { >> >> auto meaning =3D find(result); // 42 >> >> }); >> >> }); >> >> It is not enough to simply std::move the DbResult into the lambda, as >> Reactor::on_event is unable to assign to a move-only lambda as implement= ed >> with std::function. >> >> This is a recurring pattern in much concurrent code, such as work queues= , >> task graphs, or command buffers. The committee implicitly understood the >> need when it created std::packaged_task, also a type-erased polymorphic >> container, but that type is tightly bound to std::futures, which may not= be >> fit for purpose in any code base that doesn=E2=80=99t already rely on th= em. >> >> If we are developing any kind of asynchronous work queue we need >> >> 1. >> >> inheritance based polymorphism (and a mechanism to manage heap >> allocated derived objects, such as std::shared_ptr) >> 2. >> >> type-erased container with small object optimization like >> std::function (for copy-only callable types), >> std::packaged_task+std::future (for move-only callable types) >> >> >> However if any facet of our runtime precludes use of std::future -- such >> as if it provides its own future type, or does not use futures at all, w= e >> are again left without the ability to use std::unique_ptr or any other >> non-copyable type. >> >> auto unique =3D >> >> std::make_unique(=E2=80=9CDrEvil=E2=80=9D, 1000000000); >> >> auto do_bank_transfer =3D >> >> [transfer =3D std::move(unique)] (Bank* to, Bank* from) { >> >> return from->send(transfer, to); >> >> }; >> >> ThreadSafeQueue> transactions1; >> >> transactions1.emplace(do_bank_transfer); // Compile Error!! >> >> // ... >> >> ThreadSafeQueue> transactions2; >> >> ThreadSafeQueue results >> >> transactions2.emplace(do_bank_transfer); >> >> hpy::async([&] { >> >> while (!transactions2.empty()) { >> >> transactions2.top()(); >> >> results.push_back(transactions2.top().get_future()); // ?? >> >> } >> >> }); >> >> In the above example we simply present wasted human time and >> computational time due to an unnecessary synchronization with std::futur= e, >> however a more complex system may indeed need their own future >> implementation which std::packaged_task cannot interoperate with at all. >> Comparison Table >> >> std::promise p; >> >> std::unique_ptr f; >> >> std::queue> a; >> >> std::queue> b; >> >> using namespace std; >> >> >> Before Proposal >> >> After Proposal >> >> 1 >> >> auto s =3D make_shared(move(f)); >> a.emplace([s] { s->work(); }); >> >> >> // ... >> >> auto shared =3D a.top(); >> >> shared(); >> >> b.emplace([u =3D move(f)] { >> >> u->work(); >> >> }); >> >> // ... >> >> auto unique =3D move(b.top()); >> >> unique(); >> >> 2 >> >> a.emplace([r =3D f.get()] { >> >> r->work(); >> >> }); >> >> b.emplace([u =3D move(f)] { >> >> u->work(); >> >> }); >> >> 3 >> >> atomic result{nullptr}; >> >> a.emplace([&result] { >> >> result =3D new Foo; >> >> }); >> // ... >> spin_on(result); >> result->work(); >> >> auto future =3D p.get_future(); >> >> b.emplace([p =3D move(p)] { >> >> p.set_value(Foo{}); >> >> }); >> >> // ... >> >> future.get().work(); >> >> As you can see, attempts to work around the limitation of std::function >> results in unacceptable undermining of uniqueness semantics, life-time >> safety, and/or ease of use: >> >> 1. >> >> Loss of move-only semantics and extra LoC and a heap allocation per >> shared pointer. >> 2. >> >> Queue lifetime must not exceed enclosing scope lifetime (or dangling >> pointer). >> 3. >> >> Poor usability or unnecessary complexity in search of better >> performance. >> >> Alternatives >> >> Papers =E2=80=9Cstd::function and Beyond=E2=80=9C N4159 >> , =E2= =80=9CA >> polymorphic wrapper for all Callable objects (rev. 3)=E2=80=9D P0288R1 >> >> have already argued for fixing std::function to support non-copyable typ= es >> (among other issues) some time ago. Yet it seems self evident that as lo= ng >> and as widely as std::function has been in use, any change that breaks t= he >> observable surface area of std::function is a non-starter. The question = is >> would the use case we outlined herein, if overlaid onto the existing >> std::function, cause previously valid code to break, or conversely >> previously incorrect code to become easily written? >> >> Let=E2=80=99s assume we have a means of =E2=80=9Cfixing=E2=80=9D std::fu= nction to allow mixing of >> copy and move semantics. Clearly all existing code would continue to >> function as all existing target callable types are CopyConstructible and >> CopyAssignable, but what if we mix erased instances with copy and move >> semantics? How should the following operations on std::function be defin= ed >> in terms of their target callable types? >> >> Let=E2=80=99s temporarily ignore the details of memory management, and c= onsider p >> to be the underlying pointer to the erased instance. >> >> std2::function c =3D C{}; // Copy-only >> std2::function m =3D M{}; // Move-only >> >> >> Operation >> >> Definition >> >> Result >> >> c =3D std::move(m); >> >> *((M*)c.p) =3D move(*((M*)m.p)); >> >> Move >> >> m =3D std::move(c); >> >> *((C*)c.p) =3D move(*((C*)m.p)); >> >> Copy >> >> c =3D m; >> >> *((M*)c.p) =3D *((M*)m.p); >> >> Throw ??? >> >> m =3D c; >> >> *((C*)c.p) =3D *((C*)m.p); >> >> Copy >> >> Again we face an unacceptable undermining of uniqueness semantics; in >> addition we have changed the observable surface area of assignment by >> necessitating a new runtime error reporting mechanism for when erased >> targets conflicting behavior, affecting the exception-safety of existing >> code. >> Shallow v. Deep Const >> >> =E2=80=9CShallow=E2=80=9D or =E2=80=9CDeep=E2=80=9D const in a type eras= ed context means whether the >> constness of the erasing type is extended towards the erased type. For o= ur >> discussion we consider whether the the erasing container is const callab= le >> if and only if the underlying type is const callable. >> >> >> It is by now understood that the standard requires const correctness to >> imply thread-safety, and if the container operator() is const, the >> underlying callable type=E2=80=99s operator() must also be const in orde= r to hope >> of satisfying this obligation. So a shallow const container could not ad= mit >> a thread-safe call operation in general, and both N4159 >> and >> P0045R1 >> >> draw attention to the unfortunate outcome. The solution presented in tho= se >> papers it to include the constness of the callable in signature, and to >> have the constness of container operator() be conditional on the signatu= re. >> >> struct Accessor { >> >> void operator()() const; >> >> // ... >> >> } access; >> >> struct Mutator { >> >> void operator()(); >> >> // ... >> >> } mutate; >> >> std3::function a =3D access; // ok >> >> std3::function b =3D access; // ok >> >> std3::function c =3D mutate; // ok >> >> std3::function d =3D mutate; // compile error >> >> a =3D b; // compile error: target type more restrictive >> >> b =3D a; // ok >> >> b =3D c; // ok >> >> >> This proposal, while in the author=E2=80=99s opinion is highly recommend= ed >> improvement, it=E2=80=99s best presented in referenced papers, and for s= implicity=E2=80=99s >> sake isn=E2=80=99t pursued further here. >> >> Otherwise, without the above but desiring deep const semantics, we would >> wish that const containers require const callable targets. However, sinc= e >> we have erase the constness of target instance, we are left with throwin= g >> when deep const is violated, breaking existing exception-safe code. >> >> const std4::function a; // deep const >> >> std::function b =3D mutate; // non-const target >> >> a =3D b; // target copied >> >> a(); // now throws! >> Necessity and Cost >> >> We can all agree type erased containers should support as much as >> feasible the const correctness of its target types, but we began our >> argument with a specific asynchronous use case that involved deferred >> computations, often invoked on foreign threads. If this pattern as seen = =E2=80=9Cin >> the wild=E2=80=9D makes use of thread safe queues, is thread safety of t= he >> container itself actually sufficient to justify the costs associated wit= h >> extra synchronization or exception safety? Even if we can guarantee cons= t >> container only calls const target methods, we still cannot guarantee the >> target itself makes the connection between const and thread safety. >> >> Should a proposed std::unique_function emulate =E2=80=9Cbroken=E2=80=9D = shallow const >> correctness of std::function for uniformity, or is =E2=80=9Cfixing=E2=80= =9D the >> contradiction worth breaking runtime changes? And is std::unique_/functi= on >> more like a container or pointer (where const is understood to be shallo= w), >> or is it more like an opaque type (where const is generally considered t= o >> be deep)? Historically we allow container const to vary independently of >> parameterized types when they exist, suggesting std::function should rem= ain >> a shallow container. >> >> While this question is relevant to the specification of >> std::unique_function, it is ultimately orthogonal to the question of the >> need for std::unique_function to exist, which is this paper=E2=80=99s ma= in concern. >> Conclusion >> >> So long as we have move-only lambdas and type erased callable containers= , >> a move-only capable std::function is required. As uniqueness semantics o= f >> container are orthogonal to const-correctness of the interface we recomm= end >> not conflating the two, and pursuing P0045R1 >> >> for const-correctness as an independent feature request. >> References >> >> [1] https://reviews.llvm.org/D48349 >> >> [2] http://llvm.org/doxygen/FunctionExtras_8h_source.html#l00046 >> >> [3] >> https://github.com/STEllAR-GROUP/hpx/blob/master/hpx/util/unique_functio= n.hpp >> >> [4] >> https://github.com/potswa/cxx_function/blob/master/cxx_function.hpp#L119= 2 >> >> [5] >> https://github.com/Naios/function2/blob/master/include/function2/functio= n2.hpp#L1406 >> >> [6] https://github.com/facebook/folly/blob/master/folly/Function.h >> >> -- >> You received this message because you are subscribed to the Google Group= s >> "ISO C++ Standard - Future Proposals" group. >> To unsubscribe from this group and stop receiving emails from it, send a= n >> email to std-proposals+unsubscribe@isocpp.org. >> To post to this group, send email to std-proposals@isocpp.org. >> To view this discussion on the web visit >> https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/CAHn%2BA5Ny= aUQjAeudtObjn08Trz4iJ%2Bu-GGt2J5VnmtmZJiPVxg%40mail.gmail.com >> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "ISO C++ Standard - Future Proposals" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to std-proposals+unsubscribe@isocpp.org. > To post to this group, send email to std-proposals@isocpp.org. > To view this discussion on the web visit > https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/CAJH_FNW1PVw= 2BeYJCWfKNKN3_67eLY_6wG1vgNJOZx9oQS%2Bs%3DQ%40mail.gmail.com > > . > --=20 You received this message because you are subscribed to the Google Groups "= ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. To view this discussion on the web visit https://groups.google.com/a/isocpp= ..org/d/msgid/std-proposals/CAHn%2BA5NHiEO1rTKfZOryWnW-NTqLGgim6epp-Bt03iU5w= d-EEA%40mail.gmail.com. --0000000000007e6af00578e7b6b2 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Yes, I'll be at SD in order to push this paper -- thou= ght I don't know if there's room on anyone schedule. I have hope ho= wever since it's relatively uncontroversial it can yet be fast tracked.= I would very much like help with wording if you're available.

=
Cheers,

On Tue, Oct 23, 2018 at 9:14 AM Lee Howes <xrikcus@gmail.com> wrote:
Hi Ryan,
Are you planning to attend the meeti= ng and argue the case for unique_function? This is something that came up a= gain this week for us at Facebook as well, and while we work around it with= folly::Function as you noted, having to do so is not optimal. It would cer= tainly be a good thing to get a unique_function into the standard soonish.<= /div>

Lee Howes

On Mon, 22 Oct 2018 at 11:19, Ryan McDougall <mcdougall.ryan@gmail= ..com> wrote:

=E2=80=9CThis is motivated by increasing usage of thing= s like executors and the task

queue where it is useful to embed move-only types like a std:= :promise within

= a type erased function. That isn't possible without this version of a t= ype

erased funct= ion.=E2=80=9D -- Chandler Carruth[1]

Thanks

=E2=80=9Cstd::function and Beyond=E2=80=9C <= /span>N4159, =E2=80=9CQualified std::functio= n Signatures=E2=80=9D P0045R1, and = =E2=80=9CA polymorphic wrapper for all Callable objects (rev. 3)=E2=80=9D <= /span>P0288R1 et al. for laying the groundw= ork for this paper. Thank you to David Krauss and Arthur O=E2=80=99Dwyer fo= r your discussion and feedback. Thank you all authors of alternative type e= rased callable containers for your proof and inspiration.

Motivation

std::function model= s both CopyConstructible and CopyAssignable, and requires its erased target= type to declare a copy constructor. This means std::function =C2=A0is hope= lessly incompatible with std::unique_ptr, std::promise, or any other move-o= nly type. This is a functional gap felt by C++ users to the degree that the= re=E2=80=99s some half-dozen popular and high-quality implementations avail= able under the name =E2=80=9Cunique_function=E2=80=9D. [2][3][4][5][6]


C++ has many mov= e-only vocabulary types, and when introduced they impose tighter constraint= s on the interface, and can become =E2=80=9Cviral=E2=80=9D -- causing any p= reviously copyable paths to require move or forwarding operations. Consider=


class DbResult {

private:

=C2=A0std::unique_ptr<Blob> da= ta_; =C2=A0// now required!

};


class Reactor {

private:

=C2= =A0std::map<std::string, std::function<void()>> reactions_;

}= ;


reactor.on_event(=E2=80=9Cdb-result-ready=E2=80=9D, [&] {

= =C2=A0reactor.on_event(=E2=80=9Chas-towel=E2=80=9D, [result =3D std::move(d= b.result())] {

=C2=A0=C2=A0=C2=A0auto meaning =3D find(result); // 42

= =C2=A0});

});


It is not enough to simply std::move the DbResult into the lambda, as = Reactor::on_event is unable to assign to a move-only lambda as implemented = with std::function.


This is a recurring pattern in much concurrent code, such as work q= ueues, task graphs, or command buffers. The committee implicitly understood= the need when it created std::packaged_task, also a type-erased polymorphi= c container, but that type is tightly bound to std::futures, which may not = be fit for purpose in any code base that doesn=E2=80=99t already rely on th= em.


If we ar= e developing any kin= d of asynchronous work queue we need

  1. inheritance based polymorphis= m (and a mechanism to manage heap allocated derived objects, such as std::s= hared_ptr)

  2. type-erased container w= ith small object optimization= like std::function (for copy-only callable types), std::pack= aged_task+std::future (for move-only callable types)

However if any facet of = our runtime precludes use of std::future -- such as if it provides its own = future type, or does not use futures at all, we are again left without the = ability to use std::unique_ptr or any other non-copyable type.


auto uniqu= e =3D

=C2=A0std::make_unique<BankTransfer>(=E2=80=9CDrEvil=E2=80= =9D, 1000000000);


auto do_bank_transfer =3D

=C2=A0[transfer =3D st= d::move(unique)] (Bank* to, Bank* from) {

=C2=A0=C2=A0=C2=A0return from-= >send(transfer, to);

};


ThreadSafeQueue<std::function<int(= )>> transactions1;

transactions1.emplace(do_bank_transfer); =C2=A0/= / Compile Error!!


// ...


ThreadSafeQueue<std::packaged_task&= lt;int()>> transactions2;

ThreadSafeQueue<int> results=

tran= sactions2.emplace(do_bank_transfer);


hpy::async([&] {

<= p dir=3D"ltr" style=3D"line-height:1.38;margin-top:0pt;margin-bottom:0pt"><= span style=3D"font-size:11pt;font-family:"Courier New";color:rgb(= 0,0,0);background-color:transparent;font-variant-numeric:normal;font-varian= t-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"> =C2=A0wh= ile (!transactions2.empty()) {

=C2=A0=C2=A0=C2=A0transactions2.top()();<= /span>

=C2=A0}

});


In the above example we simply present wasted human time and comp= utational time due to an unnecessary synchronization with std::future, howe= ver a more complex system may indeed need their own future implementation w= hich std::packaged_task cannot interoperate with at all.

Comparison Tabl= e

std::promise<Foo> p;

std::unique_ptr<Foo> f;

<= p dir=3D"ltr" style=3D"line-height:1.2;margin-top:0pt;margin-bottom:0pt">std::queue= <std::function<void()>> a;

std::queue<std::unique_function&= lt;void()>> b;

using namespace std;



Before Proposal

After Proposal

1

auto s= =3D make_shared<Foo>(move(f));
a.emplace([s] { s->work(); });

<= br class=3D"m_-1335907127606806884m_-3924309839588613576gmail-kix-line-brea= k">auto shared =3D a.top();

shared();

b.emplace([u =3D move(f)] {

=C2=A0u->work();

});

<= p dir=3D"ltr" style=3D"line-height:1.2;margin-top:0pt;margin-bottom:0pt">// ...

au= to unique =3D move(b.top());

unique();

2

a.emplace([r =3D f.get()= ] {

=C2=A0r->work();

});

= b.emplac= e([u =3D move(f)] {

=C2=A0u->work();

});

3

atomic<Foo*&= gt; result{nullptr};

a.emplace([&result] {

=C2=A0result =3D new Foo;=


// ...
spin_on(result);
result->work();

auto future =3D p.get_future();

b.emplace([p =3D move(p)] {

= =C2=A0p.set_value(Foo{});

});

// ...

future.get().work();


As you can see, attempts to work around the limitation of std::funct= ion results in unacceptable undermining of uniqueness semantics, life-time = safety, and/or ease of use:

  1. Loss of move-only semantics and extra LoC a= nd a heap allocation per shared pointer.

  2. Queue lifetime must not exceed enclosing scope lifetime (or dangling poi= nter).

  3. Poor usability or unnecessary c= omplexity in search of better performance.

Alternatives<= /h1>

Papers =E2=80=9Cstd::f= unction and Beyond=E2=80=9C N4159, =E2= =80=9CA polymorphic wrapper for all Callable objects (rev. 3)=E2=80=9D P0288R1 have already argued for fixing= std::function to support non-copyable types (among other issues) some time= ago. Yet it seems self evident that as long and as widely as std::function= has been in use, any change that breaks the observable surface area of std= ::function is a non-starter. The question is would the use case we outlined= herein, if overlaid onto the existing std::function, cause previously vali= d code to break, or conversely previously incorrect code to become easily w= ritten?


Let= =E2=80=99s assume we have a means of =E2=80=9Cfixing=E2=80=9D std::function= to allow mixing of copy and move semantics. Clearly all existing code woul= d continue to function as all existing target callable types are CopyConstr= uctible and CopyAssignable, but what if we mix erased instances with copy a= nd move semantics? How should the following operations on std::function be = defined in terms of their target callable types?


Let=E2=80=99s temporarily ignore the = details of memory management, and consider p to be the underlying pointer to the erased instanc= e.


std2::function<void()> c =3D C{}; // Copy-only
std2::functi= on<void()> m =3D M{}; // Move-only




Again we fac= e an unacceptable undermining of uniqueness semantics; in addition we have = changed the observable surface area of assignment by necessitating a new ru= ntime error reporting mechanism for when erased targets conflicting behavio= r, affecting the exception-safety of existing code.

Shallow v. Deep Const=

=E2=80=9CShall= ow=E2=80=9D or =E2=80=9CDeep=E2=80=9D const in a type erased context means = whether the constness of the erasing type is extended towards the erased ty= pe. For our discussion we consider whether the the erasing container is con= st callable if and only if the underlying type is const callable.


It is by now understood that the standar= d requires const correctness to imply thread-safety, and if the container o= perator() is const, the underlying callable type=E2=80=99s operator() must = also be const in order to hope of satisfying this obligation. So a shallow const container coul= d not admit a thread-safe call operation in general, and both N4159 and P0045R1 draw attention to the unfortunate outcome. The solution presented in = those papers it to include the constness of the callable in signature, and = to have the constness of container operator() be conditional on the signatu= re.

<= /p>

struc= t Accessor {

=C2=A0void operator()() const;

=C2=A0// ...

} access;<= /span>


struct Mutator {

=C2=A0void operator()();

=C2=A0// ...

} muta= te;


std3::function<void() const> a =3D access; =C2=A0// ok

st= d3::function<void()> b =3D access; =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0// ok

std3::function<void()> c =3D mutate; =C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0// ok

std3::function<void() const> d =3D= mutate; =C2=A0// compile error


a =3D b; =C2=A0// compile error: targ= et type more restrictive

b =3D a; =C2=A0// ok

b =3D c; =C2=A0// ok


This proposal, while in the author= =E2=80=99s opinion is highly recommended improvement, it=E2=80=99s best pre= sented in referenced papers, and for simplicity=E2=80=99s sake isn=E2=80=99= t pursued further here.


Otherwise, without the above but desiring deep const semantics,= we would wish that const containers require const callable targets. Howeve= r, since we have erase the constness of target instance, we are left with t= hrowing when deep const is violated, breaking existing exception-safe code.=


const std4::function<void()> a; =C2=A0=C2=A0=C2=A0// deep cons= t

std::function<void()> b =3D mutate; =C2=A0// non-const target

a = =3D b; =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0// target copied

a(); =C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0// now throws!

Necessity and Cost

We can all agree type erased containers should= support as much as feasible the const correctness of its target types, but= we began our argument with a specific asynchronous use case that involved = deferred computations, often invoked on foreign threads. If this pattern as= seen =E2=80=9Cin the wild=E2=80=9D makes use of thread safe queues, is thr= ead safety of the container itself actually sufficient to justify the costs= associated with extra synchronization or exception safety? Even if we can = guarantee const container only calls const target methods, we still cannot = guarantee the target itself makes the connection between const and thread s= afety.


Shoul= d a proposed std::unique_function emulate =E2=80=9Cbroken=E2=80=9D shallow = const correctness of std::function for uniformity, or is =E2=80=9Cfixing=E2= =80=9D the contradiction worth breaking runtime changes? And is std::unique= _/function more like a container or pointer (where const is understood to b= e shallow), or is it more like an opaque type (where const is generally con= sidered to be deep)? Historically we allow container const to vary independ= ently of parameterized types when they exist, suggesting std::function shou= ld remain a shallow container.


While this question is relevant to the specification of = std::unique_function, it is ultimately orthogonal to the question of the need for std::unique_f= unction to exist, which is this paper=E2=80=99s main concern.

<= span style=3D"font-size:20pt;font-family:Arial;color:rgb(0,0,0);background-= color:transparent;font-weight:400;font-variant-numeric:normal;font-variant-= east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Conclusion<= /span>

So long as we h= ave move-only lambdas and type erased callable containers, a move-only capa= ble std::function is required. As uniqueness semantics of container are ort= hogonal to const-correctness of the interface we recommend not conflating t= he two, and pursuing = P0045R1 for con= st-correctness as an independent feature request.

References

[1] htt= ps://reviews.llvm.org/D48349

[2] http://= llvm.org/doxygen/FunctionExtras_8h_source.html#l00046

[3] https://github.com/STEllAR-GROUP/hpx/blob/master/hpx/util/= unique_function.hpp

[4] htt= ps://github.com/potswa/cxx_function/blob/master/cxx_function.hpp#L1192

[5] https://github.com/Naios/functio= n2/blob/master/include/function2/function2.hpp#L1406

[6] https://github.com/facebook/folly/blob/master/folly/Function.h=


--
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-p= roposals/CAHn%2BA5NyaUQjAeudtObjn08Trz4iJ%2Bu-GGt2J5VnmtmZJiPVxg%40mail.gma= il.com.

--
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-p= roposals/CAJH_FNW1PVw2BeYJCWfKNKN3_67eLY_6wG1vgNJOZx9oQS%2Bs%3DQ%40mail.gma= il.com.

--
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposa= ls+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit h= ttps://groups.google.com/a/isocpp.org/d/msgid/std-proposals/CAHn%2BA5NHiEO1= rTKfZOryWnW-NTqLGgim6epp-Bt03iU5wd-EEA%40mail.gmail.com.
--0000000000007e6af00578e7b6b2-- .

Operation

Definition

Result

c =3D std::move(m);

*((M*)c.p) = =3D move(*((M*)m.p));

Move

m =3D std::move(c= );

*((C*)c.p) =3D move(*((C*)m.p));

Copy

c =3D m;

*((M*)c.p) =3D *((M*)m.p);

Th= row ???

= m =3D c;

*((C*)c.p) =3D *((C*)m.p);

Copy