220 39473 article Path: news.gmane.org!.POSTED!not-for-mail From: Niall Douglas Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: D1031R1 draft 1 LLFIO with proposed C++ object model and lifetime changes for mapped memory support Date: Wed, 1 Aug 2018 11:01:22 -0700 (PDT) Lines: 1219 Approved: news@gmane.org Message-ID: References: <096f1d1c-3e2e-4df8-91ee-6f1cf4d31aad@isocpp.org> Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_17_462843701.1533146482841" X-Trace: blaine.gmane.org 1533146363 27852 195.159.176.226 (1 Aug 2018 17:59:23 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 1 Aug 2018 17:59:23 +0000 (UTC) To: ISO C++ Standard - Future Proposals Original-X-From: std-proposals+bncBDGKFT5YZADRB5HKQ7NQKGQEWZHXHDQ@isocpp.org Wed Aug 01 19:59:19 2018 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-yb0-f199.google.com ([209.85.213.199]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fkvP1-000750-0F for gclcip-std-proposals@m.gmane.org; Wed, 01 Aug 2018 19:59:15 +0200 Original-Received: by mail-yb0-f199.google.com with SMTP id u10-v6sf3443072ybu.8 for ; Wed, 01 Aug 2018 11:01:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isocpp-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:message-id:in-reply-to:references:subject:mime-version :x-original-sender:reply-to:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-subscribe:list-unsubscribe; bh=WcmI+H6/Fq0zouaKFz3ayE40P7qM/Ky/9hXSmuxZaNU=; b=gXli5FT56L8HvffBELuhVvCQW2creWA1mjdDONEvuFcbBB7nvsblYBhfjEIK1MJL7G VZ3mGx5WZ8DyqdJHyd5DZ6AoBtkG7yRn+/9bMS9JXsKTYK1oifckhefwXYfOak/d3eCM 5rtk9DrGEJUoxFff1hsU6C+Mz9ydruI76+aqxiMqwil5tzOy33YuiurzySgmg/lIWITQ Vbd40XQJjJmCS6JH2F4dnw9xWDA8SsoLNJbb+EL5BNRgOVZcwSL0D/XD2GaPJpDLw2J/ MZaIpE7vA1KjC/wg+DKXhmVbt4IGmjTqKbJFdBVjQIqC3bM6KBCwB2+yD/OACTZYcIB8 bNEA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:message-id:in-reply-to:references:subject:mime-version :x-original-sender:reply-to:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-subscribe:list-unsubscribe; bh=WcmI+H6/Fq0zouaKFz3ayE40P7qM/Ky/9hXSmuxZaNU=; b=FcdJ8dq9bdoyUyVsl1R5eRT0208S1RC0ZT4Nv2YPUFMc6dARKKs4nFkgXwtYd6InMf xgp0m/+PcBU7+Xzw+WMfkc2eJ7evx2dUV1R/+HU7rBq715HAB4Q/0/iCKi33FGiwbfBA ZJPWQddHzSGtPXPpTHEYdlMvPFJ2Gs6/sHoTCRjMHkL6ZYBcgDOTTJ1TaoysIC+sJvbg MMqWQNVWB5QZ9CHa2TmuP1qYEW7MiUToemSUQChsqbF9Hd6qL7+IT9EO+O6UCRWONWaX yVz0Huj1xKn67sqCQplcxIP8RhRnHcVcBngc7Qow0QLCBled7JtlFqU3/n+csg6/9Odz iKgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:message-id:in-reply-to:references :subject:mime-version:x-original-sender:reply-to:precedence :mailing-list:list-id:x-spam-checked-in-group:list-post:list-help :list-archive:list-subscribe:list-unsubscribe; bh=WcmI+H6/Fq0zouaKFz3ayE40P7qM/Ky/9hXSmuxZaNU=; b=TpjBDJMFWKRhuHqIx7xA4bfywKBEert6Y9sceXydTYz5kIjkvuwmWEvefe/+jVaMmf 7FSPIquEC+STGuJazJis0kku9RnG7lr4rzAFT5/VGJnzdwyjDE4aVJy0dsftD4RbPXYZ ctPXzGDd1AqvCDVG44EHELAqFHI1cwBE6fc3eRgZv4mlmeBDVADrl/nZ8LtXGwc9mCZ8 CAAwEV75VF+FKiiJ2wA725I2xFUL1T8VjMx2aCsKMMrzNhIgKZdjCxU6gqCErvdBjdYG sD8dJXqJcjB+WkwlxTiSxU4TmNRv72Y0LIHJO9OmMz9jEBRgScC0gc7TFqgZ4pSB3iOh YKwQ== X-Gm-Message-State: AOUpUlHX9LilEf+EyH/S8hWVS8p6qKQ1jYvZ+lFwGq62c/J3lNsRJCV5 6ZSQoUVB1a4r85YD8LVlXRB+Zw== X-Google-Smtp-Source: AAOMgpdlRlhUDVP9cHCItm9WUJKW5s8QdoudDZ6yCeP5IIRk05LKDb3KKZkO5zVon20Oo5bPDWSUsA== X-Received: by 2002:a0d:ea10:: with SMTP id t16-v6mr7856125ywe.37.1533146485516; Wed, 01 Aug 2018 11:01:25 -0700 (PDT) X-BeenThere: std-proposals@isocpp.org Original-Received: by 2002:a81:a703:: with SMTP id e3-v6ls2191849ywh.37.gmail; Wed, 01 Aug 2018 11:01:24 -0700 (PDT) X-Received: by 2002:a81:330a:: with SMTP id z10-v6mr443166ywz.5.1533146483555; Wed, 01 Aug 2018 11:01:23 -0700 (PDT) In-Reply-To: <096f1d1c-3e2e-4df8-91ee-6f1cf4d31aad@isocpp.org> X-Original-Sender: nialldouglas14@gmail.com Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Spam-Checked-In-Group: std-proposals@isocpp.org X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:39473 Archived-At: ------=_Part_17_462843701.1533146482841 Content-Type: multipart/alternative; boundary="----=_Part_18_1402463751.1533146482843" ------=_Part_18_1402463751.1533146482843 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable So assuming people were put off by the length of the paper, below is the=20 section I'd really like feedback upon. Niall 3 Impact on the Standard Listed at the end of this section are the in-flight WG21 papers this=20 proposal is dependent upon, and which would need to enter the standard=20 before this library can be considered. However a bigger issue involves the= =20 potential changes to the C++ object model, plus new contracts with which to= =20 indicate the limited side effects of i/o functions in order to enable=20 improved optimisation.=20 3.1 Potential changes to the C++ object model Firstly I wish to make it clear that there is the viable option of simply= =20 making no changes at all to the object model, and any placement of objects= =20 into mapped memory is declared to be undefined behaviour, as it is at=20 present5=20 .= =20 However I also think that an opportunity missed. Via memory maps, the=20 proposed low level file i/o library herein enables object lifetime to=20 exceed that of the program. It also enables an object to appear in multiple= =20 instances of concurrent C++ program execution, or indeed at multiple=20 addresses within a C++ program. And finally one can mark individual pages= =20 of memory with different access permissions than read-only or read-write=20 (e.g. no access i.e. unreachable), or kick individual pages out to swap, or= =20 throw away their contents. In other words, individual pages of memory can= =20 become unreachable with the objects within that storage still being alive.= =20 All four things are problematic for the current C++ standard=E2=80=99s obje= ct=20 model, not least that objects cannot currently outlive the program; that=20 there is no awareness of it being possible for multiple concurrent C++=20 program instances to exist in the current standard; that objects not marked= =20 with [[no_unique_address]] and which are not bits in a bitfield must=20 currently always have a single, unique address in the process; that all=20 alive objects are equally reachable from anywhere in the program; and=20 finally that objects are always available at their unique address, and are= =20 not also elsewhere.=20 Furthermore, there are reordering constraints peculiar to mapped memory if= =20 a modification to an object is to persist at all, or indeed in a form which= =20 is usable. This is because in most implementations of C++, atomic=20 reordering constraints only affect apparent ordering to CPU cores within=20 the same symmetric multiprocessing cluster, and may have no effect on the a= ctual=20 ordering of when modifications are sent to main memory and/or the storage= =20 device. Thus, if a C++ program modifies an object, and then modifies it=20 again, some of the changes of the latter modification may land in main=20 memory before some of the changes of the earlier modification, irrespective= =20 of any atomics or fences or anything else which a C++ program currently can= =20 legally do. This has obvious implications for the integrity of that object= =20 across a program restart.=20 I therefore suggest some changes to the C++ object model, listed below.=20 3.1.1 The relevant parts of the current standard There are presently four kinds of storage duration. As [basic.stc]=20 describes it:=20 The storage duration is the property of an object that defines the minimum= =20 potential lifetime of the storage containing the object. The storage=20 duration is determined by the construct used to create the object and is=20 one of the following:=20 1. static storage duration=20 2. thread storage duration=20 3. automatic storage duration=20 4. dynamic storage duration [basic.life] describes in some detail the lifetime model, and it=20 essentially reduces down to one of these possible states for each object=20 (and apologies for the over-simplification for purposes of terse=20 exposition):=20 1. Unconstructed (from P0593: and considered unreachable by the compiler= =20 during analysis), but storage is of the right size and alignment, and=20 pointers and glvalues to the storage of the object type are permitted to= be=20 used under limited circumstances.=20 2. Unalive, in the process of being constructed, for types with=20 non-trivial constructors only.=20 3. Alive, fully constructed.=20 4. Unalive, in the process of being destructed, for types with=20 non-trivial destructors only. You will note that due to their triviality, memory storing trivial types=20 such as std::byte can either be unconstructed, or fully constructed, and no= =20 lifetime state exists apart from those two. It is important to understand= =20 that currently speaking, using allocated memory in which nothing has been= =20 constructed is undefined behaviour, so the common method of reinterpret=20 casting the return from malloc() to a pointer of the type you want is=20 presently undefined behaviour.=20 This is being dealt with in [P0593=20 ] I= mplicit=20 creation of objects for low-level object manipulation. Amongst its many=20 recommendations, it makes unconstructed memory unreachable by default. You= =20 can make it reachable by (i) constructing objects into it (ii) via the=20 proposed std::bless() which tells the compiler that a region of unreachable= =20 memory (i.e. unconstructed) is now reachable (and contains alive trivial=20 objects of some form). You can restore it to being unreachable by calling= =20 the destructor of the objects, even if those objects are completely=20 trivial. P0593 doesn=E2=80=99t mention it explicitly (yet!), but given std:= :launder=E2=80=99s=20 special casing for std::byte, it seems likely that calling the destructor= =20 for std::byte on a region with unknown trivially destructible types of=20 objects in it will always mark it unreachable. And finally, P0593 names=20 various functions as always returning memory which is reachable i.e.=20 containing an array of alive objects of some unspecified trivial type.=20 This new concept of reachable vs unreachable memory is an important one,=20 and it is leaned upon heavily in the proposed extensions.=20 3.1.2 A new storage duration: mapped As always, naming is very hard, but mapped storage duration seems a=20 reasonable name for memory which can be shared between concurrently running= =20 processes, or between multiple start-stop cycles of the same program (note= =20 that a program using an object from mapped storage which it itself did not= =20 store there is undefined behaviour).=20 The storage for objects with mapped storage duration shall last for the=20 duration of the filesystem entity (see TS definitions below for precise=20 meaning) referred to by the section_handle instance which represents the=20 storage. section_handle instances may refer to filesystem entities whose=20 lifetimes automatically end when the program ends (=E2=80=98anonymous inode= s=E2=80=99), or=20 which may persist for an indeterminate period, including across subsequent= =20 executions of the C++ program.=20 Mapped storage has the most similarity to dynamic storage, but with the=20 following differences:=20 - It has allocation and alignment granularities with architecture=20 specific coarse sizes (=E2=80=98memory page=E2=80=99). 4Kb/2Mb/1Gb page = sizes are common.=20 - New allocations are guaranteed to be all bits zero on creation.=20 - It has map on first read semantics. This means that the first read=20 from a memory page can take hundreds of CPU cycles, and a TLB shootdown= =20 causes an interrupt for other CPUs.=20 - It has allocate on first write semantics. This means that the first=20 write to a memory page can take thousands or even hundreds of thousands = of=20 CPU cycles, plus a TLB shootdown.=20 - Usually, but not always, mapped storage is a memory cache of=20 equivalent storage a high latency storage device. Hence they can be=20 individually pushed to storage, their contents thrown away, deallocated,= =20 given different access permission or caching strategies, and lots of oth= er=20 interesting (i.e. potentially game changing for large STL containers)=20 operations. Individual pages can be:=20 - Read-only, read-write, or copy-on-write. These are self describing,= =20 and these are hard characteristics: violating them means program fail= ure.=20 - Committed or uncommitted. This indicates whether the page counts=20 towards the resources used by the C++ program. Uncommitted memory is= =20 inaccessible, and acts as a placeholder for later use (i.e. it is res= erved=20 address space, useful for expanding large arrays without content copy= ing).=20 - Dirty or clean. This indicates whether the page contains data not= =20 yet mirrored onto its backing storage.=20 - Allocated or unallocated. This indicates whether storage backing=20 the page has been allocated on the storage device. If not, the first = write=20 to a clean page may be very expensive as the page may need to be copi= ed by=20 the kernel and/or space allocated for it on the backing storage devic= e. =20 The storage represented by a section_handle instance can be mapped into=20 (i.e. made available to) a C++ program by creating a map_handle sourcing=20 the storage from a section_handle instance. The storage mapped by the low= =20 level map_handle shall represent unconstructed and unreachable memory, and= =20 will require the use of std::bless() or map_view to make it reachable=20 (alternatively, use the convenience class mapped on a section_handle instan= ce=20 which bundles the aforementioned low level operations on your behalf).=20 Destroying a map_view does not make the storage unreachable. Decommitting= =20 individual pages does, as does destroying a mapped or map_handle.=20 3.1.3 A new lifetime stage: unreachable I propose adding a new separate lifetime status for objects: unreachable.= =20 Under P0593, unconstructed objects are unreachable, but there is no=20 possibility for a partially constructed, alive or partially destructed=20 object to be unreachable. I propose that this new status ought to be added= =20 such that objects can now have the following lifetime states:=20 1. Unconstructed, always unreachable.=20 2. Unalive, in the process of being constructed, for types with=20 non-trivial constructors only. Always reachable.=20 3. Alive, fully constructed, reachable. Has a single, unique address in= =20 memory (unless marked with [[no_unique_address]]).=20 4. Alive, fully constructed, unreachable. Does NOT have a single, unique= =20 address in memory (it may have many, or none). May change whilst=20 unreachable (i.e. reload it after marking it reachable at some address).= =20 5. Unalive, in the process of being destructed, for types with=20 non-trivial destructors only. Always reachable. P0593 proposed these functions to mark regions as reachable:=20 // Requires: [start, (char*)start + length) denotes a region of allocated= =20 // storage that is a subset of the region of storage reachable through=20 start.=20 // Effects: implicitly creates objects within the denoted region.=20 void std::bless(void *start, size_t length);=20 =20 // Effects: create an object of implicit lifetype type T in the storage=20 // pointed to by T, while preserving the object representation.=20 template T *std::bless(void *p); Thus the obvious corrollary for marking regions as unreachable:=20 // Requires: [start, (char*)start + length) denotes a region of allocated= =20 // storage that is a subset of the region of storage reachable through=20 start.=20 // Effects: implicitly uncreates objects within the denoted region.=20 void std::unbless(void *start, size_t length);=20 =20 // Effects: uncreate an object of implicit lifetype type T in the storage= =20 // pointed to by T, while preserving the object representation.=20 template void *std::unbless(T *p);=20 A gain of this new unreachable lifetime status is that it neatly solves the= =20 problem of objects appearing at multiple addresses in a running program. We= =20 can now say that only one of those addresses can be reachable for an alive= =20 object at a time. If a program wishes to change an object=E2=80=99s current= =20 reachable address, they unbless the old location, and bless the new=20 location.=20 It should be emphasised that similarly to blessing, unblessing of trivial= =20 types causes no code emission. It simply tells the compiler what is now=20 unreachable.=20 Finally, it may seem that unreachability is a bit of a big sledgehammer to= =20 throw at this problem. However, there are a number of other problems=20 elsewhere in C++ where having unreachable but alive objects would be very= =20 useful =E2=80=93 thread local storage on a million CPU core compute resourc= e is an=20 excellent example6=20 .= =20 That said, this is a big change to lifetime, and if strong arguments are=20 made against it then I am happy to propose something more conservative.=20 3.1.4 Blessing and unblessing polymorphic objects Currently P0593 makes no mention of polymorphic objects i.e. ones with=20 vptrs in many implementations. I can see lots of reasons why it would be=20 useful to bless and unbless polymorphic objects i.e. update the vptr to the= =20 correct one for the currently running program, or clear the vptr to the=20 system null pointer.=20 Imagine, for example, a polymorphic object with mapped storage duration=20 whose constructing program instance terminates. Upon restart of the=20 program, the storage is mapped back into the program, but now the=20 polymorphic object has the wrong vptr! So we need to write the correct vptr= =20 for that object, after which all works as expected7=20 .= =20 This also applies to shared mapped memory. Extending the C++ object model= =20 to handle memory being modifiable by concurrently running C++ programs=20 would be very hard, so I propose that we don=E2=80=99t do that. Rather we s= ay that=20 objects in mapped storage can only be reachable in exactly one or none=20 running C++ programs at a time i.e. at exactly one or no address at any one= =20 time, otherwise it is undefined behaviour. Therefore, if process A wants to= =20 make available an object to process B, it unblesses it and tells process B= =20 that the object is now available for use. Process B blesses the object into= =20 reachable, and uses it. When done, it unblesses it once more. Now process A= =20 can bless it and use it again, if so desired.=20 Implied in this is that blessing and unblessing becomes able to rewrite the= =20 vptr (or whatever the C++ implementation uses to implement polymorphism).= =20 To be truthful, I am torn on this. On the one hand, if you wish to be able= =20 to bless and unbless polymorphic objects, rewriting the vptr is=20 unavoidable, and the frequency of relocating polymorphic objects in current= =20 C++ is low. On the other hand, future C++=E2=80=99s may do a lot more remap= ping of=20 memory during large array expansion in order to skip doing a memory copy,= =20 and in this circumstance every single polymorphic object in a few million= =20 item array would need their vptr=E2=80=99s needlessly rewritten, which is= =20 unnecessary work.=20 I=E2=80=99m going to err on the side of caution, and propose that the polym= orphic=20 editions of bless and unbless shall be:=20 // Effects: create an object of implicit lifetype type T in the storage=20 // pointed to by T, while preserving the object representation=20 // apart from any necessary changes to make any polymorphic=20 // objects within and including T ready for use.=20 template T *std::revive(void *p);=20 =20 // Effects: uncreate an object of implicit lifetype type T in the storage= =20 // pointed to by T, while preserving the object representation=20 // apart from any necessary changes to make any polymorphic=20 // functions within the objects within and including T unusable.= =20 template void *std::stun(T *p); Note that stun() and revive() not just change the vptrs of the type you=20 pass them, but also any vptrs of any nested types within that type. They=20 also perform blessing and unblessing, so you do not have to do that=20 separately. Calling these functions on non-polymorphic types is permitted.= =20 3.2 New attributes [[no_side_effects]] and [[no_visible_side_effects]], and= =20 new contract syntax for specifying lack of side effects It is highly important for the future efficiency of any iostreams v2 that= =20 the low level i/o functions do not cause the compiler to dump and reload=20 all state for every i/o, as they must do at present. The i/o functions=20 proposed do not modify their handle on most platforms, and on those=20 platforms we can guarantee to the compiler that there are either no side=20 effects or no visible side effects except for those objects modified by=20 parameters.=20 Reusing bless() and unbless(), I propose the following contracts syntax for= =20 the io_handle functions so we can tell the compiler what side effects the= =20 i/o functions have:=20 constexpr! void ensure_blessed(buffers_type buffers)=20 {=20 for(auto &buffer : buffers)=20 {=20 bless(buffer.data(), buffer.size());=20 }=20 }=20 =20 // This function has no side effects visible to the caller except=20 // for the objects it creates in the scatter buffer list.=20 virtual buffers_type io_handle::read(io_request reqs,=20 deadline d =3D deadline()) throws( file_io_error)=20 [[no_side_effects]]=20 [[ensures: ensure_blessed(reqs.buffers)]]=20 [[ensures: ensure_blessed(return)]]; The key thing to note here is if the caller never accesses any of the=20 scatter buffers returned by the function, the read can be optimised out=20 entirely due to the [[no_side_effects]] attribute. Obviously the compiler= =20 also does not dump and reload any state around this scatter read apart from= =20 buffers supplied and returned, despite it calling a kernel system call,=20 because we are giving a hard guarantee to the compiler that it is safe to= =20 assume no side effects apart from modifying the scatter buffers.=20 Gather write is less exciting, but straightforward:=20 virtual const_buffers_type io_handle::write(io_request= =20 reqs,=20 deadline d =3D deadline())=20 throws(file_io_error) [[no_visible_side_effects ]];=20 Here we are telling the compiler that there are side effects, just not ones= =20 visible to the caller. Hence the compiler cannot optimise out calling this= =20 function, but it can skip dumping and reloading state despite the kernel=20 system call made by the function.=20 3.2.1 I/O write reordering barriers C and C++ allow the programmer to constrain memory read and write=20 operations, specifically to constrain the extent of reordering that the=20 compiler and CPU are permitted to do. One can use atomics with a memory=20 order specified, or one can call a fence function which applies a memory=20 reordering constraint for the current thread of execution either at just=20 the compiler level, or also at the CPU level whereby the CPU is told what= =20 ordering its symmetric multiprocessing cores needs to manifest in visible= =20 side effects.=20 File i/o is no different: concurrent users of the same file or directory or= =20 file system will experience races unless they take special measures to=20 coordinate amongst themselves.=20 Given that I have proposed above that C++ standardises mapped storage=20 duration into the language itself, it might seem self evident that one=20 would also need to propose a standardised mechanism for indicating=20 reordering constraints on i/o, which are separate to those for threads. My= =20 current advice is that we should not do this =E2=80=93 yet.=20 The first reason why is that persistent memory is just around the corner,= =20 and I think it would be unwise to standardise without the industry gaining= =20 plenty of empirical experience first. So kick that decision down a few=20 standard releases.=20 Secondly, the proposed library herein does expose whatever acquire-release= =20 semantics is implemented by the host operating system for file i/o, plus=20 the advisory locking infrastructure implemented by the host, plus the=20 ability to enable write-through caching semantics. It also provides=20 barrier(), though one should not write code which relies upon it working,= =20 as it frequently does not.=20 Note that if the file is opened in non-volatile RAM storage mode, barrier()= calls=20 the appropriate architecture-specific assembler instructions to correctly= =20 barrier writes to main memory, so in this sense there is library, if not=20 language, support for persistent memory. The obvious sticker is that barrie= r()=20 is a virtual function with significant preamble and epilogue, so for=20 flushing a single cache line it is extremely inefficient relative to direct= =20 language support for this.=20 Nevertheless, one can write standards conforming code with the proposed=20 library which performs better on persistent memory than on traditional=20 storage, and my advice is that this is good enough for the next few years.= =20 --=20 You received this message because you are subscribed to the Google Groups "= ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. To view this discussion on the web visit https://groups.google.com/a/isocpp= ..org/d/msgid/std-proposals/b214ce98-2aaa-4f7f-a04a-77687422111a%40isocpp.or= g. ------=_Part_18_1402463751.1533146482843 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
So assuming people were put off by the length of the paper= , below is the section I'd really like feedback upon.

Niall


<= span class=3D"titlemark">3 Impact on the S= tandard

Listed at the end of this se= ction are the in-flight WG21 papers this proposal is dependent upon, and wh= ich would need to enter the standard before this library can be considered.= However a bigger issue involves the potential changes to the C++ object mo= del, plus new contracts with which to indicate the limited side effects of = i/o functions in order to enable improved optimisation.


3.1 Po= tential changes to the C++ object model

Firstly I wish to make it clear that there is the viable option of simp= ly making no changes at all to the object model, and any placement of objec= ts into mapped memory is declared to be undefined behaviour, as it is at pr= esent5= .


However I also think that an opportu= nity missed. Via memory maps, the proposed low level file i/o library herei= n enables object lifetime to exceed that of the program. It also enables an= object to appear in multiple instances of concurrent C++ program execution= , or indeed at multiple addresses within a C++ program. And finally one can= mark individual pages of memory with different access permissions than rea= d-only or read-write (e.g. no access i.e. unreachable), or kick individual = pages out to swap, or throw away their contents. In other words, individual= pages of memory can become unreachable wi= th the objects within that storage still being al= ive.


All four things are problematic for the current C++ standard=E2=80=99= s object model, not least that objects cannot currently outlive the program= ; that there is no awareness of it being possible for multiple concurrent C= ++ program instances to exist in the current standard; that objects not mar= ked with [[no_unique_address]] and which= are not bits in a bitfield must currently always have a single, = = = unique address in the process; that all alive objects are= equally reachable from anywhere in the program; and finally that objects a= re always available at their unique address, and are not also elsewhere.


Furtherm= ore, there are reordering constraints peculiar to mapped memory if a modifi= cation to an object is to persist at all, or indeed in a form which is usab= le. This is because in most implementations of C++, atomic reordering const= raints only affect apparent ordering to CP= U cores within the same symmetric multiprocessing cluster, and may have no = effect on the actual ordering of when modi= fications are sent to main memory and/or the storage device. Thus, if a C++= program modifies an object, and then modifies it again, some of the change= s of the latter modification may land in main memory before some of the cha= nges of the earlier modification, irrespective of any atomics or fences or = anything else which a C++ program currently can legally do. This has obviou= s implications for the integrity of that object across a program restart. <= !--l. 653-->


I there= fore suggest some changes to the C++ object model, listed below.


3.1.1 The relevant parts of the= current standard

There are presentl= y four kinds of storage duration. As [basic.stc= ] describes it:


The st= orage duration is the property of an object that defines the minimum potent= ial lifetime of the storage containin= g the object. The storage duration is determined by the construct used to create the object and is one of the fo= llowing:

  1. static storage duration =
  2. thread storage duration
  3. automatic storage duration =
  4. dy= namic storage duration

[basic.life] describes in some deta= il the lifetime model, and it essentially reduces down to one of these poss= ible states for each object (and apologies for the over-simplification for = purposes of terse exposition):

<= ol class=3D"enumerate1">
  • Unconstru= cted (from P0593: and considered unreachable by t= he compiler during analysis), but = = st= orage is of the right size and alignment, and pointers and glvalues to the = storage of the object type are permitted to be used under limited circu= mstances.
  • Unalive, in t= he process of being constructed, for types with non-trivial constructors on= ly.
  • Alive, fully constr= ucted.
  • Unalive, in the = process of being destructed, for types with non-trivial destructors only.

    You will note that due to their = triviality, memory storing trivial types such as std::byte can either be unconstructed, or fully constructed, and= no lifetime state exists apart from those two. It is important to understa= nd that currently speaking, using allocated memory in which nothing has bee= n constructed is undefined behaviour, so the common method of reinterpret c= asting the return from malloc() to a p= ointer of the type you want is presently undefined behaviour. =


    This is being dealt= with in [P0593] Implicit creation of objects for low-level object manipulation. Amongst its many recommendations, it makes unconstructed memory unreachable by default. You can make it reachable= by (i) constructing objects into it (ii) via the proposed std::bless() which tells the compiler that a region of= unreachable memory (i.e. unconstructed) is now reachable (and contains ali= ve trivial objects of some form). You can restore it to being unreachable b= y calling the destructor of the objects, even if those objects are complete= ly trivial. P0593 doesn=E2=80=99t mention it explicitly (yet!), but given <= span class=3D"fvmr8t-x-x-93">std::launder=E2=80=99s special casing f= or std::byte, it seems likely that cal= ling the destructor for std::byte on a= region with unknown trivially destructible types of objects in it will alw= ays mark it unreachable. And finally, P0593 names various functions as alwa= ys returning memory which is reachable i.e. containing an array of alive ob= jects of some unspecified trivial type.


    This new concept of reachable vs unreacha= ble memory is an important one, and it is leaned upon heavily in the propos= ed extensions.


    3.1.= 2 A new storage duration: mapped

    As alwa= ys, naming is very hard, but mapped storage durat= ion seems a reasonable name for memory which can be shared between c= oncurrently running processes, or between multiple start-stop cycles of the= same program (note that a program using an object from mapped storage whic= h it itself did not store there is undefined behaviour).

    <= p class=3D"noindent">

    The storage for objects = with mapped storage duration shall last for the duration of the filesystem = entity (see TS definitions below for precise meaning) referred to by the section_handle instance which represents= the storage. section_handle instances= may refer to filesystem entities whose lifetimes automatically end when th= e program ends (=E2=80=98anonymous inodes=E2=80=99), or which may persist f= or an indeterminate period, including across subsequent executions of the C= ++ program. = =


    Mapped storage has the most similarity to= dynamic storage, but with the following differences:

    • It has allocation and alignment granularit= ies with architecture specific coarse sizes (=E2=80=98memory page=E2=80= =99). 4Kb/2Mb/1Gb page sizes are common.
    • Ne= w allocations are guaranteed to be all bits zero on creation.
      • It has map on first read semantics. This means that the = first read from a memory page can take hundreds of CPU cycles, and a TL= B shootdown causes an interrupt for other CPUs.
    • It has allocate on first write semantics. This means that the first wr= ite to a memory page can take thousands or even hundreds of thousands o= f CPU cycles, plus a TLB shootdown.
    • Usually= , but not always, mapped storage is a memory cache of equivalent storage a = high latency storage device. Hence they can be individually pushed to s= torage, their contents thrown away, deallocated, given different access= permission or caching strategies, and lots of other interesting (i.e. = potentially game changing for large STL containers) operations. Individual = pages can be:
      • Re= ad-only, read-write, or copy-on-write. These are self describing, and these= are hard characteristics: violating them means program failure. =
      • Committed or uncommitted. This indi= cates whether the page counts towards the resources used by = the C++ program. Uncommitted memory is inaccessible, and acts as a= placeholder for later use (i.e. it is reserved address space, useful for e= xpanding large arrays without content copying).
        • Dirty or clean. This indicates whether the page contains= data not yet mirrored onto its backing storage. Allocated or unallocated. This indicates whether storag= e backing the page has been allocated on the storage device. If no= t, the first write to a clean page may be very expensive as the pa= ge may need to be copied by the kernel and/or space allocated for it = on the backing storage device.

    The storage represented by a se= ction_handle instance can be mapped into (i.e. made available to) a C++ program by creating a map_handle sourcing the storage from a section_handle instance. The storage mapped by the lo= w level map_handle shall represent unc= onstructed and unreachable memory, and will = = req= uire the use of std::bless() or map_view to make it reachable (altern= atively, use the convenience class mapped = on a section_handle instanc= e which bundles the aforementioned low level operations on your behalf).


    Destroyi= ng a map_view does not make the= storage unreachable. Decommitting individual pages does, as does destroyin= g a mapped or map_handle.


    =

    3.1.3 A new lifetime st= age: unreachable

    I propose adding a new separate lifetime status for objects: = unreachable. Under P0593, unconstructed ob= jects are unreachable, but there is no possibility for a partially construc= ted, alive or partially destructed object to be unreachable. I propose that= this new status ought to be added such that objects can now have the follo= wing lifetime states:

    1. Unconstructed, al= ways unreachable.
    2. Unali= ve, in the process of being constructed, for types with non-trivial constru= ctors only. Always reachable.
    3. Alive, fully constructed, reachable. Has a single, unique addres= s in memory (unless marked with [[no_unique= _address]]).
    4. Ali= ve, fully constructed, unreachable. Does NOT have a single, unique address = in memory (it may have many, or none). May change whilst unreachable (i= ..e. reload it after marking it reachable at some address).
      5. Unalive, in the process of being de= structed, for types with non-trivial destructors only. Always reachable= ..

    P0593 proposed these function= s to mark regions as reachable:


    =

    // Requires: [start, (char*)start + len= gth) denotes a region of allocated
    // storage that is a subset of the region of storage rea= chable through start.
    // Effects: implicitly creates objects within the denoted region.
    void     std::bless(void *= start, size_t length= );<= span style=3D"color: #000;" class=3D"styled-by-prettify">
    =C2=A0
    // Effects: c= reate an object of implicit lifetype type T in the storage
    // =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0pointed to by T, while preserving the object representation. <= span style=3D"color: #000;" class=3D"styled-by-prettify">
    template<typename T> T *std::bles= s(<= span style=3D"color: #008;" class=3D"styled-by-prettify">void *p);

    Thus the obvious corrollary for marking regions as unreachable:


    // Requires: [start, (char*)start + length) denote= s a region of allocated
    // storage that is a subset of the region of storage reachable thro= ugh start.
    // Ef= fects: implicitly uncreates objects within the denoted region.
    void std::unbless(void = *start, size_t length);
    =C2=A0
    // Effects: uncreate a= n object of implicit lifetype type T in the storage
    // =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0po= inted to by T, while preserving the object representation.
    template<typename T> void *std::unbless(T *p);


    A gain of this new unreachable lifetime status is that it nea= tly solves the problem of objects appearing at multiple addresses in a runn= ing program. We can now say that only one of those addresses can be reachab= le for an alive object at a time. If a program wishes to change an object= =E2=80=99s current reachable address, they unbless the old location, and bl= ess the new location.


    It should be emphasised that similarly to blessing, unbless= ing of trivial types causes no code emission. It simply tells the compiler = what is now unreachable.


    Finally, it may seem that unreachability is a bit of a b= ig sledgehammer to throw at this problem. However, there are a number of ot= her problems elsewhere in C++ where having unreachable but alive objects wo= uld be very useful =E2=80=93 thread local storage on a million CPU core com= pute resource is an excellent example6= . That said, this is a big change to lifetime, and if strong arguments are= made against it then I am happy to propose something more conservative.


    3.1.4 Blessing and unbl= essing polymorphic objects

    Currently= P0593 makes no mention of polymorphic objects i.e. ones with vptrs in many= implementations. I can see lots of reasons why it would be useful to bless= and unbless polymorphic objects i.e. update the vptr to the correct one fo= r the currently running program, or clear the vptr to the system null point= er.


    Im= agine, for example, a polymorphic object with mapped storage duration whose= constructing program instance terminates. Upon restart of the program, the= storage is mapped back into the program, but now the polymorphic object ha= s the wrong vptr! So we need to write the correct vptr for that object, aft= er which all works as expected7 .


    This also a= pplies to shared mapped memory. Extending the C++ object model to handle me= mory being modifiable by concurrently running C++ programs would be very ha= rd, so I propose that we don=E2=80=99t do that. Rather we say that objects = in mapped storage can only be reachable in exactly one or none running C++ = programs at a time i.e. at exactly one or no address at any one time, other= wise it is undefined = = behaviour. Therefore, if p= rocess A wants to make available an object to process B, it unblesses it and tells process B that the object is now a= vailable for use. Process B blesses the object into reachable, and uses it.= When done, it unblesses it once more. Now process A can bless it and use i= t again, if so desired.


    Implied in this is that blessing and unblessing becomes a= ble to rewrite the vptr (or whatever the C++ implementation uses to impleme= nt polymorphism). To be truthful, I am torn on this. On the one hand, if yo= u wish to be able to bless and unbless polymorphic objects, rewriting the v= ptr is unavoidable, and the frequency of relocating polymorphic objects in = current C++ is low. On the other hand, future C++=E2=80=99s may do a lot mo= re remapping of memory during large array expansion in order to skip doing = a memory copy, and in this circumstance every single polymorphic object in = a few million item array would need their vptr=E2=80=99s needlessly rewritt= en, which is unnecessary work.


    <= /p>

    I=E2=80=99m going to err on the side of caution, a= nd propose that the polymorphic editions of bless and unbless shall be:


    // Effects: create an object of implicit lifetype type T in the storage =
    // =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0pointed to by T, while preserving the object represent= ation
    =     // =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0apart from any necessary changes to make any pol= ymorphic <= br>// =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0objects within and including T ready for use= ..
    template<typename T> T *std::revive(void *p);
    =C2=A0
    // Effects: uncreate an object of implicit = lifetype type T in the storage
    // =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pointed to by T, while = preserving the object representation
    // =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0apart from any ne= cessary changes to make any polymorphic
    // =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0functions with= in the objects within and including T unusable.
    template<typename T> void *std::stun(T *p);=

    <= div class=3D"lstlisting" id=3D"listing-12">

    • Note that stun() and revive() not just change the vptrs of the type you pass them, but also any vptrs= of any nested types within that type. They also perform blessing and unble= ssing, so you do not have to do that separately. Calling these functions on= non-polymorphic types is permitted.


    3.2 New attributes [[no_side_effec= ts]] and [[no_visible_side_effects]], and new contract syntax for specifying lack of side effects

    It is highly important for the future effici= ency of any iostreams v2 that the low level i/o functions do not cause the = compiler to dump and reload all state for every i/o, as they must do at pre= sent. The i/o functions proposed do not modify their handle on most platfor= ms, and on those platforms we can guarantee to the compiler that there are = either no side effects or no visible side effects except for those objects = modified by parameters.


    Reusing bless() and = unbless(), I propose the following con= tracts syntax for the io_handle functi= ons so we can tell the compiler what side effects the i/o functions have: <= !--l. 790-->


    constexpr! void ensure_blessed(buffers_type buffers)
    {
    =C2=A0     for(auto &buffer : buffers= )
    =C2=A0     = {
    =C2=A0 =C2=A0 bless(buffer    .data(), buffer.size());
    =C2=A0     } }=
    =C2=A0
    // This func= tion has no side effects visible to the caller except
    // for the objects it creates in the = scatter buffer list.
    virtual buffers_type io_handle::read(io_reque= st<buff= ers_type> reqs,
    =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0deadline d     =3D     deadline()) throws(file_io_error)
    =C2=A0 =C2=A0     [[no_side_effects]]
    =C2=A0 =C2=A0     [[ensures: = ensure_blessed(reqs<= /span>.buffers)]]
    =C2=A0 =C2=A0     [[    ensures: ensure_blessed(return)]];

    The key thing to note here is if the caller never accesses any of = the scatter buffers returned by the function, = = t= he read can be optimised out entirely due to the [[no_side_effects]] attribute. Obviously the compiler also does = not dump and reload any state around this scatter read apart from buffers s= upplied and returned, despite it calling a kernel system call, because we a= re giving a hard guarantee to the compiler that it is safe to assume no sid= e effects apart from modifying the scatter buffers.


    Gather write is less exciting= , but straightforward:


    = virtual const_buffers_type = io_handle:= :write(io_request<const_buffers_type&= gt; reqs,
    =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 deadline= d     =3D deadline())
    =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0    thro= ws(= file_io_error) [[no_visible_side_effects]];

    =

    Here we are telling the= compiler that there are side effects, just not ones visible to the caller.= Hence the compiler cannot optimise out calling this function, but it can s= kip dumping and reloading state despite the kernel system call made by the = function.


    3.2.1 <= /span>I/O write reordering barriers

    C and C++ allow the programmer to constrain mem= ory read and write operations, specifically to constrain the extent of reor= dering that the compiler and CPU are permitted to do. One can use atomics w= ith a memory order specified, or one can call a fence function which applie= s a memory reordering constraint for the current thread of execution either= at just the compiler level, or also at the CPU level whereby the CPU is to= ld what ordering its symmetric multiprocessing cores needs to manifest in v= isible side effects.


    File i/o is no different: concurrent users of the same file = or directory or file system will experience races unless they take special = measures to coordinate amongst themselves.


    Given that I have proposed above that = C++ standardises mapped storage duration into the language itself, it might= seem self evident that one would also need to propose a standardised mecha= nism for indicating reordering constraints on i/o, which are separate to th= ose for threads. My current advice is that we should not do this =E2=80=93 yet.


    The first reason why is that persistent mem= ory is just around the corner, and I think it would be unwise to standardis= e without the industry gaining plenty of empirical experience first. So kic= k that decision down a few standard releases.

    Secondly, the proposed library herein does expose whatever acquir= e-release semantics is implemented by the host operating system for file i/= o, plus the advisory locking infrastructure implemented by the host, plus t= he ability to enable write-through caching semantics. It also provides barrier(), though one should not write cod= e which relies upon it working, as it frequently does not.


    Note that if the file = is opened in non-volatile RAM storage mode, b= arrier() calls the appropriate architecture-specific assembler instr= uctions to correctly barrier writes to main memory, so in this sense = = = there is library, if not language, support for persis= tent memory. The obvious sticker is that barr= ier() is a virtual function with significant preamble and epilogue, = so for flushing a single cache line it is extremely inefficient relative to= direct language support for this.

    N= evertheless, one can write standards conforming code with the proposed libr= ary which performs better on persistent memory than on traditional storage,= and my advice is that this is good enough for the next few years.=C2=A0

    --
    You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
    To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposa= ls+unsubscribe@isocpp.org.
    To post to this group, send email to std-proposals@isocpp.org.
    To view this discussion on the web visit https://groups.google.= com/a/isocpp.org/d/msgid/std-proposals/b214ce98-2aaa-4f7f-a04a-77687422111a= %40isocpp.org.
    ------=_Part_18_1402463751.1533146482843-- ------=_Part_17_462843701.1533146482841-- .