220 33276 <1590671.jJ8zeR8SAH@tjmaciei-mobl1> article Path: news.gmane.org!.POSTED!not-for-mail From: Thiago Macieira Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: Memory mapped files and shared memory for C++ Date: Fri, 21 Jul 2017 18:28:42 -0700 Lines: 140 Approved: news@gmane.org Message-ID: <1590671.jJ8zeR8SAH@tjmaciei-mobl1> References: <7ee99206-74e9-4309-a5af-8974994698b3@isocpp.org> Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1500686928 32228 195.159.176.226 (22 Jul 2017 01:28:48 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 22 Jul 2017 01:28:48 +0000 (UTC) To: std-proposals@isocpp.org Original-X-From: std-proposals+bncBCB4TK757YBRBTWUZLFQKGQEXMW5ATY@isocpp.org Sat Jul 22 03:28:42 2017 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-lf0-f70.google.com ([209.85.215.70]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dYjDl-00087P-Qi for gclcip-std-proposals@m.gmane.org; Sat, 22 Jul 2017 03:28:41 +0200 Original-Received: by mail-lf0-f70.google.com with SMTP id 87sf7381388lfy.9 for ; Fri, 21 Jul 2017 18:28:47 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1500686927; cv=pass; d=google.com; s=arc-20160816; b=jPtIUmLLMdWi4+gvg8edZqyJ8DK2JOLl4C9Y/rOXd2hIwITQKsDSj+wNShJJT+vIN8 QxgcDekeSrfLzIPu3uZ/Ey7nx+SrtG99YhIJwh1byUcc2SZlR8UEprT2BFLzGGJWVXY/ k7065Mv//ec3RYsyZlyiUnZKDDhBnbeNf6VgR5lN4m0I+M/pZL/5uOnYHO2wO+GvKrRf xuRzBaNM5Nke9BygJLHZN8+eynTYHSdxnk1XAvOV7fCp8287kyk7v16ViEZAB5yG8df/ LKdUY1TUjLviLlDRB/Kx87BvNQHwkQYk4tZ0eYmmyej9qvNQFbCBp/S/27vFYTpPnceZ FslA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :arc-authentication-results:arc-message-signature:dkim-signature :arc-authentication-results; bh=+vdrm5e0F9THGsqH67vsZRqzxxoCc2y3tFv22qwrEBc=; b=CVpzHXuwFmx90cHbBsu1iP/i8LjBXKg+33Xr29A9fkOAnN8RTt73QWoHGugM9nIqv3 uSq1zXc2qquPKiYLO+hgivTd2ZvDnXY2IFhoDqeqiu4HL5sL0w+V708p8ES0A41ZNokV d26TrMXXT7h//fi82bRbfe4FQJPL0djKmq6qMmIt9CDLgqoMnCBDarU74zlRuIk5MQ+D E0NETrZ7uvcE1HlsSgTenc91jqF/W9gnCHUTyRROPPuUhc1iGqZt29mB1z2gRiltuZYy YY6JqoIpMC0DIMe4RS2uDUzLQxOqAo8HtwScObfB8+0kMYMxj7wuFXcqMFtLwSM1kXT2 +uKw== ARC-Authentication-Results: i=2; mx.google.com; spf=pass (google.com: domain of thiago@macieira.org designates 78.47.120.188 as permitted sender) smtp.mailfrom=thiago@macieira.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isocpp-org.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:list-post:list-help:list-archive:list-subscribe :list-unsubscribe; bh=+vdrm5e0F9THGsqH67vsZRqzxxoCc2y3tFv22qwrEBc=; b=f+KBe8Tv8nQlJEzS93pU12Gst7wFuSu4o4jgRYyFoACtC+5KUKVAvikX6tAfvKyO/S PqCoO+xJSKUgPzenvEtE/yeKr3IZ8wxITJDTiDpl5HNPdB8wUoz2oDb+G0sMQf7ZsFZT DtJ3cXo7NgTHOV16XfG3cUKu+/5Yko2YsieXTBURIAJLeaQKqYk/NbRZWQN+DoEPiULy j4JGa2k2CQn8VD4BbN7N9BfMdpsOP4WJ1ItuaCPA5gxxLmmSipC+GEvrFPXevBuAfqpC F3s8Ut3LkoJ4UMKKERfm8ZUEsD+6EJfUxI5zdkxPM+HWCdlNwXmV2ZtUFT8GVj2Cg3hC s57A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=+vdrm5e0F9THGsqH67vsZRqzxxoCc2y3tFv22qwrEBc=; b=BMaLENjcSRt+rp723e7du351zyPwNuyWl9wTR6of0zZds5GXBZEHHJ7Nl3FyWgg/Dm LcMGihdi9Q0TuK+5kURpH+j9TJ4awOxjUaSE9C6Wa1CKDJ6vJVMCbkTVTM6BCbDM+mnt WNFxs5e6rNvSg65gyMtjx6HPEhoE8IBbgSMuUEylyiCKYtIYzR8ES9RpH3Wb1LFMa+OV CgRCdw03qHTWmeF6gRhVlveIDDGRbJj4Uc1Ib5YVYeo3Leq15DBLNoaWQDUNT5xim5JJ Fomq1BwtLG0TI1BdBlp+RYRMlXqfCOmgJmnytNd+snvCK1yTJ7F/hWuOhhaMH84qfYkI hSC X-Gm-Message-State: AIVw113hiF+vt7SRoTcsBZpKJk2gtUH2JRkCrpYGhpNRHVgFJV6kRpLB ZV5i9RY+ozV0tpy7 X-Received: by 10.46.21.15 with SMTP id s15mr1345023ljd.32.1500686927612; Fri, 21 Jul 2017 18:28:47 -0700 (PDT) X-BeenThere: std-proposals@isocpp.org Original-Received: by 10.28.45.77 with SMTP id t74ls95808wmt.4.canary-gmail; Fri, 21 Jul 2017 18:28:46 -0700 (PDT) X-Received: by 10.28.211.7 with SMTP id k7mr464418wmg.1.1500686925989; Fri, 21 Jul 2017 18:28:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1500686925; cv=none; d=google.com; s=arc-20160816; b=0V5lt/sz+oUHdYUlZ01h+JsHMLSogiz5m+0a5XCxwhGA9NI7Qa4B2D5VXZq76afFKi 3F5CH+EupEC3SHZwh3NUM4ZEWnu3dTQb07JsQonmK88HC9jcd28txHtqzk22iKaJwPF1 RCWIrhAgd1WwxjCmo5qA1XW/goonJubKwJyB7fJevvE4OUBAhCM3i5Trb3/UWlUEoZ21 sWDDNkYLYVybhFfrtJqbBlMQg5aThvb5t+w/0BltbcD0STR4H82jy9aPF+D+mmf0cOh+ J+ZxjDQiQqo8JKBlEPYYhUSROEac+utJhFG0iTE/c57f3lj0k+8tzdiOsL8zHJrJf/uD QT4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:arc-authentication-results; bh=ndiCdTw63HvCJHCKx7H+2WdRVoDkYHupMqVvNxWrrHA=; b=y2hiacDKyCQXvMiYxpZdJncZ3Vc8Fklt/G97CVxKT6YaTX1zC4ULQcJRSCBCMVMren FbZrbxbke07Y1BKmYLF5k7N4CNyXYhOdPvxJIdOJeW3eyfl3iqKDcyKaIOfsOEHwpaje T4QbunUJNeGkEuaygFd4FnYCqO7UFZRV1F6pndfFGHnTKgvhe66FRedbo4w8Qnvy1ZXN 3hDHJOCZlPOel6he7GAEUsg5GXgyxCT7gUzqYgDyGKuDy5ZDTFyb8ZDMmCS+B8zMokp8 oSEqMVbMxwozPql1uYuvw95vA0xKfERX/n9Zs0sJFjfrCpaGe6+S3BGjjCrIwChXDJja p9FA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of thiago@macieira.org designates 78.47.120.188 as permitted sender) smtp.mailfrom=thiago@macieira.org Original-Received: from gondolin.macieira.info (gondolin.macieira.info. [78.47.120.188]) by mx.google.com with ESMTP id s12si5036594wra.517.2017.07.21.18.28.45 for ; Fri, 21 Jul 2017 18:28:45 -0700 (PDT) Received-SPF: pass (google.com: domain of thiago@macieira.org designates 78.47.120.188 as permitted sender) client-ip=78.47.120.188; Original-Received: from tjmaciei-mobl1.localnet (unknown [IPv6:2601:1c0:4501:5f9f:d982:adf0:f2b:89d8]) by gondolin.macieira.info (Postfix) with ESMTPSA id 0F49011B450 for ; Fri, 21 Jul 2017 18:28:44 -0700 (PDT) In-Reply-To: <7ee99206-74e9-4309-a5af-8974994698b3@isocpp.org> X-Original-Sender: thiago@macieira.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of thiago@macieira.org designates 78.47.120.188 as permitted sender) smtp.mailfrom=thiago@macieira.org Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:33276 Archived-At: On Friday, 21 July 2017 16:26:20 PDT Alexander Zaitsev wrote: > Hello. >=20 > In 2006 Ion Gazta=C3=B1aga (Boost.Interprocess and other great libraries = author, > email: igaztanaga at gmail dot com) wrote proposal about memory mapped > files and shared memory for C++ (Document number: N2044=3D06-0114). But t= he > proposal is forgotten now. >=20 > I want to reborn it, because i think it's very important thing for C++. >=20 > Please read the proposal and discuss about it. Looks interesting, but falls short because it leaves extremely important=20 things out. * needs an exception-free API * Execute permissions are missing. Your classes support only as read-only,= =20 read-write or invalid. But memory mapped objects have more flags, especiall= y=20 the execute permission. In fact, the ability to *deny* execution is an=20 important security feature. * "copy on write" is probably the wrong name for the "private" property. It= 's=20 unintuitive that it would apply to a read-only region, but it does. It=20 indicates whether you want to see changes made by other processes or not. I= =20 advise changing the name back to "private" vs "shared". * std::size_t may be the wrong type for the map size. It's valid for all=20 modern architectures, but in theory it just has to be wide enough to hold t= he=20 size of the biggest object. We may want instead to change the wording of th= is=20 type's definition (though note it's also in POSIX).=20 * Please update the address parameter to be nullptr, not 0, and update the= =20 get_address() documentation. Do all OSes guarantee they will map where aske= d? (We can probably safely assume that mapping at nullptr is not supported by = the=20 C++ standard) * For offset_t, you may want to use the filesystem API's offset type. * For the flush() function, you need a way to explain whether it is needed = or=20 not. Most memory mappings don't need it. If you were thinking of msync(), then you're missing the arguments to that= =20 function. * I'm missing an equivalent of madvise() / posix_madvise() * std::file_mapping needs a way to refer to an already-open file, without= =20 needing to know its name (assuming it even has one). O_TMPFILE and memfd fi= les=20 on Linux don't. * what happens to the memory region if the std::file_mapping object is=20 destroyed? On one hand, RAII principles would say the mapping is undone; on= =20 another, that's not how the low-level OS works and it's often useful to ret= ain=20 a mapping forever, without having to keep a (polymorphic) object around tha= t=20 will never, ever be used again. * std::shared_memory_object needs to specify which type of shared memory it= =20 is. On Unix systems, we have two: POSIX (sys/mman.h, shm_open()) and System= V=20 (sys/shm.h, shmget()), with Linux having recently added a third (memfd). Th= ey=20 are completely disjoint namespaces and are identified differently. * POSIX shared memory has the semantics of a file, including the access=20 permissions. Please merge with the filesystem API where they've looked into= =20 that. I suspect Windows ones are similar. * the System V shared memory uses a key_t object for identification, not a= =20 "const char *name". You can use a simple narrow-character string, but you n= eed=20 to provide an "int proj_id" so it can be passed through ftok(3). * just like path names, on Windows shared memory segments are not narrow=20 characters. You need to provide a wchar_t overload for Windows. * Using truncate() for growing is weird. I know the POSIX API does that, bu= t=20 bad prior art is not good reason. What did the filesystem API choose? If gi= ven=20 the option, I'd recommend "resize". * Please write something about what happens if you map a size larger than t= he=20 object's size. Our experience in Qt is that some systems allow this,=20 automatically growing the object that backs it, while others will just cras= h=20 your application. It's ok to write that it's implementation-defined, but yo= u=20 need to write it. * for that matter, please update the API with the ability to seal memory- mappable objects. See https://lwn.net/Articles/593918/,=20 http://man7.org/linux/man-pages/man2/memfd_create.2.html. This was added to= =20 Linux specifically to avoid DoS attacks where processes of different privil= eges=20 are operating on the same shared memory object. Of course, implementation- defined whether this is supported, but it's an important security feature w= e=20 should encourage application writers to use. * you'll want to investigate cross-process thread signalling (semaphores) t= oo.=20 Eventually, we'll need to discuss std::mutex on memory-mapped regions. --=20 Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel Open Source Technology Center --=20 You received this message because you are subscribed to the Google Groups "= ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. To view this discussion on the web visit https://groups.google.com/a/isocpp= ..org/d/msgid/std-proposals/1590671.jJ8zeR8SAH%40tjmaciei-mobl1. .