220 29339 <684B5982-22CB-4357-9A1A-09F4588ADA8A@gmail.com> article Path: news.gmane.org!.POSTED!not-for-mail From: Miro Knejp Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: Re: std::byte thoughts Date: Wed, 2 Nov 2016 13:14:50 +0100 Lines: 270 Approved: news@gmane.org Message-ID: <684B5982-22CB-4357-9A1A-09F4588ADA8A@gmail.com> References: <7da80620-e4bd-40eb-a4ec-4f3c2dcb2c98@isocpp.org> <1549171.TC16PSHVVd@tjmaciei-mobl1> <2350841.Z7lUCKSumT@tjmaciei-mobl1> <3e64b220-79ae-4108-ba1c-9785494fc434@isocpp.org> <97ebfc84-c5a3-eaf5-efcb-c1bdf9b507a5@gmail.com> Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: multipart/alternative; boundary="Apple-Mail=_76CE7E6B-C53E-47AF-A878-7CA2C4CFBBA6" X-Trace: blaine.gmane.org 1478088903 27154 195.159.176.226 (2 Nov 2016 12:15:03 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 2 Nov 2016 12:15:03 +0000 (UTC) To: std-proposals@isocpp.org Original-X-From: std-proposals+bncBC6ONSXJ54LBBPNR47AAKGQECCVBOAY@isocpp.org Wed Nov 02 13:14:57 2016 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-wm0-f71.google.com ([74.125.82.71]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1c1uRP-0005k6-Fy for gclcip-std-proposals@m.gmane.org; Wed, 02 Nov 2016 13:14:51 +0100 Original-Received: by mail-wm0-f71.google.com with SMTP id m203sf9133031wma.2 for ; Wed, 02 Nov 2016 05:14:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isocpp-org.20150623.gappssmtp.com; s=20150623; h=from:message-id:mime-version:subject:date:references:to:in-reply-to :x-original-sender:x-original-authentication-results:reply-to :precedence:mailing-list:list-id:x-spam-checked-in-group:list-post :list-help:list-archive:list-subscribe:list-unsubscribe; bh=Lu0nfmaiaotRIhiXjcKWBpDsne01Hnyg/TIbLRjjGZ8=; b=mhunbEAEMbxNnbupDi/Mohh50qMeBYfEm6w9MuY5ew1p9ZV12jYgNLFyWpUquVoxN9 FKr0asWxjrPfSRlkWFNskqLbqMQEXY9aZ5l14AnfB9VtSQybD2CafO4hbvNqhxzmM/Qe IjMuwp3adZkVrdEu+YjcHlR5Y9AgT2/njKyrrsVd+lYJ/ysZdorbp8XUS/uAPOrrvNXL CWFrGIQjtB9gew8XFMJTl2KUwa6DMXwsGBTre9ce/dvW4UXx5US6SaNnyZsydj/c7+xX YZYcu6C/2zUd72SEJq4fFZ3dmrG3SBKr63uWXcV8rEeZcWFJegs6m9aGEcXqmIJfHqkr 6XOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:message-id:mime-version:subject:date :references:to:in-reply-to:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=Lu0nfmaiaotRIhiXjcKWBpDsne01Hnyg/TIbLRjjGZ8=; b=T7ovfjjF68hSULd+GLJhiAQcKgrBRWw7LU+cWE94QUR2o+pYbFO5pmS4k/GXQLtfro cvR1+O1cT8ymEXz8xppO/ME51IBUy/YDeltagFRaWz3gBgXfxW6UVfKrQTP8trJFKgXE iFyJr25FZ0fBbpsMuY5KPXQIkoQABAnaURnpKoVGRPRHQcFbqWpTc6ib/tpbwPkxmFUG k2TSkhhHBGNvIowLCJjUPD/GS/xOAkuB47FyYVflrVsz7euAeMqAWxrJwqX6EBIX515n gcN6ilkjpC1TyL7j9DrZvoqr5y+pKr3bZqm/0dkNWPJjg6GysjTxqK307+cOydKaU4UZ bozw== X-Gm-Message-State: ABUngvc4V58I/uZ00vD8cpKztKkZPawGZNHoORdBlCrRjoDT8XdqQODtvilqhS9sOQ4BLg== X-Received: by 10.28.47.210 with SMTP id v201mr303007wmv.14.1478088894279; Wed, 02 Nov 2016 05:14:54 -0700 (PDT) X-BeenThere: std-proposals@isocpp.org Original-Received: by 10.28.136.5 with SMTP id k5ls373999wmd.9.gmail; Wed, 02 Nov 2016 05:14:53 -0700 (PDT) X-Received: by 10.28.0.13 with SMTP id 13mr2574042wma.126.1478088893047; Wed, 02 Nov 2016 05:14:53 -0700 (PDT) Original-Received: from mail-wm0-x233.google.com (mail-wm0-x233.google.com. [2a00:1450:400c:c09::233]) by mx.google.com with ESMTPS id 16si2848109wmn.89.2016.11.02.05.14.52 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 02 Nov 2016 05:14:53 -0700 (PDT) Received-SPF: pass (google.com: domain of miro.knejp@gmail.com designates 2a00:1450:400c:c09::233 as permitted sender) client-ip=2a00:1450:400c:c09::233; Original-Received: by mail-wm0-x233.google.com with SMTP id p190so264459659wmp.1 for ; Wed, 02 Nov 2016 05:14:52 -0700 (PDT) X-Received: by 10.28.138.209 with SMTP id m200mr2793767wmd.89.1478088892427; Wed, 02 Nov 2016 05:14:52 -0700 (PDT) Original-Received: from [172.16.1.80] ([87.191.28.122]) by smtp.gmail.com with ESMTPSA id h10sm2370155wje.48.2016.11.02.05.14.51 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 02 Nov 2016 05:14:51 -0700 (PDT) In-Reply-To: <97ebfc84-c5a3-eaf5-efcb-c1bdf9b507a5@gmail.com> X-Mailer: Apple Mail (2.3124) X-Original-Sender: miro.knejp@gmail.com X-Original-Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com; spf=pass (google.com: domain of miro.knejp@gmail.com designates 2a00:1450:400c:c09::233 as permitted sender) smtp.mailfrom=miro.knejp@gmail.com; dmarc=pass (p=NONE dis=NONE) header.from=gmail.com Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:29339 Archived-At: --Apple-Mail=_76CE7E6B-C53E-47AF-A878-7CA2C4CFBBA6 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=UTF-8 > On 02 Nov 2016, at 10:02 , Andrey Semashev wr= ote: >=20 > On 11/02/16 08:16, Nicol Bolas wrote: >> On Wednesday, November 2, 2016 at 12:55:46 AM UTC-4, Thiago Macieira wro= te: >>=20 >> Em ter=C3=A7a-feira, 1 de novembro de 2016, =C3=A0s 20:10:28 PDT, Nic= ol Bolas >> escreveu: >> > Actually, no. `std::byte` as proposed by P0298 is explicitly *not* = a >> > typedef. It is a scoped enumeration whose underlying type is >> `unsigned >> > char`, but it is not a typedef. >> > >> > Granted, I can't agree with that design decision. I would much >> rather it be >> > a genuine type, rather than using C++ enum chicanery to get strong >> aliases. >>=20 >> That would also not be a good idea because, unsigned char is, by >> defintion, a >> byte. Why should we have (more) types that mean exactly the same >> thing, and >> this time in all platforms, by definition? >>=20 >>=20 >> Because "unsigned char" /also/ means "unsigned character". With just >> `unsigned char`, there is no way to distinguish between manipulating >> bytes and manipulating unsigned characters. >=20 > And what is "unsigned character", exactly? The standard defines a charact= er set, but leaves character encoding implementation-defined, except that i= t says that code units are representable by char. Assuming that code units = are always positive (which I don't think is mandated anywhere, but let's ke= ep things sane), you could also store code units as unsigned chars. But nei= ther char nor unsigned char represents a character, unless a code point is = equivalent to a code unit. >=20 > I think when you say "unsigned character" you should actually be saying "= code units", and at this point it's not that much different from "bytes". I= think, unsigned char should be considered as byte in every respect; if suc= h a type is added to C++, it should either be a regular typedef (std::byte_= t) or an intrinsic integral type that is equivalent to unsigned char. >=20 >> My problem with using C++ enum chicanery is that, if you use the type >> traits mechanisms to ask what `std::byte` is, it will say that it's an >> enum, not that it's an integral type. There's no reason why `byte` >> should not be an integral type. >=20 > Agreed. >=20 >> Think about it. The C++ standard allows a break in strict aliasing rules >> for `unsigned char` and `char`. Why? Not because the standard thinks >> it's reasonable for people to alias UTF-8 strings. But because that's >> the only way to pass/manipulate a byte array. And byte arrays need to be >> able to alias. >>=20 >> It's the same reason why we have `char16_t` as a distinct type from >> `uint_least16_t`. Because there is a fundamental semantic difference >> between an array of unsigned integers that are at least 16-bits in size >> and an array of UTF-16 code units. One of this is a string; the other is >> not. >>=20 >> It's time we had such a distinction for bytes. And UTF-8 code units, for >> that matter. >=20 > Thing is, unsigned char is already allowed to alias, and if we add the in= trinsic byte type that is also allowed to alias, we don't make that distinc= tion you talk about. And if we prohibit unsigned char to alias other types,= we will render lots of existing code invalid. We could add an intrinsic ch= ar8_t instead and say it will only represent narrow character code units an= d not alias other types. This way unsigned char is left as the "byte" type = and we have the other type for string processing. I think this is the real issue here: a type that is dedicated to represent = only characters and not allowed to alias anything else. Having a char* can = severely limit the compiler=E2=80=99s ability to optimize your function bec= ause of all the aliasing implications, even if you as the author know it ne= ver aliases anything because it actually *is* a string. Having a type that = clearly conveys this semantic to the compiler would be useful. --=20 You received this message because you are subscribed to the Google Groups "= ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. To view this discussion on the web visit https://groups.google.com/a/isocpp= ..org/d/msgid/std-proposals/684B5982-22CB-4357-9A1A-09F4588ADA8A%40gmail.com= .. --Apple-Mail=_76CE7E6B-C53E-47AF-A878-7CA2C4CFBBA6 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=UTF-8
On 02 Nov 2016, at 1= 0:02 , Andrey Semashev <andrey.semashev@gmail.com> wrote:

On 11/02/16 08:16, Nicol Bolas wrote:
On Wednesday, November 2, = 2016 at 12:55:46 AM UTC-4, Thiago Macieira wrote:

   Em ter=C3=A7a-feira, 1 de novembro de 2016, =C3=A0s= 20:10:28 PDT, Nicol Bolas
   escreveu:
   > Actually, no. `std::byte` as proposed by P02= 98 is explicitly *not* a
   > typedef. It i= s a scoped enumeration whose underlying type is
  &= nbsp;`unsigned
   > char`, but it is not a = typedef.
   >
  &nb= sp;> Granted, I can't agree with that design decision. I would much
   rather it be
   &g= t; a genuine type, rather than using C++ enum chicanery to get strong
   aliases.

 &nbs= p; That would also not be a good idea because, unsigned char is, by   defintion, a
   = byte. Why should we have (more) types that mean exactly the same
   thing, and
   this tim= e in all platforms, by definition?


Because "unsigned char" /also/ means "unsigned character". With just<= br class=3D"">`unsigned char`, there is no way to distinguish between manip= ulating
bytes and manipulating unsigned characters.

And what is "unsigned character", exactly? The stand= ard defines a character set, but leaves character encoding implementation-d= efined, except that it says that code units are representable by char. Assu= ming that code units are always positive (which I don't think is mandated a= nywhere, but let's keep things sane), you could also store code units as un= signed chars. But neither char nor unsigned char represents a character, un= less a code point is equivalent to a code unit.
I think when you say "unsigned character" you should actually be saying "c= ode units", and at this point it's not that much different from "bytes". I = think, unsigned char should be considered as byte in every respect; if such= a type is added to C++, it should either be a regular typedef (std::byte_t= ) or an intrinsic integral type that is equivalent to unsigned char.=

My problem with using C++ enum chicanery is that, if you use the type
traits mechanisms to ask what `std::byte` is, it will say that it= 's an
enum, not that it's an integral type. There's no reason= why `byte`
should not be an integral type.

Agreed.

Think about it. The C++ standard allows a brea= k in strict aliasing rules
for `unsigned char` and `char`. Wh= y? Not because the standard thinks
it's reasonable for people= to alias UTF-8 strings. But because that's
the only way to p= ass/manipulate a byte array. And byte arrays need to be
able = to alias.

It's the same reason why we have `ch= ar16_t` as a distinct type from
`uint_least16_t`. Because the= re is a fundamental semantic difference
between an array of u= nsigned integers that are at least 16-bits in size
and an arr= ay of UTF-16 code units. One of this is a string; the other is
not.

It's time we had such a distinction for= bytes. And UTF-8 code units, for
that matter.
=

Thing is, unsigned char is already allowed to alias, and i= f we add the intrinsic byte type that is also allowed to alias, we don't ma= ke that distinction you talk about. And if we prohibit unsigned char to ali= as other types, we will render lots of existing code invalid. We could add = an intrinsic char8_t instead and say it will only represent narrow characte= r code units and not alias other types. This way unsigned char is left as t= he "byte" type and we have the other type for string processing.
I think this is the real issue here: a type that is dedicate= d to represent only characters and not allowed to alias anything else. Havi= ng a char* can severely limit the compiler=E2=80=99s ability to optimize yo= ur function because of all the aliasing implications, even if you as the au= thor know it never aliases anything because it actually *is* a string. Havi= ng a type that clearly conveys this semantic to the compiler would be usefu= l.

--
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposa= ls+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.c= om/a/isocpp.org/d/msgid/std-proposals/684B5982-22CB-4357-9A1A-09F4588ADA8A%= 40gmail.com.
--Apple-Mail=_76CE7E6B-C53E-47AF-A878-7CA2C4CFBBA6-- .