220 29341 <2145466f-4680-4489-8728-258fd4374aff@isocpp.org> article Path: news.gmane.org!.POSTED!not-for-mail From: Nicol Bolas Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: Re: std::byte thoughts Date: Wed, 2 Nov 2016 06:55:29 -0700 (PDT) Lines: 253 Approved: news@gmane.org Message-ID: <2145466f-4680-4489-8728-258fd4374aff@isocpp.org> References: <7da80620-e4bd-40eb-a4ec-4f3c2dcb2c98@isocpp.org> <2350841.Z7lUCKSumT@tjmaciei-mobl1> <3e64b220-79ae-4108-ba1c-9785494fc434@isocpp.org> <2128372.NTAZafObDQ@tjmaciei-mobl1> Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_2423_1043699034.1478094929894" X-Trace: blaine.gmane.org 1478094943 1174 195.159.176.226 (2 Nov 2016 13:55:43 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 2 Nov 2016 13:55:43 +0000 (UTC) To: ISO C++ Standard - Future Proposals Original-X-From: std-proposals+bncBCEKFTV6ZUMBBUXA47AAKGQE4GZV6RA@isocpp.org Wed Nov 02 14:55:39 2016 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-it0-f70.google.com ([209.85.214.70]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1c1w0m-0006uo-Tn for gclcip-std-proposals@m.gmane.org; Wed, 02 Nov 2016 14:55:29 +0100 Original-Received: by mail-it0-f70.google.com with SMTP id e187sf44671385itc.0 for ; Wed, 02 Nov 2016 06:55:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isocpp-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:message-id:in-reply-to:references:subject:mime-version :x-original-sender:reply-to:precedence:mailing-list:list-id :x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=6OLJCVLZjlnI0B1De6UcsEuhnYFpdTMbZL0HNo0J5v4=; b=ZhB+NjoGoe5oNEO4dFYIqXpYW6bu04jpHwyQthfCZOSfzxpv0rpjcqUPLaNFzm/YG2 8SvKMwekBWvXM7wGql8BZtz9Uhs5/NR0Ti0srH3pAKbCMITuZxx2f2KP9q7AR7/myPL8 mkT1SBr8djBdmCdF28CCdCqYAODy7Ms6vIqTwEf9vtkLRwQe3iqCkZMfxbWbiFUpZ3mH cbnpa8iGPRYKXuiGqrmQ1UKiPmhFYUevJuDYC4dlBzyEamse67P8yZoGwf+/jNMoTaQs YZGu8iLskyOZ7ZEZHireQL4iZPDEKy/WMhlrs9bGmc5Ug8lec/BtQAMdps6E03LGhqlv kjvA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:message-id:in-reply-to:references:subject:mime-version :x-original-sender:reply-to:precedence:mailing-list:list-id :x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=6OLJCVLZjlnI0B1De6UcsEuhnYFpdTMbZL0HNo0J5v4=; b=gIAwsL4RCZ3tmG9UsBjXG327OhxXKGACTk2JBccPuEPKlv5FdQMiWTUq/pWulFVQm1 FLI3P0Eu5jj2Sj4Gf41ApP5aTZAwUG64pvbowJ3611r7qo4BBhivKIrCqS1A7iiweXMy p/ueE/XGzMt2598g4GNKB5EOinfBtpT2WfTOa600zAKc9jBry1TGJSr9PTVl9iaGm9dz uBZa+fwfjIQWsvgFGF4tHlDaF5nhIgzIgHPGKirKkGHo1tbBwsznYE5lcPMQwItAMCpo IF8MwQ/mPNSeFDat610H6t+pe+l/g8qbEoBh+rLxK5dl3MdW7NMGJdeMg/T5/Gv8loRQ gVGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:message-id:in-reply-to:references :subject:mime-version:x-original-sender:reply-to:precedence :mailing-list:list-id:x-spam-checked-in-group:list-post:list-help :list-archive:list-subscribe:list-unsubscribe; bh=6OLJCVLZjlnI0B1De6UcsEuhnYFpdTMbZL0HNo0J5v4=; b=ZqbvhMcymjyLgBjsbaGGgDZmJnb9o3xVlN346x36qcc6kEJql1Tuqd3kRwa6LCRZTr IgCvOMNPq2OBxdpwTa9o9eg/Q/ktNnJ3hNR8pTPhcN7sHjSrCo21istgM5xg1eWcBiA/ g8ziKuhu7m4gCCno4JHyjFgV6/MsqI5MUIxU8IaUUwoZbRWbvNRhkUoDcoujI04IuqWu sMGh4L15af2UfxeKmeZbDw0TeeJ3kWwPDfthmR5gm7KHQmzlUPDJyV5e2reU+MPcfBeD q1QxSYZCKLp1KtygCbxYggqA8Ay097YGCFPR+sMYSC86gZ49rQ70PqIyJLnEExPnLRyB +54A== X-Gm-Message-State: ABUngvdTscbiYhRVxA3AOmsxf0IZmVcv8V/DCYF45sj82SiG3FZOt968BynnHeEkgYMKQw== X-Received: by 10.36.33.131 with SMTP id e125mr3035039ita.27.1478094931126; Wed, 02 Nov 2016 06:55:31 -0700 (PDT) X-BeenThere: std-proposals@isocpp.org Original-Received: by 10.157.13.110 with SMTP id 101ls17308871oti.25.gmail; Wed, 02 Nov 2016 06:55:30 -0700 (PDT) X-Received: by 10.157.32.11 with SMTP id n11mr271965ota.4.1478094930331; Wed, 02 Nov 2016 06:55:30 -0700 (PDT) In-Reply-To: <2128372.NTAZafObDQ@tjmaciei-mobl1> X-Original-Sender: jmckesson@gmail.com Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:29341 Archived-At: ------=_Part_2423_1043699034.1478094929894 Content-Type: multipart/alternative; boundary="----=_Part_2424_1028791155.1478094929894" ------=_Part_2424_1028791155.1478094929894 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Wednesday, November 2, 2016 at 2:43:35 AM UTC-4, Thiago Macieira wrote: > > Em ter=C3=A7a-feira, 1 de novembro de 2016, =C3=A0s 22:16:32 PDT, Nicol B= olas=20 > escreveu:=20 > > On Wednesday, November 2, 2016 at 12:55:46 AM UTC-4, Thiago Macieira=20 > wrote:=20 > > > That would also not be a good idea because, unsigned char is, by=20 > > > defintion, a=20 > > > byte. Why should we have (more) types that mean exactly the same=20 > thing,=20 > > > and=20 > > > this time in all platforms, by definition?=20 > >=20 > > Because "unsigned char" *also* means "unsigned character". With just=20 > > `unsigned char`, there is no way to distinguish between manipulating=20 > bytes=20 > > and manipulating unsigned characters.=20 > >=20 > > That's what `byte` is for, as a type: a way to semantically=20 > differentiate=20 > > between operations on bytes and operations on characters. The types can= =20 > be=20 > > inter-convertible, numerically speaking, but they don't mean the same= =20 > thing.=20 > > I'm sorry, I don't agree that there's a distinction in the first place.= =20 > Bytes=20 > are used more often than just copying around. If you add, subtract, shift= =20 > left=20 > or right, perform bitwise operations, etc, you need the value. If I need= =20 > the=20 > value, then a zero is a zero is a zero, a 0x40 is still a 0x40.=20 > > Also, I can assign 'a' to any integer type. Maybe this was the main issue= :=20 > that single-quote character literals automatically convert to integral,= =20 > instead of staying a character. We're 40 years too late to change this,= =20 > though=20 > (since B).=20 > > > My problem with using C++ enum chicanery is that, if you use the type= =20 > > traits mechanisms to ask what `std::byte` is, it will say that it's an= =20 > > enum, not that it's an integral type. There's no reason why `byte`=20 > should=20 > > not be an integral type.=20 > > Agreed, but I'm going to go further and say that it's pretty useless for = a=20 > lot=20 > of use-cases. I need a value of a lot of operations and an enum won't giv= e=20 > it=20 > to me unless I cast it to a suitable integral in the first place --=20 > unsigned=20 > char (that is, a *real* byte).=20 > > This week I've been spending time working on hashing algorithms, notably= =20 > SipHash (btw, implementations should reconsider their std::hash=20 > algorithms).=20 > In order to implement it, I needed to access the byte array and do=20 > byte-level=20 > operations like rotating left, XOR, and additions. Not only would=20 > std::byte=20 > not work for me, I fail to see how the operations I'm doing are any=20 > different=20 > than the operations on an unsigned char. > P0298's std::byte permits all of those operations via operator overloading. > It's the same reason why we have `char16_t` as a distinct type from=20 > > `uint_least16_t`. Because there is a fundamental semantic difference=20 > > between an array of unsigned integers that are at least 16-bits in size= =20 > and=20 > > an array of UTF-16 code units. One of this is a string; the other is=20 > not.=20 > > The only benefit I see there is allowing overloading.=20 > > But while that may be true, what's the point of an *unsigned* char? If yo= u=20 > want to do character operations, you use char. If you're using unsigned= =20 > char,=20 > that's because you want a byte, plain and simple. By this argument, we=20 > already=20 > have the distinction between character operations and byte operations. > You must not do much UTF-8 work. Because `char` can be signed or unsigned,= =20 the only reasonable way to manipulate UTF-8 code units is to use `unsigned= =20 char`. While a signed `char` is required to be able to store UTF-8 code=20 units, you don't want to invoke implementation-defined behavior when=20 bitshifting signed types. So you have to use `unsigned char`. So it's hardly unreasonable to pass UTF-8 strings around via `unsigned=20 char*`, rather than `char*`. --=20 You received this message because you are subscribed to the Google Groups "= ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. To view this discussion on the web visit https://groups.google.com/a/isocpp= ..org/d/msgid/std-proposals/2145466f-4680-4489-8728-258fd4374aff%40isocpp.or= g. ------=_Part_2424_1028791155.1478094929894 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On Wednesday, November 2, 2016 at 2:43:35 AM UTC-4, Thiago= Macieira wrote:
Em ter=C3=A7a-= feira, 1 de novembro de 2016, =C3=A0s 22:16:32 PDT, Nicol Bolas escreveu:
> On Wednesday, November 2, 2016 at 12:55:46 AM UTC-4, Thiago Maciei= ra wrote:
> > That would also not be a good idea because, unsigned char is,= by
> > defintion, a
> > byte. Why should we have (more) types that mean exactly the s= ame thing,
> > and
> > this time in all platforms, by definition?
>=20
> Because "unsigned char" *also* means "unsigned char= acter". With just
> `unsigned char`, there is no way to distinguish between manipulati= ng bytes
> and manipulating unsigned characters.
>=20
> That's what `byte` is for, as a type: a way to semantically di= fferentiate
> between operations on bytes and operations on characters. The type= s can be
> inter-convertible, numerically speaking, but they don't mean t= he same thing.

I'm sorry, I don't agree that there's a distinction in the = first place. Bytes=20
are used more often than just copying around. If you add, subtract, shi= ft left=20
or right, perform bitwise operations, etc, you need the value. If I nee= d the=20
value, then a zero is a zero is a zero, a 0x40 is still a 0x40.

Also, I can assign 'a' to any integer type. Maybe this was the = main issue:=20
that single-quote character literals automatically convert to integral,= =20
instead of staying a character. We're 40 years too late to change t= his, though=20
(since B).

> My problem with using C++ enum chicanery is that, if you use the t= ype
> traits mechanisms to ask what `std::byte` is, it will say that it&= #39;s an
> enum, not that it's an integral type. There's no reason wh= y `byte` should
> not be an integral type.

Agreed, but I'm going to go further and say that it's pretty us= eless for a lot=20
of use-cases. I need a value of a lot of operations and an enum won'= ;t give it=20
to me unless I cast it to a suitable integral in the first place -- uns= igned=20
char (that is, a *real* byte).

This week I've been spending time working on hashing algorithms, no= tably=20
SipHash (btw, implementations should reconsider their std::hash algorit= hms).=20
In order to implement it, I needed to access the byte array and do byte= -level=20
operations like rotating left, XOR, and additions. Not only would std::= byte=20
not work for me, I fail to see how the operations I'm doing are any= different=20
than the operations on an unsigned char.

P0298= 's std::byte permits all of those operations via operator overloading.<= br>
> It's the same reason why we have `char16_t` as a distinct type fro= m
> `uint_least16_t`. Because there is a fundamental semantic differen= ce
> between an array of unsigned integers that are at least 16-bits in= size and
> an array of UTF-16 code units. One of this is a string; the other = is not.

The only benefit I see there is allowing overloading.

But while that may be true, what's the point of an *unsigned* char?= If you=20
want to do character operations, you use char. If you're using unsi= gned char,=20
that's because you want a byte, plain and simple. By this argument,= we already=20
have the distinction between character operations and byte operations.<= br>

You must not do much UTF-8 work. Because `char` ca= n be signed or unsigned, the only reasonable way to manipulate UTF-8 code u= nits is to use `unsigned char`. While a signed `char` is required to be abl= e to store UTF-8 code units, you don't want to invoke implementation-de= fined behavior when bitshifting signed types. So you have to use `unsigned = char`.

So it's hardly unreasonable to pass UTF-8 strings around = via `unsigned char*`, rather than `char*`.

--
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposa= ls+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.= com/a/isocpp.org/d/msgid/std-proposals/2145466f-4680-4489-8728-258fd4374aff= %40isocpp.org.
------=_Part_2424_1028791155.1478094929894-- ------=_Part_2423_1043699034.1478094929894-- .