220 26338 <66a5b78e-5445-40c8-80e7-c550416bf0cf@isocpp.org> article Path: news.gmane.org!not-for-mail From: Nicol Bolas Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: Re: Comments on P0372R0, A type for utf-8 data Date: Wed, 15 Jun 2016 08:56:32 -0700 (PDT) Lines: 211 Approved: news@gmane.org Message-ID: <66a5b78e-5445-40c8-80e7-c550416bf0cf@isocpp.org> References: <5760C3A8.5060804@honermann.net> <5b62dc8c-b02e-46ec-91cd-2598965a73ff@isocpp.org> <5760CF11.20807@honermann.net> Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_694_1215804870.1466006192419" X-Trace: ger.gmane.org 1466006200 30629 80.91.229.3 (15 Jun 2016 15:56:40 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 15 Jun 2016 15:56:40 +0000 (UTC) Cc: bigcheesegs@gmail.com, dccitaliano@gmail.com To: ISO C++ Standard - Future Proposals Original-X-From: std-proposals+bncBCEKFTV6ZUMBBMPVQW5QKGQESPLQEMI@isocpp.org Wed Jun 15 17:56:37 2016 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-it0-f72.google.com ([209.85.214.72]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bDDBD-0001i8-KB for gclcip-std-proposals@m.gmane.org; Wed, 15 Jun 2016 17:56:35 +0200 Original-Received: by mail-it0-f72.google.com with SMTP id z189sf55707662itg.2 for ; Wed, 15 Jun 2016 08:56:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isocpp-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:message-id:in-reply-to:references:subject :mime-version:x-original-sender:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=8+uQekuTosjtM8ulY7qLbOGbL6tILrWUvrTx+pg4CeM=; b=AvsqfuJMAKGsEPviVHKkRkx1NCJfTpcQ96K4cjHDl7AVR2SJMacOUyD0Em9RT4SP+X axfOlgUnCcn7eTN4dEtVpHAYoFc2bvQoBK5KMT5J5uCQq2MPdnehUPydfrrQVy9FysDw tP4uNxwYhFIKKfFf0o8K/EjYzmJoDTeYfi6BqV36ODaoy9JnNr5fKYE8xBgDh5g+8sas 0Qa7t5XzVNOgFzoA+fOF2TfrWXZjyBo+x4NRZD+iJprc8HNUtetoc9o7Ih0gf5MCy8l6 lsde2PsvTbyXNLyfK45HXCdN5ywGixdkJZDM9CCIXRKM/uhUxxLu3cgbWnPoJpWLjMvm Scnw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:message-id:in-reply-to:references:subject :mime-version:x-original-sender:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=8+uQekuTosjtM8ulY7qLbOGbL6tILrWUvrTx+pg4CeM=; b=gQ1uprXEGRQDdLGbxU0yxm6zuqCyJ4PolMDpwMte3x00ZpXGOO6B+CCIxJH3p+L3qp mvmH8JvDnTMO26VgiVA3wwU81TgWLuVtQaoRjH1Rf3EIYqZ8KpjChvEmPWrymE0yIxbS x0p22KkkP0cnSbtJaXhy5v3pstdKhqZ8IR5AimRk08YjRqd/f7hDO8XyyJc00HGQwkij JZonpgX5A0canWFDJgq1hqfD2GxBiPaVkeUywa0V6vqZ7P1RZsnm246hPqodO7hrsh9l ie4soXbCCxvyen0cFvOpy+f2AOoqxmWxjNBIQqBsiq/YGysC6k1KtPVwaEnqfx2gKldE 04zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:message-id:in-reply-to :references:subject:mime-version:x-original-sender:reply-to :precedence:mailing-list:list-id:x-spam-checked-in-group:list-post :list-help:list-archive:list-subscribe:list-unsubscribe; bh=8+uQekuTosjtM8ulY7qLbOGbL6tILrWUvrTx+pg4CeM=; b=aWfyF9avZ3eBoLpK6W4ROvYRDmUQAaY6/bHDDF5mnwaAlkNKo6unWpLfchILVAkmq5 7bSgWMmy+/eUW5566+aA+3YjiQAWgjQjdcfhHqFQhIFkz4aAdiTYepCty3Yq7FHEUYsx 0mMQlYupHWjPtG/MDqi6wgwL9ACRX/fxSRhyCJOGsnz9DeIOxjD1W7m5gDH6Yk+LQVsM 2JXYU8i0Q9JmbjBfzXxKMxFYUKiRUTP33Cn3Ccz8N01iPO+5AbvBiY7smw2UGwLExdhy khbHdqyqRT0+ti86Zd7uIefhe+0LwM0KlyTxD0wLT0f97MW5H144XRR1qByG5ajT9kef baDg== X-Gm-Message-State: ALyK8tJAshJ9Vs732541lAldin+StbA7oHNEj1wJcCofXOUfYNmfd8M3VcHEaXgpYO4aMA== X-Received: by 10.157.24.90 with SMTP id t26mr21299904ott.10.1466006194571; Wed, 15 Jun 2016 08:56:34 -0700 (PDT) X-BeenThere: std-proposals@isocpp.org Original-Received: by 10.107.27.145 with SMTP id b139ls296693iob.47.gmail; Wed, 15 Jun 2016 08:56:33 -0700 (PDT) X-Received: by 10.36.113.199 with SMTP id n190mr11697itc.0.1466006193448; Wed, 15 Jun 2016 08:56:33 -0700 (PDT) In-Reply-To: <5760CF11.20807@honermann.net> X-Original-Sender: jmckesson@gmail.com Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Spam-Checked-In-Group: std-proposals@isocpp.org X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:26338 Archived-At: ------=_Part_694_1215804870.1466006192419 Content-Type: multipart/alternative; boundary="----=_Part_695_869998225.1466006192419" ------=_Part_695_869998225.1466006192419 Content-Type: text/plain; charset=UTF-8 On Tuesday, June 14, 2016 at 11:44:19 PM UTC-4, Tom Honermann wrote: > > On 06/14/2016 11:32 PM, Nicol Bolas wrote: > > On Tuesday, June 14, 2016 at 10:55:38 PM UTC-4, Tom Honermann wrote: >> >> First, thank you for writing this paper! It has been on my todo list to >> write such a proposal, but alas... >> >> I spoke with Richard Smith about such a proposal in Jacksonville and he >> mentioned a further justification for supporting a char8_t type - >> optimization. Today, compilers are limited in optimizing code involving >> char and unsigned char glvalues because these types are allowed to alias >> objects of other types (C++14 3.10 [basic.lval] p10). If a char8_t type >> were to be added that adhered to strict aliasing, then compilers could >> more aggressively optimize code involving it. I think this may be a >> benefit worth adding to the paper. >> > > I'm quite certain that the proposal makes this illegal: > > const char8_t *str = "Some String"; > > > I would hope so. > > `char8_t` is meant for UTF-8 strings *only*. And most people's strings > are narrow character strings; on specific platforms, this may work out to > being UTF-8, but there is no guarantee of that. We need to differentiate > between narrow character strings and UTF-8 encoded strings at the type > level. > > The last thing we want is to encourage people to do this: > > auto str = (const char8_t *)"Some String"; > > > I agree. > > If people start trying doing casts like that to take advantage of more > aggressive optimizations, then we'll be right back where we were before: we > won't know if a string *really is* UTF-8 or not. > > Solving the "char as byte array and string" problem is important. But we > shouldn't suggest that `char8_t` constitutes such a solution. > > > I don't think the ability to abuse a feature should be sufficient > justification to not add it. I did not intend to suggest that char8_t be > used to circumvent existing aliasing rules. Rather, that giving it strict > aliasing behavior would enable optimizations for UTF-8 data. That could > potentially provide some motivation towards using UTF-8 strings in > preference to narrow strings. > Right, but it already has that. `char8_t`, based on the "unique, unsigned type" statement in the proposal, is a different type from `char` and `unsigned char`. It has the same value representation as those two, but the way strict aliasing is defined already does not allow `char8_t*` to alias with other types. Just as it doesn't allow `char16_t*` or `char32_t*` to do so. The same goes for enums who use `char` as their underlying types; arrays of them are not `char*`s to the strict aliasing rules. The strict aliasing rules do not care what the underlying type of something is. What I'm saying is that we shouldn't *advertise* this as a selling point of the feature. It shouldn't be listed in the motivation section, for example. Otherwise you will encourage people to abuse it. -- You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/66a5b78e-5445-40c8-80e7-c550416bf0cf%40isocpp.org. ------=_Part_695_869998225.1466006192419 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On Tuesday, June 14, 2016 at 11:44:19 PM UTC-4, Tom Honerm= ann wrote:
=20 =20 =20
On 06/14/2016 11:32 PM, Nicol Bolas wrote:
On Tuesday, June 14, 2016 at 10:55:38 PM UTC-4, Tom Honermann wrote:
First, thank you for writing this paper! =C2=A0It has been on my todo li= st to
write such a proposal, but alas...

I spoke with Richard Smith about such a proposal in Jacksonville and he
mentioned a further justification for supporting a char8_t type -
optimization. =C2=A0Today, compilers are limited in optimizing co= de involving
char and unsigned char glvalues because these types are allowed to alias
objects of other types (C++14 3.10 [basic.lval] p10). =C2=A0If a char8_t type
were to be added that adhered to strict aliasing, then compilers could
more aggressively optimize code involving it. =C2=A0I think this may be a
benefit worth adding to the paper.

I'm quite certain that the proposal makes this illegal:

const char8_t *str =3D "Some String";

I would hope so.

`char8_t` is meant for UTF-8 strings only. And most people's strings are narrow character strings; on specific platforms, this may work out to being UTF-8, but there is no guarantee of that. We need to differentiate between narrow character strings and UTF-8 encoded strings at the type level.
The last thing we want is to encourage people to do this:

auto str =3D (const char8_t *)"Some String";

I agree.

If people start trying doing casts like that to take advantage of more aggressive optimizations, then we'll be right back where we were before: we won't know if a string really is UTF-8 or not.

Solving the "char as byte array and string" problem is important. But we shouldn't suggest that `char8_t` constitute= s such a solution.

I don't think the ability to abuse a feature should be sufficient justification to not add it.=C2=A0 I did not intend to suggest that char8_t be used to circumvent existing aliasing rules.=C2=A0 Rather, th= at giving it strict aliasing behavior would enable optimizations for UTF-8 data.=C2=A0 That could potentially provide some motivation toward= s using UTF-8 strings in preference to narrow strings.

Right, but it already has that. `char8_t`, based on the "= unique, unsigned type" statement in the proposal, is a different type = from `char` and `unsigned char`. It has the same value representation as th= ose two, but the way strict aliasing is defined already does not allow `cha= r8_t*` to alias with other types. Just as it doesn't allow `char16_t*` = or `char32_t*` to do so. The same goes for enums who use `char` as their un= derlying types; arrays of them are not `char*`s to the strict aliasing rule= s.

The strict aliasing rules do not care what the underlying type of= something is.

What I'm saying is that we shouldn't adver= tise this as a selling point of the feature. It shouldn't be listed= in the motivation section, for example. Otherwise you will encourage peopl= e to abuse it.

--
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposa= ls+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.= com/a/isocpp.org/d/msgid/std-proposals/66a5b78e-5445-40c8-80e7-c550416bf0cf= %40isocpp.org.
------=_Part_695_869998225.1466006192419-- ------=_Part_694_1215804870.1466006192419-- .