220 29349 <0092ba40-3c2a-23ef-5a60-2512037b1489@honermann.net> article Path: news.gmane.org!.POSTED!not-for-mail From: Tom Honermann Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: Re: std::byte thoughts Date: Wed, 2 Nov 2016 11:14:26 -0400 Lines: 43 Approved: news@gmane.org Message-ID: <0092ba40-3c2a-23ef-5a60-2512037b1489@honermann.net> References: <7da80620-e4bd-40eb-a4ec-4f3c2dcb2c98@isocpp.org> <1549171.TC16PSHVVd@tjmaciei-mobl1> <2350841.Z7lUCKSumT@tjmaciei-mobl1> <3e64b220-79ae-4108-ba1c-9785494fc434@isocpp.org> <97ebfc84-c5a3-eaf5-efcb-c1bdf9b507a5@gmail.com> <5933359a-3f59-33df-10db-e643bfa39c55@honermann.net> Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed X-Trace: blaine.gmane.org 1478099680 17080 195.159.176.226 (2 Nov 2016 15:14:40 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 2 Nov 2016 15:14:40 +0000 (UTC) User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 To: std-proposals@isocpp.org Original-X-From: std-proposals+bncBDBNXSHG6UDBBVEF5DAAKGQE6XPP2KQ@isocpp.org Wed Nov 02 16:14:36 2016 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-vk0-f71.google.com ([209.85.213.71]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1c1xFC-0002gw-6e for gclcip-std-proposals@m.gmane.org; Wed, 02 Nov 2016 16:14:26 +0100 Original-Received: by mail-vk0-f71.google.com with SMTP id 19sf11521888vko.0 for ; Wed, 02 Nov 2016 08:14:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isocpp-org.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:x-original-sender:x-original-authentication-results :reply-to:precedence:mailing-list:list-id:x-spam-checked-in-group :list-post:list-help:list-archive:list-subscribe:list-unsubscribe; bh=dfr7vunVu04KFZesawq2EjMp8mpAgVjuo0Q0uVI+ET4=; b=w9TSk09TadrljYQRbmhxLnX71ayk6+iOA1ijCG/kXTGOnQE6AIBC8TF5NudurIraxg ujOyEmVrx1O1wEgRbOCfo4WkPpqTHJD3vtjOgw5n8TXS+ConbFD0PYxnzRiGBbXfq1FA UM52zKbsinx0M6Qjcqvs4UJPOBRmXGguLYFj6TKOa+DAoPRQ/Mss4iNHUKRTN8MCDsZR KiaSTzBS6ZPNAz1l8AwnpK7fJVL8td6D0zekmfHciuz7NrKEnHa5AarDiWapRL3vUP4R OeuLdqs/0AJlU4XDSiUGu6vF53K/2xMRfES1EejQyqtm1HYUMLRmHZGvM4++hC+/2sry UaVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=dfr7vunVu04KFZesawq2EjMp8mpAgVjuo0Q0uVI+ET4=; b=TnVPSJT7fEcYCfUEhxan0hJ4en+j+KsgICLsAiT5SA2I2+g/XfLrEbJqDYXeu9o1Re paPfH9gbz0gXlgS+k9fRxC8Vu8ctLEao8kdoAfccoyLcpV5PMMTga1a8WnOKT3aaxG+5 eKUe3DKP/pn9dAZNIEEZKOmMz4/bKgAjtUCLSDuygpbe81uPRqVnzEUveG4Y2ogHtnaG FOqxAZ9n5oWEExk880wGXQmw1t3zbf0xRw48x5wSyPAp36iT+xuxodOnUtrycIHKK2ce ku7XVoWl5NbguI4Cb2eIn8AQFH5XAwsvbcr7chJDjVbmqVKTOCH1Jmer0eP/1MSbPvdC sO2w== X-Gm-Message-State: ABUngvc3Ub5iFxWWZJ3lUydpcT/KG0fY5TZihhSiWTCEMmcaJkQbjaTq0xm+GBBXFnZG7Q== X-Received: by 10.31.14.1 with SMTP id 1mr1014640vko.31.1478099668930; Wed, 02 Nov 2016 08:14:28 -0700 (PDT) X-BeenThere: std-proposals@isocpp.org Original-Received: by 10.157.40.182 with SMTP id s51ls18143967ota.14.gmail; Wed, 02 Nov 2016 08:14:28 -0700 (PDT) X-Received: by 10.37.44.135 with SMTP id s129mr3036325ybs.40.1478099668052; Wed, 02 Nov 2016 08:14:28 -0700 (PDT) Original-Received: from smtp107.iad3a.emailsrvr.com (smtp107.iad3a.emailsrvr.com. [173.203.187.107]) by mx.google.com with ESMTPS id f131si1522017qkb.7.2016.11.02.08.14.27 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 02 Nov 2016 08:14:27 -0700 (PDT) Received-SPF: neutral (google.com: 173.203.187.107 is neither permitted nor denied by best guess record for domain of tom@honermann.net) client-ip=173.203.187.107; Original-Received: from smtp38.relay.iad3a.emailsrvr.com (localhost [127.0.0.1]) by smtp38.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id C587256C4 for ; Wed, 2 Nov 2016 11:14:27 -0400 (EDT) X-Auth-ID: tom@honermann.net Original-Received: by smtp38.relay.iad3a.emailsrvr.com (Authenticated sender: tom-AT-honermann.net) with ESMTPSA id B44E45E20 for ; Wed, 2 Nov 2016 11:14:27 -0400 (EDT) X-Sender-Id: tom@honermann.net Original-Received: from [10.12.196.149] (pool-71-176-249-9.rcmdva.fios.verizon.net [71.176.249.9]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA) by 0.0.0.0:587 (trex/5.7.7); Wed, 02 Nov 2016 11:14:27 -0400 In-Reply-To: X-Original-Sender: tom@honermann.net X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 173.203.187.107 is neither permitted nor denied by best guess record for domain of tom@honermann.net) smtp.mailfrom=tom@honermann.net Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:29349 Archived-At: On 11/2/2016 10:50 AM, Andrey Semashev wrote: > On 11/02/16 17:01, Tom Honermann wrote: >> On 11/02/2016 05:02 AM, Andrey Semashev wrote: >>> Assuming that code units are always positive (which I don't think is >>> mandated anywhere, but let's keep things sane), >> >> I don't know of any encodings that specify negative values for code >> units or code points. However, the mapping of UTF-8 code units to char >> results in negative code unit values in UTF-8 strings for >> implementations that use a signed 8-bit char. So, in practice, sanity >> does not always prevail. This is one of the motivations for P0482R0 >> . > > AFAIR, Unicode does not define negative code units. The fact that code > units greater than 0x7f cannot be represented in 8-bit signed `char` > simply means overflow and implementation-defined behavior. In other > words, UTF-8 code units cannot be represented by `char` on such > platforms. The standard specifies behavior a little stronger than that: C++14 [basic.fundamental] 3.9.1p1: "For each value i of type unsigned char in the range 0 to 255 inclusive, there exists a value j of type char such that the result of an integral conversion (4.7) from i to char is j, and the result of an integral conversion from j to unsigned char is i." So, arguably, UTF-8 code units with value greater than 0x7f must be representable in char, though their values are mapped in an implementation defined way. Obtaining the UTF-8 code unit value requires conversion to unsigned char. This means that it is possible to fully work with UTF-8 data, including UTF-8 character and string literals, though it is quite frustrating and error prone. Tom. -- You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/0092ba40-3c2a-23ef-5a60-2512037b1489%40honermann.net. .