220 26406 <1facff45-59ee-47dc-a7df-bbc72a32cca8@isocpp.org> article Path: news.gmane.org!not-for-mail From: asorenji@gmail.com Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: iswalpha and locales Date: Sun, 26 Jun 2016 07:03:49 -0700 (PDT) Lines: 154 Approved: news@gmane.org Message-ID: <1facff45-59ee-47dc-a7df-bbc72a32cca8@isocpp.org> References: <5057e854-b2ea-4c39-81c7-367bc3e54080@isocpp.org> <84d2b9e6-a29b-4c79-8646-2f00dbc5087b@isocpp.org> <1550791.ovymrLsGO4@tjmaciei-mobl1> <84d2f2a0-0fc4-4526-9826-8062bb443869@isocpp.org> <8ef0b8d3-3c57-4e54-8478-b81c2bd4207a@isocpp.org> <4d975edd-1c9c-468d-b852-28e2a2108931@isocpp.org> Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_160_1301924111.1466949829655" X-Trace: ger.gmane.org 1466949834 10211 80.91.229.3 (26 Jun 2016 14:03:54 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 26 Jun 2016 14:03:54 +0000 (UTC) Cc: asorenji@gmail.com To: ISO C++ Standard - Future Proposals Original-X-From: std-proposals+bncBDSYF7ODTYPBBRWBX65QKGQE6WB6CBY@isocpp.org Sun Jun 26 16:03:53 2016 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-vk0-f71.google.com ([209.85.213.71]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bHAfA-00037n-OW for gclcip-std-proposals@m.gmane.org; Sun, 26 Jun 2016 16:03:52 +0200 Original-Received: by mail-vk0-f71.google.com with SMTP id x7sf246091087vka.0 for ; Sun, 26 Jun 2016 07:03:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isocpp-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:message-id:in-reply-to:references:subject :mime-version:x-original-sender:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=q2xjT6Scn47teQLrAZ/HVmPL1xVPN6OYdzQDFL3a7e4=; b=VC9zwMhjN3Xs9tEYK6fCkWawQBkmqT5Us/npeORkTqt7myNxv+jOOK+cy+su3crqEn sLBUmiiadxVkVymDPACugngxUUmuwmyequIlDbTnBztG5zIJd3ud/fnPYnyr29stEu+b QPys2q888TeUcUU5LBFgCOVPtOtWZlOtpgWm2sfyrMSFL82+/eTPXIQCWOZ78ixmWjko 8wGx3z0aPhVnpwMlAVkcKhyJUz/RllsLYiyEOyy677XW95X8DbgLmJ9nlySo56eSfHGP J3MZuob0VmMDaOFfJ/EKuFsf9l5I9/ZIemAvjHoBOk6UbmlJKsgb1Dw5f8XdLqnmv1vC Q8vA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:message-id:in-reply-to:references:subject :mime-version:x-original-sender:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=q2xjT6Scn47teQLrAZ/HVmPL1xVPN6OYdzQDFL3a7e4=; b=FQjRnbCiJOBpL5nb4U4MjGCrVET5F17L7rjqnSjNNWRp8C9YtLlzzOW6SaCRwzUy/u OrUyMraQbqrsEykPa9+oqu5yt9WBREjgn8zt+swcSdUF3IprY3aR6eWI62xy/S735iNI PK66IzA04PVIQbzfzr90hmV3BNygPGcRaXj9Qc9IiQob66Mif8lN4/f1F436ovixOvs0 CW/EVJpU0vOH3VowTN5uTUIlFGD++8PGJ0rFhWNFEyaZvSD6FzJG4F0maCDYpVLtllUZ 5mjwKKvDyh0tR5XGllXj/8H8agBgnfwDLiNm9KtRkS+w6HQJQuAMKpY5dRF1dSo6Vowr HUkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:message-id:in-reply-to :references:subject:mime-version:x-original-sender:reply-to :precedence:mailing-list:list-id:x-spam-checked-in-group:list-post :list-help:list-archive:list-subscribe:list-unsubscribe; bh=q2xjT6Scn47teQLrAZ/HVmPL1xVPN6OYdzQDFL3a7e4=; b=DMAp8z+83qRnTUcRMVWU3e8cqtka5yHVk3E2ZnnhIWG5f6pP9xSvnSa23AGrMIOYG5 5cGFz19LxWS36B4A/XMpiphlL1UWz9SW4ymrBNXHwKnWnPW0WL4jrcpUcIJa0VTN7mNb kbVSUV0IPiDT8fYh7pKnrXrobKCQ+AE2FHszgcdul0ei0djHX7pX0wKVCFY7gmRAIxJx FHIZuxNQRw77mwBfnKYSC5BcWrFJF0IyaBbGZjjl4nrnPfTjIDwxwH85kSMAjyv+T83o e5XKFhe7Ci2OJ4tzvXJgA0LKNurRZo9SkIoNKj4BkUvqq3OFL9FSy/NjvTUeuaRbV+g5 GT8g== X-Gm-Message-State: ALyK8tL/NVO4k8zzYTNSlH4QAr9IAtDEKro4QsCUJdKCMMQdwqhV4xdCPXTsYkJfvr+pMw== X-Received: by 10.237.34.131 with SMTP id p3mr12798053qtc.19.1466949831483; Sun, 26 Jun 2016 07:03:51 -0700 (PDT) X-BeenThere: std-proposals@isocpp.org Original-Received: by 10.107.142.83 with SMTP id q80ls3105025iod.72.gmail; Sun, 26 Jun 2016 07:03:50 -0700 (PDT) X-Received: by 10.36.57.15 with SMTP id l15mr153174ita.7.1466949830537; Sun, 26 Jun 2016 07:03:50 -0700 (PDT) In-Reply-To: <4d975edd-1c9c-468d-b852-28e2a2108931@isocpp.org> X-Original-Sender: AsoRenji@gmail.com Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Spam-Checked-In-Group: std-proposals@isocpp.org X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:26406 Archived-At: ------=_Part_160_1301924111.1466949829655 Content-Type: multipart/alternative; boundary="----=_Part_161_325493963.1466949829656" ------=_Part_161_325493963.1466949829656 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable =D0=B2=D0=BE=D1=81=D0=BA=D1=80=D0=B5=D1=81=D0=B5=D0=BD=D1=8C=D0=B5, 26 =D0= =B8=D1=8E=D0=BD=D1=8F 2016 =D0=B3., 16:08:45 UTC+3 =D0=BF=D0=BE=D0=BB=D1=8C= =D0=B7=D0=BE=D0=B2=D0=B0=D1=82=D0=B5=D0=BB=D1=8C Nicol Bolas=20 =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB: > > On Sunday, June 26, 2016 at 2:04:59 AM UTC-4, asor...@gmail.com wrote: >> >> =D0=B2=D0=BE=D1=81=D0=BA=D1=80=D0=B5=D1=81=D0=B5=D0=BD=D1=8C=D0=B5, 26 = =D0=B8=D1=8E=D0=BD=D1=8F 2016 =D0=B3., 8:52:16 UTC+3 =D0=BF=D0=BE=D0=BB=D1= =8C=D0=B7=D0=BE=D0=B2=D0=B0=D1=82=D0=B5=D0=BB=D1=8C Patrice Roy=20 >> =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB: >>> >>> 18MB means we get away from many platforms where we are today. It's 9= =20 >>> times the size of my whole current project (with debug info in!) which = runs=20 >>> on embedded devices. Please remember that we want Unicode support, but = not=20 >>> at the cost of not being able to support target platforms which are ver= y=20 >>> much alive today. >>> >>> Unicode using codespace from 0 to 0x10FFFF. Therefore we need at most= =20 >> 0x110000 bits for single predicate function. 18 MB enough for hundred=20 >> predicates. I'm don't think you really need so many. >> > > Again, let's forget that most of that range is not actually assigned and= =20 > therefore takes up 0 bits. > > Not all of the properties in the Unicode tables are *binary*. Indeed,=20 > most are not. Case-conversion, for example, cannot be binary. It has to= =20 > specify how you go from codepoint X to one or more codepoints YZW. For ea= ch=20 > codepoint. That cannot take up a single bit per codepoint. > > Lets concentrate to predicate (return bool value) functions. I'm don't=20 believe that isalpha or isupper have this sort of problems. Although we=20 should decide what is "isupper" means in languages without upper and lower= =20 characters. In Japanese language for example. > That being said, I firmly believe that 18MB is much larger than the=20 > Unicode tables *need* to be. That there must be clever ways to make that= =20 > table much smaller (on the order of hundreds of kilobytes rather than=20 > megabytes). But as of yet, I have not undertaken the task of *proving*=20 > that, so that doesn't mean much. > > In my answer to Jeffrey Yasskin, I'm compress tables to 38 KB. But price= =20 of this - binary search with eight (log2(273)) comparison. And eight=20 problems with branch prediction unit. At least in desktop I'm prefer avoid= =20 this. > --=20 You received this message because you are subscribed to the Google Groups "= ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. To view this discussion on the web visit https://groups.google.com/a/isocpp= ..org/d/msgid/std-proposals/1facff45-59ee-47dc-a7df-bbc72a32cca8%40isocpp.or= g. ------=_Part_161_325493963.1466949829656 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


=D0=B2=D0=BE=D1=81=D0=BA=D1=80=D0=B5=D1=81=D0=B5= =D0=BD=D1=8C=D0=B5, 26 =D0=B8=D1=8E=D0=BD=D1=8F 2016 =D0=B3., 16:08:45 UTC+= 3 =D0=BF=D0=BE=D0=BB=D1=8C=D0=B7=D0=BE=D0=B2=D0=B0=D1=82=D0=B5=D0=BB=D1=8C = Nicol Bolas =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB:
On Sunday, June 26, 2016 at 2:04:= 59 AM UTC-4, asor...@gmail.com wrote:
=D0=B2=D0=BE=D1=81=D0=BA=D1=80=D0=B5=D1=81=D0=B5= =D0=BD=D1=8C=D0=B5, 26 =D0=B8=D1=8E=D0=BD=D1=8F 2016 =D0=B3., 8:52:16 UTC+3= =D0=BF=D0=BE=D0=BB=D1=8C=D0=B7=D0=BE=D0=B2=D0=B0=D1=82=D0=B5=D0=BB=D1=8C P= atrice Roy =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB:
18MB means we get away from many p= latforms where we are today. It's 9 times the size of my whole current = project (with debug info in!) which runs on embedded devices. Please rememb= er that we want Unicode support, but not at the cost of not being able to s= upport target platforms which are very much alive today.

Unicode using codespace from 0 to 0x10FFFF. Therefore we= need at most 0x110000 bits for single predicate function. 18 MB enough for= hundred predicates. I'm don't think you really need so many.

Again, let's forget that most of that r= ange is not actually assigned and therefore takes up 0 bits.

Not all= of the properties in the Unicode tables are binary. Indeed, most ar= e not. Case-conversion, for example, cannot be binary. It has to specify ho= w you go from codepoint X to one or more codepoints YZW. For each codepoint= .. That cannot take up a single bit per codepoint.

Lets concentrate to predicate (return bool value) functions. I&= #39;m don't believe that isalpha or isupper have this sort of problems.= Although we should decide what is "isupper" means in languages w= ithout upper and lower characters. In Japanese language for example.
<= blockquote class=3D"gmail_quote" style=3D"margin: 0;margin-left: 0.8ex;bord= er-left: 1px #ccc solid;padding-left: 1ex;">
That bein= g said, I firmly believe that 18MB is much larger than the Unicode tables <= i>need to be. That there must be clever ways to make that table much sm= aller (on the order of hundreds of kilobytes rather than megabytes). But as= of yet, I have not undertaken the task of proving that, so that doe= sn't mean much.

In my answer to Je= ffrey Yasskin, I'm compress tables to 38 KB. But price of this - binary= search with eight (log2(273)) comparison. And eight problems with branch p= rediction unit. At least in desktop I'm prefer avoid this.

--
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposa= ls+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.= com/a/isocpp.org/d/msgid/std-proposals/1facff45-59ee-47dc-a7df-bbc72a32cca8= %40isocpp.org.
------=_Part_161_325493963.1466949829656-- ------=_Part_160_1301924111.1466949829655-- .