220 26407 article Path: news.gmane.org!not-for-mail From: Nicol Bolas Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: iswalpha and locales Date: Sun, 26 Jun 2016 08:12:51 -0700 (PDT) Lines: 144 Approved: news@gmane.org Message-ID: References: <5057e854-b2ea-4c39-81c7-367bc3e54080@isocpp.org> <84d2b9e6-a29b-4c79-8646-2f00dbc5087b@isocpp.org> <1550791.ovymrLsGO4@tjmaciei-mobl1> Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_3692_2020040332.1466953971433" X-Trace: ger.gmane.org 1466953983 3752 80.91.229.3 (26 Jun 2016 15:13:03 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 26 Jun 2016 15:13:03 +0000 (UTC) Cc: asorenji@gmail.com To: ISO C++ Standard - Future Proposals Original-X-From: std-proposals+bncBCEKFTV6ZUMBB5HBX65QKGQEMZOC43I@isocpp.org Sun Jun 26 17:12:55 2016 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-it0-f69.google.com ([209.85.214.69]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bHBjy-0000yV-ID for gclcip-std-proposals@m.gmane.org; Sun, 26 Jun 2016 17:12:54 +0200 Original-Received: by mail-it0-f69.google.com with SMTP id g8sf141740778itb.2 for ; Sun, 26 Jun 2016 08:12:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isocpp-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:message-id:in-reply-to:references:subject :mime-version:x-original-sender:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=uFMILhQHKlSTq7k+lOUSbNTO9llwBAGmeDUvdoCTEYc=; b=b7TFy9GGk7h4DDYZXci+gdH56h/lRwF4WrJcDBLBA4nIqV3pW8R+qxGVLgWJruXt3V qAeXCvIbjo4AUuD0mU2ubSMgJyQ2mQKrfAs1KWSNZ5zkQQJOlyXVM4uSHaMRqMDNS0ME 6IlhTpMtZc8QIxYsug5Y3NYwDksy+HNTdS3HL6BDg8Vmcr/T93X4GqVBGYxCT8SoBIcz g8l27j7TwryDXmPqrVsRen2w93db1iFhla/BSU2Wy5lmiXd4FlpHMIzIZFYosJlX9/30 DefXz471GXN6kGzL8l7GjHHQNNhk5MaAQ6j4Rd+4m+XIuDk4DFAGAgEiEVLtGh3xe3py hQFw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:message-id:in-reply-to:references:subject :mime-version:x-original-sender:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=uFMILhQHKlSTq7k+lOUSbNTO9llwBAGmeDUvdoCTEYc=; b=a+RN5IDboZzYLQ9iolIKr/9tW4uK6CcVNnhxX6diNm1PmdAsuaFi9bjL7TV4ZVYRmn OygZI2Z0AxAr3yO5GPJybgmwX4XusZrleOcH18XjOCQdpLr7A79GjbbveOsHJSeWPLBB eB3mI3v2L2+JKYrL3iyxVRappxFjoItn1kqHm45GzV4wKNPrvZM1/SwY2KOQra4WGSyo 5M6y3zOXS7W3LGll6RNTGpRL9dNaB88+c1fOwqjI8HDdWEp5i9raKOJfCSPUe3zloxSA jU9NPu/0Kx+qG9xgBXUND1OmZQHO+cdUj+bZrkqlvKwy1dOvkAMcdSjpSccV71Xh0LbY Ng/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:message-id:in-reply-to :references:subject:mime-version:x-original-sender:reply-to :precedence:mailing-list:list-id:x-spam-checked-in-group:list-post :list-help:list-archive:list-subscribe:list-unsubscribe; bh=uFMILhQHKlSTq7k+lOUSbNTO9llwBAGmeDUvdoCTEYc=; b=VY46qfWxIputPS6rlLRkK5PE3BBahQWWA6DcvpyTHABNbrUEuxJ5hjKSpRD3/00m+H 2R4VlyoeJthmT6V64NHjx8FI9rj2CZGBQ/AaRXxLOoJ65XiBALdlcQ7D8qkLJMn0lNw2 TME5cSHkLMmKqEtQd+klpaF519DVj+BGtFy1QnfHred1cpKfAlUYlD1zqGK1ylBN5R48 gBen1xJRaAEc73Zz0KNZtnisX9Lj8wixwN5HQek8vh48sXKh/J8PbcTU9NR4nqDOIp4D whS72ZCJejxHqdipAKe54fEmey/v4NQbSdUHgwQ8vuJMuDRJMU2J501U9JGzJVg4MRFp D7dg== X-Gm-Message-State: ALyK8tL3pkhAALINXNdIw9X101bH9h+dDbcYJlAbkvWa6A8T2obp7w/q0x807YvT731n6Q== X-Received: by 10.107.57.215 with SMTP id g206mr7795282ioa.8.1466953973310; Sun, 26 Jun 2016 08:12:53 -0700 (PDT) X-BeenThere: std-proposals@isocpp.org Original-Received: by 10.36.184.2 with SMTP id m2ls681073ite.20.canary; Sun, 26 Jun 2016 08:12:52 -0700 (PDT) X-Received: by 10.36.17.15 with SMTP id 15mr156739itf.10.1466953972194; Sun, 26 Jun 2016 08:12:52 -0700 (PDT) In-Reply-To: X-Original-Sender: jmckesson@gmail.com Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Spam-Checked-In-Group: std-proposals@isocpp.org X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:26407 Archived-At: ------=_Part_3692_2020040332.1466953971433 Content-Type: multipart/alternative; boundary="----=_Part_3693_1607879778.1466953971433" ------=_Part_3693_1607879778.1466953971433 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Sunday, June 26, 2016 at 10:03:02 AM UTC-4, asor...@gmail.com wrote: > > =D0=B2=D0=BE=D1=81=D0=BA=D1=80=D0=B5=D1=81=D0=B5=D0=BD=D1=8C=D0=B5, 26 = =D0=B8=D1=8E=D0=BD=D1=8F 2016 =D0=B3., 11:28:35 UTC+3 =D0=BF=D0=BE=D0=BB=D1= =8C=D0=B7=D0=BE=D0=B2=D0=B0=D1=82=D0=B5=D0=BB=D1=8C Jeffrey Yasskin=20 > =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB: >> >> >> ICU does have ways to subset its data tables to include only the parts= =20 >> you use. The proposal author should probably validate that to show us=20 >> what kinds of subsets are already possible, but it'll also be possible= =20 >> to add new subsets if the C++ library wants to make finer-grained=20 >> distinctions.=20 >> >> Okay, lets use subset model. Unicode use 273 unicode blocks, 271792=20 > codes total . We= =20 > can write something like this: > struct u32_subset > { > int32_t lower_bound,size; > const int8_t*table; > bool operator<(int32_t code)const{return lower_bound }; > bool isu32alpha(int32_t code) > { > static const u32_subset unicode_blocks[273]=3D{/*some large table*/}; > > const=20 > u32_subset&subset=3D*std::lower_bound(unicode_blocks,unicode_blocks+273,c= ode); > size_t offset=3Dcode-subset.lower_bound; > return offset } > > u32_subset request 4 byte for lower_bound, 4 byte for size, and 8 bytes= =20 > for table pointer. 273*16=3D4368 bytes total. > All 273 tables request 271792 bites, or 33974 bytes total. > 4368+33974=3D38342 bytes total. > 38 KB is still very big and your 4+ GB desktop can't afford this? > 1: The world of C++ is greater than the world of "4+GB desktop" computers. 2: Why are you using `lower_bound` for a table? That's like making an=20 `unordered_map`, then using a range-for loop to search for an item by its= =20 key. You should be able to get the exact block index for a Unicode=20 codepoint with some simple mathematics. 3: 38KB is still *way* too big for this information. --=20 You received this message because you are subscribed to the Google Groups "= ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. To view this discussion on the web visit https://groups.google.com/a/isocpp= ..org/d/msgid/std-proposals/ad31eadd-08a3-4968-9fa9-a957106a2472%40isocpp.or= g. ------=_Part_3693_1607879778.1466953971433 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On Sunday, June 26, 2016 at 10:03:02 AM UTC-4, asor...@gma= il.com wrote:
= =D0=B2=D0=BE=D1=81=D0=BA=D1=80=D0=B5=D1=81=D0=B5=D0=BD=D1=8C=D0=B5, 26 =D0= =B8=D1=8E=D0=BD=D1=8F 2016 =D0=B3., 11:28:35 UTC+3 =D0=BF=D0=BE=D0=BB=D1=8C= =D0=B7=D0=BE=D0=B2=D0=B0=D1=82=D0=B5=D0=BB=D1=8C Jeffrey Yasskin =D0=BD=D0= =B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB:

ICU does have ways to subset its data tables to include only the parts
you use. The proposal author should probably validate that to show us
what kinds of subsets are already possible, but it'll also be possi= ble
to add new subsets if the C++ library wants to make finer-grained
distinctions.

Okay, lets use subset model. Unicode use 273 unicode = blocks, 271792 codes total. We can write something like this= :
struct=C2=A0u32_subset
{
=C2=A0 =C2=A0 int32_t=C2= =A0lower_bound,size;
=C2=A0 =C2=A0 const int8_t*table;
= =C2=A0 =C2=A0 bool operator<(int32_t code)const{return lower_bound<co= de;}
};
bool isu32alpha(int32_t code)
{
=C2= =A0 =C2=A0 static const u32_subset unicode_blocks[273]=3D{/*some large tabl= e*/};

=C2=A0 =C2=A0 const u32_subset&subset=3D= *std::lower_bound(unicode_blocks,unicode_blocks+273,code);
<= div>=C2=A0 =C2=A0 size_t offset=3Dcode-subset.lower_bound;
= =C2=A0 =C2=A0 return offset<subset.size?subset.table[offset/8]&= (1<<(offset&7)):false;
}

u32= _subset request 4 byte for=C2=A0lower_bound, 4 byte for size, and 8 bytes f= or table pointer. 273*16=3D4368 bytes total.
All 273 tables reque= st 271792 bites, or 33974 bytes total.
4368+33974=3D38342 bytes t= otal.
38 KB is still very big and your 4+ GB desktop can't af= ford this?

1: The world of C++ is greater = than the world of "4+GB desktop" computers.

2: Why are you= using `lower_bound` for a table? That's like making an `unordered_map`= , then using a range-for loop to search for an item by its key. You should = be able to get the exact block index for a Unicode codepoint with some simp= le mathematics.

3: 38KB is still way too big for this informa= tion.

--
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposa= ls+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.= com/a/isocpp.org/d/msgid/std-proposals/ad31eadd-08a3-4968-9fa9-a957106a2472= %40isocpp.org.
------=_Part_3693_1607879778.1466953971433-- ------=_Part_3692_2020040332.1466953971433-- .