From -5954950272290718799 X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: f78e5,b1fa62568eb1d2a7 X-Google-Attributes: gidf78e5,public X-Google-ArrivalTime: 2002-02-02 08:28:43 PST Path: archiver1.google.com!news2.google.com!news1.google.com!newsfeed.stanford.edu!newsfeeds.belnet.be!news.belnet.be!colt.net!dispose.news.demon.net!news.demon.co.uk!demon!mail2news.demon.co.uk!not-for-mail From: "Early Ehlinger" Newsgroups: comp.std.c++ Subject: Re: Unicode vs. Char Date: Sat, 2 Feb 2002 16:25:37 GMT Organization: WebUseNet Corp. - "ReInventing The UseNet" Approved: Fergus Henderson , moderator of comp.std.c++ Message-ID: <0hU68.89357$h31.5940930@e420r-atl1.usenetserver.com> References: X-Trace: mail2news.demon.co.uk 1012667142 mail2news:28987 mail2news mail2news.demon.co.uk X-Complaints-To: abuse@demon.net X-Mail2News-Path: news.demon.net!mulga.cs.mu.oz.au X-Authentication-Warning: mulga.cs.mu.OZ.AU: fjh set sender to devnull@stump.algebra.com using -f X-Robomod: STUMP, ichudov@algebra.com (Igor Chudov) X-Priority: 3 X-MSMail-Priority: Normal X-Newsreader: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Abuse-Info: Please be sure to forward a copy of ALL headers X-Abuse-Info: Otherwise we will be unable to process your complaint properly. NNTP-Posting-Date: Sat, 02 Feb 2002 11:15:56 EST Lines: 127 Xref: archiver1.google.com comp.std.c++:9362 "Hans Aberg" wrote: > This kind of questions have been discussed in comp.std.c++. In brief (and > perhaps follow-ups will say that I am wrong): My apologies for beating a dead horse... > It is not possible to deprecate "char", because almost all code would break. Deprecate does not mean "remove." It means to warn of possible future obsolescence. Therefore, deprecating char would not break any existing code; it would merely warn people that char is considered to be a bad design choice in light of the new types that would presumably be added. And before it is mentioned, warn does not necessarily mean that the compiler would issue warnings, just that the Standard would effectively say, "hey guys, your old buddy char is on its death bed." > But one may think of introducing new types that do rely on a special type > of binary representation: > > A byte type with exactly 8 bits, and say a type "unichar", encoding > probably UTF-32, with additional macros telling if the compiler can > support them. (Some say that the presence of these types may create > overheads they do not want to have.) Certainly. Exactly what I was asking for, although what may prove better would be a Standard Library extension similar to bit vector whereby the developer could say something like this: std::sized_int< 16 , std::big_endian > int16be = 32; std::sized_int< 32 , std::little_endian > int32le = 123; std::sized_unsigned_int< 64 , std::big_endian > int64be = 4097; A vendor could provide specializations such that values described as being std::big_endian effectively behave like built-in types of the same size on big-endian processors, while their std::little_endian counterparts behave like built-ins on little-endian processors. Furthermore, the standard could be written such that the big_endian/little_endian effects only matter when transmitting the object to the "outside world"; in memory, the layout could be whatever is appropriate to the platform. As an example of where this could be useful, consider IP addresses, which are 32-bit integers, transmitted in big-endian format. Today, you have to be very careful when writing socket code to use htonX / ntohX the appropriate number of times. A socket library based on std::sized_int could simply do this: typedef std::sized_int< 32 , std::big_endian > ip_address_t; And the library could use whatever accessor/modifiers std::sized_int provides to change the value of a specific address. Before it is mentioned, I'm well aware that all of this could be done today without a change to the Standard (well, except for putting sized_int into std::). I'm merely suggesting that this is one of those things that would enhance the Standard Library and make distributed computing considerably easier to do. Perhaps in my copious free time I will attempt to write these templates and submit them to boost, unless somebody else (hopefully) beats me to it. There seem to be some templates in boost today for selecting appropriate types based on minimum bit-size requirements, but no apparent luck as far as being able to specify a specific binary layout. > One should note that the Unicode character numbers do not rely on any > binary computer representation. So it might be possible to merely specify > that a unichar should be able to hold all Unicode characters, with UTF-32 > and other translations being present. I think though that it is going to > be very complicated to use variable width characters internally in a > program (and slow, due to alignment cutoffs that the CPU will have to > perform), so that suggests one should use UTF-32 and nothing else. > > There is also the high/low "endian" issue when dealing with Unicode > representations. This probably belongs to the distributed programming > chapter. I think specifically requesting Unicode would probably be a mistake. Consider how bad a choice it would have been for the Standard to require that std::string have a refcounted implementation. It was quite wise to write the Standard to allow refcounted std::string, while not requiring it. Consider what happens if the Standard requires Unicode and then it turns out that the Unicode design is fundamentally flawed. Or what if another international character encoding comes into favor, and C++ is tied to Unicode? Note, though that a sized_X template could help in this regard too. You could have something like this: typedef std::sized_char < 16 , std::character_encoding< std::UTF_16 > , std::big_endian > char_utf16_le; typedef std::sized_char < 32 , std::character_encoding< std::UTF_32 > , std::little_endian > char_utf32_le; Again, a vendor would be free to use magic to make such sized_char values effectively turn into built-in types. > One model that comes to my mind in order to resolve this problem is to > number all bits in the computer (instead of words), and specify the > representation with respect to that. In this model, the high/low endian > representations are two different encodings in this binary model. Eek. As convenient as this might be, it would also be terribly inefficient as the basis for memory management. Of course, there's no reason a library couldn't include the ability to access individual bits, at least using the virtual address space, say read( void* byte , int bit ), write ( void* byte , int bit ). But that can be done without any library help using &, |, ^, etc. -- Early Ehlinger --- [ comp.std.c++ is moderated. To submit articles, try just posting with ] [ your news-reader. If that fails, use mailto:std-c++@ncar.ucar.edu ] [ --- Please see the FAQ before posting. --- ] [ FAQ: http://www.research.att.com/~austern/csc/faq.html ]