220 12564 <53FF3BC0.5080802@gmx.net> article Path: news.gmane.org!not-for-mail From: Jens Maurer Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: Cryptographic hash functions reloaded [was Interest in cryptographic functions within the standard library] Date: Thu, 28 Aug 2014 16:25:04 +0200 Lines: 142 Approved: news@gmane.org Message-ID: <53FF3BC0.5080802@gmx.net> References: <53F8F620.3090701@gmx.de> <53F989E9.5090701@gmx.net> <53FE5E8A.2090003@gmx.net> <53FE7CFB.1090509@gmail.com> Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 X-Trace: ger.gmane.org 1409235922 9306 80.91.229.3 (28 Aug 2014 14:25:22 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 28 Aug 2014 14:25:22 +0000 (UTC) To: std-proposals@isocpp.org Original-X-From: std-proposals+bncBDPMTYGK64JRBQ7X7SPQKGQEIJ3H36A@isocpp.org Thu Aug 28 16:25:15 2014 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-la0-f71.google.com ([209.85.215.71]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1XN0dR-0006Ze-LH for gclcip-std-proposals@m.gmane.org; Thu, 28 Aug 2014 16:25:09 +0200 Original-Received: by mail-la0-f71.google.com with SMTP id pi18sf887514lab.2 for ; Thu, 28 Aug 2014 07:25:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:list-post:list-help:list-archive:list-subscribe :list-unsubscribe:content-type; bh=vRMeQCVW5Fcgr9Zp0eD6b3ntWUK3pK/Pqa6d21Hjy+8=; b=MAb72F4WvLxxVTMAbJyNbd2d2A+nOnp7lmoYSGcz1vHGCXJoDdilEN+GYbvnz0GgKP 8v9UOo7Vu8Vlq2DVxKnn+l7BBA5sA6KbMlQ3zd4KoqZM4lxx5IC/6KsYD1A6IQAoY8/X ZA9A3OBOPTys/RfyuPI+V69DAKDyxgV5eEGOxHBjtwkd3XlWBgzNcsEFOpo5a8l4oxTE WK2DouuiAeAEr83Ea6kW5zjPUQFWsX3jeqTx/U73wcpEt+4qmQaANXARCVErnBBc+yuq uWUrmyDKBE9lurS62lviy06lf5fl/WBWSvp2eUtiMV30ixXF7rRzrJUtNltMp03V4H8G I16A== X-Gm-Message-State: ALoCoQkdOIhlMoP7Z4rLS2cTtI+zENiONa8EIMnLFJ7EZ8Zot0BudkZvbctKrVfTE1t1d+jB8pQD X-Received: by 10.180.105.3 with SMTP id gi3mr3323135wib.3.1409235908188; Thu, 28 Aug 2014 07:25:08 -0700 (PDT) X-BeenThere: std-proposals@isocpp.org Original-Received: by 10.152.4.3 with SMTP id g3ls144055lag.17.gmail; Thu, 28 Aug 2014 07:25:06 -0700 (PDT) X-Received: by 10.112.78.38 with SMTP id y6mr4445159lbw.94.1409235906826; Thu, 28 Aug 2014 07:25:06 -0700 (PDT) Original-Received: from mout.gmx.net (mout.gmx.net. [212.227.17.22]) by mx.google.com with ESMTPS id q9si5629405lbe.47.2014.08.28.07.25.06 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 28 Aug 2014 07:25:06 -0700 (PDT) Received-SPF: pass (google.com: domain of Jens.Maurer@gmx.net designates 212.227.17.22 as permitted sender) client-ip=212.227.17.22; Original-Received: from [192.168.2.100] ([178.7.39.200]) by mail.gmx.com (mrgmx103) with ESMTPSA (Nemesis) id 0LjquD-1WlSpM3Rwf-00bwUx for ; Thu, 28 Aug 2014 16:25:06 +0200 User-Agent: Mozilla/5.0 (X11; Linux i686; rv:6.0.1) Gecko/20110830 Thunderbird/6.0.1 In-Reply-To: <53FE7CFB.1090509@gmail.com> X-Provags-ID: V03:K0:NziCy83boBKahMP87oRlWZePgiJbyQsa7RoMv1LmpgVI/aNRCdh e1I/MvPLqZUcauDOHNSCe2GzhyshxeFH+coooXKDcNk1yNWWrAtUbjF7xmiLIAu2iOmG2lZ fb4GtB4M8I5LU5r8t3Es06d5gYGGq0C2+zdYc8kgrgy06hMilz/WkKBnbQbLV8KHXqEEy3j 42B7Ro+YHzxXsnw8eawog== X-UI-Out-Filterresults: notjunk:1; X-Original-Sender: Jens.Maurer@gmx.net X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of Jens.Maurer@gmx.net designates 212.227.17.22 as permitted sender) smtp.mail=Jens.Maurer@gmx.net Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:12564 Archived-At: On 08/28/2014 02:51 AM, Miro Knejp wrote: > Maybe this whole issue simply shows that algorithms with strict size > requirements should not be defined in terms of char, short, int but > int8_t, int16_t, and so on. If hash_append has (by default) only > overloads for exactly sized types then the compiler should pick the > correct one when the user feeds it unsized types like int. On machines > where no int8_t exists no hash_append overload for int8_t exists. If we go that route, we should use "uint8_t" etc, not the signed variants, which have more platform freedom. And, for a hash algorithm that operates on octets, what does it mean to hash a uint16_t value, call it x? I think the definition should be hash the two octets obtained by x & 0xff and (x>>8) & 0xff, in sequence but that should be stated explicitly, and also how the user can get that result if he chooses to directly call the hash algorithm. > Where the same source code is to produce equal hashes for the same data > structures on different machines then the only portable way is to use > the explicitly sized types. That is honestly the *only* way to be sure > of consistent hash values and should maybe be added as a note somewhere. I don't think so. If you strictly think about portably serializing values to octets, the types on the platforms (other than the integer/floating-point distinction) becomes meaningless, and you should only think about how to represent values in a portable manner, using (for example) a value-dependent variable-length integer representation. (I'm not saying this is the only choice, but we shouldn't claim hash algorithms such as sha256 if the results aren't really portable.) > Though I think "char" (not "signed char" or "unsigned char" as those are > 3 different types) should be treated implementation-defined as the > standard uses this type only for character values in strings. This is not exactly true. Standard filestreams use these for (possibly) binary data, too. (Unfortunately, in my opinion.) > On a > machine where char has more than 8 bit only the implementation knows > which of these bits are representative of the value of a string > character/codepoint. Same goes for wchar_t. Note that for "unsigned char", all bits contribute to the value (3.9.1p1 basic.fundamental). (No padding allowed.) That also holds if "char" happens to be unsigned. > Regarding the void* question, I think Howard's is_contiguously_hashable > covers this nicely. If your type has no padding in it, specialize the > trait. If it does, provide your own hash_append overload and feed it the > required members. I think a good set of predefined overloads for > hash_append would be > > hash_append(Hasher&, const T&) // enable if is_contiguously_hashable Hasher> is true Other than for nerd-ness, why is it easier to partially specialize is_contiguously_hashable instead of overloading void hash_append(Hasher& h, const my_T& value) { std::hash_append_contiguous(h, value); } ? The latter seems easier to read, and makes it easier to provide a different answer for a different Hasher. > hash_append(Hasher&, array_view) // enable if hash_append(Hasher, T) > is well-formed > hash_append(Hasher&, basic_string_view) // enable if > hash_append(Hasher, Char) is well-formed Yes, this is basic decomposition. We should have something like that. (I'd be happy to omit the enable_if dance; you'll simply get an error if T doesn't have a hash_append(), which is good.) Question: Do you hash the length of the string or array separately, or just the elements? In the latter case, hashing two empty strings is indistinguishable from hashing three empty strings. This seems undesirable for crypto-hashes. > Having is_contiguously_hashable predefined as true for the > types [u]int[8|16|32|64]_t, char, wchar_t and char[16|32]_t allows the > implementation to select which integral types are acceptable and can > then internally provide specializations with tag dispatching. Using > unsized types like short or int would pick the proper overload depending > on what int##_t typedefs alias to. So would std::hash_append() overloads that each take one of the scalar types mentioned above. (The standard library can avoid duplicate overloads for the uintX_t typedefs conflicting with char etc.) I don't see why we need an is_contiguously_hashable trait. > A further alternative would be to provide a third parameter in the form > hash_append(Hasher&, const T&, valid_bits), so even on architectures > without direct int8_t support one could use ints for storage and only > mask the leading N bits as relevant for the hash. For scalar T, this should be the job of the standard library implementation. I don't see a need for valid_bits with a generic "T". That said, I'm not opposed to adding hash_append(Hasher&, const std::bitset&); where N is divisible by 8. This allows to pass an octet without concerns about C++ built-in types at all. > This should make any need for a hash_append(void*, size_t) overload > obsolete. The Hasher itself needs a (void* p, size_t n) overload where n > denotes the number of valid OCTETS pointed to by p. Then it's hard to call that function in portable code, because sizeof(int) = 1 on some platforms where "int" is 32 bits (and "char", too). > hash_append() has > then already taken care of endianess and other details by applying > Hasher's traits. The nice hing here is that if a type has no padding and > the user *decided that endianess, etc. does not matter* then enabling > is_contiguously_hashable makes hash_append() feed the entire structure > to (void*, size_t). I believe it's a user-level policy decision for his struct type T to determine whether it can be hashed contiguously, possibly depending on the hasher (think two hash tables keyed off on different things, both pointing to T objects). And that policy decision is best left to the user's overload of hash_append(T). Jens -- --- You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. Visit this group at http://groups.google.com/a/isocpp.org/group/std-proposals/. .