220 14543 article Path: news.gmane.org!not-for-mail From: Jeremy Maitin-Shepard Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: Comments on n3976 (array_view) Date: Fri, 14 Nov 2014 12:51:52 -0800 Lines: 997 Approved: news@gmane.org Message-ID: References: <81f95afc-7a67-45cd-950c-433528b2b8db@isocpp.org> <93261ea6-195a-4838-8283-397a538e9444@isocpp.org> Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=f46d044280b66ef7ac0507d7ccf7 X-Trace: ger.gmane.org 1415998324 8368 80.91.229.3 (14 Nov 2014 20:52:04 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Fri, 14 Nov 2014 20:52:04 +0000 (UTC) To: std-proposals@isocpp.org Original-X-From: std-proposals+bncBCYZZ4E5WQCRB2OWTGRQKGQEDCEUEMQ@isocpp.org Fri Nov 14 21:51:56 2014 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-wi0-f198.google.com ([209.85.212.198]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1XpNqU-0002aq-P2 for gclcip-std-proposals@m.gmane.org; Fri, 14 Nov 2014 21:51:54 +0100 Original-Received: by mail-wi0-f198.google.com with SMTP id n3sf1468198wiv.9 for ; Fri, 14 Nov 2014 12:51:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:content-type:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:list-post:list-help:list-archive:list-subscribe :list-unsubscribe; bh=T8VrJuaMinXmXIWo1N0edyKRUTbiBl7/PIKRSY9ZkrM=; b=LZSEHuallGPs9XJbfuG4arcTV2Za18TG1X1MQYQVihQkR8iC+tOhLMNcpwDLP4P9D3 VrHDc/YPCUZLzb3PpLRr8McwEpb0EVPdrBk7Um9oGVjV5XDu+KHqvkqlESuzTT6JGCA9 7rUe65WS+dY//6M2oVOeUMdGGPCnUycanzcMntQGTbPv//tphkwVvJhRKaLZXTvKt2JR IqepwLD6P1xFakJKRQ9Cks81wVlsQ8CBPW6BMr9oVe95FQEmj5VTAAwP0ifbXRz97Jp5 G8A0jl9IDnFSRNJ0yz0tI89cWr/DpZlMwy3FGLa8kG+nYSA17irwWEQ63NnIJ/fO4dyv 7rGw== X-Gm-Message-State: ALoCoQlnYnWDpWdwPf5vmFErH28fBNkmuoXLibKNGdRRV2fwyBHCYAFKOWUUQrI0LLdki9gNrcRq X-Received: by 10.180.94.3 with SMTP id cy3mr1491149wib.7.1415998314372; Fri, 14 Nov 2014 12:51:54 -0800 (PST) X-BeenThere: std-proposals@isocpp.org Original-Received: by 10.180.74.73 with SMTP id r9ls408270wiv.3.canary; Fri, 14 Nov 2014 12:51:53 -0800 (PST) X-Received: by 10.194.87.131 with SMTP id ay3mr17636453wjb.66.1415998313080; Fri, 14 Nov 2014 12:51:53 -0800 (PST) Original-Received: from mail-wi0-x234.google.com (mail-wi0-x234.google.com. [2a00:1450:400c:c05::234]) by mx.google.com with ESMTPS id fu1si50711077wjb.120.2014.11.14.12.51.53 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 14 Nov 2014 12:51:53 -0800 (PST) Received-SPF: pass (google.com: domain of jeremybms@gmail.com designates 2a00:1450:400c:c05::234 as permitted sender) client-ip=2a00:1450:400c:c05::234; Original-Received: by mail-wi0-f180.google.com with SMTP id hi2so3889596wib.7 for ; Fri, 14 Nov 2014 12:51:53 -0800 (PST) X-Received: by 10.180.101.200 with SMTP id fi8mr10643585wib.77.1415998312738; Fri, 14 Nov 2014 12:51:52 -0800 (PST) Original-Sender: jeremybms@gmail.com Original-Received: by 10.216.158.193 with HTTP; Fri, 14 Nov 2014 12:51:52 -0800 (PST) In-Reply-To: X-Original-Sender: jeremy@jeremyms.com X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jeremybms@gmail.com designates 2a00:1450:400c:c05::234 as permitted sender) smtp.mail=jeremybms@gmail.com; dkim=pass header.i=@gmail.com Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:14543 Archived-At: --f46d044280b66ef7ac0507d7ccf7 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable If the size is known at compile time, an actual reference to std::array (or a multi-dimensional static-sized array counterpart) would be much better, as this eliminates the need for an extra type, and makes reference decay do the right thing. If the current language rules do not permit the casting of an arbitrary pointer T *x where [x, x+size) is a valid range, to be a pointer to std::array*, then I think the language rules should be changed, as this is clearly extremely useful and any loss of optimization possibilities are probably minimal. On Fri, Nov 14, 2014 at 12:29 PM, wrote: > As a small addition, has anyone considered perhaps calling the class > buffer_view > > and then leaving the name *array_view* for a compile-time version of this > class (to match std::array)? > > I don't know if a fully constexpr-qualified version of the buffer_view > class will have the same benefit as a compile-time defined > array_view > > class (with *strided* variants for buffer_view and array_view). > > > On Friday, June 27, 2014 1:00:54 AM UTC-4, =C5=81ukasz Mendakiewicz wrote= : >> >> I do understand your rationale in many of the points you are raising, an= d >> I am aware of other existing libraries, which in some cases made differe= nt >> design decisions. However many of the choices are mutually exclusive and= in >> my opinion it is impossible to converge on the "single best design". Man= y >> of the design decisions in our proposal are also compromises. >> >> I disagree with your judgment that we should strive to find an ultimate >> solution before moving forward with any proposal. Lack of array_view or = a >> similar type is clearly a gap in C++ Standard, which many libraries are >> addressing in different ways. We took the first stab in trying to addres= s >> this deficiency with putting forward the type we are familiar with, have >> worked with successfully for over 2 years, and in the process we were al= so >> fixing all issues we were aware of. It surely is not rock solid -- nothi= ng >> is -- and many use-cases might have benefit from tailoring details >> differently to cater each of them, but we have to start somewhere. >> Importantly, we are not "standardizing" array_view yet. If it passes LWG >> voting on the next Committee meeting it will be merely appended to one o= f >> TSs (unless anything changes, this should be Library Fundamentals TS) an= d >> on track to be considered for C++17. The three years lead time is intend= ed >> exactly to gather the wider practical experience you are concerned about= .. >> >> I hope this helps to clarify our position. >> >> On Monday, June 16, 2014 5:49:39 PM UTC-7, Jeremy Maitin-Shepard wrote: >> >>> On Sun, Jun 15, 2014 at 9:38 PM, =C5=81ukasz Mendakiewicz < >>> l.menda...@live.com> wrote: >>> >>>> Thank you Jeffrey for adding me on the thread and for your initial >>>> response -- which I almost universally agree with. >>>> >>>> Jeremy, thank you for your detailed feedback. First, I believe that >>>> most of the disagreement stems from the approach to the proposal. You'= ve >>>> commented that "it doesn't make sense to standardize a partial solutio= n; >>>> better to work out a good, complete interface outside the standard". T= his >>>> is opposite to the "crawl, walk, run" philosophy we have taken when wr= iting >>>> this document up. We recognize that we have only explored some parts o= f the >>>> design space, because as you have rightly observed our experience is >>>> limited to C++ AMP, which constitutes only a specific data parallel >>>> approach to algorithms. Of course we believe that this is a solid >>>> foundation that can be generalized and built upon, hence the proposal.= But >>>> we deliberately avoided any design decisions which may be controversia= l or >>>> result in a prolonged tug of war, instead intending for subsequent >>>> proposals (not necessarily authored by us) to fill the gaps. Certainly= , >>>> should anything being proposed now be a possible hindrance for any >>>> reasonable extension, it must be addressed sooner than later, however = I >>>> think that none of the lacks pointed by you belong to this category. >>>> >>> I think it is basically impossible to nail down even a "minimal" >>> interface without considering the design of a complete library. If you >>> looked at 10 different, widely-used C++ multidimensional array librarie= s >>> and saw that all of their interfaces shared a given attribute, then >>> standardizing that attribute of the interface would likely be a safe >>> choice. Obviously that is not the case here, though. >>> >>> Some examples of design options: >>> - Make array_view parameterized by the pointer type, rather than the >>> value type. This allows it to hold smart pointers of various types, or= a >>> pointer that represents GPU memory, or a smart pointer that represents = GPU >>> memory, etc. If the pointer is to GPU memory, reading and writing won'= t >>> work (and won't be allowed at compile-time), but the indexing, slicing, >>> sectioning and other operations on the array_view representation itself= can >>> still be used. This is the decision I took in my own in-house library. >>> Allowing array_view to hold a smart pointer means you don't need a sepa= rate >>> type to represent an array_view with ownership. (Sectioning and slicin= g >>> produces a result with the pointer decayed to a regular pointer.) You = can >>> also use a __restrict__ pointer for an array_view type used as a functi= on >>> parameter to indicate lack of aliasing. >>> >>> - Make bounds and index just generic fixed-size vector types. In this >>> case iterators on bounds should iterate over the components, rather tha= n >>> iterate over coordinates within the bounds. >>> >>> - More generally, the proposal distinguishes between array_view and >>> strided_array_view, but doesn't allow for more fine-grained information= to >>> be provided at compile time, such as the number of trailing contiguous >>> dimensions, alignment information, etc. >>> >>>> >>> Some particular issues with the proposed design include: >>>> >>> >>>> >>> - No iterators are provided for array_view or strided_array_view, = a >>>> clear >>>> >>> omission in the interface. For a multidimensional array, there ar= e >>>> two >>>> >>> obvious types of iteration that may be desired: >>>> >>> 1. iterating over just the first dimension; >>>> >>> 2. iterating over a flat view of all of the elements. For a >>>> strided >>>> >>> array, iterating over a flat view cannot be done efficiently >>>> without >>>> >>> segmented/hierarchical iterators. >>>> >>> Which of these should be the default (i.e. provided through >>>> begin() and >>>> >>> end() rather than through some adaptor) is not clear, >>>> >> >>>> >> From the Zen of Python, "In the face of ambiguity, refuse the >>>> >> temptation to guess." >>>> > >>>> > That's true, but no adaptors to provide the functionality >>>> unambiguously are provided either. For the particular case of a >>>> one-dimensional array, there is no ambiguity, and not being able to ea= sily >>>> use array_view with standard algorithms or range-based for is a seriou= s >>>> usability issue. This will only become worse if/when standard algorit= hm >>>> overloads that take ranges rather than iterator pairs are added. >>>> >>>> To the list of possible alternatives, I would add an option that was >>>> proposed somewhere in the prior discussion on the proposal -- >>>> "hierarchical" array_view/elemental iterator, i.e. an iterator that fo= r >>>> array_view with Rank > 1 iterates over array_view;= and >>>> over T for Rank =3D=3D 1. >>>> >>> Yes, that is a better way of expressing what I meant by "iterating over >>> just the first dimension". I believe this makes the most sense, but th= en >>> it becomes desirable to have size() be the size of the first dimension = for >>> consistency with the iterators, and instead add a num_elements() that >>> returns the total number of elements. The correct definition of size()= is >>> hard to decide without figuring out how iterators should work. >>> >>> The other issue is that if the iterators are hierarchical, you need a >>> new hierarchical for_each that effectively expands to N nested for loop= s >>> for N-dimensional arrays and which can operate on multiple arrays (of t= he >>> same size). This way you can express element-wise operations without a= ny >>> overhead. >>> >>>> It is however worth pointing out that in the first round of review wit= h >>>> LEWG at Issaquah meeting, the vocal unchallenged feedback for iterator= s was >>>> "I do not want iterators for the views", thus we did not put much effo= rt >>>> into contemplating adding any. Even though the choice for Rank =3D=3D = 1 is >>>> "obvious", we consider this a corner case for array_view, and would ra= ther >>>> have a holistic approach. >>>> There remains the possibility to add some form of the iterator as a >>>> future extension. >>>> >>> Well, you can take my comments as a challenge to that view. Rank =3D= =3D 1 >>> may be a "corner" case, but is likely to be used more frequently than a= ll >>> other cases combined. A 1-dimensional array view that can't be used wi= th >>> for-range loops, boost (or std?) range adapters, etc. is a non-starter. >>> >>>> >>> - No data() access member for strided_array_view. This is >>>> presumably >>>> >>> omitted because the memory is non-contiguous, but given that >>>> >>> strided_array_view is a thin wrapper, there has to be a convenient >>>> way to >>>> >>> get at the underlying memory. >>>> >> >>>> >> Why "has to"? >>>> > >>>> > Because the user might want to convert the array to another >>>> representation for use by another piece of code/library, e.g. FFTW or >>>> Eigen, CUDA memcpy, etc. Under the proposed interface, for a 3-dimens= ional >>>> stided array view, you have to use &arr[0][0][0]. This is even worse = than >>>> having to do &vec[0] for std::vector, since the same expression cannot= be >>>> used independent of the dimensionality. >>>> >>>> data() on strided_array_view could be harmful, as there is no guarante= e >>>> of contiguity, which a developer / function template might assume. Ple= ase >>>> note that for every other case in the Standard, the range [data(), >>>> data()+size()) is valid/safe, thus providing other semantics here woul= d be >>>> dubious. >>>> Granted, one can perform &arr[0][0][0] and run afoul, but that is no >>>> different than e.g. &my_list.front(). >>>> It could be possible to provide "unsafe_data()" interface, which would >>>> have to be consumed along with stride(), however I do not see much ben= efit >>>> here over just using the "proper" interface -- bounds() and operator[]= (). >>>> >>> Calling it noncontiguous_data() is fine, though certainly then >>> array_view should have an noncontiguous_data() (in addition to data()) = as >>> well for consistency. I don't think unsafe_data is really the right na= me, >>> though the functionality is more important than the name. &arr[0][0][0= ] >>> has the disadvantage that it is more verbose and doesn't allow for writ= ing >>> generic code that works with any number of dimensions. Just using the >>> "proper" interface isn't a solution. A fundamental component like >>> multidimensional arrays cannot be a "walled garden". The (pointer, bou= nds, >>> strides) representation, which numerous other libraries like FFTW use a= s >>> well, should be transparent. >>> >>>> >>> - Slicing is limited: it is not possible to take a slice in a >>>> dimension >>>> >>> other than the first. >>>> >> >>>> >> I think that's called "section()". >>>> > >>>> > Well, that takes a "section" but does not reduce the dimensionality >>>> like "slice". [...] Certainly there is a large design space for genera= l >>>> subscripting and it almost certainly requires a fair amount of templat= e >>>> metaprogramming, but this is very useful functionality >>>> >>>> Yes, slicing and sectioning are slightly different. It is however >>>> important to note that slicing in its current form always guarantees t= he >>>> contiguity of the result, thus the return type may be array_view. The = more >>>> general cases (as you are describing) would result in a potentially st= rided >>>> resulting view, which would need to be typed strided_array_view. This = is >>>> why it is important IMO to distinguish these two cases. >>>> In the current form of the proposal we decided to provide the minimal >>>> design -- only the first version, which in our opinion might be handy = for >>>> users used to C-style multidimensional addressing (i.e. arr[3][1][4] v >>>> arr[{3, 1, 4}]), which actually you used above :). >>>> The latter generic case can be trivially added as an extension to the >>>> interface. I agree it may be useful, and if the current design is acce= pted >>>> for the TS, I encourage you to propose such extension (or nag us for f= ollow >>>> ups :)). >>>> >>>> >>> - It is not clear whether it is really useful to have bounds and >>>> index be >>>> >>> separate types (though should probably at least be explicitly >>>> convertible to >>>> >>> each other). >>>> >> >>>> >> It adds a bit of type safety. Adding two bounds, for instance, >>>> doesn't >>>> >> make a lot of sense. >>>> > >>>> > I agree that type safety is in principle useful, but it is impossibl= e >>>> to predict what relationships might hold in any given program, and so = there >>>> is the question of whether the safety it adds is worth the cost of hav= ing >>>> to explicitly work around it in the cases that it is too restrictive. >>>> Certainly if they are separate types there needs to be a way to >>>> (explicitly) convert. >>>> >>>> I agree with both of you :). >>>> These are separate types, because their meaning and semantics are >>>> different. It was also discussed in N3851, Appendix, II. The mental mo= del I >>>> use for reasoning about them is: index ~ vector and bounds ~ point (mo= re >>>> specifically: a zero-bound, axis-aligned rectangle described by its ma= ximum >>>> point), which makes the set of available operators apparent. >>>> I understand the explicit conversions between these two types, even >>>> though not theoretically sound, might be handy in practice and should = be >>>> added. >>>> >>> If we follow that logic, a point is a vector from the origin. Therefor= e >>> index should just be Vector and bounds could have a >>> Vector member. However, it seems like it would be simpler= to >>> just have bounds be Vector as well and save the trouble of= the >>> explicit conversions. What type of error does the bounds/index distinc= tion >>> prevent? More generally, I am not sure why we shouldn't think of an in= dex >>> as a point as well. If we want the last element in an array x, we have= : >>> x.bounds() - 1 (or x.bounds() - {1,1, ...}). We want to use this as a= n >>> index, but the current operator definitions would have it be a bounds. >>> >>>> > As a specific example, consider if we want to compute the bounds of = a >>>> valid convolution if we have an array x and a filter k. The bounds of= the >>>> result are: x.bounds - k.bounds + 1. None of the arithmetic operators >>>> defined in the proposal help me with that; I'd just have to write a lo= op. >>>> Even if the arithmetic operators weren't limited by the bounds/index >>>> distinction, there would still be no way to create a constant-valued >>>> vector, i.e. to handle the constant 1. >>>> >>>> Explicit conversions would help with the first part, right? I hope you >>>> are also not suggesting any such conversion should be implicit? >>>> However I do not really understand what do you mean in the second part= , >>>> "a constant-valued vector". We purposefully allow to create both bound= s<1> >>>> and index<1> from a scalar; and disallow such operation for Rank > 1 >>>> (mostly because the semantics are not obvious, e.g. for rank 2, should >>>> index<2>{16} be {0, 16}, {16, 0} or {16, 16}). >>>> So net-net, in your example you should be able to do: x.bounds - >>>> index<2>{k.bounds} + {1, 1}. It a little more long-winded, but IMO eas= ier >>>> to comprehend than the original. >>>> >>> I don't think the explicit conversion to index adds to the clarity. >>> Also, the problem with {1, 1} is that it depends on the dimensionality. >>> The interface should be designed to be convenient for generic code. I >>> would suggest a Scalar type generated by a wrapper function scalar, = from >>> which bounds and index can be implicitly constructed and which can be u= sed >>> in all of the operators. >>> >>>> > Another related issue I didn't notice before: the only way to >>>> construct an index or bounds type is from an initializer_list. There = is no >>>> (explicit) conversion from std::array, for instance. It is quite like= ly >>>> that an index or bounds value would be computed by some other code, th= at >>>> may use a different representation. >>>> >>>> Yes, we should add (const value_type* begin, const value_type* end) >>>> constructors for both types. Or it should be possible to create >>>> initializer_list explicitly from such pair which would solve this issu= e >>>> universally. >>>> >>> Okay, though initializer_list is not compile-time sized. >>> >>> >>>> >>> Furthermore, bounds and index essentially represent a >>>> >>> fixed-size discrete vector type, i.e. an augmented interface for >>>> std::array. >>>> >>> Arguably the interface of std::array should just be extended to ad= d >>>> >>> component-wise operators instead. Also, a much more comprehensive >>>> set of >>>> >>> component-wise operations should be provided. A statically-sized >>>> vector >>>> >>> type is sorely needed in C++, but it does not make sense to >>>> standardize >>>> >>> these limited, single-purposes types when they could be, and shoul= d >>>> be, much >>>> >>> more general. >>>> >> >>>> >> We don't necessarily want to go for the most general possible type >>>> all at once >>>> > >>>> > Certainly there is a large design space, but that is a reason to be >>>> cautious about standardizing anything. [...] >>>> >>>> There are two separate issue raised here: >>>> - Why novel types -- this is primarily to introduce "vocabulary" types >>>> and allow to discriminate index and bounds, as they have different >>>> semantics, as discussed before. This was also touched upon in the >>>> previously referred N3851, Appendix, II. >>>> - Why so little functionality -- I agree with Jeffrey on this, and I >>>> have described more of the philosophy in the introduction. I would be >>>> interested however what exactly you mean by "a much more comprehensive= set >>>> of component-wise operations". >>>> >>> Almost anything could be useful (and I am thinking of the perspective o= f >>> a general fixed-size vector type, but these all apply to index/bounds a= s >>> well), but some particular examples: min, max, <, <=3D, =3D=3D, !=3D, >= =3D, >. The >>> relational operations need to return a vector of bool, which isn't >>> compatible with index/bounds currently since they aren't parameterized = by >>> the type. Obviously, though, it wouldn't make sense to call them index= or >>> bounds if they are parameterized by a type. >>> >>> >>>> Having said that, I assume this is something that can be added later a= s >>>> a pure extension, correct? >>>> >>> Some of them, like min and max, could be unintrusive, though neither >>> index nor bounds would be a particularly meaningful choice as the retur= n >>> type. >>> >>> >>>> > Given the long turn around time for revisions to C++ >>>> >>>> The Committee is certainly trying to get better at quicker iterations, >>>> let's not be discouraged by the ghosts of the past... >>>> >>>> > Other omissions from index and bounds include: >>>> > 1. no data() access to the components >>>> > 2. no iterators over components for index or bound >>>> >>>> index and bounds are not containers and should not be used as such. Fo= r >>>> the limited number of scenario where I want to use/modify indeterminat= e >>>> number of components, it was always sufficient for me to use a raw loo= p, >>>> and operator[]... >>>> In case I'm wrong, these as well should be easy to add later as >>>> extensions. >>>> >>>> > for_each(x_array, y_array, x_array.bounds(), [&](auto &x, auto &y, >>>> auto pos) { >>>> > if (x !=3D y) std::cout << "mismatch at position " << pos << >>>> std::endl; >>>> > }); >>>> >>>> I assume the idea here is that such "for_each" uses begin(X), end(X) >>>> for all provided arguments, passing dereferenced iterators to the func= tion >>>> object. If so, I do not see how the current semantics of bounds_iterat= or >>>> would get into way here? It should just work as is, no? >>>> >>> The idea is that it would generate N nested for loops for N-dimensional >>> array views, rather than an less efficient single for loop. >>> >>> >>>> Actually, it should be possible to write the following with just the >>>> current proposal and the current standard for_each: >>>> auto x_av =3D array_view{x_array}; >>>> auto y_av =3D array_view{y_array}; >>>> for_each(begin(x_av.bounds()), end(x_av.bounds()), [&](index pos) = { >>>> if(x_av[pos] !=3D y_av[pos]) std::cout << "mismatch at position " << >>>> pos << std::endl; >>>> }); >>>> >>> Yes, but note that the iteration over bounds is more expensive and the >>> multiplication with all of the strides has to happen for every position= .. >>> >>> >>>> > From my perspective, the only way to properly explore this design >>>> space for multidimensional arrays/containers/ranges is through an acti= vely >>>> developed and widely used open source library. >>>> Our (Microsoft) prototype implementation is in the pipeline to be >>>> published as an open source project. It should happen really soon, and= I >>>> hope it will help to convey the discussed ideas. >>>> >>> >>> That's good news. I think the best library is likely to be produced if >>> the users of the library and the developers of the library are one and = the >>> same. Issues become apparent and the library interface can be tweaked = to >>> correct them very quickly that way. >>> >> -- > > --- > You received this message because you are subscribed to a topic in the > Google Groups "ISO C++ Standard - Future Proposals" group. > To unsubscribe from this topic, visit > https://groups.google.com/a/isocpp.org/d/topic/std-proposals/xzb1d5KUxMU/= unsubscribe > . > To unsubscribe from this group and all its topics, send an email to > std-proposals+unsubscribe@isocpp.org. > To post to this group, send email to std-proposals@isocpp.org. > Visit this group at > http://groups.google.com/a/isocpp.org/group/std-proposals/. > --=20 ---=20 You received this message because you are subscribed to the Google Groups "= ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. Visit this group at http://groups.google.com/a/isocpp.org/group/std-proposa= ls/. --f46d044280b66ef7ac0507d7ccf7 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
If the size is known at compile time, an actual reference = to std::array (or a multi-dimensional static-sized array counterpart) would= be much better, as this eliminates the need for an extra type, and makes r= eference decay do the right thing.=C2=A0 If the current language rules do n= ot permit the casting of an arbitrary pointer T *x where [x, x+size) is a v= alid range, to be a pointer to std::array<T,size>*, then I think the = language rules should be changed, as this is clearly extremely useful and a= ny loss of optimization possibilities are probably minimal.

On Fri, Nov 14, 2014 at= 12:29 PM, <phdofthehouse@gmail.com> wrote:
As a small addition, has anyone c= onsidered perhaps calling the class
buffer_view<typenam= e T, size_type dimensions>

and then leaving the name = array_view for a compile-time version of this class (to match std::arra= y)?

I don't know if a fully constexpr-qualified version of the b= uffer_view class will have the same benefit as a compile-time defined
array_view<<= span style=3D"color:#008">typename T, size_type<= /span>... dime= nsion_sizes>
=
class (with strided variants for buffer_view and array_view).


On Friday, June 27, 2014 1:00:54 AM UTC-4, =C5= =81ukasz Mendakiewicz wrote:

I do understand your rationale in many of the points you are r= aising, and I am aware of other existing libraries, which in some cases mad= e different design decisions. However many of the choices are mutually excl= usive and in my opinion it is impossible to converge on the "single be= st design". Many of the design decisions in our proposal are also comp= romises.

I disagree with your judgment that we should strive to find = an ultimate solution before moving forward with any proposal. Lack of array= _view or a similar type is clearly a gap in C++ Standard, which many librar= ies are addressing in different ways. We took the first stab in trying to a= ddress this deficiency with putting forward the type we are familiar with, = have worked with successfully for over 2 years, and in the process we were = also fixing all issues we were aware of. It surely is not rock solid -- not= hing is -- and many use-cases might have benefit from tailoring details dif= ferently to cater each of them, but we have to start somewhere.

Imp= ortantly, we are not "standardizing" array_view yet. If it passes= LWG voting on the next Committee meeting it will be merely appended to one= of TSs (unless anything changes, this should be Library Fundamentals TS) a= nd on track to be considered for C++17. The three years lead time is intend= ed exactly to gather the wider practical experience you are concerned about= ..

I hope this helps to clarify our position.
=

On Monday, June 16, 2014 5:49:39 PM UTC-7, Jeremy Maitin-Shepard w= rote:
On Sun, Jun 15, 2014 at 9:38 P= M, =C5=81ukasz Mendakiewicz <l.menda...@live.com> wrote:

Thank you Jeffrey for adding me on the= thread and for your initial response -- which I almost universally agree w= ith.

Jeremy, thank you for your detailed feedback. First, I believe that most= of the disagreement stems from the approach to the proposal. You've co= mmented=C2=A0that "it doesn't make sense to standardize a partial = solution; better to work out a good, complete interface outside the standar= d". This is opposite to the "crawl, walk, run" philosophy we= have taken when writing this document up. We recognize that we have only e= xplored some parts of the design space, because as you have rightly observe= d our experience is limited to C++ AMP, which constitutes only a specific d= ata parallel approach to algorithms. Of course we believe that this is a so= lid foundation that can be generalized and built upon, hence the proposal. = But we deliberately avoided any design decisions which may be controversial= or result in a prolonged tug of war, instead intending for subsequent prop= osals (not necessarily authored by us) to fill the gaps. Certainly, should = anything being proposed now be a possible hindrance for any reasonable exte= nsion, it must be addressed sooner than later, however I think that none of= the lacks pointed by you belong to this category.

I think it is basically impossible to nail down eve= n a "minimal" interface without considering the design of a compl= ete library.=C2=A0 If you looked at 10 different, widely-used C++ multidime= nsional array libraries and saw that all of their interfaces shared a given= attribute, then standardizing that attribute of the interface would likely= be a safe choice.=C2=A0 Obviously that is not the case here, though.

Some examples of design options:
- Make array_view parame= terized by the pointer type, rather than the value type.=C2=A0 This allows = it to hold smart pointers of various types, or a pointer that represents GP= U memory, or a smart pointer that represents GPU memory, etc.=C2=A0 If the = pointer is to GPU memory, reading and writing won't work (and won't= be allowed at compile-time), but the indexing, slicing, sectioning and oth= er operations on the array_view representation itself can still be used.=C2= =A0 This is the decision I took in my own in-house library.=C2=A0 Allowing = array_view to hold a smart pointer means you don't need a separate type= to represent an array_view with ownership.=C2=A0 (Sectioning and slicing p= roduces a result with the pointer decayed to a regular pointer.)=C2=A0 You = can also use a __restrict__ pointer for an array_view type used as a functi= on parameter to indicate lack of aliasing.

- Make bounds and index just generic fixed-size vector types= ..=C2=A0 In this case iterators on bounds should iterate over the components= , rather than iterate over coordinates within the bounds.
=C2= =A0
- More generally, the proposal distinguishes between array_view = and strided_array_view, but doesn't allow for more fine-grained informa= tion to be provided at compile time, such as the number of trailing contigu= ous dimensions, alignment information, etc.

>>> Some particula= r issues with the proposed design include:
>>>
>>> - No iterators are provided for array_view or strided_array_vi= ew, a clear
>>> omission in the interface.=C2=A0 For a multidi= mensional array, there are two
>>> obvious types of iteration = that may be desired:
>>>=C2=A0=C2=A0 1. iterating over just the first dimension;
&g= t;>>=C2=A0=C2=A0 2. iterating over a flat view of all of the elements= ..=C2=A0 For a strided
>>> array, iterating over a flat view ca= nnot be done efficiently without
>>> segmented/hierarchical iterators.
>>>=C2=A0=C2=A0= Which of these should be the default (i.e. provided through begin() and >>> end() rather than through some adaptor) is not clear,
&g= t;>
>> From the Zen of Python, "In the face of ambiguity, refuse the=
>> temptation to guess."
>
> That's true, = but no adaptors to provide the functionality unambiguously are provided eit= her.=C2=A0 For the particular case of a one-dimensional array, there is no = ambiguity, and not being able to easily use array_view with standard algori= thms or range-based for is a serious usability issue.=C2=A0 This will only = become worse if/when standard algorithm overloads that take ranges rather t= han iterator pairs are added.

To the list of possible alternatives, I would add an option that w= as proposed somewhere in the prior discussion on the proposal -- "hier= archical" array_view/elemental iterator, i.e. an iterator that for arr= ay_view<T, Rank> with Rank > 1 iterates over array_view<T, Rank= -1>; and over T for Rank =3D=3D 1.

Yes, that is a better way of expressing what I = meant by "iterating over just the first dimension".=C2=A0 I belie= ve this makes the most sense, but then it becomes desirable to have size() = be the size of the first dimension for consistency with the iterators, and = instead add a num_elements() that returns the total number of elements.=C2= =A0 The correct definition of size() is hard to decide without figuring out= how iterators should work.

The other issue is that if the iterators are hierarchical, y= ou need a new hierarchical for_each that effectively expands to N nested fo= r loops for N-dimensional arrays and which can operate on multiple arrays (= of the same size).=C2=A0 This way you can express element-wise operations w= ithout any overhead.

It is however worth pointing out= that in the first round of review with LEWG at Issaquah meeting, the vocal= unchallenged feedback for iterators was "I do not want iterators for = the views", thus we did not put much effort into contemplating adding = any. Even though the choice for Rank =3D=3D 1 is "obvious", we co= nsider this a corner case for array_view, and would rather have a holistic = approach.
There remains the possibility to add some form of the iterator as a future = extension.

Well, you can take my comments as a c= hallenge to that view.=C2=A0 Rank =3D=3D 1 may be a "corner" case= , but is likely to be used more frequently than all other cases combined.= =C2=A0 A 1-dimensional array view that can't be used with for-range loo= ps, boost (or std?) range adapters, etc. is a non-starter.

>>> - No data() ac= cess member for strided_array_view.=C2=A0 This is presumably
>>> omitted because the memory is non-contiguous, but given that <= br>>>> strided_array_view is a thin wrapper, there has to be a con= venient way to
>>> get at the underlying memory.
>><= br> >> Why "has to"?
>
> Because the user might wa= nt to convert the array to another representation for use by another piece = of code/library, e.g. FFTW or Eigen, CUDA memcpy, etc.=C2=A0 Under the prop= osed interface, for a 3-dimensional stided array view, you have to use &= ;arr[0][0][0].=C2=A0 This is even worse than having to do &vec[0] for s= td::vector, since the same expression cannot be used independent of the dim= ensionality.

data() on strided_array_view could be harmful, as there is no guar= antee of contiguity, which a developer / function template might assume. Pl= ease note that for every other case in the Standard, the range [data(), dat= a()+size()) is valid/safe, thus providing other semantics here would be dub= ious.
Granted, one can perform &arr[0][0][0] and run afoul, but that is no di= fferent than e.g. &my_list.front().
It could be possible to provide = "unsafe_data()" interface, which would have to be consumed along = with stride(), however I do not see much benefit here over just using the &= quot;proper" interface -- bounds() and operator[]().

Calling it noncontiguous_data() is fine, though cer= tainly then array_view should have an noncontiguous_data() (in addition to = data()) as well for consistency.=C2=A0 I don't think unsafe_data is rea= lly the right name, though the functionality is more important than the nam= e.=C2=A0 &arr[0][0][0] has the disadvantage that it is more verbose and= doesn't allow for writing generic code that works with any number of d= imensions.=C2=A0 Just using the "proper" interface isn't a so= lution.=C2=A0 A fundamental component like multidimensional arrays cannot b= e a "walled garden".=C2=A0 The (pointer, bounds, strides) represe= ntation, which numerous other libraries like FFTW use as well, should be tr= ansparent.

>>> - Slicing = is limited: it is not possible to take a slice in a dimension
>>> other than the first.
>>
>> I think that= 9;s called "section()".
>
> Well, that takes a= "section" but does not reduce the dimensionality like "slic= e". [...] Certainly there is a large design space for general subscrip= ting and it almost certainly requires a fair amount of template metaprogram= ming, but this is very useful functionality

Yes, slicing and sectioning are slightly different. It is however im= portant to note that slicing in its current form always guarantees the cont= iguity of the result, thus the return type may be array_view. The more gene= ral cases (as you are describing) would result in a potentially strided res= ulting view, which would need to be typed strided_array_view. This is why i= t is important IMO to distinguish these two cases.
In the current form of the proposal we decided to provide the minimal desig= n -- only the first version, which in our opinion might be handy for users = used to C-style multidimensional addressing (i.e. arr[3][1][4] v arr[{3, 1,= 4}]), which actually you used above :).
The latter generic case can be trivially added as an extension to the inter= face. I agree it may be useful, and if the current design is accepted for t= he TS, I encourage you to propose such extension (or nag us for follow ups = :)).

>>> - It is not clear whether it is really useful to have = bounds and index be
>>> separate types (though should probably= at least be explicitly convertible to
>>> each other).
>>
>> It adds a bit of type safety. Adding two bounds, for i= nstance, doesn't
>> make a lot of sense.
>
> I a= gree that type safety is in principle useful, but it is impossible to predi= ct what relationships might hold in any given program, and so there is the = question of whether the safety it adds is worth the cost of having to expli= citly work around it in the cases that it is too restrictive.=C2=A0 Certain= ly if they are separate types there needs to be a way to (explicitly) conve= rt.

I agree with both of you :).
These are separate types, because = their meaning and semantics are different. It was also discussed in N3851, = Appendix, II. The mental model I use for reasoning about them is: index ~ v= ector and bounds ~ point (more specifically: a zero-bound, axis-aligned rec= tangle described by its maximum point), which makes the set of available op= erators apparent.
I understand the explicit conversions between these two types, even though = not theoretically sound, might be handy in practice and should be added.

If we follow that logic, a point is a vector from = the origin.=C2=A0 Therefore index should just be Vector<ptrdiff_t,N> = and bounds<N> could have a Vector<ptrdiff_t,N> member.=C2=A0 Ho= wever, it seems like it would be simpler to just have bounds be Vector<p= trdiff_t,N> as well and save the trouble of the explicit conversions.=C2= =A0 What type of error does the bounds/index distinction prevent?=C2=A0 Mor= e generally, I am not sure why we shouldn't think of an index as a poin= t as well.=C2=A0 If we want the last element in an array x, we have : x.bou= nds() - 1=C2=A0 (or x.bounds() - {1,1, ...}).=C2=A0 We want to use this as = an index, but the current operator definitions would have it be a bounds.

> As a specific example,= consider if we want to compute the bounds of a valid convolution if we hav= e an array x and a filter k.=C2=A0 The bounds of the result are: x.bounds -= k.bounds + 1.=C2=A0 None of the arithmetic operators defined in the propos= al help me with that; I'd just have to write a loop.=C2=A0 Even if the = arithmetic operators weren't limited by the bounds/index distinction, t= here would still be no way to create a constant-valued vector, i.e. to hand= le the constant 1.

Explicit conversions would help with the first part, right? I hope= you are also not suggesting any such conversion should be implicit?
How= ever I do not really understand what do you mean in the second part, "= a constant-valued vector". We purposefully allow to create both bounds= <1> and index<1> from a scalar; and disallow such operation for= Rank > 1 (mostly because the semantics are not obvious, e.g. for rank 2= , should index<2>{16} be {0, 16}, {16, 0} or {16, 16}).
So net-net, in your example you should be able to do: x.bounds - index<2= >{k.bounds} + {1, 1}. It a little more long-winded, but IMO easier to co= mprehend than the original.

I don't think th= e explicit conversion to index adds to the clarity.=C2=A0 Also, the problem= with {1, 1} is that it depends on the dimensionality.=C2=A0 The interface = should be designed to be convenient for generic code.=C2=A0 I would suggest= a Scalar<T> type generated by a wrapper function scalar, from which = bounds and index can be implicitly constructed and which can be used in all= of the operators.

> Another related issue = I didn't notice before: the only way to construct an index or bounds ty= pe is from an initializer_list.=C2=A0 There is no (explicit) conversion fro= m std::array, for instance.=C2=A0 It is quite likely that an index or bound= s value would be computed by some other code, that may use a different repr= esentation.

Yes, we should add (const value_type* begin, const value_type* end= ) constructors for both types. <rant>Or it should be possible to crea= te initializer_list explicitly from such pair which would solve this issue = universally.</rant>

Okay, though initializer_list is not compile-time s= ized.
=C2=A0
>>> Furthermore, bounds and index essentially represent a
>= >> fixed-size discrete vector type, i.e. an augmented interface for s= td::array.
>>> Arguably the interface of std::array should jus= t be extended to add
>>> component-wise operators instead.=C2=A0 Also, a much more comp= rehensive set of
>>> component-wise operations should be provi= ded.=C2=A0 A statically-sized vector
>>> type is sorely needed= in C++, but it does not make sense to standardize
>>> these limited, single-purposes types when they could be, and s= hould be, much
>>> more general.
>>
>> We d= on't necessarily want to go for the most general possible type all at o= nce
>
> Certainly there is a large design space, but that is a r= eason to be cautious about standardizing anything. [...]

There are= two separate issue raised here:
- Why novel types -- this is primarily = to introduce "vocabulary" types and allow to discriminate index a= nd bounds, as they have different semantics, as discussed before. This was = also touched upon in the previously referred N3851, Appendix, II.
- Why so little functionality -- I agree with Jeffrey on this, and I have d= escribed more of the philosophy in the introduction. I would be interested = however what exactly you mean by "a much more comprehensive set of com= ponent-wise operations".

Almost anything could be useful (and I am thinking = of the perspective of a general fixed-size vector type, but these all apply= to index/bounds as well), but some particular examples: min, max, <, &l= t;=3D, =3D=3D, !=3D, >=3D, >.=C2=A0 The relational operations need to= return a vector of bool, which isn't compatible with index/bounds curr= ently since they aren't parameterized by the type.=C2=A0 Obviously, tho= ugh, it wouldn't make sense to call them index or bounds if they are pa= rameterized by a type.
=C2=A0

Having said tha= t, I assume this is something that can be added later as a pure extension, = correct?

Some of them, like min and max, could be= unintrusive, though neither index nor bounds would be a particularly meani= ngful choice as the return type.
=C2=A0

> Given the long turn around time for revisions= to C++

The Committee is certainly trying to get better at quic= ker iterations, let's not be discouraged by the ghosts of the past...

> Other omissions from index and bounds include:
>=C2=A0 1= .. no data() access to the components
>=C2=A0 2. no iterators over com= ponents for index or bound

index and bounds are not containers = and should not be used as such. For the limited number of scenario where I = want to use/modify indeterminate number of components, it was always suffic= ient for me to use a raw loop, and operator[]...
In case I'm wrong, these as well should be easy to add later as extensi= ons.

> for_each(x_array, y_array, x_array.bounds(), [&](a= uto &x, auto &y, auto pos) {
>=C2=A0=C2=A0 if (x !=3D y) std:= :cout << "mismatch at position " << pos << std:= :endl;
> });

I assume the idea here is that such "for_each&quo= t; uses begin(X), end(X) for all provided arguments, passing dereferenced i= terators to the function object. If so, I do not see how the current semant= ics of bounds_iterator would get into way here? It should just work as is, = no?

The idea is that it would generate N nested for= loops for N-dimensional array views, rather than an less efficient single = for loop.
=C2=A0

Actually, it should be possible to write the following = with just the current proposal and the current standard for_each:
=C2=A0= auto x_av =3D array_view<N>{x_array};
=C2=A0auto y_av =3D array_vi= ew<N>{y_array};
=C2=A0for_each(begin(x_av.bounds()), end(x_av.bounds()), [&](ind= ex<N> pos) {
=C2=A0 if(x_av[pos] !=3D y_av[pos]) std::cout <<= ; "mismatch at position " << pos << std::endl;
=C2= =A0});

Yes, but note that the iteration over bounds is more expensive and the= multiplication with all of the strides has to happen for every position.
=C2=A0

> From my perspective, the only way to properly expl= ore this design space for multidimensional arrays/containers/ranges is thro= ugh an actively developed and widely used open source library.

Our = (Microsoft) prototype implementation is in the pipeline to be published as = an open source project. It should happen really soon, and I hope it will he= lp to convey the discussed ideas.

That's good news.=C2=A0 I think = the best library is likely to be produced if the users of the library and t= he developers of the library are one and the same.=C2=A0 Issues become appa= rent and the library interface can be tweaked to correct them very quickly = that way.

--

---
You received this message because you are subscribed to a topic in the Goog= le Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this topic, visit https://groups.google.com/a/isocpp.org/d/topic/std-proposals/xzb1d5KUxMU= /unsubscribe.
To unsubscribe from this group and all its topics, send an email to std-prop= osals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
Visit this group at http://groups.google.com/a/isocpp.org/gro= up/std-proposals/.

--

---
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposa= ls+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
Visit this group at http://groups.google.com/a/isocpp.org/group/std-proposals/<= /a>.
--f46d044280b66ef7ac0507d7ccf7-- .