220 6747 <1403ae04-1e3c-4d90-8641-dbb078f5af99@isocpp.org> article Path: news.gmane.org!not-for-mail From: David Krauss Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: A tool to check if breaking changes will actually break any code out there Date: Sat, 28 Sep 2013 17:59:06 -0700 (PDT) Lines: 218 Approved: news@gmane.org Message-ID: <1403ae04-1e3c-4d90-8641-dbb078f5af99@isocpp.org> References: Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_1342_21540792.1380416346997" X-Trace: ger.gmane.org 1380416348 29839 80.91.229.3 (29 Sep 2013 00:59:08 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 29 Sep 2013 00:59:08 +0000 (UTC) To: std-proposals@isocpp.org Original-X-From: std-proposals+bncBCW25A7E3QCRBXHWTWJAKGQE3JD2JMA@isocpp.org Sun Sep 29 02:59:12 2013 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-oa0-f69.google.com ([209.85.219.69]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1VQ5Lq-00039d-9b for gclcip-std-proposals@m.gmane.org; Sun, 29 Sep 2013 02:59:10 +0200 Original-Received: by mail-oa0-f69.google.com with SMTP id n10sf13056556oag.8 for ; Sat, 28 Sep 2013 17:59:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:message-id:in-reply-to:references:subject:mime-version :x-original-sender:reply-to:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-subscribe:list-unsubscribe :content-type; bh=tmbM9yQKVBBOKkQ5ytNHG8Rq3fGwztN5BtBLfcdufxw=; b=jV8nPYz8EgTVII3tm2tMCDBsCEH8rhjCWv4wN6CxyDA2F4thouCzgOKeFKDNo/4jZR 64nyAqk5vc0etNGXjjrI9NQkm+YqO3sgIRjqwvj5p+BPsDmHKTa8NppS1zjlr+BBr2bA +cynDDlqcQ+gDOEk094qS/e5gHBLzR9I6snzMOKQEoe6GEHXxzc+1v5vVo5s6gMPdvM3 DlVS5/iEGEV2HJ+pfaQ5OQ4r0QuNbnAqk4dmxO+BoQ5dPCSe2CaiVF43+yBYbY6X59GD yznmAhPNiX89oeeXRGBJD6pcDBIJoXKhrrpu5vqcXDvOxyLdWXCKLRVFmIO2WApcbe5O qkIw== X-Received: by 10.42.84.6 with SMTP id j6mr8777504icl.31.1380416349065; Sat, 28 Sep 2013 17:59:09 -0700 (PDT) X-BeenThere: std-proposals@isocpp.org Original-Received: by 10.50.41.100 with SMTP id e4ls1138975igl.9.canary; Sat, 28 Sep 2013 17:59:08 -0700 (PDT) X-Received: by 10.50.111.200 with SMTP id ik8mr298714igb.7.1380416348653; Sat, 28 Sep 2013 17:59:08 -0700 (PDT) In-Reply-To: X-Original-Sender: potswa@gmail.com Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:6747 Archived-At: ------=_Part_1342_21540792.1380416346997 Content-Type: text/plain; charset=ISO-8859-1 On Friday, September 27, 2013 5:40:26 PM UTC+8, Bengt Gustafsson wrote: > > Having now followed this group for a few weeks I have noticed a great fear > of breaking someone's code when introducing new features. While this is of > course important in general it can be taken too far. Adding overloads of > methods is a typical case. Here is a recent example: > > Someone suggested a new stringstream::str() const && overload to grab the > string buffer out of a stringstream that will not be used anymore. The > problem is that if someone has written move(mystringstream).str() today > they may rely on the non-existance of a const&& overload and continue using > the stream afterwards. This use of move() is clearly ill-adviced as the > notion of move() is that it renders the moved object unusable. Still, for > fear that someone may rely on the move() being a nop in this case the > suggestion was quite strongly advocated against. > If the user wrote that, then they intended for move() not to be a no-op. That's an idiomatic rvalue member access. Perhaps the code wasn't *tested*to work as they intended, and perhaps they added something later to use the moved-from object, and now there's a bug per their intent that isn't getting caught. That they could predict the no-op is besides the point; such is the nature of "unspecified state." Part of that proposal is that O(1) may not be guaranteed if the implementation doesn't/can't reassign ownership of the underlying buffer. > My suggestion is to work towards creating a tool that can do a statistical > analysis of different potentially code breaking standard changes by > scanning a large base of source code. This would be a non-trivial task of > course, but I think today doable. > What advance in computer science makes this doable? > Clang can be used as a basis for the analysis itself and there are plently > of places with lots of source code like github and sourceforge. > Volume of source isn't the answer. People write in idioms, and tend not to be very adventurous. My current project doesn't even *parse* in Clang. > The main obstacle I see is to be able to automate the build environment > setup. It is quite easy to download a project from sourceforge but to get > all the include directories etc. set up to compile it not standardized. > Even if only cmake projects are considered there is usually a lot of > handiwork required to get projects compilable. > > There are of course a lot of large and important code bases that are not > reached by this type of tool as they are proprietary. As an extension, > however, it would be possible to offer some kind of satellite setup where a > company with a large and valuable code base (and possibly a say in the > standards committe) could have their own scanning server which receives > jobs from the central tool and only delivers the statistics back, say > "there were 4 uses of move(stringstream).str() in 176563466 lines of code". > As the statistics is human readable the risk of leaking sensitive > information can be controlled. > You could sell such a product. Companies could write-up their own feedback and send it to official channels such as proposal authors. A good value proposition can make money for everyone. But the initial investment here sounds pretty high. > The other problem is how to define a test. Ideally in the example you > would test for uses of move(stringstream).str() where the stringstream was > actually used after this statement. This would require quite a lot of > analysis and probably some specific C++ code written in the Clang code > base. Just detecting move(stringstream).str() could maybe be done by some > kind of general pattern matching in the AST, while just detecting > move(stringstream) would be simple but useless as it would get myriads of > uninteresting hits. > An AST search index would be a nice tool for proposal authors. But most code hasn't been fully updated to C++11. What we do now is theorize about what could go wrong, and that's not such a bad way to predict the future :v) . I don't think move(ss).str() probably ever literally occurs unless the user is intending a move operation and "living with" the copy for now. Finding problems would be more a matter of finding all instances of move(x).y where x is type-dependent and multiplying by the probability of an argument into a generic function being a stringstream. The necessary statistics may not be very well-developed. Academic review might turn up some kind of analysis techniques. > Another example where such code scanning would be interesting is the > introduction of names like Object into the std namespace which will happen > if some concepts lite suggestions get accepted into the standard. Here I > think that a scan would detect lots of code that would break severly... A > scan could tell if I'm right in this or if it is a non-issue. > RIP Google Code Search. -- --- You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. Visit this group at http://groups.google.com/a/isocpp.org/group/std-proposals/. ------=_Part_1342_21540792.1380416346997 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

On Friday, September 27, 2013 5:40:26 PM UTC+8, Bengt = Gustafsson wrote:
Having now followed this group for a few weeks I have noticed a great f= ear of breaking someone's code when introducing new features. While this is= of course important in general it can be taken too far. Adding overloads o= f methods is a typical case. Here is a recent example:

S= omeone suggested a new stringstream::str() const && overload to gra= b the string buffer out of a stringstream that will not be used anymore. Th= e problem is that if someone has written move(mystringstream).str() today t= hey may rely on the non-existance of a const&& overload and continu= e using the stream afterwards. This use of move() is clearly ill-adviced as= the notion of move() is that it renders the moved object unusable. Still, = for fear that someone may rely on the move() being a nop in this case the s= uggestion was quite strongly advocated against.

If the user wrote that, then they intended for move() not to be a no-= op. That's an idiomatic rvalue member access. Perhaps the code wasn't te= sted to work as they intended, and perhaps they added something later t= o use the moved-from object, and now there's a bug per their intent that is= n't getting caught. That they could predict the no-op is besides the point;= such is the nature of "unspecified state." Part of that proposal is that O= (1) may not be guaranteed if the implementation doesn't/can't reassign owne= rship of the underlying buffer.
 
My suggestion is to work towards cre= ating a tool that can do a statistical analysis of different potentially co= de breaking standard changes by scanning a large base of source code. This = would be a non-trivial task of course, but I think today doable.

What advance in computer science makes this doable?<= br> 
Clang can be used as a basis for the analysis itself and there are = plently of places with lots of source code like github and sourceforge.

Volume of source isn't the answer. People wri= te in idioms, and tend not to be very adventurous. My current project doesn= 't even parse in Clang.
 
The main obstacle I see is to be abl= e to automate the build environment setup. It is quite easy to download a p= roject from sourceforge but to get all the include directories etc. set up = to compile it not standardized. Even if only cmake projects are considered = there is usually a lot of handiwork required to get projects compilable.

There are of course a lot of large and important cod= e bases that are not reached by this type of tool as they are proprietary. = As an extension, however, it would be possible to offer some kind of satell= ite setup where a company with a large and valuable code base (and possibly= a say in the standards committe) could have their own scanning server whic= h receives jobs from the central tool and only delivers the statistics back= , say "there were 4 uses of move(stringstream).str() in 176563466 lines of = code". As the statistics is human readable the risk of leaking sensitive in= formation can be controlled.

You could sel= l such a product. Companies could write-up their own feedback and send it t= o official channels such as proposal authors.

A good value propositi= on can make money for everyone. But the initial investment here sounds pret= ty high.
 
The other problem is how to define a test. Ideally in the e= xample you would test for uses of move(stringstream).str() where the string= stream was actually used after this statement. This would require quite a l= ot of analysis and probably some specific C++ code written in the Clang cod= e base. Just detecting move(stringstream).str() could maybe be done by some= kind of general pattern matching in the AST, while just detecting move(str= ingstream) would be simple but useless as it would get myriads of uninteres= ting hits.

An AST search index would be a = nice tool for proposal authors. But most code hasn't been fully updated to = C++11. What we do now is theorize about what could go wrong, and that's not= such a bad way to predict the future :v) .

I don't think move(ss).str() probably ev= er literally occurs unless the user is intending a move operation and "livi= ng with" the copy for now. Finding problems would be more a matter of findi= ng all instances of mov= e(x).y where x is type-dependent and multiplying by the probability = of an argument into a generic function being a stringstream.

The nec= essary statistics may not be very well-developed. Academic review might tur= n up some kind of analysis techniques.
 
Another example where such co= de scanning would be interesting is the introduction of names like Object i= nto the std namespace which will happen if some concepts lite suggestions g= et accepted into the standard. Here I think that a scan  would detect = lots of code that would break severly... A scan could tell if I'm right in = this or if it is a non-issue.

RIP Google C= ode Search.
 

--
 
---
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
Visit this group at http://groups.google.com/a/isocpp.org/group/std-proposals/<= /a>.
------=_Part_1342_21540792.1380416346997-- .