220 24824 <29edd7b0-671f-49ea-8538-46070981228f@isocpp.org> article Path: news.gmane.org!not-for-mail From: FrankHB1989 Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: Modules are hard for build tools Date: Sun, 28 Feb 2016 22:42:41 -0800 (PST) Lines: 291 Approved: news@gmane.org Message-ID: <29edd7b0-671f-49ea-8538-46070981228f@isocpp.org> References: Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_792_197808656.1456728161680" X-Trace: ger.gmane.org 1456728171 24197 80.91.229.3 (29 Feb 2016 06:42:51 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 29 Feb 2016 06:42:51 +0000 (UTC) To: ISO C++ Standard - Future Proposals Original-X-From: std-proposals+bncBCTJVBPG3QIBBYWQZ63AKGQEMU2B4RI@isocpp.org Mon Feb 29 07:42:44 2016 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-pf0-f199.google.com ([209.85.192.199]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1aaHXX-0008JF-SB for gclcip-std-proposals@m.gmane.org; Mon, 29 Feb 2016 07:42:44 +0100 Original-Received: by mail-pf0-f199.google.com with SMTP id 124sf138777301pfg.0 for ; Sun, 28 Feb 2016 22:42:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isocpp-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:message-id:in-reply-to:references:subject:mime-version :x-original-sender:reply-to:precedence:mailing-list:list-id :x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=5UCfJPkyacHSmGTIiJHOLR4yT2I7wieCGdieUPQS3SM=; b=iuYp1eCbYmBTfFTcdkj9FvCtFH3ki6dYZSpOt379RFWFfkd++aTeqgzkmfsQ/RSdj4 3+9H2a1JfuMCSXkHq3YE4WtBvdNJS2hpoS19aTMbc3jLccQr7E8YLG3ZNStn+lBh/sw6 t1F86Vgh/hiPzcKEe5Pgh9CkQXlGLQRHvzDys+8+yItlQbNh1GI3kd4slnHQm5k/FQPO YN30mZukUQrr+pPvPEuDFv0mR1wk5svEjcyi2o54Cn7GXgQQ+J5WeEinP/PmFl0l2Wcp hEpnkoWbrIctudm1C7pjvOQbnt50oongjb81HXyxMfbGg/Knxd0o2i4YxrcBV6AjR2ul o66g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:message-id:in-reply-to:references:subject:mime-version :x-original-sender:reply-to:precedence:mailing-list:list-id :x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=5UCfJPkyacHSmGTIiJHOLR4yT2I7wieCGdieUPQS3SM=; b=RHdbzh0WWkP0f5GAHK0G9pkO1IbJvg2hEtADrufi66Ewq8b3iv8pRNyP3mwI9e4Z6F 7TJj5QnsiLmwEwvO7wOJ92axMTYR8OvIEFWxsvQH0Iu/zBHZWdkI77OQLDF4G6Cd7BMV P1u41K0gajzrf3wwK6uKI36LwVl9taftrGAgOYTQ7QMWAa2JtmP04WxCKsk0bz9euy4U PFa0s9U17YT3LsXtuGxyfxgDF8rZfuq8z7+1OZBqsuo0+gZFSm8YsE260oZw5u2EyVJK HxnYLbEl778oCYJktynaNdR47hUmj9Hab7i21wjnBdi8gKTFJX/sjxKNwy3QDsW8CzLT xdVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:message-id:in-reply-to:references :subject:mime-version:x-original-sender:reply-to:precedence :mailing-list:list-id:x-spam-checked-in-group:list-post:list-help :list-archive:list-subscribe:list-unsubscribe; bh=5UCfJPkyacHSmGTIiJHOLR4yT2I7wieCGdieUPQS3SM=; b=L4g0He4nkHsJj0+NncYPzSqZpqiPmnBrOQhy6K4HBsW37mjopHouMYNY+ISQkAPCW/ +2SrdtzIwLASGZBUmHAx/DnWI/6w5RQ5HujF5BzpGQgcm3UzaFgzHQ0q5ymHXN8o+mXO MfaiTwKrDMkMZKU/YtKHh3oevSFm0MVQEUKwLtwh7nxs5mizQAgS1JR9p/PHqWf1ULLh kYDCtV0orDqnIBSj6cG41A4pLdcWDcu9i9VBQaQvYQ+CRWxGMulrNhLoKEzrVPkv8/v2 KkFUVTImwdJ9HsT/vaySKDsG0CchGlvUWlgWJsc5NBO9UeMd/3hLjSB9nhP/WMqWpFWI osbQ== X-Gm-Message-State: AD7BkJKyiBHnE+vQtIMsW1N2gxr8pKISgJ+ZCvltdfpojyxA8cbEJODu51pK1bXYGAvVxA== X-Received: by 10.66.63.68 with SMTP id e4mr11536872pas.40.1456728162898; Sun, 28 Feb 2016 22:42:42 -0800 (PST) X-BeenThere: std-proposals@isocpp.org Original-Received: by 10.107.186.10 with SMTP id k10ls233343iof.46.gmail; Sun, 28 Feb 2016 22:42:42 -0800 (PST) X-Received: by 10.50.102.106 with SMTP id fn10mr133253igb.5.1456728162136; Sun, 28 Feb 2016 22:42:42 -0800 (PST) In-Reply-To: X-Original-Sender: frankhb1989@gmail.com Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Spam-Checked-In-Group: std-proposals@isocpp.org X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:24824 Archived-At: ------=_Part_792_197808656.1456728161680 Content-Type: multipart/alternative; boundary="----=_Part_793_1406381613.1456728161680" ------=_Part_793_1406381613.1456728161680 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable =E5=9C=A8 2016=E5=B9=B42=E6=9C=8829=E6=97=A5=E6=98=9F=E6=9C=9F=E4=B8=80 UTC= +8=E4=B8=8A=E5=8D=889:41:06=EF=BC=8CSean Middleditch=E5=86=99=E9=81=93=EF= =BC=9A > > I'll start off by saying that I don't think that Abyx's worries are a=20 > significant problem. Yes, build systems will need to change, but they've= =20 > needed to change for 20 years now. If modules are the impetus for the=20 > community to finally make building C++ not ridiculous, that's just fine= =20 > with me. I'm actually really hoping that modules are the final catalyst f= or=20 > the community to realize that CMake is this generations' Autotools. ;) > > That said, I do think it's worthwhile that folks at least understand wher= e=20 > Abyx is coming from. > > On Friday, February 26, 2016 at 11:57:44 AM UTC-8, Richard Smith wrote: > >> On Fri, Feb 26, 2016 at 7:23 AM, Abyx wrote:=20 >> > > 1) You manually list the dependencies between your libraries in your=20 > > build rules. This is often a good thing for your code health, but may=20 > > not be right for everyone. =20 > > > We're not talking about libraries! We're talking about translation units= =20 > and modules. > > A single library could be made up of dozens and dozens (if not hundreds)= =20 > of TUs and modules. > =20 > >> 2) Your build process scans for import statements. This is not really=20 >> any harder than scanning for #includes; no real parsing is required,=20 >> at most, you need tokenize and maybe preprocess (if you want 'import=20 >> MACRO_NAME' to work). The same is true for #includes. =20 > > > This is much harder than scanning for #include files. No good modern C++= =20 > build chain actually has a separate "scan for includes" step anymore. :) > Do you mean scan by searching the text? Otherwise, the compiler may do some= =20 "scan" work (as you said below). It certainly should not be the right way. However, there are tools do that= =20 in practice, e.g. Code::Blocks. (yep, not so modern & good...)=20 =20 > > With headers, the dependency collection is just part of building. MSC,=20 > GCC, etc. all have flags that allow them to generate dependency chains=20 > while compiling, e.g. -MM -MF on GCC. The order of compilation of= =20 > TUs is usually irrelevant; in fact, it's safe to build a dependent withou= t=20 > even knowing the dependency exists, so long as all the dependee files=20 > already exist. The dependencies are used only for checking whether files= =20 > are out of date relative to another file. > > In fact, A.cpp and B.cpp can be compiled in any order, or compiled in=20 > parallel, or compiled on two different machines, even if B.cpp has a=20 > dependency on A.h. > > Agreed. And g++ -MMD as a single step works as well, though the result=20 (makefile deps) may need additional parsing process, but not hard. I've=20 actually implemented it in several lines=20 =20 with aid of my parsing library utilities and integrated it into my own=20 implemented-in-one-file build tool=20 .. =20 > With modules, this is no longer the case. The dependency graph must be=20 > present before any compilation begins, as building properly is now=20 > dependent on per-TU build artifacts (the .ifc files). Distributed builds= =20 > require synchronization of the artifacts or island solving to ensure that= =20 > files with dependency chains are all built on the same build node. This i= s=20 > because B.cpp no longer would depend on A.h, but instead would depend on= =20 > A.ifc, which doesn't even exist until after A.cpp is compiled. > > Note in particular that this also means that with modules we've just=20 > serialized a previously parallel process, which is theoretically a build= =20 > deoptimization - seemingly the exact opposite of what many of us are=20 > expecting or want out of modules! > > That said, your option 3 is the obvious solution. It's roughly what most= =20 > other modern languages with module systems do, and C++ implementations an= d=20 > the build tools we use are going to have to modernize to adopt to a world= =20 > where the compiler plays a more integral role in the build chain than it= =20 > did before. > =20 > >> >> 3) You ask your compiler to implicitly build (and cache) module=20 >> interfaces on demand. This requires that your compiler has some way to= =20 >> map from an imported module name to the relevant interface file(s).=20 >> That could happen via some implementation-defined means (such as=20 >> Clang's module map files) or by making the module names directly=20 >> correspond to module interface files (as suggested in=20 >> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0273r0.pdf=20 >> >> ).=20 > > I'm worrying there are many details not clear to average C++ users. And=20 comparing to the complexity of the specification, the whole system still=20 seems to be too weak in functionality. Are there anything like *structures*= ,=20 *signatures* and *functors* in module systems of ML-like languages proposed= ? =20 --=20 You received this message because you are subscribed to the Google Groups "= ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. To view this discussion on the web visit https://groups.google.com/a/isocpp= ..org/d/msgid/std-proposals/29edd7b0-671f-49ea-8538-46070981228f%40isocpp.or= g. ------=_Part_793_1406381613.1456728161680 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


=E5=9C=A8 2016=E5=B9=B42=E6=9C=8829=E6=97=A5=E6=98= =9F=E6=9C=9F=E4=B8=80 UTC+8=E4=B8=8A=E5=8D=889:41:06=EF=BC=8CSean Middledit= ch=E5=86=99=E9=81=93=EF=BC=9A
<= div dir=3D"ltr">
I'll start off by saying that I don't think th= at Abyx's worries are a significant problem. Yes, build systems will ne= ed to change, but they've needed to change for 20 years now. If modules= are the impetus for the community to finally make building C++ not ridicul= ous, that's just fine with me. I'm actually really hoping that modu= les are the final catalyst for the community to realize that CMake is this = generations' Autotools. ;)

That said, I do thi= nk it's worthwhile that folks at least understand where Abyx is coming = from.

On Friday, February 26, 2016 at 11:57:44= AM UTC-8, Richard Smith wrote:
On Fri, Feb 26, 2016 at 7:23 AM, Abyx <fl....@= gmail.com> wrote:

1) You manually list the d= ependencies between your libraries in your=C2=A0
build rules. This is often a good thing for your code health, but may=C2= =A0
not be right for everyone.=C2=A0=C2=A0

We're not talking about libraries! We'= re talking about translation units and modules.

A = single library could be made up of dozens and dozens (if not hundreds) of T= Us and modules.
=C2=A0
2) Your build process scans for import statements. This is not really
any harder than scanning for #includes; no real parsing is required,
at most, you need tokenize and maybe preprocess (if you want 'impor= t
MACRO_NAME' to work). The same is true for #includes. =C2=A0

This is much harder than scanning for #include fi= les. No good modern C++ build chain actually has a separate "scan for = includes" step anymore. :)
Do you mea= n scan by searching the text? Otherwise, the compiler may do some "sca= n" work (as you said below).
It certainly should not be the right w= ay. However, there are tools do=20 that in practice, e.g. Code::Blocks. (yep, not so modern & good...) =C2=A0

With headers, the dependency collection = is just part of building. MSC, GCC, etc. all have flags that allow them to = generate dependency chains while compiling, e.g. -MM -MF <file> on GC= C. The order of compilation of TUs is usually irrelevant; in fact, it's= safe to build a dependent without even knowing the dependency exists, so l= ong as all the dependee files already exist. The dependencies are used only= for checking whether files are out of date relative to another file.
=

In fact, A.cpp and B.cpp can be compiled in any order, = or compiled in parallel, or compiled on two different machines, even if B.c= pp has a dependency on A.h.

Agr= eed. And g++ -MMD as a single step works as well, though the result=20 (makefile deps) may need additional parsing process, but not hard. I've= actually=20 implemented it in several line= s with aid of my parsing library utilities and integrated it into my own implemented-in-one-file build tool.
=C2= =A0
With modules, this is no longer the case. The dependency graph = must be present before any compilation begins, as building properly is now = dependent on per-TU build artifacts (the .ifc files). Distributed builds re= quire synchronization of the artifacts or island solving to ensure that fil= es with dependency chains are all built on the same build node. This is bec= ause B.cpp no longer would depend on A.h, but instead would depend on A.ifc= , which doesn't even exist until after A.cpp is compiled.

Note in particular that this also means that with modules w= e've just serialized a previously parallel process, which is theoretica= lly a build deoptimization - seemingly the exact opposite of what many of u= s are expecting or want out of modules!

That said,= your option 3 is the obvious solution. It's roughly what most other mo= dern languages with module systems do, and C++ implementations and the buil= d tools we use are going to have to modernize to adopt to a world where the= compiler plays a more integral role in the build chain than it did before.=
=C2=A0

3) You ask your compiler to implicitly build (and cache) module
interfaces on demand. This requires that your compiler has some way to
map from an imported module name to the relevant interface file(s).
That could happen via some implementation-defined means (such as
Clang's module map files) or by making the module names directly
correspond to module interface files (as suggested in
http://www.open-std.org/jtc1/sc22/wg21/docs/p= apers/2016/p0273r0.pdf).=C2=A0

I'm worrying there are many details not clear to average C++ users= .. And comparing to the complexity of the specification, the whole system st= ill seems to be too weak in functionality. Are there anything like struc= tures, signatures and functors in module systems of ML-li= ke languages proposed?

=C2=A0

--
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposa= ls+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.= com/a/isocpp.org/d/msgid/std-proposals/29edd7b0-671f-49ea-8538-46070981228f= %40isocpp.org.
------=_Part_793_1406381613.1456728161680-- ------=_Part_792_197808656.1456728161680-- .