220 40204 article Path: news.gmane.org!.POSTED!not-for-mail From: Richard Hodges Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: Include file content as data at compile-time Date: Fri, 21 Sep 2018 11:44:52 +0200 Lines: 273 Approved: news@gmane.org Message-ID: References: <2314a9d2-9cc4-4054-ad3a-3e31a2094fc0@isocpp.org> Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="0000000000007601a605765e7e7f" X-Trace: blaine.gmane.org 1537522981 12042 195.159.176.226 (21 Sep 2018 09:43:01 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 21 Sep 2018 09:43:01 +0000 (UTC) To: std-proposals@isocpp.org Original-X-From: std-proposals+bncBD4PBM7UWAHRBIP3SLOQKGQE5OH73BQ@isocpp.org Fri Sep 21 11:42:57 2018 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-io1-f71.google.com ([209.85.166.71]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g3Hxf-00030J-Mx for gclcip-std-proposals@m.gmane.org; Fri, 21 Sep 2018 11:42:56 +0200 Original-Received: by mail-io1-f71.google.com with SMTP id l24-v6sf19458109iok.21 for ; Fri, 21 Sep 2018 02:45:06 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1537523106; cv=pass; d=google.com; s=arc-20160816; b=TUF74U8FNBInsoetQdibkfGsZhZhBbt/PG84VxrBA8ywTtKyjseVC21wgCzWg/nDGv KSb7waQwz1s590JtbRrP08ph2jir0w1B0ndJkAhMOXHrOC1kXXMmJFuD3cuJwPtASP1y ZjTRIQS6kkzF8geiuTgm/gbit6nixMbYCpRv0cH58nTHXyMYi+3qPho7Gi2iND5JSBCX ZsDBU5XdnGEDGN7jA7Kgo0vaPG4ToLV6LTJ84KSRFvgcZXZQ3nQkwLwwmd0fjw9ePsfm fnKLKI810/lqFctZmfSPZ0urNGLbYB+yKOcM10xUMJWxjanEBMyL30nuCTY48wGo4LPN RKNQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:to:subject:message-id:date :from:in-reply-to:references:mime-version:dkim-signature; bh=cs3G/yyzEei95Wo3JWa77YodM7gpOHiggstsHVml2SA=; b=tXfieJKkAfx4EJ/vULM7/uqf/797AlAli14X4NhaVjwD9vjBl+jqSUJOCF+SEH30xq 0F6lFucDsgajuQJMxrwXLS5UIvuTCFUNMR+YVUwHhItmG+uLgGosfWtnbTJ1d6pifWB2 PdXsmhAHpXL4uC0TyofjvK9Edw+2Lah6sw7iT6ODsPjmyEbw2m6stNMe/C/jkTitN2zZ GyQLorQfGlTZhFMqx2kBC6w6tE58rQfVSCvpEo54uyOmfyQl5R+Zs3I/byfeeWXifLmQ g77kTDTQ2WVRms1Cpd+RllSn3tN39bJErPQRJL9M3X7Mvp3Cx3CfQr0sRVy+2qz1Rywa dxFw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=rLOt1DT+; spf=pass (google.com: domain of hodges.r@gmail.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=hodges.r@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isocpp-org.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :x-original-sender:x-original-authentication-results:reply-to :precedence:mailing-list:list-id:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=cs3G/yyzEei95Wo3JWa77YodM7gpOHiggstsHVml2SA=; b=A6xgqPNtGHFz2O3dVOWqjhcg8q9dpJVEKbCQsbfUvurFqJESkFSif2dclQnDxcw2HU K3f4DbHgyJAChBD/ls/hTvkpeH8ez9uShazDh5q10N/5atdFlZ8W1sPpuDJzI6jFBG/N NPWZXhmMRdYrlSLF+8sB/GmkV5ItNcfznZadpUyU4vUTQhFJrMUUbj3u43BUT3erKiwO gSJSEGp0ywD26FzaCB337vLgWz6wst5+CYgtChpTMr9AjW/ssUwjFJDQOSE7cFTfeYEK bnLIYxcCQQMIA/MapI0MaHg+g3SP1nCuDXIiu0b4zv+9BChqD7RLTBifAIzHWhs4fpei 6nXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=cs3G/yyzEei95Wo3JWa77YodM7gpOHiggstsHVml2SA=; b=AVebIfp/uQCG92gVtUoPabvU3ntFqRB49y1JvnjuSvkAx2cpMg8sIDGAbf98Arl5kQ AEDIdeTvl66W6DTvRzQUsLX9aQLYYUhYwwkkDudgtceS7rs0YBen1y2EMTjJERV3MZZF tlDV7yGWyW88P/5MRENl+9H4vW74Q/SoVA4XTJLeYKIms4DZhJdIw3uECqb8WOsb980/ 9CUyvTNpp4kmvoDYIJpaFOTXkt8VO1LYLh/rld/2hp6vSp/I9qWkrZVTVCtZIojXbmX0 WBvryj/k9Z9Z0B4qFCwD4zpuHVEZTtMu64F7iKTugPGS0N7f+OYKXLA3h3PsLcBJhcMw E3CA== X-Gm-Message-State: APzg51AOyNK1blW0/bAxkTX1RD0mez9+uWs0M8idv7FQ4SxlAD/qO4pj SUr/NkQyxoW4Lo8NgPB3NhjyfA== X-Google-Smtp-Source: ANB0VdZTfMB03j0vuwQeHsUnZHRUdhKhg/w9bohQN/jSazo7BwyugFoPnh43/iLNyBUMJbkR0WqJaA== X-Received: by 2002:a24:6fd3:: with SMTP id x202-v6mr4881002itb.34.1537523106077; Fri, 21 Sep 2018 02:45:06 -0700 (PDT) X-BeenThere: std-proposals@isocpp.org Original-Received: by 2002:a24:f64a:: with SMTP id u71-v6ls748067ith.3.canary-gmail; Fri, 21 Sep 2018 02:45:04 -0700 (PDT) X-Received: by 2002:a24:918d:: with SMTP id i135-v6mr2064699ite.98.1537523104784; Fri, 21 Sep 2018 02:45:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537523104; cv=none; d=google.com; s=arc-20160816; b=d/TVIsSvUXpHZoN3JL6Ifv/EjEN5Q7S22STBVtx/GxlFP3LISbo9pGdFqJlyItXtKR 7CrKg4yVQ4QYg7pKsI5EHOJYLyqOmcRmexEu63xtpkMRyS3tVl0Vg59I39XhwS/gl+8/ sOindgTjUJ2Av7t+uBfyLyPanEAUO9aTEP74KNZ3jrJD49pEZYfeFVV+NMBtPa9xWVxW kF0ZOi4Sy08Z2TkPuQ2G/GQs8Wg1/r5ch6tyWqi+N8FSEBkoLj0PGl5qCOJ2IlYzTEr1 0iiAj3pbY8SpmTo5wSQ7IoUQGr5jLcpylEYb4S0ecSrwh+gA64hwIdeQ6hsHimJKbhyu bDFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :dkim-signature; bh=udVmb1dEfWRrQ3vCRg6GLIsS2EbjzEI3bcXLvNJM7/w=; b=fnHbRFP3bS578dqcE2GP7QJPRBfdjTTvoWN53abryZtzT8Qs4qRrxhqgOny5UalK+O YLvjEYrnnn3jGerIwM9Gl7JgfxMpa0DGgmilOsDytGqLF/LQI/nbGke+0zt75xbFxxXH OvSMX6uIss/bHvSweMhYSz5FXjPcgthi0btxwxYBCaJMvEC5wubz3aGCk2lbAaYx4tTL kH8bAaNZHdq1c+lzDRcLmtxPdfMCgDpJhZsCXomMxgzE/ETobwQEDXZ8Ux+wGbi370QS HGzGDiI5D+ggPKNbOOb6nyIZIKyAyKnj7OzPcbXIt8GtA02isoC4lBH4QZ5zyoMcqOjG 9coA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=rLOt1DT+; spf=pass (google.com: domain of hodges.r@gmail.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=hodges.r@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Original-Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id f130-v6sor891523jab.38.2018.09.21.02.45.04 for (Google Transport Security); Fri, 21 Sep 2018 02:45:04 -0700 (PDT) Received-SPF: pass (google.com: domain of hodges.r@gmail.com designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; X-Received: by 2002:a02:b38c:: with SMTP id p12-v6mr38108382jan.79.1537523104218; Fri, 21 Sep 2018 02:45:04 -0700 (PDT) In-Reply-To: <2314a9d2-9cc4-4054-ad3a-3e31a2094fc0@isocpp.org> X-Original-Sender: hodges.r@gmail.com X-Original-Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=rLOt1DT+; spf=pass (google.com: domain of hodges.r@gmail.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=hodges.r@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Spam-Checked-In-Group: std-proposals@isocpp.org X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:40204 Archived-At: --0000000000007601a605765e7e7f Content-Type: text/plain; charset="UTF-8" There have been a few times when I've wanted something like this. My workaround was to use the configure_file feature of cmake. Although this is a little unsatisfactory, as the debugger is then tracking the wrong file. Shouldn't this be a preprocessor feature rather than a language feature? e.g. const char text[] = #include_quoted_ascii "data/hello.txt" ; or even: const std::u8string text = #pragma include(file="data/hello.txt", encoding="utf-8", prefix="r\"", postfix="\"") ; On Fri, 21 Sep 2018 at 11:15, Sidney Congard wrote: > Hi, > > I want to propose a new feature similar to what have already been discuted > here : > > https://groups.google.com/a/isocpp.org/forum/#!topic/std-proposals/b6ncBojU8wI%5B1-25%5D > > Sorry if this proposal already exists or if other topics have been opened > about this > subject since then. > > What do you think about it ? > > > Proposal > > This proposal aims to allow to embed file data in the library or > executable built by > adding a new preprocessor command, behaving similarly to R"~( #include )~" > : > > // Content of : 'Hello' > > // Expands to a string literal. Equivalent to : > // constexpr char text[] = "Hello" "\0"; > constexpr char text[] = __INCLUDE_TEXT() "\0"; > > // Expands to a char sequence. Equivalent to : > // constexpr uint8_t data[] = { 'H','e','l','l','o', }; > constexpr uint8_t data[] = { __INCLUDE_DATA() }; > > > Motivation > > The use cases would mainly be : > - to parse DSLs at compile-time (eg. with Boost.Metaparse). > - to reduce dependencies and/or load time by incorporating files into > the library or > executable built. > > Several solutions exist to incorporate file content (resources or code) > into the target > built (for libraries or executables). > > But, these solutions ether don't work at compile-time, are not fully > portable and/or > need to modify the build process (incbin, xxd, scripts for file > manipulation). > > > Tradeoffs > > Performances could be improved if this proposal was integrated in the > langage, but > it does currently not have means to load files at compile-time AFAIK > whereas the > preprocessor has this with #include. > > Two macro are proposed for conveniance, to manipulate string literal to > arrays. > Note the trailing coma for __INCLUDE_DATA : this syntax is less natural > but it > would allow embed empty files (which #include do). > > Multi-byte types are not supported because of the endianness issue. It can > be > problematic to decode Unicode (apart UTF-8). 'std::byte' can't be > isntancied from > chars even if it can be the most suitable type to design raw bytes. > > -- > You received this message because you are subscribed to the Google Groups > "ISO C++ Standard - Future Proposals" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to std-proposals+unsubscribe@isocpp.org. > To post to this group, send email to std-proposals@isocpp.org. > To view this discussion on the web visit > https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/2314a9d2-9cc4-4054-ad3a-3e31a2094fc0%40isocpp.org > > . > -- You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/CALvx3haGrqjjhU-iPQwKtrCTmPYvX8WEQBeD%3DNbdQUYXdkw%2BLA%40mail.gmail.com. --0000000000007601a605765e7e7f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
There have been a few times when I've wanted something= like this. My workaround was to use the configure_file feature of cmake.= =C2=A0
Although this is a little unsatisfactory, as the debugger is the= n tracking the wrong file.

Shouldn't this be a prepr= ocessor feature rather than a language feature?

e.= g.

const char = text[] =3D=C2=A0
#incl= ude_quoted_ascii "data/hello.txt"
;

or even:
=
const std::u8string= text =3D=C2=A0
#pragm= a include(file=3D"data/hello.txt", encoding=3D"utf-8", = prefix=3D"r\"", postfix=3D"\"")
<= div>;




=

On Fri, 21 Sep 2018 at 11:15, Sidney Congard <sidney.congard@gmail.com> wrote:
Hi,

I want to propose a new= feature similar to what have already been discuted here :
https://groups.google.com/a/isocpp.org/forum= /#!topic/std-proposals/b6ncBojU8wI%5B1-25%5D

Sorry if this = proposal already exists or if other topics have been opened about this
subject since then.

What do you think ab= out it ?


Propo= sal

This proposal aims to allow to embed file data= in the library or executable built by
adding a new preprocessor = command, behaving similarly to R"~( #include )~" :

=
// Content of <data/hello.t= xt> : 'Hello'

// Expands to a string li= teral. Equivalent to :
// constexpr char text[] =3D &= quot;Hello" "\0";
constexpr char text[] =3D __INCLU= DE_TEXT(<data/hello.txt>) "\0";
// Expands to a char sequence. Equivalent to :
<= /span>// constexpr uint8_t data[] =3D { 'H','e','l= 9;,'l','o', };
constexpr uint= 8_t data[] =3D { __IN= CLUDE_DATA(<data/hello.txt>) };
<= br>

Motivation

The use cases would mainly be :
=C2=A0 - to par= se DSLs at compile-time (eg. with Boost.Metaparse).
=C2=A0 - = to reduce dependencies and/or load time by incorporating files into the lib= rary or
=C2=A0=C2=A0=C2=A0 executable built.

Several solutions exist to incorporate file content (resources or = code) into the target
built (for libraries or executables).
But, these solutions ether don't work at compile-time, are not fully = portable and/or
need to modify the build process (incbin, xxd, sc= ripts for file manipulation).


Tradeoffs

Performances could be= improved if this proposal was integrated in the langage, but
it = does currently not have means to load files at compile-time AFAIK whereas t= he
preprocessor has this with #include.

<= div>Two macro are proposed for conveniance, to manipulate string literal to= arrays.
Note the trailing coma for __INCLUDE_DATA : this syntax = is less natural but it
would allow embed empty files (which #incl= ude do).

Multi-byte types are not supported be= cause of the endianness issue. It can be
problematic to decode Un= icode (apart UTF-8). 'std::byte' can't be isntancied from
=
chars even if it can be the most suitable type to design raw bytes.

--
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit = https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/2314a9d2-9cc4-= 4054-ad3a-3e31a2094fc0%40isocpp.org.

--
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposa= ls+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/CALvx3haGrqjj= hU-iPQwKtrCTmPYvX8WEQBeD%3DNbdQUYXdkw%2BLA%40mail.gmail.com.
--0000000000007601a605765e7e7f-- .