220 41359 <1d2bd16a-c2f9-4baa-b2d3-4b0241de8992@isocpp.org> article Path: news.gmane.org!.POSTED!not-for-mail From: joel.g.hemphill@gmail.com Newsgroups: gmane.comp.lang.c++.isocpp.proposals Subject: Re: Optimizing Power Function for Integral Types Date: Fri, 18 Jan 2019 17:12:17 -0800 (PST) Lines: 742 Approved: news@gmane.org Message-ID: <1d2bd16a-c2f9-4baa-b2d3-4b0241de8992@isocpp.org> References: Reply-To: std-proposals@isocpp.org NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_1242_234203149.1547860338011" X-Trace: blaine.gmane.org 1547860219 21506 195.159.176.226 (19 Jan 2019 01:10:19 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 19 Jan 2019 01:10:19 +0000 (UTC) To: ISO C++ Standard - Future Proposals Original-X-From: std-proposals+bncBC4Y55PCR4BBB47SRHRAKGQEUIIT7JA@isocpp.org Sat Jan 19 02:10:15 2019 Return-path: Envelope-to: gclcip-std-proposals@m.gmane.org Original-Received: from mail-yw1-f71.google.com ([209.85.161.71]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gkf9K-0005MW-Bu for gclcip-std-proposals@m.gmane.org; Sat, 19 Jan 2019 02:10:14 +0100 Original-Received: by mail-yw1-f71.google.com with SMTP id l7sf7908400ywh.16 for ; Fri, 18 Jan 2019 17:12:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isocpp-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:message-id:in-reply-to:references:subject:mime-version :x-original-sender:reply-to:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-subscribe:list-unsubscribe; bh=lF/TdA0KEYBL3c1xdWGwSgj1wBffmjGKabHmQmW0tbc=; b=UmUzYOyquYG8h7wUrxzckZKxCH+pV5zaqxIHNMm2H9lwFm2LwhFd5HUgPTAbIZCzkK SiXvhHQ/yFYhkliX1Sve5U4eVzii5YondeodpsoTGarghUWZZPzm8o6EYdDOxIswdjwr 1e31bOeBIzGBolaUw9DBWmdJO7//rJ8T1mmPw25UZhHW1xuAgqQnyRMfoj00mhzL4LP/ 9g34Wy1ZacSaWLK6YTAHccxocVjxmoseeDEGPYl8Eq0xjsxjjZY8lVea+1jGfoO++AtS noktAry0k7ck0yGmr1tgyh1qqz9AAyiMng2GQlNyfBqJaN2t1zeOAtw/y7zrDn5WVbvl 5DQg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:message-id:in-reply-to:references:subject:mime-version :x-original-sender:reply-to:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-subscribe:list-unsubscribe; bh=lF/TdA0KEYBL3c1xdWGwSgj1wBffmjGKabHmQmW0tbc=; b=RU3pdcHq2ZSA4TuLzG76kLi6hH2afPaBNZbwyMO6mfcs1tuSfpBvKquLSs9dWoP21/ lUgkjzeSgdmV8/toZWYGfsRYnUC0bvS0k6dl8fJza124mtOa0Jp4+uXzBcj9fAnnWMB5 NhNpIAJKc0wQNioq255M/o3TeuMKyd9CUr7rqLfwP9kOOVvX+VCQAeGMBqQKf+xvU3jH rCK/pBIOrVYRyW7KddHxaaN1Joi5HLQGDFE0szpWEBqGJ4r1vtrVG8HWKVW2Y4iKjYSA GUlMDz07gTDzdGwudqhXtEK0SoBU2USbmX60V+Ge4ALy/QZf42G2nYEQjo+Gls0Lu1d8 wTTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:message-id:in-reply-to:references :subject:mime-version:x-original-sender:reply-to:precedence :mailing-list:list-id:x-spam-checked-in-group:list-post:list-help :list-archive:list-subscribe:list-unsubscribe; bh=lF/TdA0KEYBL3c1xdWGwSgj1wBffmjGKabHmQmW0tbc=; b=baMKpB4NgzpVV/GorqVmWqPLVpWRIOuRv6E/WJPoUkr8bnfmk8Kuzq7y7vvl8lty63 9m9UCHCQ6lO+y84/X9JLuvXd1ecpK0oL33C4vZUj0zIU77TWS2xcAgHppIVGabGPkfTK co8ISu13tXUwdRG8DZTTlvHjuxJI3jWS9hn+RXWxbGjIvnL7F2EWbEqHH8ecV6uUwldT j9h9VuqLGP46Aic1PX3EbfXpMJKpVXaICwZt8onJJMd6XG5ToCV0vlf3iR3Z9KnsA3Fx QAWpA0JRRGLpkfCdLANNblY1fvzyf+hgejXJHJHj8K1j+6MC2GcbMVyf7d6fqmU/M5FO 0y8g== X-Gm-Message-State: AJcUukdkdVKYkEErocgVv0R4ZmPkPi5t9MmB0qZwEGc9smcW1aa0q8BT H6cI/cBMClWP3/ZuGkQJeMcNMw== X-Google-Smtp-Source: ALg8bN5X4PKtW3P8yga1oZvm2DVJMfuN2r54v6yaLNdn3gwWYqVyWKjsymvLd8Z3k76LMzxBpdl+gw== X-Received: by 2002:a25:e692:: with SMTP id d140mr4611653ybh.102.1547860340170; Fri, 18 Jan 2019 17:12:20 -0800 (PST) X-BeenThere: std-proposals@isocpp.org Original-Received: by 2002:a25:c042:: with SMTP id c63ls299254ybf.4.gmail; Fri, 18 Jan 2019 17:12:19 -0800 (PST) X-Received: by 2002:a25:380a:: with SMTP id f10mr88981yba.4.1547860338810; Fri, 18 Jan 2019 17:12:18 -0800 (PST) In-Reply-To: X-Original-Sender: Joel.G.Hemphill@gmail.com Precedence: list Mailing-list: list std-proposals@isocpp.org; contact std-proposals+owners@isocpp.org List-ID: X-Spam-Checked-In-Group: std-proposals@isocpp.org X-Google-Group-Id: 399137483710 List-Post: , List-Help: , List-Archive: List-Subscribe: , List-Unsubscribe: , Xref: news.gmane.org gmane.comp.lang.c++.isocpp.proposals:41359 Archived-At: ------=_Part_1242_234203149.1547860338011 Content-Type: multipart/alternative; boundary="----=_Part_1243_674698673.1547860338012" ------=_Part_1243_674698673.1547860338012 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable The case you pointed out of using the double-type of the return for=20 'pow(int, int)' would be a major concern. Perhaps a workaround would be the= =20 requirement of an explicit template? i.e. "double x =3D std::pow(10, 20);"= =20 would use overload 7, whereas "std::pow(10, 20)" would return=20 int? Alternatively, as Jake Arkinstall suggested, a new syntax for 'pow' as= =20 'power' might be in order, in which case, such contingencies wouldn't be as= =20 large a concern. I would also argue that the above code segment is poor=20 practice, as it is relying upon the pow function to convert the two=20 integers to doubles. More explicit typing would be preferred in most coding= =20 environments. Another solution, as is implemented with powl and powf, is to= =20 use a single-letter abbreviation to explicitly define the types used. i.e.= =20 powi. As for the graph; I mentioned in a reply to my original posting that I have= =20 since found an issue with the testing method I used for graphing the=20 execution times for pow and powi. I have since done other tests as shown=20 below [avg. of 20000 runs for powi, and 500 for pow (since the execution is= =20 longer, I decided to neglect the sample-count somewhat)]: [image: Untitled2.png] As you can see, powi does experience a gradual increase in execution length= =20 with the increasing exponents (since it needs to do at worst [if an=20 implementation were to use a simple linear approach, O(n) multiplications])= .. While the normal method does appear to be O(1), the execution lengths for= =20 floating-point operations are significant enough to negate those benefits= =20 with the ranges of 32 and 64-bit integer types. Also below, I have included the collected data, which was gathered using=20 "std::chrono::high_resolution_clock::now()" surrounding for-loops executing= =20 the functions within the base-value ranges for each exponent. Power STL Custom=20 > -19 152.4881 5.922077=20 > -18 156.3487 5.497594=20 > -17 141.8481 5.546061=20 > -16 145.8614 5.683295=20 > -15 145.8326 6.041384=20 > -14 149.2215 5.701708=20 > -13 145.5323 5.59141=20 > -12 149.3403 5.602157=20 > -11 146.3427 5.974374=20 > -10 152.5245 6.180647=20 > -9 146.4068 5.772337=20 > -8 143.3287 5.782097=20 > -7 152.735 5.678149=20 > -6 146.4267 5.743723=20 > -5 145.5064 5.750488=20 > -4 150.1708 5.772985=20 > -3 142.524 6.569755=20 > -2 140.4772 5.600695=20 > -1 143.6424 6.281142=20 > 0 107.1507 3.854533=20 > 1 140.658 6.386001=20 > 2 147.3853 6.076429=20 > 3 146.9267 8.102243=20 > 4 141.4524 6.984409=20 > 5 136.9645 10.44516=20 > 6 139.3443 8.883989=20 > 7 142.125 7.988396=20 > 8 141.0955 7.625333=20 > 9 140.6131 12.97227=20 > 10 138.1168 11.3224=20 > 11 143.1305 10.61643=20 > 12 139.4992 10.38626=20 > 13 141.5211 11.2896=20 > 14 141.0866 10.22915=20 > 15 140.7223 9.948738=20 > 16 139.7512 9.173968=20 > 17 138.1409 15.03408=20 > 18 141.6834 13.97868=20 > 19 153.7554 14.77899=20 On Friday, January 18, 2019 at 2:38:49 PM UTC-8, Arthur O'Dwyer wrote: > > On Thursday, January 17, 2019 at 11:02:13 AM UTC-5, joel.g....@gmail.com= =20 > wrote: >> >> As of C++11, per overload 7 of the 'pow' function ( >> https://en.cppreference.com/w/cpp/numeric/math/pow), integral types can= =20 >> be used as parameters, but they will be cast to double before calculatio= n.=20 >> I believe that an integral-specific overload should be made, which makes= =20 >> use of faster integer-operations available to CPUs. >> >> As a disclaimer, I am still a rather inexperienced programmer (College= =20 >> Sophomore), and I my have some misunderstands as to what would fall unde= r=20 >> the ISO standard and what would fall under compiler-implementations. As = I=20 >> understand it, the line about "If any argument has integral type, it is= =20 >> cast to double." is a component of the ISO standard. >> >> As stated on isocpp.org's FAQ section, this idea is based on existing=20 >> practices. As discussed here ( >> https://stackoverflow.com/questions/2398442/why-isnt-int-powint-base-int= -exponent-in-the-standard-c-libraries),=20 >> multiple implementations of this function have been made on a per-user= =20 >> basis, due to its general simplicity and lack of inclusion in previous= =20 >> standards. In the below testing, I created a function using x86 asm. It = is=20 >> worth noting that even with more standard C++ approaches, as testified i= n=20 >> the stackoverflow discussion above, a significant performance improvemen= t=20 >> can be made. >> >> For the sake of brevity, the data-table and source have been withheld,= =20 >> but if it is relevant to the conversation, I would be more than willing = to=20 >> provide them. >> >> Do any of you agree that this is a useful addition to the C++ standards? >> >> *Performance Testing:* >> Tests were run on an Intel Core i3 6006U @ 2.0GHz >> >> Custom Function (VS 2017) :=20 >> __declspec(naked) __int32 __fastcall powi(__int32 base, __int32 exp) {= =20 >> __asm { ... } } >> >> Test Range: >> >> base: [-46340, 46340] (int-truncation of the sqrt of MAX_INT) >> >> exp: [-19, 19] (highest power of 3, that does not overflow the int type) >> >> >> Results: (Approx. x20 avg. improvement) >> >> [image: Untitled.png] >> >> >> > I am skeptical of this graph's reality. It looks too neat to me, with an= =20 > identical dip at n^0 and a constant gap between the two lines even on a l= og=20 > scale. (I could believe a constant gap on a linear scale, if what we were= =20 > witnessing is the constant cost of two int-to-double conversions. But not= a=20 > constant gap on a log scale!) > Do you have a link to your exact test cases and the software that produce= d=20 > this graph, such that other people could reproduce your experiment from= =20 > scratch on their own hardware? > > Another thing that makes me skeptical of this graph is the weirdly=20 > constant and very low latencies for n^-19 through n^-1 (which are display= ed=20 > as -n^19 to -n^1 on the X-axis, but I assume that's a typo). Solving n^-= 19=20 > (a.k.a. 1/n^19) in integers is just "return (n =3D=3D 1) ? 1 : (n =3D=3D = -1) ? -1 :=20 > 0;", right? Whereas solving n^-19 in floating-point ought to require the= =20 > exact same "log, multiply, exp" that would be required for n^19. > > As for adding a new overload of std::pow =E2=80=94 no, you'll never get a= way with=20 > that. > > double x =3D std::pow(10, 20); > > currently has well-defined behavior =E2=80=94 it yields 1e+20. You're pr= oposing=20 > to make it have undefined behavior instead. I don't think that's going t= o=20 > fly. > > You'd have better luck asking GCC and Clang to optimize `__builtin_pow`= =20 > for small integers. Which I think they might already do? > https://godbolt.org/z/7lPPLf > Looks like maybe they special-case std::pow(1, n) but nothing else. > > HTH, > Arthur > --=20 You received this message because you are subscribed to the Google Groups "= ISO C++ Standard - Future Proposals" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposals+unsubscribe@isocpp.org. To post to this group, send email to std-proposals@isocpp.org. To view this discussion on the web visit https://groups.google.com/a/isocpp= ..org/d/msgid/std-proposals/1d2bd16a-c2f9-4baa-b2d3-4b0241de8992%40isocpp.or= g. ------=_Part_1243_674698673.1547860338012 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
The case you pointed out of using the double-type of the r= eturn for 'pow(int, int)' would be a major concern. Perhaps a worka= round would be the requirement of an explicit template? i.e. "double x= =3D std::pow(10, 20);" would use overload 7, whereas "std::pow&l= t;int, int>(10, 20)" would return int? Alternatively, as Jake Arkin= stall suggested, a new syntax for 'pow' as 'power' might be= in order, in which case, such contingencies wouldn't be as large a con= cern. I would also argue that the above code segment is poor practice, as i= t is relying upon the pow function to convert the two integers to doubles. = More explicit typing would be preferred in most coding environments. Anothe= r solution, as is implemented with powl and powf, is to use a single-letter= abbreviation to explicitly define the types used. i.e. powi.

As for the graph; I mentioned in a reply to my original posting that = I have since found an issue with the testing method I used for graphing the= execution times for pow and powi. I have since done other tests as shown b= elow [avg. of 20000 runs for powi, and 500 for pow (since the execution is = longer, I decided to neglect the sample-count somewhat)]:

3D"Untitled2.png"

=


As you can see= , powi does experience a gradual increase in execution length with the incr= easing exponents (since it needs to do at worst [if an implementation were = to use a simple linear approach, O(n) multiplications]).


While the normal method does = appear to be O(1), the execution lengths for floating-point operations are = significant enough to negate those benefits with the ranges of 32 and 64-bi= t integer types.


Also below, I have included the collected data, which was gathered us= ing "std::chrono::high_resolution_clock::now()" surrounding for-l= oops executing the functions within the base-value ranges for each exponent= ..


Power STL Custom
-19 152.4881 5.922077
-18 156.3487 5.497594
-17 141.8481 5.546061
-16 145.8614 5.683295
-15 145.8326 6.041384
-14 149.2215 5.701708
-13 145.5323 5.59141
-12 149.3403 5.602157
-11 146.3427 5.974374
-10 152.5245 6.180647
-9 146.4068 5.772337
-8 143.3287 5.782097
-7 152.735 5.678149
-6 146.4267 5.743723
-5 145.5064 5.750488
-4 150.1708 5.772985
-3 142.524 6.569755
-2 140.4772 5.600695
-1 143.6424 6.281142
0 107.1507 3.854533
1 140.658 6.386001
2 147.3853 6.076429
3 146.9267 8.102243
4 141.4524 6.984409
5 136.9645 10.44516
6 139.3443 8.883989
7 142.125 7.988396
8 141.0955 7.625333
9 140.6131 12.97227
10 138.1168 11.3224
11 143.1305 10.61643
12 139.4992 10.38626
13 141.5211 11.2896
14 141.0866 10.22915
15 140.7223 9.948738
16 139.7512 9.173968
17 138.1409 15.03408
18 141.6834 13.97868
19 153.7554 14.77899


On Friday, January 18, 2019= at 2:38:49 PM UTC-8, Arthur O'Dwyer wrote:
On Thursday, January 17, 2019 at 11:02:13= AM UTC-5, joel.g....@gmail.com wrote:

As stated on isocpp.org's = FAQ section, this idea is based on existing practices. As discussed here (<= /span>https://stackov= erflow.com/questions/2398442/why-isnt-int-powint-base-int-exponen= t-in-the-standard-c-libraries), m= ultiple implementations=C2=A0of this function have been made on a per-user = basis, due to its general simplicity and lack of inclusion in previous stan= dards. In the below testing, I created a function using x86 asm. It is wort= h noting that even with more standard C++ approaches, as testified in the s= tackoverflow discussion above, a significant performance improvement can be= made.

For the sake of brevity, the data-table and s= ource have been withheld, but if it is relevant=C2=A0to the conversation, I= would be more than willing to provide them.

Do any= of you agree that this is a useful addition to the C++ standards?

Performance Testing:
Tests were run on an Intel Core i3 6006U @ 2.0GHz

Custom Function (VS 2017) :
__declspec(naked) __int32 __= fastcall powi(__int32 base, __int32 exp) { __asm { ... } }

Test Range:
base: [-= 46340, 46340]=C2=A0 (int-truncation of the sqrt of MAX_INT)
exp: [-1= 9, 19] (highest power of 3, that does not overflow the int type)

Results: (Approx. x20 avg. improvement)

3D"Untitled.png"



I am = skeptical of this graph's reality. It looks too neat to me, with an ide= ntical dip at n^0 and a constant gap between the two lines even on a log sc= ale. (I could believe a constant gap on a linear scale, if what we were wit= nessing is the constant cost of two int-to-double conversions. But not a co= nstant gap on a log scale!)
Do you have a link to your exact test= cases and the software that produced this graph, such that other people co= uld reproduce your experiment from scratch on their own hardware?

Another thing that makes me skeptical of this graph is the = weirdly constant and very low latencies for n^-19 through n^-1 (which are d= isplayed as -n^19 to -n^1 on the X-axis, but I assume that's a typo). = =C2=A0Solving n^-19 (a.k.a. 1/n^19) in integers is just "return (n =3D= =3D 1) ? 1 : (n =3D=3D -1) ? -1 : 0;", right? Whereas solving n^-19 in= floating-point ought to require the exact same "log, multiply, exp&qu= ot; that would be required for n^19.

As for adding= a new overload of std::pow =E2=80=94 no, you'll never get away with th= at.

=C2=A0 =C2=A0 double x =3D std::pow(10, 20);

currently has well-defined behavior =E2=80=94 i= t yields 1e+20. =C2=A0You're proposing to make it have undefined behavi= or instead. =C2=A0I don't think that's going to fly.

=
You'd have better luck asking GCC and Clang to optimize `__b= uiltin_pow` for small integers. Which I think they might already do?
<= div>https://godbolt.org/z/7lPPLf
Looks like maybe the= y special-case std::pow(1, n) but nothing else.

HT= H,
Arthur

--
You received this message because you are subscribed to the Google Groups &= quot;ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to std-proposa= ls+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.= com/a/isocpp.org/d/msgid/std-proposals/1d2bd16a-c2f9-4baa-b2d3-4b0241de8992= %40isocpp.org.
------=_Part_1243_674698673.1547860338012-- ------=_Part_1242_234203149.1547860338011 Content-Type: image/png; name=Untitled2.png Content-Transfer-Encoding: base64 Content-Disposition: inline; filename=Untitled2.png X-Attachment-Id: 48451991-147f-4022-b5a2-fd68fb1a3cbe Content-ID: <48451991-147f-4022-b5a2-fd68fb1a3cbe> iVBORw0KGgoAAAANSUhEUgAAAfUAAAEfCAIAAADX9ciiAAAYyklEQVR42u3dXY8kV3nAcX8UfwWu fJcrbgJSRHJBGAVF5gKhCHEDGgSy0MoRG0HA0eBIK4LIIuMQRzAQIDFaI8cKlm2W8Rteg9fGlrN4 gWSJvYbFXu+yMZtnpnqqq+utq6uqX6r691fJ7u2pl1Onu/7nqXNO13PLhQsXXgAAjItw+y3xv5sA gHERbud3AOB3AAC/AwD4HQDA7wAAfgcAfgcA8DsAgN8BAPwOAOB3AAC/AwC/AwD4HQDA78CquLi/ u7Ozu39RTYDf+X143jm7tzPLZtmsULydnb2z4/L70TFytV9y2jOnf/j31X5QM6XU5PE7Nsc7NTrI /SkRyyoV2sDvqyvOyr2Z+yDjn/nDl9bAasuZuD1ThsM3Oh1+9e0T+J3fV2/U7fZ7g9Nbu9+Pvms9 fwT8zu9o7vfjgGp6X3/8x9n76qIqClda2a5Lti60MHkP5bWQ7XIornZ8lEbRa/lfZs6kukKqylNa UYWzrKiNRkesOon69Rb3e8kWM29lz2G+t+uboPpPoexY1V/I6rqd7DSzQvLXtK61Ffw+dr9nLor8 DXXr+D3nn+zfckWIP+3u7uZ6etJ/Vm9YJfYe/D6nQqb/jH9NXudrY7asNbUx94h1fqxdr0X8Xlo1 Wb2WnHnLO4zaT6H6WMXy19btscczn1Lh35vTp8jvWIbf89H07NXSyO+Jlcqvz/zqM3896jje35t1 YdV+sv+ePWK/fq+qkOoehzq/19fGvI9gruKraqFN/0yuqNm1F+0ZmbN+7adQvW1pPVfXbf4o+dse guf38fs931tQFy6VdlPkHFPWyV8lvLN7k76JkkC4ePHmW4k5oWhpARv4vaJCag5Z4/dFamNOK1L1 gTbvoZpn6XzbW7ixaaz4Ln6vPlZZL05N3c7r+1vGEAH4fSR+r/hT6VWTv36T1xOxH75xtEGZUso9 3cTvbeP3Or+X22Ce32tqo5vfs61ZScfFwuOrudMt2WXD/vcu/TPVxyrze8058ju/8/tS/F4TVU03 Tqf1TV7MKqX26luX3zcsfq8+35bzZ9KPoHou46TdrS/iop9PRclyxxK/8zvW7Pd5vaLpP89OZ22f PeqCP7vXuCe6td9Lr/FGfp/TaHTpf+/q95LaaDs/cuL1+qnqTXrjayP4+k+h8lgd+9/5nd/5vWYy Y8MrPDcJpGROyOHme3t7s329u/mrrWT6SuWwZGO5FNy709Tv+fJMi1M7bl1fGw2a2NK+9bymyiaw tpr/fvRR7O3t5pva3I1VxdkVu41yUXpJjeU+hepjVc07rapbfud3fq8LHqd94PPmv5df3JUzPPJd xpU3/dn9zLk1Xyh4TM+q2OffrEJ2aiqqfJZ/aW008XvJicwOIFcMRZb6fe5jJcqHN8sehzDH7/Xl nPcplJew5AtZXbf8zu/AhjfIDAR+B8ZGriMG4HdgNMG7X9CD3wEA/D7wgG69N+yZJ3ocv+wYYcY5 6YEA+H0M1Pyuv6ELcw+kam3TdCZC8x2lv0eaffBUJ8PzO8DvI4m/M9OGj3+R31yPWRdOGooF3Zru YXrcBQSbPC2m2E403L4quxC/A/w+guA9+2zG1K97TR1dotIFZ1ike8gUpbmgp2u28nvpquJ3gN9H Er/PPBh30fi5uwu7+b2igVmgTMW1+R3Dv5wb37v2N3C1cFnXdae8Tf3vJc9zzQb1Kzx+9gfmJuEB S/R7DwNX3UbOug/dtR843PL5Mx0D2B5j+oGWfwAhzNKCOHc/q/3kWg4ddRy4mlm71Z1/x6G7TgOH /M7vSw8ibpp9hJ4vktUNXN1cUs/q4u1Ti4HDbRpf7RgFdJ1/st4CdNzc7COzj9Z78bYfOuo8cLUE v7cObBb86m9R/N5xAkn3+SfrLUC3zc0+Mvtoq28g1jly1mHgcNt+39RlAkn3+SfrLUCXzc0+6n32 kRkgfUdrK+1bG0bPqucTYOlBxBhiqHUaaqtngPA7vwMbfuNoBsjMjhZpnoyctd98W/3eMfTsHrmu twCD3nyA5TcDpOfBGyNn4neG4veN2dwMkJvtbx+MnLXdnN8ZSu2tvPwrZ7gzQNAF/e8AwO8jCdw7 /S6l4+ZrL8CgN1d7ak/t8XvdLWLb+u24+doLMOjN1Z7aU3vid82/4MnmTl/tbWv/zGQyadvK7rj5 2gsw6M3VntpTe/wOAOB3AOD3kZCZCNzmEa8dN197AQa9udpTe2rP8wlq8Mscv2tSezbflvJv4fx3 XxO1p/ZsvhXl39L5M23vkjpuvvYCDHpztaf21J7+GQAAvwMAvwMA+B0AMFK/H03WzIz3TkcHMk+f rhonqPkTAGCNfg+Z7+6fzWQPmEnaciTt4/9PVr6Ys3vFn/rj9ttv77LCejdXPMVTPMVbY/w+mx1m xu+Hr8r/WLNd2nLs7u/vHcf2x3F+JiVM08lDt9xyy80OK6x3c8VTPMVTvI3xe6bLJZ/EfVG/70xj +8nLNN5fJCOlb4niKZ7i8Xs/fk/EfPjfoze7xO+F5JGp15PwvVGXzi0AMEA2zu9FZS/H79lbBf0z iqd4iid+X4nfU99Oxkxbjq828fvNJl01viWKp3iKx+8tmHlUQtpHnpsfma6UmVkzeZn/UzO/zzw/ c15HvG+J4ime4vH7OLn11lu7rLDezRVP8RRP8fgdALYXfgcAfgcA8DsAgN8BAPwOAOB3AOB3fgcA fgcA8DsAYGR+z+fnK3/ejPx8ADAovxfy8808QTL3xnry8wEAv3eI4Kuf4772/HwAwO99+L3wwOC1 5+cDAH7vye+z+T3Wnp8PAPi9Z79vSH6+ixcvvgAAgyLEtXl+nw6ZHr9ad34+ABC/t6CYn69kyuN6 8/PV8Mqlq6e+/fLHTj37b4/+t+8HAH4fPK9duf7NH/zyA5958p0ffSRdHnn2VTUDgN8HyfUbbz9w cOmTX/ppVuvp8t4TB+F93xIA/D4kDs5f/tx9L77r44/lnB7vnLz3hfd/+onknx879axvCYC188iz ry6qoy31ewTmxWg96i5i+d9dvRErPHfhSqr+b/7gl9tTM/c9eDHq4dS3X17SjUvyHV3e/oGxyj3R Eb/PJ6v1D3zmyZDa/7x2LbfOPWd+nkb0L/3ijdHXyY9f+k12+OE9d5yNGrh+4+0eD5FW6ZL2D4y1 syENN/m9kd8jhN/bf6le3BFppm3AiE0UofTJe1+oGoHoax5R1PZS9x/EvVdy/6HlwGgIR0UwlFwv 7//0E/zeGxHUpzUbelrBEdPZmZ+778UIqFsU+Mv3X4jNw9fhzeJNSZEQYnqOSVgdBUiHH9LmrctU olDtidPns/1g/e4/OfEodvZEPnTX01GZvsMYjYLiqmlyRfP7Ajz01K9XMF2ydHZm8omGr5t4KspZ Ogso9hktU5S8GM/mOmRiiVYh6RaPlaM8WV0mXn7uwpUWMXV6G5QcInbe4/4juom2sPTmI25pHzi4 5DuM4co9jYTiYmnRS8zv80n10ft0yfrZmdklotGIx4tHTwL20uHi4vKRLzwT0Xo4tNghE6Iv3i6E mmPnuSlGEYk3DyJizSh5um2E2D3u/+D85WzLkTYScZRco5WMmQ+CpH8puQO758zPY4nTjI/Gvci2 EV/a9NqJa6RF6MPvTS3c+3TJ+tmZEYxH0J0Lb1P9xV+jSFUBe7wZfwpNhM2bSD+OEivX31sUA+Qm /T+hpGw/TNVRSvdfL7hoFIv3OrF+eg3Ei+yh43W7y2OVxAk2+chiHbOPtsE56ZchnBCXQLv98Hsj Gk6XjKsu3Bq2yvUvN1myszOzvS5VI5/FgcqIhXPCjW/JI8++Gk1F0Ya5Dpkm3SAL9f9EjaXtU5N+ kqr9lzZIuRYxClBsaaImc1VX34yt92KOz27RL0x8zbZhWtd2yj17LXTpY+T3ptRMl4zWNa7PbEdE 86VqdmZOVREsVwV3ScDepKskdhLKS/xY2iEzl9ik5jTT/p/sdK6Foo/6/RebtPhQ6vte4trI3iRF I7ppkW9uFCRKG3UYb0alJbcv8ZFFsWs+/dbBHTaT7O1sxx/frCz/av69IeZfzU6XDMVH1cfVVexj aeimubMzSx0dF3xyc1AasK/sbqZh/0+7QaFXLl2tF1zUQIi74QzI2Fu2zYgitRNienOW3Gn10sGa 65iKc5zbzx6Vn52MlA7PjH4kOZ382m5q2VDIjh7Fl7/j3laTfzV59zC3XuZpkcPLv5qdq1QzEBra 3ZKQqr7/p8V0ruWVMzfoGg1zXDyhiblDr1U3Zx1PLZqK7Kh4fK8WEnQcPao9F1vEDiPmWNJgcuw2 bd5WPPxbnPy60NSyIXYSJP1v3Xe4kue/3zx+iO/x832Hm381O10ye13FhxF/2uZRr1z/Twh002oj TFTaPIcpIiLO6j6sMffmrPXPsqJaciMNcfR2dRVbRbFzs6fiHEP9fYkvbhcW6nuMW5Cwf1KfsUSj FbXapB0tpWby69ypZdnWPQoQsX/2uVJVS6yQNGBJ+eOijm2XPc6RxEnZM40K7GXPq/F7/GOSIrWr 3zcg/2oSCcaVH1dpWMDEtQERFihOqWy+hEpSNbe7AuMyzrYxoebut3phhxBc0Vyh2ua9WMWmOk6w Xd/j3HkEuda0qjEunfwaZzR3allW6FUzC9ot/bZeaT0Xx1T6+vX1Kvw+fdlH/N5L/lX5+baZ/3j0 p/d895mTX3nyrz7/o7mX9J/d8din/uHJr93/zOM/Pp9sm7z/x7uP/uS55xc9dOxtOnnpK08+fe58 j+f19QfOffCz+TP6k08+duc/Pvnwj56r2fCHT5z/1wef/buvPfWxux//8xM/rKqK2//m7N/e+1Ss +e//+ZNk/Vg+ceqJqMbicRsucbg4aOwn9pnURnw0f/HX+TLEUc48/JNsmeMTiTd7b346LlEVsSRn FEucS5xXemrp1y+qMSqzdA9RkzXfig3MzzeTWCntWVlv/lUg2wkQgVjyWIh0EkvNzVkaKS86yhfr p2H78ubjx55Lf1qRDeejJBE5JqdcH6THyUa8HHFxwxA1GRtPwueIcJOnZcTSLo6umvyaHRWomVqW PffYT6w5t9rjWEn5kwg9HW/oqwGo+iliMm7Xe0fQqvrfc36WfxWDJe0nzf0ct/mGcSUvu5CJ+NpZ 9T13nD1x+nxsvoy+x9hntBaJ92ualiaTX3NeTqeWLST0RQvfY+uV9Ck1fE7U5vi9JP9qwc+bm38V mNuHnk6TXaiXPO01XuWYTVU4nxN6SCpsFQH+in/oW9T9QpNfN40mDUAyHaP0kVCD8DswZiKoTHXZ PPI6OH+5RauwpHD+Q3c9nUxwDBN51MGI4XdgYdIO2eazJNPOme4/WgH4HVgWcfe90CzJuBNPQ34T asHvwOby0i/eSCd4NOlFTX8W95EvPKP2wO/ARrPQLMn0SZYb+wxL8DuACc1nSWY7ZzbkgTzgdwCV NJ8l+cDBJZ0z4HdgMDSfJZk+sqb1I8kAfgdWSpNZktlmwExz8DswDJrMkkw7Z/rK3AvwO7B0msyS bPFLKGDD/Z7Pz1d8pMwQ8/MBOepnSb525XraACwprRKwYr8X8vNd3N/LPTZymPn5gBz1syQjZk8z NqgrjCZ+r3g+8LGyh5ufD8hSP0sy7ZwZffJr8Pv0zUHn5wNSamZJxj91zmB7/D6T0mm9+fmAvqga QU1n15y81/wFjN3v2d70tefnk38VffH3//L0ZAbk3Y9n309Tkn7t/mfUEnphA/Ov3pxJzjR9Q34+ jIHSWZLZzpmBpiKC+L2UQn6+mTcmzpafD6OhOEvynjM/1zmD8fbPAFtDcZZkmgnv4Pxl9QN+B4ZK bpbkK5eupnmrdc6A34EBk5slmXbORFyvcsDvwLDJzpJMu+N1zoDfgcGTznZP5f7eEweqBfwODJ50 lmS67O2/pFrA78AYSCP35nm3AX4HBkA6S1LnDPgdGBXpLMlYvnz/BRUCfgdGQnaW5HMXrqgQ8Dsw HiJsr0/HCvA7AGBD/Z7Pv1pMqSr/KgAMzu8l+Vfzj/yVfxUAhhm/zzzAt5iyQ/5VAOB3+VcBYDvi 917yr8rPB0B+vgH5Pd1I/wwArCb/am/jq/KvAsA6/V7Iv1qWUlX+VQAYZvwOAOB3AAC/AwD4HQD4 XUUAAL8DAPgdAMDvAAB+BwDwOwDwOwCA3xdj+kiazANk5OcDgIH7ffowsOOHQ8rPBwCj8/vhK/n5 AGAUfs90uSTelZ8PAEbi90TMh/89Tu+x3vx8AMDv/QTvm5afT/5VAPKv9tQ5c+zbyZip/HwAMIr+ mUzuvIyV5ecDgKH7HQDA7wAAfgcA8DsA8DsAgN8BAPwOAOB3AAC/AwD4HQD4HQDA7wCArfV72fPE 5F8FgIH7PfuE4Nwb8q8CwHD9XsjTIf8qAIzC7xnv7hw/q13+VQAYhd9n8zetPf+q/HwA5Ofr2e8b kn8VAMTvfTAdMj1+Jf8qAIzB72VTHuVfBYAx+B0AwO8AAH4HAPA7APA7AIDfAQD8DgDgdwAAvwMA +B0A+B0AwO9NOHpATPbZkPLzAcAY/J7kXso8TUx+PgAYgd+ThzweP/9Rfj4AGIffw8STFHpd/S4/ HwBsjt+nmu4jfu8lPx8A8Hsvet+Z4ahnZb35+eRfBSD/aq+c7Ti+Kj8fAGy43+XnA4BR+R0AwO8A AH4HAPA7APA7AIDfAQD8DgDgdwAAvwMA+B0A+B0AwO8AgC3ze/GRYfKvAsDw/X5xfy/3WGD5VwFg HPF7Jo4/VLb8qwAwLr8fC1v+VQAYk9+nul57/lX5+QDIz9d318zN5fk93Uj/DACsyO+Z5HvTN+Rf BYCh+z0z7pl2oMi/CgAjiN8BYKv5w43rVx/64pXTH4zlja9/Il7H8vufPfr7lx//v//9r+LK8X4s 186dSdaMTZJt+R0ANoi3Dr7x+mff+dqn3lG//PZLf3n5ztvq1+F3AJsXwL515erDpyMC/d0/f/Qw dH358Xhn9Gd9/bmHfvOFP51r9uYLvwPYIN5+/Zdvfu/zl0/+UdFWr9/17rHq/sYr56Ixy55s1MBb j/3TYcfLzx7N9rpE2F6slqQ3Jupt0pNz1F1z41fP8zuAEsle/f7dh9b46ofDFxFXFrt9+3fcr55/ 41sneoxeOy55bz58OvFmVE6/VR2N1ozZ77wtKn/1DRi/AyPn2rkz4fQq5R3K7rsnk7jyDzeu93XQ CFFz0WtyrCjMtae+E3ot/nUTlgilkwagfqlqHt7+3avxp9w+o4WL99fy0fM7MOaAvcmwXkmEexTj x/LWwTcmCmtsqNB3scf5ja9/4sYr50oD/E3W/aJLbnQ0qnHRHhV+B7aLZLZcRIvpJLmkzzqWkGPi 3+y9f1XAHm8ehs9HU+5iJ4uqv0mEm9tn+C5uDvrt/ejY5iXVFfcrSQUmxf7tqff1fB9w6n1xlLWf L79jzAHs5GI++EZyMYcWUxN1mYMcL0KsC3WnRvwb+4lb9SaCzgp90dkXRWvHOxHIl0o2Djo50LdO FEf5OvV0x0Ef+uLIhkybNA9xAxQf9IYUeEv9Hp/BCG4GLUvqgZ07Bzm7clzqNQOVhzMlvn/3QuFh j9PpkoB9oUvjxq+eTxQW0j9UWDIke/qDzUt1KLho/Prryge/LwaLWXqfmPHm9z4fNo9vV+g+pB9u bd5ONGlIDkdBj3vD6/1bE7CD31fBevPz8dE2LKG8xH0hu+wk4snSdg5ycnveojcjovgoSdIdNDdA ToVeOiwJbLDf5efD8PnDW1euPfWdaB5Kf7mThtJvfOvEtXNn1jVDDvy+ar8vIz+f5HxYIxGVR5if jOtcvvO2K1/9cH2/PMDvjfPzeeI7AAw9fq94/vsCyfkAgN8H5Pd0i0b9M/KvApB/dWmCX1p+Pl01 AHBzjfMj+83Pty85HwBsiN8BAPwOAOB3AAC/AwC/AwD4HQDA7wAAfgcA8DsA8Du/AwC/AwD4HQDA 7wAAfgcA8DsA8LuKwKYxmwbgYvYJ/0fv7u5u0JP+q3MWAPwO5IU5TeAy9XhN3q71MU05dviyUaEK qScBfscW+r3Cixvj95KiZss2fZ0G+flcY5mbk+PNDlOV7e8dv3e8gjYB/I6RGL5otA30e1koXub3 mfJmN8q2D+nrw7NP0xBPXmbuEwB+x3I5OH/5/Z9+4p0ffaSvJfb2wMGlguWn/mzt99//7NHX73r3 a596Ry9L7OraU99ZzO8zDVbleUxbgwap5AF+x7LoV+7J8q6PP1YTILeWXY9yT5bLd95WIvPa/pmb 076Y3V1+B79jk7nvwYu9+31v/6Wbs+OUWau1lt3Vh0/36/c3v3sye48x7Tc/fDUt5pHOS/S/s1PW PzPtgeF38DtGS2YwMuO0WdmVrbEWpgOkhdLt7SdN1cwEz3TENNtzszMrfX4HvwMA+B0A+B0AwO8A AH4HAPA7AIDfAQD8DgD8DgDgdwAAvwMA+B0AwO8AAH4HAH4HAPA7AGCD/P7iiy++AAAYF+H2/wfV NRpJA0xVBAAAAABJRU5ErkJggg== ------=_Part_1242_234203149.1547860338011-- .