clang-cl, #include <intrin.h>, _mm_shuffle_epi8 compiler error

[Nigel Tao]

I'm trying to upgrade third_party/wuffs. The latest version has some
new SIMD code.

Linux and Mac compile but Windows doesn't.

The SIMD code is behind #ifdef guards but clearly the guard conditions
aren't 100% correct.

I don't have a Windows box, though, and I don't have much Windows dev
experience, so I would appreciate some guidance. I'm otherwise just
guessing and my best edit-compile cycle (I don't even need to run,
just compile) is high latency, as it involves firing up a trybot.

The clang-cl compiler error
(https://chromium-review.googlesource.com/c/chromium/src/+/2731168/2)
looks like

----
[527/27538] CC obj/third_party/wuffs/wuffs/wuffs-v0.3.obj
FAILED: obj/third_party/wuffs/wuffs/wuffs-v0.3.obj
C:\b\s\w\ir\cache\goma\client\gomacc.exe
..\..\third_party\llvm-build\Release+Asserts\bin\clang-cl.e...(too
long)
../../third_party/wuffs/src/release/c/wuffs-v0.3.c(15490,9): error:
implicit declaration of function '_mm_shuffle_epi8' is invalid in C99
[-Werror,-Wimplicit-function-declaration]
x = _mm_shuffle_epi8(x, shuffle);
^
----

The relevant parts of wuffs-v0.3.c (including line 15490) is:

----
#if defined(WUFFS_CONFIG__AVOID_CPU_ARCH)
// No-op.
#else
#if defined(_MSC_VER)
#if defined(_M_X64)
#include <intrin.h>
#define WUFFS_BASE__CPU_ARCH__X86_64
#endif // defined(__x86_64__)
#endif // defined(_MSC_VER)
#endif // defined(WUFFS_CONFIG__AVOID_CPU_ARCH)

// etc etc etc

#if defined(WUFFS_BASE__CPU_ARCH__X86_64)
#if defined(__GNUC__)
__attribute__((target("sse4.2")))
#endif
static uint64_t //
wuffs_base__pixel_swizzler__swap_rgbx_bgrx__sse42(etc etc) {
// etc
__m128i shuffle = _mm_set_epi8(+0x0F, +0x0C, +0x0D, +0x0E, //
+0x0B, +0x08, +0x09, +0x0A, //
+0x07, +0x04, +0x05, +0x06, //
+0x03, +0x00, +0x01, +0x02);

while (n >= 4) {
__m128i x;
x = _mm_lddqu_si128((const __m128i*)(const void*)s);
x = _mm_shuffle_epi8(x, shuffle); // This is line 15490.
_mm_storeu_si128((__m128i*)(void*)d, x);
// etc
}
// etc
}
#endif // defined(WUFFS_BASE__CPU_ARCH__X86_64)
----

AFAICT the first compiler error message is on the _mm_shuffle_epi8
line (an SSSE3 function). On earlier lines, there's an _mm_set_epi8
call (SSE2) and an _mm_lddqu_si128 call (SSE3). I can rationalize the
_mm_set_epi8 call compiling (X86_64 hardware implies SSE2) but I am a
little surprised that the _mm_lddqu_si128 call (SSE3) apparently
compiled successfully when _mm_shuffle_epi8 doesn't (SSSE3).

Does clang-cl support `__attribute__((target("sse4.2")))`, since clang
certainly does? I had a quick search and couldn't find an answer. In
any case, I'm not sure if that's a viable approach if cl (not
clang-cl) doesn' support it.

I didn't find anything in
https://docs.microsoft.com/en-us/cpp/preprocessor/predefined-macros?view=msvc-160
for SSE3 (or e.g. SSE4.2). I guess I could gate on #ifdef(__AVX__) but
I don't know if e.g. Chromium/Win is built with /arch:AVX or similar.
(Again, I'm not a Windows dev so I'm guessing).

Also, I don't really want a compile-time check for
particular-flavor-of-SIMD support (e.g. SSE3 support), I want a
runtime check. On Linux, I don't require -msse3, but the
SSE3-requiring intrinsics still compile because of the
attribute-target. There is a separate runtime cpuid check whether the
CPU is SSE3 capable. I'd like the same mechanism for Windows (whether
cl or clang-cl) but I don't know what the Windows/MSVC terms are that
I should be searching for.

FWIW, patch set 3 is all green, after I disable the SIMD code paths on
Windows (see the BUILD.gn diff that defines
WUFFS_CONFIG__AVOID_CPU_ARCH to avoid "CPU-architecture specific
code", i.e. SIMD code), but this is obviously not an ideal solution.
https://chromium-review.googlesource.com/c/chromium/src/+/2731168/3

FWIW, the same Wuffs code compiles fine for Google's internal build
system, even building with MSVC, but I have no idea what compiler
flags that corresponds to, other than e.g. the _mm_shuffle_epi8 call
is actually still compiled (not skipped during preprocessing).

[Takuto Ikuta]

Maybe you need to extract code using SSSE3 to separate file and tweak -m flag for the file.

https://source.chromium.org/chromium/chromium/src/+/master:build/config/win/BUILD.gn;l=112;drc=cbbfb16cbc550d4bf2555c7f061324d854126116

There is

https://docs.microsoft.com/en-us/cpp/intrinsics/cpuid-cpuidex?view=msvc-160

for runtime check.

I found

https://clang.llvm.org/docs/AttributeReference.html#target

but runtime resolution seems only for ELF.

I also recommend to use http://shortn/_0Mc4SHFwzI

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
    http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAEdON6a1EWi%3DHk_DbLXbe8O4_4_cRW7ixjY-%3DOigZ6n77PHurA%40mail.gmail.com.

[Nico Weber]

https://chromium.googlesource.com/chromium/src/+/master/docs/win_cross.md should help you with most questions.

cl.exe doesn't have a dedicated flag for SSSE3 or SSE4.2 (https://docs.microsoft.com/en-us/cpp/build/reference/arch-x64?view=msvc-160). clang-cl does, it exposes the -mssse3 / -msse4.2 flags of the gcc-compat driver for this purpose. There are a few examples for how to use them in the tree: https://source.chromium.org/chromium/chromium/src/+/master:skia/BUILD.gn;l=726?q=msse4%20file:%5C.gn&ss=chromium

I'm not sure, but I think the `__attribute__((target("sse4.2")))` relies on ifuncs, which are an ELF thing, so it likely doesn't work on COFF. So you need to make a build target with cc files built for sse4.2 and make the distinction per file, not per function, like tikuta suggests.

[David Benjamin]

I'd expect the target attributes to be usable on a per-function basis, provided you don't have multiple copies of the same function. If you have multiple copies, it does the fancy ifunc thing. If you just have the one copy, it seems to work as you'd expect without any trickery.

That said, I've only done local experiments, not actually landed code with it, and I'm not sure how clang-cl behaves. (I've occasionally looked at intrinsics and target attributes for BoringSSL, but compilers still emit terrible intrinsics code, so we haven't switched anything to it.) But if it doesn't work, I think that's worth fixing in clang. Doing it at the file level seems like it'd risk ODR problems if the SSE4.2 file included bits of the STL and emitted an SSE4.2-using copy of some inline function which wins over the normal copy.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAMGbLiGbuWaEm-8ibV_vz9EiQvooRxuF1_EwCTKaT-%3Dm1FSc5A%40mail.gmail.com.

[Nico Weber]

Yes, that's a problem. Behold this amazing workaround for that: https://skia-review.googlesource.com/c/skia/+/5089/8/include/private/SkSafe_math.h#37 :D

[David Benjamin]

Oww. Worth trying single-function target attributes there maybe? FWIW, this CL cross-compiled just fine on my machine. Running the CQ now to confirm:

https://chromium-review.googlesource.com/c/chromium/src/+/2764229

[Nigel Tao]

On Tue, Mar 16, 2021 at 10:58 PM Takuto Ikuta <tik...@chromium.org> wrote:
> Maybe you need to extract code using SSSE3 to separate file

I forgot to mention that third/party wuffs ships as a "single file C
library", also known as a ["header file
library"](https://github.com/nothings/stb/blob/master/docs/stb_howto.txt).
That doesn't play well with extracting code to separate files. :-/

> and tweak -m flag for the file.
> https://source.chromium.org/chromium/chromium/src/+/master:build/config/win/BUILD.gn;l=112;drc=cbbfb16cbc550d4bf2555c7f061324d854126116

Ah, that explains the "I am a little surprised that the

_mm_lddqu_si128 call (SSE3) apparently compiled successfully when

_mm_shuffle_epi8 doesn't (SSSE3)" mystery.

That //build/config/win/BUILD.gn snippet is:

# Chrome currently requires SSE3. Clang supports targeting any Intel
# microarchitecture. MSVC only supports a subset of architectures, and the
# next step after SSE2 will be AVX.
if (current_cpu == "x86" || current_cpu == "x64") {
cflags += [ "-msse3" ]
}

I don't feel empowered to unilaterally change "Chrome currently
requires SSE3" to "Chrome currently requires SSSE3 (with *three*
'S's)", or to SSE4.2.

Hmm...

> There is
> https://docs.microsoft.com/en-us/cpp/intrinsics/cpuid-cpuidex?view=msvc-160
> for runtime check.

Yeah, I already said that I'm doing a separate cpuid-based runtime
check. I wasn't clear, though, whether that was clang/gcc only. It's
not, and there's a _MSC_VER one too.

> I found
> https://clang.llvm.org/docs/AttributeReference.html#target
> but runtime resolution seems only for ELF.

I'm not using multi-versioning (multiple implementations of the same
function name), which IIUC is an ELF-specific feature. I just want the
compiler to accept e.g. SSE3 intrinsics *for that attributed function*
(but not e.g. when the compiler magically vectorizes loops in the
other 99% of the file).

[David Benjamin]

FWIW, https://chromium-review.googlesource.com/c/chromium/src/+/2764229's CQ run has since come through clean, so target attributes seem okay? Though you'll probably want appropriate ifdefs around it to be MSVC-compatible and all.

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAEdON6bij4M_PV3_76vDGgLXE9q%2BSuj4cmkzc181xWNKuTQRUw%40mail.gmail.com.

[Nigel Tao]

On Wed, Mar 17, 2021 at 9:10 AM David Benjamin <davi...@chromium.org> wrote:
> FWIW, https://chromium-review.googlesource.com/c/chromium/src/+/2764229's CQ run has since come through clean, so target attributes seem okay? Though you'll probably want appropriate ifdefs around it to be MSVC-compatible and all.

Interesting. My attribute-targets are currently #if-guarded by:

#if defined(__GNUC__)
__attribute__((target("sse4.2")))
#endif

Is there a __GNUC__ or _MSC_VER style macro that clang-cl defines but
cl doesn't? Alternatively, is there a list of built-in clang-cl
macros?

[Nico Weber]

Yes, __clang__: `if defined(__GNUC__) || defined(__clang__)`.

pwnall looked at SSE3 vs SSSE3 and enough machines support the former but not the latter that we decided to not require SSSE3 at this point (data recent, from last year).

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
    http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAEdON6axk%2BADQFYqQBwmCL%2BtGNMpRUNTDBD1VAdAj-aKvZJ_2Q%40mail.gmail.com.

[Nigel Tao]

On Wed, Mar 17, 2021 at 12:21 AM Nico Weber <tha...@chromium.org> wrote:
> https://chromium.googlesource.com/chromium/src/+/master/docs/win_cross.md should help you with most questions.

The "Cross-compiling Chrome/win" doc says:

----
gclient sync, follow instructions on screen...
These commands create a zip file named <hash value>.zip. Then, to use
the generated file in a Linux or Mac host, the following environment
variables need to be set:
export DEPOT_TOOLS_WIN_TOOLCHAIN_BASE_URL=<path/to/sdk/zip/file>
----

OK... I think "gclient sync" downloaded a zip file. But I don't know
where on the local file system it downloaded it to, and hence what to
set DEPOT_TOOLS_WIN_TOOLCHAIN_BASE_URL to. The "gclient sync" output
(short extract and full text below) mention
"/tmp/tmp9Iia3U/20d5f2553f.zip" but that file doesn't exist any more.
Where would you expect it to be?

I also ran "download_from_google_storage --config" but AFAICT that
just created a .boto file.

Short extract:
----
Running hooks: 12% (13/101) win_toolchain
________ running 'vpython src/build/vs_toolchain.py update --force' in
'/usr/local/google/home/nigeltao/chromium'
Windows toolchain out of date or doesn't exist, updating (Pro)...
current_hashes:
desired_hash: 20d5f2553f
Copying gs://chrome-wintoolchain/20d5f2553f.zip...
/ [1 files][ 1.2 GiB/ 1.2 GiB] 76.6 MiB/s
Operation completed over 1 objects/1.2 GiB.
Extracting /tmp/tmp9Iia3U/20d5f2553f.zip...
Calculating hash of toolchain in vs_files/20d5f2553f. Please wait...
Hook 'vpython src/build/vs_toolchain.py update --force' took 148.82 secs
Running hooks: 14% (15/101) clang
----

Full text:
----
$ gclient sync
use_relative_hooks is deprecated, please remove it from DEPS. (it was
merged in use_relative_paths)
Syncing projects: 100% (237/237), done.
Running hooks: 12% (13/101) win_toolchain
________ running 'vpython src/build/vs_toolchain.py update --force' in
'/usr/local/google/home/nigeltao/chromium'
Windows toolchain out of date or doesn't exist, updating (Pro)...
current_hashes:
desired_hash: 20d5f2553f
Copying gs://chrome-wintoolchain/20d5f2553f.zip...
/ [1 files][ 1.2 GiB/ 1.2 GiB] 76.6 MiB/s
Operation completed over 1 objects/1.2 GiB.
Extracting /tmp/tmp9Iia3U/20d5f2553f.zip...
Calculating hash of toolchain in vs_files/20d5f2553f. Please wait...
Hook 'vpython src/build/vs_toolchain.py update --force' took 148.82 secs
Running hooks: 14% (15/101) clang
________ running 'vpython src/tools/clang/scripts/update.py' in
'/usr/local/google/home/nigeltao/chromium'
Downloading https://commondatastorage.googleapis.com/chromium-browser-clang/Linux_x64/clang-llvmorg-13-init-3462-gfe5c2c3c-2.tgz
.......... Done.
Downloading https://commondatastorage.googleapis.com/chromium-browser-clang/Win/clang-llvmorg-13-init-3462-gfe5c2c3c-2.tgz
.......... Done.
Running hooks: 28% (29/101) rc_linux
________ running 'vpython
src/third_party/depot_tools/download_from_google_storage.py
--no_resume --no_auth --bucket chromium-browser-clang/rc -s
src/build/toolchain/win/rc/linux64/rc.sha1' in
'/usr/local/google/home/nigeltao/chromium'
0> Downloading src/build/toolchain/win/rc/linux64/rc@2d0c766039264dc2514d005a42f074af4838a446...
Downloading 1 files took 7.998627 second(s)
Running hooks: 32% (33/101) apache_win32
________ running 'vpython
src/third_party/depot_tools/download_from_google_storage.py
--no_resume --directory --recursive --no_auth --num_threads=16
--bucket chromium-apache-win32 src/third_party/apache-win32' in
'/usr/local/google/home/nigeltao/chromium'
5> Downloading src/third_party/apache-win32/modules/mod_cgi.so@9fe78b0bc590613933f10ed037216449a9854080...
15> Downloading
src/third_party/apache-win32/bin/api-ms-win-core-file-l2-1-0.dll@402b7b8f8dcfd321b1d12fc85a1ee5137a5569b2...
10> Downloading
src/third_party/apache-win32/modules/mod_authz_host.so@b6db200f1e4d8921f5ea13867902926b37de99a0...
12> Downloading
src/third_party/apache-win32/modules/mod_access_compat.so@15cb4ef42fe094ab865859900be14421741b4e74...
11> Downloading
src/third_party/apache-win32/modules/mod_mime.so@27a07dd6f2cc214a22bf6beedbcb4624aa594623...
8> Downloading src/third_party/apache-win32/modules/php7apache2_4.dll@454aedd338235a5618c34401077e639dde69dbef...
1> Downloading src/third_party/apache-win32/modules/mod_rewrite.so@83c43f08fc010f43345843818666c82254d79cd2...
3> Downloading src/third_party/apache-win32/modules/mod_ssl.so@03e42dcbd0064b2e3be7952df01849a870464db3...
6> Downloading src/third_party/apache-win32/modules/mod_headers.so@fb6530a211bfa3859dbcc3330febf2eed5a5d579...
0> Downloading src/third_party/apache-win32/modules/mod_autoindex.so@f4ff8a89d66963f79c0dc42ed4002080f7c5b49f...
2> Downloading src/third_party/apache-win32/modules/mod_env.so@632c51d3ebb008eb072362eab76d0c435f385882...
7> Downloading src/third_party/apache-win32/modules/mod_alias.so@a1ce8466c07eda65049e96360af7d6052363fabe...
9> Downloading src/third_party/apache-win32/modules/mod_log_config.so@a7ceec0a4432a1b353d7d27191064596eab01414...
4> Downloading src/third_party/apache-win32/modules/mod_asis.so@c66f70fa6dd25400787ec7816a55743bbb2cae90...
13> Downloading
src/third_party/apache-win32/modules/mod_authz_core.so@c549d87dbb75923d9df934bef591438fd629ffe1...
14> Downloading
src/third_party/apache-win32/modules/mod_authn_core.so@cbc0f27732a85e19057d2eaa99c787348fae637c...
10> Downloading
src/third_party/apache-win32/bin/api-ms-win-core-processthreads-l1-1-0.dll@50699041060d14576ed7bacbd44be9af80eb902a...
4> Downloading src/third_party/apache-win32/bin/api-ms-win-crt-string-l1-1-0.dll@7f389e6f2d6e5beb2a3baf622a0c0ea24bc4de60...
11> Downloading
src/third_party/apache-win32/bin/api-ms-win-core-interlocked-l1-1-0.dll@f779cdef9ded19402aa72958085213d6671ca572...
6> Downloading src/third_party/apache-win32/bin/php7ts.dll@82a55130eaa2138b7e1f8ddf609552ac2f679fc9...
7> Downloading src/third_party/apache-win32/bin/openssl.exe@2dba80e3402fb7bbeb5889eef2fd9815260bc4f8...
13> Downloading
src/third_party/apache-win32/bin/api-ms-win-crt-filesystem-l1-1-0.dll@78fa03c89ea12ff93fa499c38673039cc2d55d40...
14> Downloading
src/third_party/apache-win32/modules/mod_authn_core.so@cbc0f27732a85e19057d2eaa99c787348fae637c...
10> Downloading
src/third_party/apache-win32/bin/api-ms-win-core-processthreads-l1-1-0.dll@50699041060d14576ed7bacbd44be9af80eb902a...
4> Downloading src/third_party/apache-win32/bin/api-ms-win-crt-string-l1-1-0.dll@7f389e6f2d6e5beb2a3baf622a0c0ea24bc4de60...
11> Downloading
src/third_party/apache-win32/bin/api-ms-win-core-interlocked-l1-1-0.dll@f779cdef9ded19402aa72958085213d6671ca572...
6> Downloading src/third_party/apache-win32/bin/php7ts.dll@82a55130eaa2138b7e1f8ddf609552ac2f679fc9...
7> Downloading src/third_party/apache-win32/bin/openssl.exe@2dba80e3402fb7bbeb5889eef2fd9815260bc4f8...
13> Downloading
src/third_party/apache-win32/bin/api-ms-win-crt-filesystem-l1-1-0.dll@78fa03c89ea12ff93fa499c38673039cc2d55d40...
9> Downloading src/third_party/apache-win32/bin/api-ms-win-core-profile-l1-1-0.dll@e7e0b18a40a35bd8b0766ac72253de827432e148...
8> Downloading src/third_party/apache-win32/bin/ApacheMonitor.exe@a09af35e49d42ecc0aa6ef43203a709f7732bdd5...
2> Downloading src/third_party/apache-win32/bin/api-ms-win-crt-conio-l1-1-0.dll@49002b58cb0df2ee8d868dec335133cf225657df...
15> Downloading
src/third_party/apache-win32/bin/api-ms-win-crt-utility-l1-1-0.dll@eaa07829d012206ac55fb1af5cc6a35f341d22be...
12> Downloading
src/third_party/apache-win32/bin/api-ms-win-core-memory-l1-1-0.dll@9c03356cf48112563bb845479f40bf27b293e95e...
1> Downloading src/third_party/apache-win32/bin/api-ms-win-core-file-l1-1-0.dll@9acbeef0ac510c179b319ca69cd5378d0e70504d...
5> Downloading src/third_party/apache-win32/bin/api-ms-win-crt-runtime-l1-1-0.dll@a19acefa3f95d1b565650fdbc40ef98c793358e9...
14> Downloading
src/third_party/apache-win32/bin/api-ms-win-core-string-l1-1-0.dll@637e4a9946691f76e6deb69bdc21c210921d6f07...
3> Downloading src/third_party/apache-win32/bin/libcrypto-1_1-x64.dll@3b42f0b6560d1eff56e3a00756366bf260f1d608...
0> Downloading src/third_party/apache-win32/bin/api-ms-win-core-datetime-l1-1-0.dll@4940d5b92b6b80a40371f8df073bf3eb406f5658...
4> Downloading src/third_party/apache-win32/bin/pcre.dll@35481488fbc34b113885466ab3e34b6615648045...
5> Downloading src/third_party/apache-win32/bin/api-ms-win-crt-convert-l1-1-0.dll@c84e41fdcc4ca89a76ae683cb390a9b86500d3ca...
13> Downloading
src/third_party/apache-win32/bin/api-ms-win-core-processenvironment-l1-1-0.dll@2745259f4dbbefbf6b570ee36d224abdb18719bc...
2> Downloading src/third_party/apache-win32/bin/api-ms-win-core-handle-l1-1-0.dll@a2e2a40cea25ea4fd64b8deaf4fbe4a2db94107a...
1> Downloading src/third_party/apache-win32/bin/api-ms-win-core-synch-l1-1-0.dll@5584c189216a17228cca6cd07037aaa9a8603241...
11> Downloading
src/third_party/apache-win32/bin/api-ms-win-crt-heap-l1-1-0.dll@60b4cf246c5f414fc1cd12f506c41a1043d473ee...
9> Downloading src/third_party/apache-win32/bin/api-ms-win-crt-stdio-l1-1-0.dll@982b5da1c1f5b9d74af6243885bcba605d54df8c...
10> Downloading
src/third_party/apache-win32/bin/api-ms-win-core-errorhandling-l1-1-0.dll@51cbb7ba47802dc630c2507750432c55f5979c27...
14> Downloading
src/third_party/apache-win32/bin/api-ms-win-core-sysinfo-l1-1-0.dll@f20ae25484a1c1b43748a1f0c422f48f092ad2c1...
15> Downloading
src/third_party/apache-win32/bin/api-ms-win-core-heap-l1-1-0.dll@b4310929ccb82dd3c3a779cab68f1f9f368076f2...
0> Downloading src/third_party/apache-win32/bin/ucrtbase.dll@4189f4459c54e69c6d3155a82524bda7549a75a6...
8> Downloading src/third_party/apache-win32/bin/api-ms-win-core-console-l1-1-0.dll@724f4f91041ad595e365b724a0348c83acf12bbb...
12> Downloading
src/third_party/apache-win32/bin/api-ms-win-core-namedpipe-l1-1-0.dll@cb59f1fe73c17446eb196fc0dd7d944a0cd9d81f...
7> Downloading src/third_party/apache-win32/bin/libapr-1.dll@d7083018a6f19148ffedc3d09416282bc69babd6...
3> Downloading src/third_party/apache-win32/bin/api-ms-win-crt-locale-l1-1-0.dll@9c1df49a8dbdc8496ac6057f886f5c17b2c39e3e...
6> Downloading src/third_party/apache-win32/bin/api-ms-win-core-util-l1-1-0.dll@1e1a5ab47e4c2b3c32c81690b94954b7612bb493...
5> Downloading src/third_party/apache-win32/bin/api-ms-win-core-localization-l1-2-0.dll@9874398548891f6a08fc06437996f84eb7495783...
9> Downloading src/third_party/apache-win32/bin/api-ms-win-core-processthreads-l1-1-1.dll@0bffb9ed366853e7019452644d26e8e8f236241b...
2> Downloading src/third_party/apache-win32/bin/libapriconv-1.dll@18998f83f0115d6ebd67be781ec7b94e2fe67550...
13> Downloading
src/third_party/apache-win32/bin/api-ms-win-core-timezone-l1-1-0.dll@4bf13db65943e708690d6256d7ddd421cc1cc72b...
14> Downloading
src/third_party/apache-win32/bin/api-ms-win-crt-environment-l1-1-0.dll@9a4818897251cacb7fe1c6fe1be3e854985186ad...
11> Downloading
src/third_party/apache-win32/bin/libhttpd.dll@4ea4b5bfa0e3301fd326f9970efacfff028c5144...
10> Downloading
src/third_party/apache-win32/bin/vcruntime140.dll@aaccc47a71fbcadf980932874056c9570a824890...
1> Downloading src/third_party/apache-win32/bin/api-ms-win-crt-time-l1-1-0.dll@ee815a158baacb357d9e074c0755b6f6c286b625...
8> Downloading src/third_party/apache-win32/bin/zlib1.dll@bae633d523b5905725d50d59297d9c26997802f5...
15> Downloading
src/third_party/apache-win32/bin/api-ms-win-core-debug-l1-1-0.dll@e7c8a6c29c3158f8b332eea5c33c3b1e044b5f73...
12> Downloading
src/third_party/apache-win32/bin/libssl-1_1-x64.dll@a771c64188a388109c0f88a0b72593ca63b734fa...
4> Downloading src/third_party/apache-win32/bin/api-ms-win-core-rtlsupport-l1-1-0.dll@24f37d46dfc0ef303ef04abf9956241af55d25c9...
7> Downloading src/third_party/apache-win32/bin/api-ms-win-core-libraryloader-l1-1-0.dll@47143a66b4a2e2ba019bf1fd07bcca9cfb8bb117...
0> Downloading src/third_party/apache-win32/bin/libaprutil-1.dll@ad35bc0f054af6a4e02f2e9a55c9213def6cb2cf...
3> Downloading src/third_party/apache-win32/bin/api-ms-win-core-synch-l1-2-0.dll@a9aebbbb73b7b846b051325d7572f2398f5986ee...
6> Downloading src/third_party/apache-win32/bin/api-ms-win-crt-process-l1-1-0.dll@ec96f7beeaec14d3b6c437b97b4a18a365534b9b...
9> Downloading src/third_party/apache-win32/bin/api-ms-win-crt-multibyte-l1-1-0.dll@91eef52c557aefd0fde27e8df4e3c3b7f99862f2...
5> Downloading src/third_party/apache-win32/bin/api-ms-win-core-file-l1-2-0.dll@04669214375b25e2dc8a3635484e6eeb206bc4eb...
14> Downloading
src/third_party/apache-win32/bin/httpd.exe@db1e5983c9b91253eb253bfd6c3f498bc0d86230...
13> Downloading
src/third_party/apache-win32/bin/api-ms-win-crt-math-l1-1-0.dll@8b35ec4676bd96c2c4508dc5f98ca471b22deed7...
2> Downloading src/third_party/apache-win32/bin/api-ms-win-crt-private-l1-1-0.dll@0c33cfe40edd278a692c2e73e941184fd24286d9...
Downloading 69 files took 37.936652 second(s)
Hook 'vpython src/third_party/depot_tools/download_from_google_storage.py
--no_resume --directory --recursive --no_auth --num_threads=16
--bucket chromium-apache-win32 src/third_party/apache-win32' took
39.76 secs
Running hooks: 100% (101/101), done.
----

[David Benjamin]

If you're a Googler, I don't think you need to worry about the DEPOT_TOOLS_WIN_TOOLCHAIN_BASE_URL step. That's part of the package_from_installed.py step. The gs://chrome-wintoolchain/ download otherwise is sufficient. I think you're all set to cross-compile. :-)

(Hrm, I don't think downloading third_party/apache-win32 is actually necessary for cross-compiles. I'll go switch that to checking host_os...)

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
    http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAEdON6ZuA0VvqwFh5Bim0T1JswieatecU9fm0sExS-puD7eHMA%40mail.gmail.com.

[Nico Weber]

On Wed, Mar 17, 2021 at 9:27 AM David Benjamin <davi...@chromium.org> wrote:

If you're a Googler, I don't think you need to worry about the DEPOT_TOOLS_WIN_TOOLCHAIN_BASE_URL step. That's part of the package_from_installed.py step. The gs://chrome-wintoolchain/ download otherwise is sufficient. I think you're all set to cross-compile. :-)

(Hrm, I don't think downloading third_party/apache-win32 is actually necessary for cross-compiles. I'll go switch that to checking host_os...)

(It's not needed for building, but it's needed for toggling tests on swarming: https://source.chromium.org/chromium/chromium/src/+/master:BUILD.gn;l=1126?q=apache%20file:%5C.gn&ss=chromium So please keep that as-is.)

[Nigel Tao]

On Thu, Mar 18, 2021 at 12:27 AM David Benjamin <davi...@chromium.org> wrote:
> If you're a Googler, I don't think you need to worry about the DEPOT_TOOLS_WIN_TOOLCHAIN_BASE_URL step.

Ah, I mis-read the doc. It says:

---
If you're at Google, blahA
If you are not at Google, blahB
These commands create a zip file named <hash value>.zip. Then, blahC;
blahD; etc.
---

I thought "not at Google" only applied to blahB, not to blahC, blahD, etc.

I have sent out http://crrev.com/c/2770998 to add some curly brackets.

[Reid Kleckner]

On Tue, Mar 16, 2021 at 3:10 PM David Benjamin <davi...@chromium.org> wrote:

FWIW, https://chromium-review.googlesource.com/c/chromium/src/+/2764229's CQ run has since come through clean, so target attributes seem okay? Though you'll probably want appropriate ifdefs around it to be MSVC-compatible and all.

This looks like it is all sorted, but I still wanted to endorse the use of __attribute__((target())) to solve this kind of problem. This use case should be supported on all platforms. It side steps the inline function ODR problems mentioned previously in the thread, and satisfies wuffs's single-file implementation requirement.

[Nigel Tao]

On Thu, Mar 18, 2021 at 12:48 PM Reid Kleckner <r...@google.com> wrote:
> I still wanted to endorse the use of __attribute__((target())) to solve this kind of problem. This use case should be supported on all platforms.

Yeah, __attribute__((target(etc))) does exactly what I want, guarded
by "#if defined(__GNUC__) || defined(__clang__)". I still need to
figure out what to do in upstream Wuffs for cl.exe (not clang-cl), but
that's not Chromium's problem.

Thanks everyone for their help.

One final tidbit, in case anyone from the future gets here by
searching for the same symptoms... I also had to augment
#include <intrin.h>
with
#include <immintrin.h> // AVX, AVX2, FMA, POPCNT
#include <nmmintrin.h> // SSE4.2
#include <wmmintrin.h> // AES, PCLMUL
per David Benjamin's example. <intrin.h> is nominally a kitchen-sink
include but I guess it wasn't written with attribute-target in mind. I
also still need to #include it for __cpuid.

[Dale Curtis]

FWIW, zhaoliang.ma tried the target attribute mechanism for some media SIMD code and is finding it much slower than a separate build file version. We're still debugging on https://bugs.chromium.org/p/chromium/issues/detail?id=1191301

- dale

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
    http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAEdON6YUt3_WcddUWRst3tZTfwhSmk%2BgO5paV%2Bd6-ZeFbG%3DU%2Bw%40mail.gmail.com.