nw75 windows build failing due to adapter.mojom.js

[Steve Durham]

Hi,

sorry for all the posts but really really trying to get a windows x64 build that actually finishes.
I've tried duplicating exactly what buildbot is doing but there must be something missing.

Currently I cannot get past this one, and really not sure how the buildbot work can be sidestepping it:

[44/43209] ACTION //chrome/browser:resources_grit(//build/toolchain/win:win_clang_x64)
FAILED: gen/chrome/browser/resources_grit.d.stamp gen/chrome/grit/browser_resources.h gen/chrome/browser_resources.pak gen/chrome/browser_resources.pak.info
E:/depot_tools/bootstrap-2@3_8_10_chromium_26_bin/python3/bin/python3.exe ../../tools/grit/grit.py -i ../../chrome/browser/nwjs_resources.grd build -o gen/chrome --depdir . --depfile gen/chrome/browser/resources_grit.d --write-only-new=1 --depend-on-stamp -D DEVTOOLS_GRD_PATH=gen/third_party/devtools-frontend/src/front_end/devtools_resources -D SHARED_INTERMEDIATE_DIR=gen -D _google_chrome=false -D _is_chrome_for_testing_branded=false -D chromeos_ash=false -D chromeos_lacros=false -D reven=false -D toolkit_views=true -D use_aura=true -D use_ozone=false -D use_titlecase=false -D nwjs_sdk=false -D scale_factors=2x -E root_gen_dir=gen -E root_src_dir=../../ -E CHROMIUM_BUILD=chromium -D nwjs_sdk=false -D chrome_root_store_supported=true -D enable_arcore=false -D enable_printing=true -D enable_extensions=true -D enable_hangout_services_extension=false -D enable_nacl=false -D enable_dice_support=true -D enable_pdf=true -D enable_print_preview=true -D enable_supervised_users=false -D enable_vr=true -D enable_webui_certificate_viewer=true -D enable_webui_tab_strip=true -D is_official_build=false -D safe_browsing_mode=0 -D optimize_webui=true -D use_nss_certs=false -E additional_modules_list_file=gen/chrome/browser/internal/additional_modules_list.txt --brotli brotli.exe -f gen/tools/gritsettings/default_resource_ids -p ../../tools/gritsettings/startup_resources_win.txt --assert-file-list obj/chrome/browser/resources_expected_outputs.txt
Error processing node <?xml version="1.0" encoding="UTF-8"?>
<include compress="gzip" file="${root_gen_dir}\device\bluetooth\public\mojom\adapter.mojom.js" name="IDR_BLUETOOTH_ADAPTER_MOJO_JS" type="BINDATA" use_base_dir="false" />: [Errno 2] No such file or directory: '..\\..\\out\\nw\\gen\\device\\bluetooth\\public\\mojom\\adapter.mojom.js'
Traceback (most recent call last):
  File "../../tools/grit/grit.py", line 28, in <module>
    sys.exit(grit.grit_runner.Main(sys.argv[1:]))
  File "E:\nwjs2\src\tools\grit\grit\grit_runner.py", line 313, in Main
    return toolobject.Run(options, args[1:])
  File "E:\nwjs2\src\tools\grit\grit\tool\build.py", line 272, in Run
    self.Process()
  File "E:\nwjs2\src\tools\grit\grit\tool\build.py", line 407, in Process
    self.ProcessNode(self.res, output, tmpfile)
  File "E:\nwjs2\src\tools\grit\grit\tool\build.py", line 331, in ProcessNode
    formatted = formatter(node, output_node.GetLanguage(), output_dir=base_dir)
  File "E:\nwjs2\src\tools\grit\grit\format\data_pack.py", line 87, in Format
    value = node.GetDataPackValue(lang, util.BINARY)
  File "E:\nwjs2\src\tools\grit\grit\node\include.py", line 109, in GetDataPackValue
    data = util.ReadFile(filename, util.BINARY)
  File "E:\nwjs2\src\tools\grit\grit\util.py", line 210, in ReadFile
    with open(filename, mode, encoding=encoding) as f:
FileNotFoundError: [Errno 2] No such file or directory: '..\\..\\out\\nw\\gen\\device\\bluetooth\\public\\mojom\\adapter.mojom.js'
[69/43203] ACTION //chrome/browser/resources/connectors_internals:build_ts(//build/toolchain/win:win_clang_x64)
ERROR: The process "44756" not found.
[70/43203] ACTION //chrome/browser/resources/browsing_topics:build_ts(//build/toolchain/win:win_clang_x64)
ERROR: The process "46168" not found.
[71/43203] ACTION //chrome/browser/resources/commander:build_ts(//build/toolchain/win:win_clang_x64)
ERROR: The process "38888" not found.
[77/43203] ACTION //chrome/browser/resources/discards:build_ts(//build/toolchain/win:win_clang_x64)
ERROR: The process "45956" not found.
ninja: build stopped: subcommand failed.

Is it possible the buildbot worker has something cached from previous runs?
If I build chromium itself (successfully) there is no trace of any "adapter.mojom.js" in the same subdirectory.

[Nick Hogle]

Hi Steve,

I ran into this same problem, and was able to work around it by forcing it to use browser_resources.grd instead of nwjs_resources.grd:

  diff --git a/chrome/browser/BUILD.gn b/chrome/browser/BUILD.gn
  index f85c583def12..7a9ea00e065a 100644
  --- a/chrome/browser/BUILD.gn
  +++ b/chrome/browser/BUILD.gn
  @@ -7705,7 +7705,8 @@ grit("resources") {
     if (nwjs_sdk) {
       source = "browser_resources.grd"
     } else {
  -    source = "nwjs_resources.grd"
  +    #source = "nwjs_resources.grd"
  +    source = "browser_resources.grd"

I'm don't think this is the "correct" fix, but it got my resources to compile.  Perhaps the "real" fix is for nwjs_resources.grd to get updated with any appropriate changes that have been made to browser_resources.grd? 

I assume that these two different resources files offer two different sets of browser resources for SDK vs non-SDK builds, because e.g. the non-SDK build wouldn't need DevTools bundled with it.  However, looking through the Git histories for these two files, it's unclear to me when/how the two files stay "in sync" with each-other.  They don't appear to have anything in common.  When browser_resources.grd updates (due to upstream Chromium changes), I would expect to see a similar diff being applied to nwjs_resources.grd, but doesn't seem to be the case.  

I do see that every time the files change, Roger Wang's name is on the commit, perhaps he knows what's up?

I hope this helps!

Best,

-Nick

[Steve Durham]

Thanks,

Yeah late yesterday I found this in some post somewhere, and finally got a build to finish doing that change.

I spent some time digging through commits as well, and it seems the BLUETOOTH* stuff was removed back in 2018.
I too don't see any correlation between the two files.

I will now move on to getting a component and debug build, which is what I really want.

We need to actively start making changes, especially on Mac.

It would be nice to get some more complete info on the buildbot setup, the build logs only tell part of the story I think.

[Nick Hogle]

@Steve Durham: Were you able to get Node to compile?  I ran this command: 

python3 src/third_party/node-nw/tools/gyp/gyp_main.py -I src/third_party/node-nw/common.gypi -D msvs_version=2021 -D build_type=Release_x64 src/third_party/node-nw/node.gyp

But it looks like it's generating MSVC solution files, not ninja build files, so I can't use ninja to build. (Also getting errors about some missing files...)

DEBUG: Generating MSVC Solution: src\third_party\node-nw\node.sln
DEBUG: Generating MSVC Solution: src\third_party\node-nw\deps\nghttp2\nghttp2.sln
DEBUG: Generating MSVC Solution: src\third_party\node-nw\deps\googletest\googletest.sln
DEBUG: Generating MSVC Solution: src\third_party\node-nw\deps\cares\cares.sln
DEBUG: Generating MSVC Solution: src\third_party\node-nw\deps\ada\ada.sln
DEBUG: Generating MSVC Solution: src\third_party\node-nw\deps\brotli\brotli.sln
DEBUG: Generating MSVC Solution: src\third_party\node-nw\deps\base64\base64.sln
DEBUG: Generating MSVC Solution: src\third_party\node-nw\deps\uvwasi\uvwasi.sln
DEBUG: Generating MSVC Solution: src\third_party\node-nw\deps\histogram\histogram.sln
DEBUG: Generating MSVC Solution: src\third_party\node-nw\deps\llhttp\llhttp.sln
DEBUG: Generating MSVC Solution: src\third_party\node-nw\deps\zlib\zlib.sln
DEBUG: Generating MSVC Solution: src\third_party\node-nw\deps\simdutf\simdutf.sln
DEBUG: Generating MSVC Solution: src\third_party\node-nw\deps\uv\uv.sln
DEBUG: Generating MSVC Solution: src\third_party\icu\icu.sln
DEBUG: Generating MSVC Solution: src\third_party\node-nw\deps\openssl\openssl.sln
Warning: Missing input files:
src\third_party\icu\windows\icudt.dll
src\third_party\node-nw\deps\cares\src\lib\ares_library_init.h

Here are my GYP environment variables, which I tried to set up similar to the BuildBot:

GYP_CHROMIUM_NO_ACTION=0

GYP_DEFINES="target_arch=x64 building_nw=1 icu_use_data_file_flag=1 host_arch=x64 clang=1 nwjs_sdk=1 disable_nacl=0"
GYP_GENERATOR=ninja
GYP_GENERATOR_FLAGS=output_dir=out

Not sure what's going on...

Cheers,

-Nick

[Nick Hogle]

... Actually, I have Visual Studio 2022 installed, as per the Chromium pre-requisite instructions:

But still, when I pass that to the gyp_main.py, I still end up with .sln file outputs only:

src/third_party/node-nw/tools/gyp/gyp_main.py -I src/third_party/node-nw/common.gypi -D msvs_version=2022 -D build_type=Release_x64 src/third_party/node-nw/node.gyp

Cheers,

-Nick

[Steve Durham]

It did work, I ran *exactly* what was in buildbot, cuz I wanted to get anything at all to finish before I started experimenting with more.
I have VS 2019, 2017, 2022 all installed but stuck with exactly what buildbot had for this run.

I also put the env vars in the system env rather than trying to set somehow by script while running.
I'll post what I had.

[Steve Durham]

GYP_CHROMIUM_NO_ACTION 0

GYP_DEFINES target_arch=x64 building_nw=1 icu_use_data_file_flag=1 host_arch=x64 clang=1 buildtype=Official

GYP_GENERATOR_FLAGS output_dir=out

GYP_GENERATORS ninja

GYP_MSVS_OVERRIDE_PATH C:/Program Files (x86)/Microsoft Visual Studio/2019/Community

GYP_MSVS_VERSION 2017

I have some script files for each step (since I'd been running them in various combos 1000 times)

args.gn:

is_debug = false
is_component_ffmpeg = true
target_cpu = "x64"
v8_win64_unwinding_info = true
use_allocator_shim = false
use_partition_alloc_as_malloc = false
enable_backup_ref_ptr_support = false
enable_mte_checked_ptr_support = false
nwjs_sdk = false
proprietary_codecs = true
ffmpeg_branding = "Chromium"

n3_vars.cmd file:
SET NWJSDIR=E:\n3
SET NWJSBRANCH=nw75
SET NWJSHASH=e62efed4c9c686f8e21c04aeaab7e964bfed41da

n3_nw_reset.cmd:
call %~dp0n3_vars.cmd
cd %NWJSDIR%\src\content\nw

call git --version

call git clean -f -f -d -x
call git fetch -t https://github.com/nwjs/nw.js %NWJSBRANCH%
call git reset --hard %NWJSHASH% --
call git branch -M %NWJSBRANCH%
call git rev-parse HEAD

n3_update.cmd:

call %~dp0n3_vars.cmd
cd %NWJSDIR%\src
call python content/nw/tools/sync.py

n3_gyp.cmd:
call %~dp0n3_vars.cmd
cd %NWJSDIR%
call python3 src/third_party/node-nw/tools/gyp/gyp_main.py -I src/third_party/node-nw/common.gypi -D msvs_version=2021 -D build_type=Release_x64 src/third_party/node-nw/node.gyp

The last step of copying and finishing I did by hand, straight from the build instructions, seemed to work and I could launch nwjs.

I'll probably be repeating all this soon and I would really like to get this to work with a --depth 1 set of checkouts, but LASTCHANGE needs some hacking I haven't figured out yet.

[Steve Durham]

Oh yeah original checkout

call %~dp0n3_vars.cmd
cd %NWJSDIR%
call git clone -b %NWJSBRANCH% -v --progress https://github.com/nwjs/chromium.src.git src
call git clone -b %NWJSBRANCH% -v --progress https://github.com/nwjs/v8 src/v8
call git clone -b %NWJSBRANCH% -v --progress https://github.com/nwjs/nw.js src/content/nw
call git clone -b %NWJSBRANCH% -v --progress https://github.com/nwjs/node src/third_party/node-nw

and I think I manually did gclient sync --with_branch_heads.

I'll be running this entire sequence again soon probably so I can script it from start to finish.