Bypass odr-violation on Fuchsia workstation

[rheza shandikri]

Hello Fuchsia Team,

Good day!

I was able to build Fuchsia workstation with custom local buiild Chromium  with ASAN enabled, but when start Chromium, error occurs for "ERROR: AddressSanitizer: odr-violation (0x20315f6bf040)". 

I had tested several following configuration build but all of them failed to start chromium due odr-violation.

1. fx set workstation_eng.qemu-x64  --release
   
2. fx set workstation_eng.qemu-x64 --variant asan --variant host_asan --release 
   
3. fx set workstation_eng.qemu-x64 --variant asan-ubsan --variant host_asan-ubsan --release

My question: 

• How to bypass "ASAN_OPTIONS=detect_odr_violation=0" on the fuchsia workstation? in normal conventional machines such as Linux,Windows and Mac, can set the ASAN_OPTIONS=detect_odr_violation=0 into the environment path. 

Following is the error log:

[00082.636397][115392][115394][klog] INFO: =================================================================
[00082.636507][115392][115394][klog] ERROR: ERROR: AddressSanitizer: odr-violation (0x20315f6bf040):
[00082.655518][115392][115394][klog] INFO:   [1] size=4 'media::CdmPromiseTraits<>::kType' ../../media/base/cdm_promise.cc
[00082.662572][115392][115394][klog] INFO:   [2] size=4 'media::CdmPromiseTraits<>::kType' ../../media/base/cdm_promise.cc
[00082.662601][115392][115394][klog] INFO: These globals were registered at these points:
[00082.662624][115392][115394][klog] INFO:   [1]:
[00082.663336][115392][115394][klog] INFO:    #0    0x000022d89a5f12ea in <libclang_rt.asan.so>+0x562ea
[00082.663340][115392][115394][klog] INFO:    #1    0x000022d89a5f2169 in <libclang_rt.asan.so>+0x57169
[00082.663341][115392][115394][klog] INFO:    #2    0x0000437fcef48f9f in do_init_fini(dso*) ../../zircon/third_party/ulib/musl/ldso/dynlink.c:1492 <libc.so>+0x8ff9f
[00082.663342][115392][115394][klog] INFO:    #3    0x0000437fcef48c23 in __libc_start_init() ../../zircon/third_party/ulib/musl/ldso/dynlink.c:1516 <libc.so>+0x8fc23
[00082.663343][115392][115394][klog] INFO:    #4    0x0000437fcef5cd0d in start_main(const start_params*) ../../zircon/third_party/ulib/musl/src/env/__libc_start_main.c:141 <libc.so>+0xa3d0d
[00082.663343][115392][115394][klog] INFO:    #5    0x0000437fcef5d338 in __libc_start_main(zx_handle_t, int (*)(int, char**, char**)) ../../zircon/third_party/ulib/musl/src/env/__libc_start_main.c:219 <libc.so>+0xa4338
[00082.663344][115392][115394][klog] INFO:
[00082.663372][115392][115394][klog] INFO:   [2]:
[00082.663478][115392][115394][klog] INFO:    #0    0x000022d89a5f12ea in <libclang_rt.asan.so>+0x562ea
[00082.663482][115392][115394][klog] INFO:    #1    0x000022d89a5f2169 in <libclang_rt.asan.so>+0x57169
[00082.663483][115392][115394][klog] INFO:    #2    0x0000437fcef48f9f in do_init_fini(dso*) ../../zircon/third_party/ulib/musl/ldso/dynlink.c:1492 <libc.so>+0x8ff9f
[00082.663484][115392][115394][klog] INFO:    #3    0x0000437fcef48c23 in __libc_start_init() ../../zircon/third_party/ulib/musl/ldso/dynlink.c:1516 <libc.so>+0x8fc23
[00082.663484][115392][115394][klog] INFO:    #4    0x0000437fcef5cd0d in start_main(const start_params*) ../../zircon/third_party/ulib/musl/src/env/__libc_start_main.c:141 <libc.so>+0xa3d0d
[00082.663485][115392][115394][klog] INFO:    #5    0x0000437fcef5d338 in __libc_start_main(zx_handle_t, int (*)(int, char**, char**)) ../../zircon/third_party/ulib/musl/src/env/__libc_start_main.c:219 <libc.so>+0xa4338
[00082.663486][115392][115394][klog] INFO:
[00082.663514][115392][115394][klog] INFO: HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_odr_violation=0
[00082.663594][115392][115394][klog] INFO: SUMMARY: AddressSanitizer: odr-violation: global 'media::CdmPromiseTraits<>::kType' at ../../media/base/cdm_promise.cc

[Shai Barack]

If you're just looking to hack around this then start here:

build/config/sanitizers/sanitizer_default_options.gni

--
All posts must follow the Fuchsia Code of Conduct https://fuchsia.dev/fuchsia-src/CODE_OF_CONDUCT or may be removed.
---
You received this message because you are subscribed to the Google Groups "discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@fuchsia.dev.
To view this discussion on the web visit https://groups.google.com/a/fuchsia.dev/d/msgid/discuss/92b8908d-4963-4e96-a10b-bf555c59873bn%40fuchsia.dev.

[Shai Barack]

Can you print your env from your Chromium process before it has a chance to crash so you can verify that the asan options that you set did in fact make it through? 

On Mon, Oct 31, 2022, 6:18 PM rheza shandikri <rhez...@gmail.com> wrote:

Hello Shai,

>>>If you're just looking to hack around this then start here:

>>>build/config/sanitizers/sanitizer_default_options.gni

Yes I did append "detect_odr_violation=0" on build/config/sanitizers/sanitizer_default_options.gni before post the question on mailing list and rebuild fuchsia from beginning but the ODR error still exists. 

My build steps as follow:

(1) download fuchsia from source and fx set workstation_eng.qemu-x64 --variant asan --variant host_asan --release 

(2) download chromium source and build with args.gn is_asan=true on ~/chromium/src/out/asan

(3) edit  ~/fuchsia/src/chromium/build_args.gni

change chromium_build_dir = "~/chromium/src/out/asan" 

(4) edit build/config/sanitizers/sanitizer_default_options.gni and add "detect_odr_violation=0"

...

  asan_default_options = [
    "detect_stack_use_after_return=1", 

    "detect_odr_violation=0",

    # TODO(fxbug.dev/67985): Using default quarantine size of 256MiB causes
    # asan bots to OOM, this reduces the per-app quarantine size until we
    # investigate other possible fixes.
    "quarantine_size_mb=64",
  ]  ...

(5) fx build

(6) launch with ffx emu start workstation.qemu-x64 --gpu swiftshader_indirect --net tap

Can you confirm above steps is correct? or can you share me pointer where the step missing?

Thanks

[rheza shandikri]

Hello Shai,

>>>If you're just looking to hack around this then start here:

>>>build/config/sanitizers/sanitizer_default_options.gni

Yes I did append "detect_odr_violation=0" on build/config/sanitizers/sanitizer_default_options.gni before post the question on mailing list and rebuild fuchsia from beginning but the ODR error still exists. 

My build steps as follow:

(1) download fuchsia from source and fx set workstation_eng.qemu-x64 --variant asan --variant host_asan --release 

(2) download chromium source and build with args.gn is_asan=true on ~/chromium/src/out/asan

(3) edit  ~/fuchsia/src/chromium/build_args.gni

change chromium_build_dir = "~/chromium/src/out/asan" 

(4) edit build/config/sanitizers/sanitizer_default_options.gni and add "detect_odr_violation=0"

...

  asan_default_options = [
    "detect_stack_use_after_return=1", 

    "detect_odr_violation=0",

    # TODO(fxbug.dev/67985): Using default quarantine size of 256MiB causes
    # asan bots to OOM, this reduces the per-app quarantine size until we
    # investigate other possible fixes.
    "quarantine_size_mb=64",
  ]  ...

(5) fx build

(6) launch with ffx emu start workstation.qemu-x64 --gpu swiftshader_indirect --net tap

Can you confirm above steps is correct? or can you share me pointer where the step missing?

Thanks

On Monday, October 31, 2022 at 10:28:43 PM UTC+7 sha...@google.com wrote:

[rheza shandikri]

Hello Shai,

I pasted the syslog to https://pastebin.com/MqiBqJeK.

Please let me know if it is correct.

[Shai Barack]

The value of asan_options looks correct. I now suspect that it doesn't matter.

Please check my understanding - you build Fuchsia from Fuchsia sources, and the Chromium binary from Chromium sources, correct?

For the next step in your investigation I suggest tracing through the build definitions to see where the asan options end up, and then figure out whether they actually affect your Chromium binary or only affect binaries built from the Fuchsia source tree.

[Roland McGrath]

The GN build integration for `asan_default_options` and the like affects only executables that are built in the Fuchsia build.  Those options are embedded as defaults in each executable built.

Things like Chromium come from separate builds, so if they have ASan builds and support embedding default options, those details are specific to the Chromium build system and wholly separate from the Fuchsia build system.  I don't know Chromium's details off hand, but you may get some advice on chromi...@chromium.org about how to configure that.

The other approach that works at the runtime level is to set `ASAN_OPTIONS` in the process environment.  However, I'm not at all sure how to set environment variables like that for a process launched as a component. Folks here may be able to give you some advice about modifying `.cml` files for the Chromium-based components, or something like that.

To view this discussion on the web visit https://groups.google.com/a/fuchsia.dev/d/msgid/discuss/79167cad-f0ac-4d23-9c7f-d76b14eea739n%40fuchsia.dev.

[rheza shandikri]

Hello Folks,

>>>Please check my understanding - you build Fuchsia from Fuchsia sources, and the Chromium binary from Chromium sources, correct?

Yes this is correct.

>>>For the next step in your investigation I suggest tracing through the build definitions to see where the asan options end up, and then figure out whether they actually affect your Chromium binary or only affect binaries built from the Fuchsia source tree.

Yes, thank you for the advice, will look into it. 

>>>but you may get some advice on chromi...@chromium.org about how to configure that.

Yes I will ask in chromium-dev mailing list, thank you for the advice.