[Announcement] Cobalt Device Certification

[Daniel Juyung Seo]

Hi everyone,

"Cobalt Device Certification" guideline describes how to submit Cobalt(https://cobalt.foo) based device for the certification.

 

Build Types

1. Use qa build(CSP/NavigationWhitelist disabled by the parameters) for the certification. (Use --csp_mode=disable --disable_navigation_whitelist)

2. If you are not comfortable with the performance with qa build, you should submit gold build device additionally.

 

MSE/EME/Progressive Test Cases

Cobalt 9 and 11 support very up-to-date MSE/EME W3C spec. Cobalt based device should pass 2018 MSE/EME/Progressive test cases even for the 2017 certification.

While all the test cases marked as "(Optional)" can be ignored, some test cases can be ignored for 2017 certification.

• MSE: https://yt-media-test.appspot.com/2018.html

• 2017 certification: Ignore "MSE (Opus)"

• EME: https://yt-media-test.appspot.com/2018.html?test_type=encryptedmedia-test

• 2017 certification: Ignore "6. WidevineH264MultiMediaKeySessions"

• Progressive: https://yt-media-test.appspot.com/2018.html?test_type=progressive-test

 

*note: When you launch Qual-E on Cobalt, MSE/EME/Progressive test cases links automatically point to 2018 test cases.

Thanks,

Daniel

[5490 Fu]

hi Daniel Juyung Seo,

we build cobalt "Import Cobalt 9.76656" with qa (x64)  and test with below command.

./cobalt --url=https://qual-e.appspot.com --csp_mode=disable --disable_navigation_whitelist

we found two failure items "CSSPageRule.style" and "CSSStyleRule.style" in  "Conformance | DOM CSS".

are they necessary ?

Daniel Juyung Seo於 2017年8月3日星期四 UTC+8下午1時05分32秒寫道：

[Daniel Juyung Seo]

Hello, 

Basically you need to pass all the conformance tests.

Good news is that this is not your issue. This specific issues can be fixed by Qual-E code change. Please track this issue tracker for the follow-ups. https://issuetracker.google.com/issues/62847266

I will fix this ASAP.

Daniel

•  Daniel Juyung Seo 서주영     •  Partner Technology Manager (APAC) / 차장   •  Partner Engineering, YouTube in the Living Room, Google   •  Learn more about Cobalt application runtime

You received this message because you are subscribed to the Google Groups "cobalt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cobalt-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Wu Jefferson]

Hi Daniel,

  Would this CSS issue affect device certification if we submit before a fix arrives?

Thanks.

/Jefferson

[Daniel Juyung Seo]

No, unless you use old version of Cobalt because we know Cobalt supports that feature correctly.

Daniel

•  Daniel Juyung Seo 서주영     •  Partner Technology Manager (APAC) / 차장   •  Partner Engineering, YouTube in the Living Room, Google   •  Learn more about Cobalt application runtime

You received this message because you are subscribed to the Google Groups "cobalt-dev" group.

[Daniel Juyung Seo]

Hi everyone,

"SSL Test Cases" description is added to "Cobalt Device Certification" page on our partner site.

SSL Test Cases

Try to load the following URLs with qa build using "--csp_mode=disable --disable_navigation_whitelist" parameters.

• https://self-signed.badssl.com/: Connection needs to be rejected and show system error dialog due to the self signed certificate.
• https://expired.badssl.com/: Connection needs to be rejected and show system error dialog due to the expired certificate.
• https://sha256.badssl.com/, https://cert-test.sandbox.google.com/: The pages need to be loaded correctly.

Thanks,

Daniel

[Daniel Juyung Seo]

Hi everyone,

"Cobalt Versions" description is added to "Cobalt Device Certification" page on our partner site.

Cobalt Versions

Use up-to-date Cobalt versions from the Cobalt open source(https://cobalt.googlesource.com/cobalt/). Each branch head includes the critical fixes. For example, starting from August 16th, 2017, use Cobalt >= 9.76656 for Cobalt 9 and Cobalt >= 11.84380 for Cobalt 11.

Thanks,

Daniel

[Daniel Juyung Seo]

Hi,

Please refer the "Cobalt Device Certification" page. The following two pages should *not* work and you should not use --ignore_certificate_errors when you submit the device for the certification.

• https://self-signed.badssl.com/: Connection needs to be rejected and show system error dialog due to the self signed certificate.
• https://expired.badssl.com/: Connection needs to be rejected and show system error dialog due to the expired certificate.

I will investigate the WideVine test page but what kind of errors did you get?

Daniel

•  Daniel Juyung Seo 서주영     •  Partner Technology Manager (APAC) / 차장   •  Partner Engineering, YouTube in the Living Room, Google   •  Learn more about Cobalt application runtime

On Thu, Aug 17, 2017 at 11:58 AM, 5490 Fu <fu....@gmail.com> wrote:

hi daniel,

we test below three url on our cobalt(RELEASE_9 9.76656) x11x64 but not work.

./cobalt --url=https://qual-e.appspot.com/wv_license_request.html --csp_mode=disable --disable_navigation_whitelist

./cobalt --url=https://self-signed.badssl.com/ --csp_mode=disable --disable_navigation_whitelist 

./cobalt --url=https://expired.badssl.com/ --csp_mode=disable --disable_navigation_whitelist 

and we append a parameter "--ignore_certificate_errors", below two url can work.

./cobalt --url=https://self-signed.badssl.com/ --csp_mode=disable --disable_navigation_whitelist --ignore_certificate_errors

./cobalt --url=https://expired.badssl.com/ --csp_mode=disable --disable_navigation_whitelist --ignore_certificate_errors

in "Cobalt Device Certification" page you specify the parameters "--csp_mode=disable --disable_navigation_whitelist " only, not include "--ignore_certificate_errors".

is it necessary?

could you guide us how to test "https://qual-e.appspot.com/wv_license_request.html" ?

Daniel Juyung Seo於 2017年8月16日星期三 UTC+8下午8時37分10秒寫道：

You received this message because you are subscribed to the Google Groups "cobalt-dev" group.

[Daniel Juyung Seo]

Hi everyone,

"Submit NPLB Test Result" description is added to "Cobalt Device Certification" page on the partner site.

Submit NPLB Test Result

During the pre-certification step, submit the result of NPLB tests.

• Compile: ex) $ ninja -C out/linux-x64x11_gold nplb
• Run: ex) $ out/linux-x64x11_gold/nplb 2>&1 | tee nplb_result.txt
• Submit 'nplb_result.txt' file along with other pre-certification test data.

Thanks,

Daniel

[5490 Fu]

hi Daniel,

did you get our error log file that we test  WideVine page?

we re-send log file again?

attached file"wv.log" log below command.

./cobalt --url=https://qual-e.appspot.com/wv_license_request.html --csp_mode=disable --disable_navigation_whitelist

thanks.

[Daniel Juyung Seo]

Hi everyone,

Please use 'devel' build instead of 'gold' build for the NPLB test.

Submit NPLB Test Result

During the pre-certification step, submit the result of NPLB tests.

• Compile: ex) $ ninja -C out/linux-x64x11_devel nplb
• Run: ex) $ out/linux-x64x11_devel/nplb 2>&1 | tee nplb_result.txt

• Submit 'nplb_result.txt' file along with other pre-certification test data.

Thanks,

Daniel

•  Daniel Juyung Seo 서주영     •  Partner Technology Manager (APAC) / 차장   •  Partner Engineering, YouTube in the Living Room, Google   •  Learn more about Cobalt application runtime

[Wu Jefferson]

hi Daniel,

 Does cobalt certification also need to test for heartbleed? We notice that cobalt already comes with its own openssl package.

Thanks.

/Jefferson

[Daniel Juyung Seo]

Hi everyone,

I added an alert to disable the two parameters when launching the devices in the markets.

https://sites.google.com/a/google.com/youtube-leanback-partners/certification/cobalt

Build Types

1. Use qa build(CSP/NavigationWhitelist disabled by the parameters) for the certification. (Use --csp_mode=disable --disable_navigation_whitelist)

• Alert! These parameters are required solely for the certification purpose. You should remove them when launching the devices in the market.

1. If you are not comfortable with the performance with qa build, you should submit gold build device additionally.

Daniel

•  Daniel Juyung Seo 서주영     •  Partner Technology Manager (APAC) / 차장   •  Partner Engineering, YouTube in the Living Room, Google   •  Learn more about Cobalt application runtime

[weizh...@163.com]

hi Daniel,

 

we build cobalt "Import Cobalt 11.99532" with qa (x64)  and test with below command.

./cobalt --url=https://2018-dot-qual-e.appspot.com/ --csp_mode=disable --disable_navigation_whitelist

we found two failure items "CSP script-src url" and "CSP Level 2 - script-src nonce" in  "Conformance | Functionality & MSE/EME".

It seems to confict with "--csp_mode=disable" comand. If deleted  "--csp_mode=disable" comand, these two  items are sucess.

If these two  items about CSP necessary ? 

在 2017年8月3日星期四 UTC+8下午1:05:32，Daniel Juyung Seo写道：

[Daniel Juyung Seo]

Hi,

Those two tests need to be run without "--csp_moe=disable". I will separate out these two tests from Conformance test. As the build for the certification is expected to have "--csp_mode=disable" parameter, this result needs to be reported by the pre-certification report. I will update the Cobalt certification document accordingly once the job is done (ETA: 2~3 days)

Daniel

•  Daniel Juyung Seo 서주영     •  Partner Technology Manager (APAC) / 차장   •  Partner Engineering, YouTube in the Living Room, Google   •  Learn more about Cobalt application runtime

You received this message because you are subscribed to the Google Groups "cobalt-dev" group.

[dutx...@gmail.com]

Hi Daniel,

    I have a question to be confirmed. Youtube 2018 pre-certification requires that Widevine license server values on test page should match User Agent values,when navigating to https://qual-e.appspot.com/wv_license_request.html using cobalt.

 There are several Widevine license server values , the company_name and modle_name must be same with User Agent according to the pre-cetification requirement, but whether the other fields  which only get from licence, like platform_name or device_id have a certain format requirement?

   

   Thanks

在 2017年8月3日星期四 UTC+8下午1:05:32，Daniel Juyung Seo写道：

Hi everyone,

[Daniel Juyung Seo]

Hi,

I am sorry but please reach out to your YouTube technical account manager for the certification related questions.

Daniel

•  Daniel Juyung Seo 서주영     •  Partner Technology Manager (APAC) / 차장   •  Partner Engineering, YouTube in the Living Room, Google   •  Learn more about Cobalt application runtime

You received this message because you are subscribed to the Google Groups "cobalt-dev" group.

[nshin...@gmail.com]

Hi Daniel, 

Can you post the full and elaborated process on how certification/Pre-certification tests are done, I don't have any idea about how it is done, please help.

Thanks in advance. 

[Daniel Juyung Seo]

Hi,

Can you please reach out to your dedicated YouTube technical account manager?

There is a website for the YouTube Living Room certification.

This mailing list is for Open Source Cobalt in general.

Thanks,

Daniel

•  Daniel Juyung Seo     •  Team Lead of Partner Engineering   •  YouTube in the Living Room, Google

You received this message because you are subscribed to the Google Groups "cobalt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cobalt-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cobalt-dev/1b953003-c06b-4b33-92c6-e9028f16ca05%40googlegroups.com.