Running Perf benchmarks

[Sohan Jyoti Ghosh]

Hi !

I was trying to run some of the perf benchmarks for rendering.

But, it seems there is some extra permission required from the Cloud storage and even with that it doesnt work out of the box.

└─ $ ▶ ./run_benchmark rendering.desktop --browser=system
(WARNING) 2022-07-28 19:39:52,317 benchmark._GetTimelineBasedMeasurementOptions:251  The following TBMv3 metrics have been selected to run: uma_metrics. Please note that TBMv3 is an experimental feature in active development, and may not be supported in the future in its current form. Follow crbug.com/1012687 for updates and to discuss your use case before deciding to rely on this feature.
(CRITICAL) 2022-07-28 19:39:53,680 page_test_results.InterruptBenchmark:226  Benchmark execution interrupted: PermissionError('Attempted to access a file from Cloud Storage but you don\'t have permission. To configure your credentials:\n  1. Run "/home/sohan/Chromium/src/third_party/catapult/third_party/gsutil/gsutil config" and follow its instructions.\n  2. If you have a @google.com account, use that account.\n  3. For the project-id, just enter 0.')

Traceback (most recent call last):
  RunBenchmark at /home/sohan/Chromium/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py:469
    RunStorySet(
  RunStorySet at /home/sohan/Chromium/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py:298
    cloud_storage.GetFilesInDirectoryIfChanged(directory,
  GetFilesInDirectoryIfChanged at /home/sohan/Chromium/src/third_party/catapult/common/py_utils/py_utils/cloud_storage.py:611
    GetIfChanged(path_name, bucket)
  GetIfChanged at /home/sohan/Chromium/src/third_party/catapult/common/py_utils/py_utils/cloud_storage.py:583
    _GetLocked(bucket, expected_hash, file_path)
  _GetLocked at /home/sohan/Chromium/src/third_party/catapult/common/py_utils/py_utils/cloud_storage.py:444
    _RunCommand(['cp', url, partial_download_path.name])
  _RunCommand at /home/sohan/Chromium/src/third_party/catapult/common/py_utils/py_utils/cloud_storage.py:161
    raise GetErrorObjectForCloudStorageStderr(stderr.decode('utf-8'))
py_utils.cloud_storage.PermissionError: Attempted to access a file from Cloud Storage but you don't have permission. To configure your credentials:
  1. Run "/home/sohan/Chromium/src/third_party/catapult/third_party/gsutil/gsutil config" and follow its instructions.
  2. If you have a @google.com account, use that account.
  3. For the project-id, just enter 0.

Locals:
  args       : ['/home/sohan/.vpython-root/194c9f/bin/python', '/home/sohan/Chromium/src/third_party/catapult/third_party/gsutil/gsutil', 'cp', 'gs://chrome-partner-telemetry/454ecb15fa314b2f30cdca066e9878d436d91354', '/home/sohan/Chromium/src/tools/perf/page_sets/simple_canvas/resources/tmpx0i05vf9']
  gsutil     : <subprocess.Popen object at 0x7f80ecf58f10>
  gsutil_env : None
  stderr     : b'AccessDeniedException: 403 soh...@chromium.org does not have storage.objects.list access to the Google Cloud Storage bucket.\n'
  stdout     : b''

[  PASSED  ] 0 tests.

(WARNING) 2022-07-28 19:39:53,682 processor.ProcessResults:77  No test results to process.
(WARNING) 2022-07-28 19:39:54,275 render_histograms_viewer.ReadExistingResults:49  Failed to extract previous results from HTML output
View results at file:///home/sohan/Chromium/src/tools/perf/results.html

Even after trying to setup my credential as instructed, I get the same error.

Is there something I am missing here. Is it only for @google.com  accounts ?

Any pointers ?

Br,

Sohan

[Sohan Jyoti Ghosh]

Hi John,

Thank you for the clarification.

I tried using the PinPoint service too, for checking perf impact of CL. But, it seems it has a similar access issue for individual contributors.

I will try to post a message to the telemetry group and check.

Thanks again.

Br,

Sohan

On Tue, Aug 2, 2022 at 4:55 AM John Chen <john...@chromium.org> wrote:

Hi Sohan,

Since your email was displayed in the error message, it meant that you were already authenticated to the cloud storage, but the account you used didn't have the necessary permissions. Some of the data required to run benchmarks are only available to people at Google or one of the companies that work closely with Chrome team. If you work for such a company, please use your company email for access, or post a message to https://groups.google.com/a/chromium.org/g/telemetry for further assistance. If you don't work for such a company, then unfortunately you won't be able to run some benchmarks.

Regards,

John

[John Chen]

Hi Sohan,

Since your email was displayed in the error message, it meant that you were already authenticated to the cloud storage, but the account you used didn't have the necessary permissions. Some of the data required to run benchmarks are only available to people at Google or one of the companies that work closely with Chrome team. If you work for such a company, please use your company email for access, or post a message to https://groups.google.com/a/chromium.org/g/telemetry for further assistance. If you don't work for such a company, then unfortunately you won't be able to run some benchmarks.

Regards,

John

On Thursday, July 28, 2022 at 9:54:02 AM UTC-7 Sohan Jyoti Ghosh wrote: