vpython3 error on chromeos-amd64-generic-rel

113 views
Skip to first unread message

David Benjamin

unread,
Sep 13, 2021, 9:05:38 PM9/13/21
to pyt...@chromium.org
Hi Python folks,

I'm trying to get SpawnedTestServer working with Python 3. I've mostly gotten it working, but the chromeos-amd64-generic-rel bot is upset at me. (Also the Android bots, but I've almost certainly just messed up the remote test server bits. Still need to poke at it.)

The relevant bits of the CL are here. It seems to run fine on most of the bots, except on chromeos-amd64-generic-rel.
https://chromium-review.googlesource.com/c/chromium/src/+/3156548/8/net/test/python_utils.cc

There, I get an error from vpython3, and I'm not sure what to do with it:
[E2021-09-13T16:22:33.523488-07:00 11746 0 annotate.go:241] goroutine 1:
[E2021-09-13T16:22:33.523709-07:00 11746 0 annotate.go:241] #0 go.chromium.org/luci/vpython/python/find.go:93 - python.Find()
[E2021-09-13T16:22:33.523912-07:00 11746 0 annotate.go:241]   annotation #0:
[E2021-09-13T16:22:33.524420-07:00 11746 0 annotate.go:241]     reason: no Python found
[E2021-09-13T16:22:33.524626-07:00 11746 0 annotate.go:241]   annotation #1:
[E2021-09-13T16:22:33.524847-07:00 11746 0 annotate.go:241]     reason: could not find appropriate executable for: "python3.8"
[E2021-09-13T16:22:33.525329-07:00 11746 0 annotate.go:241]   annotation #2:
[E2021-09-13T16:22:33.525553-07:00 11746 0 annotate.go:241]     reason: could not find target in system
[E2021-09-13T16:22:33.525760-07:00 11746 0 annotate.go:241]     internal reason: target(python3.8)/dirs([/usr/local/cros_test/out/Release/../../vpython_dir_linux_amd64 /usr/local/cros_test/out/Release/../../vpython_dir_linux_amd64/bin/ 

Full error here:

Does anyone on this list know what might be up here? That it works on all the other bots makes me wonder if something's odd with that bot somehow.

David

Takuto Ikuta

unread,
Sep 13, 2021, 9:20:37 PM9/13/21
to David Benjamin, python
Hi David,

Where do you see the error in the link? Also the link is a build on linux-chromeos-rel, not chromeos-amd64-generic-rel?
 
Does anyone on this list know what might be up here? That it works on all the other bots makes me wonder if something's odd with that bot somehow.

David

--
You received this message because you are subscribed to the Google Groups "python" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python+un...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/python/CAF8qwaB4Fq6B4t8p-i_ODfwXQxhzx2TjoJwikL0yWv3HBKSRMA%40mail.gmail.com.

David Benjamin

unread,
Sep 13, 2021, 9:24:15 PM9/13/21
to Takuto Ikuta, python
On Mon, Sep 13, 2021 at 9:20 PM Takuto Ikuta <tik...@chromium.org> wrote:
Hi David,

On Tue, Sep 14, 2021 at 10:05 AM David Benjamin <davi...@chromium.org> wrote:
Hi Python folks,

I'm trying to get SpawnedTestServer working with Python 3. I've mostly gotten it working, but the chromeos-amd64-generic-rel bot is upset at me. (Also the Android bots, but I've almost certainly just messed up the remote test server bits. Still need to poke at it.)

The relevant bits of the CL are here. It seems to run fine on most of the bots, except on chromeos-amd64-generic-rel.
https://chromium-review.googlesource.com/c/chromium/src/+/3156548/8/net/test/python_utils.cc

There, I get an error from vpython3, and I'm not sure what to do with it:
[E2021-09-13T16:22:33.523488-07:00 11746 0 annotate.go:241] goroutine 1:
[E2021-09-13T16:22:33.523709-07:00 11746 0 annotate.go:241] #0 go.chromium.org/luci/vpython/python/find.go:93 - python.Find()
[E2021-09-13T16:22:33.523912-07:00 11746 0 annotate.go:241]   annotation #0:
[E2021-09-13T16:22:33.524420-07:00 11746 0 annotate.go:241]     reason: no Python found
[E2021-09-13T16:22:33.524626-07:00 11746 0 annotate.go:241]   annotation #1:
[E2021-09-13T16:22:33.524847-07:00 11746 0 annotate.go:241]     reason: could not find appropriate executable for: "python3.8"
[E2021-09-13T16:22:33.525329-07:00 11746 0 annotate.go:241]   annotation #2:
[E2021-09-13T16:22:33.525553-07:00 11746 0 annotate.go:241]     reason: could not find target in system
[E2021-09-13T16:22:33.525760-07:00 11746 0 annotate.go:241]     internal reason: target(python3.8)/dirs([/usr/local/cros_test/out/Release/../../vpython_dir_linux_amd64 /usr/local/cros_test/out/Release/../../vpython_dir_linux_amd64/bin/ 

Full error here:


Where do you see the error in the link? Also the link is a build on linux-chromeos-rel, not chromeos-amd64-generic-rel?

Oops sorry, I must have copied the wrong URL. :-) Try this one:

Takuto Ikuta

unread,
Sep 13, 2021, 9:30:19 PM9/13/21
to David Benjamin, Ben Pastene, python
+Ben Pastene 

I think that test task is configured to have python2 package only around in

Ben, could you help David to pass the test?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/python/CAF8qwaA6rKXuJehJ%2BSrN5Up7Dw0beX7S%3DwK5jEmEXkGgyh_MMA%40mail.gmail.com.

Ben Pastene

unread,
Sep 14, 2021, 12:45:55 PM9/14/21
to Takuto Ikuta, David Benjamin, python
Correct. We plop down a python2 binary onto the CrOS VM during tests, but no python3 binary.

You should be able to either replace the python2 package in the file Takuto linked with a python3 version when you make the local_test_server switch. Or maybe just add python3 in addition to the python2 we're already deploying. Either option should work.

Here's the list of the available py3 packages we have:

David Benjamin

unread,
Sep 14, 2021, 2:46:56 PM9/14/21
to Ben Pastene, Takuto Ikuta, python
Thanks! I'll take a look at that. Although, it is remarkably difficult to even find the list of versions in that page. I've filed https://bugs.chromium.org/p/chromium/issues/detail?id=1249552

David Benjamin

unread,
Sep 14, 2021, 5:04:09 PM9/14/21
to Ben Pastene, Takuto Ikuta, python
Progress, I think. Now it's upset that there is no space left on the device:
https://ci.chromium.org/ui/p/chromium/builders/try/chromeos-amd64-generic-rel/b8836084245518598705/test-results

I did have to ship both, albeit for very silly reasons. net/test/python_utils.h exposes a GetPythonCommand, which is used by random tests in chrome/browser/media. So my CL keeps both GetPythonCommand and GetPython3Command. I can probably just move GetPythonCommand into the two call sites in chrome/browser/media.

While I see if that works around it... is it expected that a second vpython gets us so close to running out of disk on that bot?

Ben Pastene

unread,
Sep 14, 2021, 5:17:17 PM9/14/21
to David Benjamin, Takuto Ikuta, python
Maybe not expected, but definitely not surprising. CrOS VMs have always been tight on disk space, and it's not terribly uncommon for a new suite/test to push against the limits.

Although it's possible, increasing the partition sizes in the VMs have historically been nontrivial. So if we can safely get away with replacing the py2 binary with the py3 binary instead of shipping both, I'd suggest trying that first. But LMK if you run into complications there and I'll see what we can do.

David Benjamin

unread,
Sep 14, 2021, 7:03:16 PM9/14/21
to Ben Pastene, Takuto Ikuta, python
Reply all
Reply to author
Forward
0 new messages