dazel + bazel + buildgrid: The remote cache is in an invalid state as it served a failed action.

[aa...@aaronbentley.com]

I am trying out BuildGrid with Bazel, and I consistently get "The remote cache is in an invalid state as it served a failed action."

When I run my build command the first time, it fails due to a missing file. All subsequent runs fail with "The remote cache is in an invalid state as it served a failed action.", and this is the thing I am most concerned about.

Is this a known issue?

This is with Bazel 0.19.0 and BuildGrid 0.0.2.

(If you're not familiar with Dazel, it's wrapper to run Bazel in Docker, and supports docker-compose.)

Here's my controller config:
server:
- !channel
port: 50051
insecure_mode: true

instances:
- name: main

storages:
- !disk-storage &main-storage
path: /tmp/buildgrid

services:
- !action-cache &main-action
storage: *main-storage
max_cached_refs: 256
allow_updates: true

- !reference-cache
storage: *main-storage
max_cached_refs: 512
allow_updates: true

- !execution
storage: *main-storage
action_cache: *main-action

- !cas
storage: *main-storage

- !bytestream
storage: *main-storage

Here's my docker-compose:
---
version: "3"

services:
bgd-controller:
image: ssimwavedocker/buildgrid:08-Nov-2018
volumes:
- ./config:/buildgrid/config:ro
environment:
- LC_ALL=C.UTF-8
- LANG=C.UTF-8
ports:
- 50051:50051
command: ["/buildgrid/config/controller.yml"]

bgd-worker1:
image: ssimwavedocker/buildgrid:08-Nov-2018
depends_on:
- bgd-controller
environment:
- LC_ALL=C.UTF-8
- LANG=C.UTF-8
entrypoint: ["bgd", "bot"]
command: ["--verbose", "--remote", "http://bgd-controller:50051", "--remote-cas", "http://bgd-controller:50051", "--parent", "main", "host-tools"]

[Jakob Buchgraber]

+Finn, Martin, Juerg

Hi Aaron,

thanks for the report. There seems to be an incompatibility problem between Bazel's implementation of the API and BuildGrids implementation. Bazel doesn't allow failed actions to be cached, while BuildGrid apparently does cache failed actions. Does that sound accurate Finn, Martin and Juerg?

Thanks,

Jakob

Jakob Buchgraber

Software Engineer

Google Germany GmbH

Erika-Mann-Straße 33

80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Diese E-Mail ist vertraulich. Falls sie diese fälschlicherweise erhalten haben sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail an die falsche Person gesendet wurde.

    

This e-mail is confidential. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/9e56f54a-70ed-4465-a9e5-3609f09a91ef%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[martin.b...@codethink.co.uk]

Hello Aaron, hello Jakob,

Indeed, it seems like BuildGrid is caching a failed action and this should not happened. I haven't seen that recently but I haven't tested Bazel 0.19.0. Feel free to open on issue on BuildGrid's bugtracker. Sharing the code you are trying to build would greatly help, if that's possible.

Cheers,

Martin

[Finn Ball]

Hi,

At the moment it is assumed that if the lease fails due to the bot, then the result is not cached. If the bot manages to execute the command, then it will cache this result.

Thanks,
Finn