Goma server is now open source

[Fumitoshi Ukai (鵜飼文敏)]

Hi,

We released goma client on January last year.  Today, we’re pleased to announce the release of open source goma server code at https://chromium.googlesource.com/infra/goma/server (go.chromium.org/goma/server), which uses Remote Execution API as backend. (i.e. Goma server is a client of Remote Execution API).

We’re testing this with Remote Build Execution (Alpha), Cloud Storage and Redis on Google Cloud Platform.

Historically, it has been built up on our own execution worker, but we’re working to converge API to Remote Execution API and planning to change goma client to talk RE API directly in future.

If you're interested, Yoshisato of Goma team will talk about Goma internal in BlinkOn 10 Day 2. Please come to our session to learn more about Goma.

Cheers,

ukai, on behalf of goma team

--

FAQ?

Q: Will Google provide a managed service?

A: We're currently in closed beta with a few trusted external contributors and are evaluating the feasibility of wider availability.  

Q: Will there be mac and windows support?

A: Current implementation works on Linux only but will send an update if or when that changes.

[br...@amazon.com]

Fantastic news!  Is there any ETA for an update on https://cloud.google.com/remote-build-execution/docs/tutorial-using-goma-with-rbe with detailed instructions on using Goma with Google Cloud RBE?

[Erik Staab]

bryct: the instructions should be available now if you join https://groups.google.com/forum/#!forum/rbe-alpha-customers (they weren't published initially).

And congratulations to Goma team on open sourcing the server!

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/b5c80408-3313-4d2b-8c0e-5ef2300156f4%40chromium.org.

[Victor Costan]

Congratulations on getting the Goma open-sourcing work across the finish line!

    Victor

--

--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAE8n1ndHuCbGYfci6EuLPrsLG7h9aOZ3VfX%2BzEXOFRpyk3UPgQ%40mail.gmail.com.

[Ion Lupascu]

Thank for this good news!

[傅立盛]

Please ,whether it can be solved,I don't know what the problem is ？

1. ./remoteexec_proxy --port 7005 --remoteexec-addr 127.0.0.1 --remote-instance-name goma1  --whitelisted-users x...@gmail.com

2.

root@fls-VB:~# $HOME/goma/goma_ctl.py start

I0529 12:39:51.625942 29420 mypath_helper_linux.cc:22] lstat failed. dir=/run/user/0: No such file or directory [2]

E0529 12:39:51.626085 29420 mypath.cc:204] bad username:unknown

using /tmp/goma_unknown as tmpdir

GOMA version 6f3884c89b6a0544d1fb07ac7879f5d8e3fac473@1558929100

compiler proxy (pid=29419,11615) status: http://127.0.0.1:8088 error: access to backend servers was rejected.

Failed to start compiler_proxy.

Login as x...@gmail.com

Current user is not registered with Goma service at http://localhost:7005. Unable to use Goma

Error：(authentication? configure ?)

2019-05-29T20:39:56.420+0800    WARN    auth/client.go:72       permission denied: internal error       {"logging.googleapis.com/trace": "c798a4131fa55eb0d7856c1ecf629214", "logging.googleapis.com/spanId": "142fff90f821b9c2"}

在 2019年4月10日星期三 UTC+8下午12:43:42，Fumitoshi Ukai写道：

[Yoshisato Yanagisawa]

2019年5月30日(木) 8:12 傅立盛 <flsy...@gmail.com>:

Please ,whether it can be solved,I don't know what the problem is ？

1. ./remoteexec_proxy --port 7005 --remoteexec-addr 127.0.0.1 --remote-instance-name goma1  --whitelisted-users x...@gmail.com

2.

root@fls-VB:~# $HOME/goma/goma_ctl.py start 

I0529 12:39:51.625942 29420 mypath_helper_linux.cc:22] lstat failed. dir=/run/user/0: No such file or directory [2]

E0529 12:39:51.626085 29420 mypath.cc:204] bad username:unknown

using /tmp/goma_unknown as tmpdir

Please do not use Goma client with root user.  It is discouraged for security reason, and we won't support such use case.

 

GOMA version 6f3884c89b6a0544d1fb07ac7879f5d8e3fac473@1558929100

compiler proxy (pid=29419,11615) status: http://127.0.0.1:8088 error: access to backend servers was rejected.

Failed to start compiler_proxy.

Login as x...@gmail.com

Current user is not registered with Goma service at http://localhost:7005. Unable to use Goma

 

Error：(authentication? configure ?)

2019-05-29T20:39:56.420+0800    WARN    auth/client.go:72       permission denied: internal error       {"logging.googleapis.com/trace": "c798a4131fa55eb0d7856c1ecf629214", "logging.googleapis.com/spanId": "142fff90f821b9c2"}

"internal error" sounds like coming from here:

https://chromium.googlesource.com/infra/goma/server/+/030736e0c390c356f8361c64a9a45e4111f765cc/auth/service.go#169

Do you see ERROR log to explain the reason of token info error?

 

在 2019年4月10日星期三 UTC+8下午12:43:42，Fumitoshi Ukai写道：

Hi,

We released goma client on January last year.  Today, we’re pleased to announce the release of open source goma server code at https://chromium.googlesource.com/infra/goma/server (go.chromium.org/goma/server), which uses Remote Execution API as backend. (i.e. Goma server is a client of Remote Execution API).

We’re testing this with Remote Build Execution (Alpha), Cloud Storage and Redis on Google Cloud Platform.

Historically, it has been built up on our own execution worker, but we’re working to converge API to Remote Execution API and planning to change goma client to talk RE API directly in future.

If you're interested, Yoshisato of Goma team will talk about Goma internal in BlinkOn 10 Day 2. Please come to our session to learn more about Goma.

Cheers,

ukai, on behalf of goma team

--

FAQ?

Q: Will Google provide a managed service?

A: We're currently in closed beta with a few trusted external contributors and are evaluating the feasibility of wider availability.  

Q: Will there be mac and windows support?

A: Current implementation works on Linux only but will send an update if or when that changes.

--

--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/691b17bc-0b9a-4b66-9601-1e3bfd6ab7f3%40chromium.org.

[傅立盛]

Hello,

Thanks ,your answer . 

Current  Goma client is not registered with Goma service.

Following some questions:

Current use browser to get Goma OAuth2  token code . Whether the Goma server has a list of users config file , needs to match with the client user ?

1. Is there an example  ?  As for ,is Auth token  configuration file case?

在 2019年5月30日星期四 UTC+8上午8:45:13，Yoshisato Yanagisawa写道：

To unsubscribe from this group and stop receiving emails from it, send an email to chromi...@chromium.org.

[Yoshisato Yanagisawa]

2019年5月30日(木) 20:03 傅立盛 <flsy...@gmail.com>:

Hello,

Thanks ,your answer . 

Current  Goma client is not registered with Goma service.

Following some questions:

Current use browser to get Goma OAuth2  token code . Whether the Goma server has a list of users config file , needs to match with the client user ?

1. Is there an example  ?  As for ,is Auth token  configuration file case?

I do not think you need web browser to get OAuth2 refresh token.  If you want to do that, please set --browser option.

i.e.

$ goma_auth.py login --browser

If you use remotexec_proxy, you might need to give users' email via its command line argument.  The parameter accepts comma separated values.

https://chromium.googlesource.com/infra/goma/server/+/030736e0c390c356f8361c64a9a45e4111f765cc/cmd/remoteexec_proxy/main.go#65

e.g.

$ {GOPATH}/bin/remoteexec_proxy \

  --port [GOMA_FRONTEND_PORT] \
  --remoteexec-addr remotebuildexecution.googleapis.com:443 \
  --remote-instance-name \
    projects/[PROJECT_ID]/instances/default_instance \
  --service-account-json [SERVICE_ACCOUNT_KEY] \
  --platform-container-image [DOCKER_IMAGE] \

--whitelisted-users f...@gmail.com,b...@gmail.com,b...@gmail.com

If you want to make remoteexec_proxy read whitelist from the file, you may need to modify the following code:

https://chromium.googlesource.com/infra/goma/server/+/030736e0c390c356f8361c64a9a45e4111f765cc/cmd/remoteexec_proxy/main.go#221

remoteexec_proxy allows the access if an email for OAuth2 access token matches the whitelist.  The email address come from which user you logged in with "goma_auth.py login".

I hope I answered your questions.

 

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/873ec53c-6fd3-46e6-9c6b-08a82c2c1369%40chromium.org.