Help with custom rule (go_rule)

[duarte....@gmail.com]

I'm trying to write a custom rule for [gqlgen][1]. The idea is to run it to generate Go code from a GraphQL schema.

My intended usage is:

```python

gqlgen(

    name = "gql-gen-foo",

    schemas = ["schemas/schema.graphql"],

    visibility = ["//visibility:public"],

)

```

"name" is the name of the rule, on which I want other rules to depend; "schemas" is the set of input files.

So far I have:

```python

load(

    "@io_bazel_rules_go//go:def.bzl",

    _go_context = "go_context",

    _go_rule = "go_rule",

)

def _gqlgen_impl(ctx):

    go = _go_context(ctx)

    args = ["run github.com/99designs/gqlgen --config"] + [ctx.attr.config]

    ctx.actions.run(

        inputs = ctx.attr.schemas,

        outputs = [ctx.actions.declare_file(ctx.attr.name)],

        arguments = args,

        progress_message = "Generating GraphQL models and runtime from %s" % ctx.attr.config,

        executable = go.go,

    )

_gqlgen = _go_rule(

    implementation = _gqlgen_impl,

    attrs = {

        "config": attr.string(

            default = "gqlgen.yml",

            doc = "The gqlgen filename",

        ),

        "schemas": attr.label_list(

            allow_files = [".graphql"],

            doc = "The schema file location",

        ),

    },

    executable = True,

)

def gqlgen(**kwargs):

    tags = kwargs.get("tags", [])

    if "manual" not in tags:

        tags.append("manual")

        kwargs["tags"] = tags

    _gqlgen(**kwargs)

```

My immediate issue is that Bazel complains that the schemas are not `Files`:

```

expected type 'File' for 'inputs' element but got type 'Target' instead

``` 

What's the right approach to specify the input files?

Is this the right approach to generate a rule that executes a command?

Finally, is it okay to have the output file not exist in the filesystem, but rather be a label on which other rules can depend?

Pointers to relevant documentation are appreciated. Thanks!

  [1]: https://github.com/99designs/gqlgen

[Paul Johnston]

The label list for schemas will be on the "files' attribute of the ctx object when your implementation function is invoked, so you'll want to write:

inputs = ctx.files.schemas

instead of:

inputs = ctx.attr.schemas

[duarte....@gmail.com]

Thanks! I've made some progress, but Bazel is now exiting on an unhandled exception.

The code evolved to [1]. When executing bazel build //package:gql-gen, I get:

Internal error thrown during build. Printing stack trace: java.lang.RuntimeException: Unrecoverable error while evaluating node '//package:gql-gen BuildConfigurationValue.Key[b67bdc290fb31b127eacf7a9889db436] false' (requested by nodes )

at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:528)

at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:399)

at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(Unknown Source)

at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)

at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source)

at java.base/java.util.concurrent.ForkJoinPool.scan(Unknown Source)

at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)

at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)

Caused by: java.lang.ClassCastException: class com.google.devtools.build.lib.actions.Artifact$SourceArtifact cannot be cast to class com.google.devtools.build.lib.actions.Artifact$DerivedArtifact (com.google.devtools.build.lib.actions.Artifact$SourceArtifact and com.google.devtools.build.lib.actions.Artifact$DerivedArtifact are in unnamed module of loader 'app')

at com.google.devtools.build.lib.actions.Actions.assignOwnersAndMaybeFilterSharedActionsAndThrowIfConflict(Actions.java:228)

at com.google.devtools.build.lib.actions.Actions.assignOwnersAndFilterSharedActionsAndThrowActionConflict(Actions.java:157)

at com.google.devtools.build.lib.analysis.RuleConfiguredTargetBuilder.build(RuleConfiguredTargetBuilder.java:227)

at com.google.devtools.build.lib.analysis.skylark.SkylarkRuleConfiguredTargetUtil.createTarget(SkylarkRuleConfiguredTargetUtil.java:226)

at com.google.devtools.build.lib.analysis.skylark.SkylarkRuleConfiguredTargetUtil.buildRule(SkylarkRuleConfiguredTargetUtil.java:148)

at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createRule(ConfiguredTargetFactory.java:315)

at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createConfiguredTarget(ConfiguredTargetFactory.java:182)

at com.google.devtools.build.lib.skyframe.SkyframeBuildView.createConfiguredTarget(SkyframeBuildView.java:776)

at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.createConfiguredTarget(ConfiguredTargetFunction.java:891)

at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.compute(ConfiguredTargetFunction.java:351)

at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:451)

... 7 more

Is this a known issue? 

Thanks.

[1] https://gist.github.com/duarten/5c8ca4b45f593e54a435f92f279ec71f

[Alex Humesky]

I don't think we have an issue for this crash, I filed https://github.com/bazelbuild/bazel/issues/9182

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/4e435da7-6b8c-43fa-94f5-da46af52b37c%40googlegroups.com.

[Alex Humesky]

Looking at the code history, it looks like this may have been introduced in 0.27, so one workaround is to try to use 0.26 if possible

[Alex Humesky]

Looking through the provided repro code, the thing that might be triggering this bug is using the parser, resolver, and model attributes as outputs:

https://gist.github.com/duarten/5c8ca4b45f593e54a435f92f279ec71f#file-gqlgen-bzl-L84-L98

Those are expected to be inputs to the rule, not outputs. For that you want to use attr.output or attr.output_list:

https://docs.bazel.build/versions/master/skylark/lib/attr.html#output

https://docs.bazel.build/versions/master/skylark/lib/attr.html#output_list

more information is here:

https://github.com/bazelbuild/bazel/issues/7977

Also, unrelatedly, it's not totally clear to me that you need to use go_rule() and go_context(), since all that appears to need to happen here is writing a file and calling the binary at @com_github_99designs_gqlgen//:gqlgen

[Duarte Nunes]

Thanks, Alex! Changing those attributes to be outputs fixed this particular issue.

I'm using go_rule() and go_context() to access to the go tool at runtime. There's probably a better way to do that which I'm missing.

Writing the actions as

ctx.actions.run(
        tools = [],
        inputs = inputs,
        outputs = out_files,
        arguments = ["--config", config_file.path],
        executable = ctx.executable._gqlgen,
    )

Results in:

 gqlgen failed: error executing command
  (cd /private/var/tmp/_bazel_duarten/18d4d4fe625c2f83e91dbc80b9b94787/sandbox/darwin-sandbox/372/execroot/playground && \
  exec env - \
  bazel-out/host/bin/external/com_github_99designs_gqlgen/darwin_amd64_stripped/gqlgen --config bazel-out/darwin-fastbuild/bin/package/gql-gen.yml)

modelgen: 'go list' driver requires 'go', but executable file not found in $PATH

I realized I need to pass both go.go as a tool and go.env, like so:

env = {
        "PATH": go.env["PATH"] + ":{}/bin".format(go.env["GOROOT"]),
}

env.update(go.env)

ctx.actions.run(
        tools = [go.go],
        inputs = inputs,
        outputs = out_files,
        arguments = ["--config", config_file.path],
        executable = ctx.executable._gqlgen,
        env = go.env,
    )

This solves the previous error and gets me to:

 gqlgen failed: error executing command
  (cd /private/var/tmp/_bazel_duarten/18d4d4fe625c2f83e91dbc80b9b94787/sandbox/darwin-sandbox/373/execroot/playground && \
  exec env - \
    CGO_ENABLED=1 \
    GO11MODULE=on \
    GOARCH=amd64 \
    GOOS=darwin \
    GOROOT=external/go_sdk \
    GOROOT_FINAL=GOROOT \
    PATH='$PATH:external/go_sdk/bin/' \
  bazel-out/host/bin/external/com_github_99designs_gqlgen/darwin_amd64_stripped/gqlgen --config bazel-out/darwin-fastbuild/bin/package/gql-gen.yml)

modelgen: could not determine GOARCH and Go compiler

Executing 

  ctx.actions.run_shell(
        outputs = out_files,
        command = """
            export GOCACHE=$PWD/.gocache &&
            go list -f "{{context.GOARCH}} {{context.Compiler}}" -- unsafe
        """,
    )

Does give me the correct result, so it's not clear to me what's happening with "ctx.actions.run".

[Alex Humesky]

I don't know much about go, so unfortunately the specifics are beyond me, but in general the "binary rules" like cc_binary, java_binary, py_binary, etc, are supposed to produce things that you can "just run". It looks like @com_github_99designs_gqlgen//:gqlgen is a go_binary, so theoretically you should be able to do things like:

bazel run @com_github_99designs_gqlgen//:gqlgen -- <args>

and things like:

genrule(

  name = "gen_foo",

  outs = ["foo"],

  tools = ["@com_github_99designs_gqlgen//:gqlgen"],

  cmd = "$(location @com_github_99designs_gqlgen//:gqlgen) <args>",

)

What happens when you do the "bazel run" above?

[Duarte Nunes]

Those fail a little bit further with "merging failed: unable to find type github.com/99designs/gqlgen/graphql.Boolean". I'm don't know much about go either, but it seems some imports can't be satisfied. 

It works when I just do  bazel-bin/external/com_github_99designs_gqlgen/linux_amd64_stripped/gqlgen <args>.

Adding this debug information to the action:

   env = {
            "GOCACHE": "/tmp",
            "GOPACKAGESPRINTGOLISTERRORS": "1",
            "GOPACKAGESDEBUG": "1"
    }
    env.update(go.env)
    env["PATH"] = go.env["PATH"] + ":{}/bin".format(go.env["GOROOT"])

    ctx.actions.run(
        tools = [go.go],
        inputs = inputs,
        outputs = out_files,
        arguments = ["--config", config_file.path],
        executable = ctx.executable._gqlgen,

        env = env,
    )

I can see that:

GOROOT=external/go_sdk GOPATH= GO111MODULE= PWD= go [list -f {{context.GOARCH}} {{context.Compiler}} -- unsafe] stderr: <<can't load package: package unsafe: cannot find package "unsafe" in any of:
        external/go_sdk/src/unsafe (from $GOROOT)
        ($GOPATH not set. For more details see: 'go help gopath')

>>
modelgen: could not determine GOARCH and Go compiler

Again it seems to be an issue of unmet dependencies.

[Alex Humesky]

Perhaps try reaching out to bazel-go-discuss, someone there might know what's up:

https://groups.google.com/forum/#!forum/bazel-go-discuss

[Duarte Nunes]

Thanks! I'll give it a try.

[satee...@uber.com]

Duarte,

How did you resolve "merging failed: unable to find type github.com/99designs/gqlgen/graphql.Boolean"?

[Duarte Nunes]

I don't recall coming across that error. I've since stopped using gqlgen, so maybe something changed in the versions in-between.

--
You received this message because you are subscribed to a topic in the Google Groups "bazel-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/bazel-discuss/6I6Af4JgtgE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/ce6cd134-48f2-4fa5-bc2d-cf30390d2035o%40googlegroups.com.