Tool for programmatically adding field options in .proto files

[Mark Hoffman]

Hi,

I have the tasks of retrofitting a large repo of .proto files to include a field option with a default value to every field, except for enums. We can't assume if the field option is not present that it has a default, for security reasons. Before I go off an write a parser to handle this one time task, I was justing wondering something might already exist. One other issue, I need to preserve the comments in the .proto files. 

Thanks.

[Adam Cozzette]

I don't know of any existing tool for doing this, but you could try something like this:

1. Run "protoc --descriptor_set_out=descriptors.out ..." to parse each .proto file and serialize the descriptors to a file.

2. Write a simple program to read the serialized descriptors from step 1 and determine which fields you need to edit.

3. Use a regex search-and-replace to edit the necessary fields in the original .proto files. This step may be the hardest because I can imagine it may be tough to come up with a regex that can handle all formatting variations, but you could also try passing --include_source_info in step 1 and that would give you information about the location of each field in the source .proto file.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.
To post to this group, send email to prot...@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[Josh Humphries]

One possible solution is to use a bits of the protoreflect library for Go that I've written:

https://godoc.org/github.com/jhump/protoreflect/desc/protoparse

When parsing, you'll want to use the flag to include source code info. This will preserve a little more comment info than descriptor produced by protoc. FWIW, the descriptor with source info can be had from protoc with an invocation like so:
protoc -o outputfile --include_source_info --include_imports some.proto

Once the files are parsed, you can easily recursively visit all of the fields and modify the field descriptors to add a default value.

https://godoc.org/github.com/jhump/protoreflect/desc/protoprint
Once the option is added, you can write the modified descriptors back to source files with this package. This will preserve a lot of comment info (though admittedly not all).

The main downside is that this in no way preserves formatting from the original file. The formatting produced isn't bad, but it doesn't quite look like hand-written code due to the extra whitespace you'll see (blank line between every element). To preserve formatting, you probably will need to write a custom parser.

YMMV

----
Josh Humphries
jh...@bluegosling.com

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe@googlegroups.com.

[Mark Hoffman]

Thanks for the suggestions. I started playing with just using regexes to rewrite the .proto files and this seems to work.