Generated Javascript Protobufs and Grunt

[gunnar...@noaa.gov]

I've got some javascript generated from protoc and I'm trying to compile all those files together with closure-compiler so I can use them in a client app.

I'm using Grunt and closure compiler and can provide my gruntfile and generated javascript if it would help.

My problem is that the closure compiler gives me a namespace error for each type defined in the protobuf.

ERROR - namespace "proto.gov.noaa.alaskafisheries.demoperson.protos.Person" cannot be provided twice
goog.provide('proto.gov.noaa.alaskafisheries.demoperson.protos.Person');
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

And so on for each type defined in the .proto.

So far as I can tell these are only provided once in the generated file, and I've pared down my compilation attempt to that single file + the closure library and protobuf library files.

I've googled the heck out of it and I'm stumped.  I've gotten everything working with https://github.com/dcodeIO/ProtoBuf.js/ but I'd like to be able to use google's generated code rather than bring in another library just for the javascript client.

[Adam Cozzette]

If you could share your gruntfile that would be great. I would be interested to know in particular the protoc command used to generate the Javascript. Also it would be good to verify that the generated code for that proto did not somehow get pulled into the protobuf build and end up being part of the actual protobuf library.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe@googlegroups.com.
To post to this group, send email to prot...@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[gunnar...@noaa.gov]

Sure thing!

Gruntfile.js:

module.exports = function(grunt){

  

  require('google-closure-compiler').grunt(grunt);

  grunt.initConfig({

    pkg: grunt.file.readJSON('package.json'),

    'closure-compiler': {

      my_target: {

        files: {

          'target/full.js': ['js/**.js']

        },

        options: {

          js: [

            'node_modules/google-closure-library/**.js', 

            '!node_modules/google-closure-library/**_test.js', 

            '!node_modules/google-closure-library/**_perf.js', 

            '!node_modules/google-closure-library/**tester.js', 

            '!node_modules/google-closure-library/**promise/testsuiteadapter.js', 

            '!node_modules/google-closure-library/**osapi/osapi.js', 

            '!node_modules/google-closure-library/**svgpan/svgpan.js', 

            '!node_modules/google-closure-library/**alltests.js', 

            '!node_modules/google-closure-library/**node_modules**.js', 

            '!node_modules/google-closure-library/**protractor_spec.js', 

            '!node_modules/google-closure-library/**protractor.conf.js', 

            '!node_modules/google-closure-library/**browser_capabilities.js', 

            '!node_modules/google-closure-library/doc/**.js', 

            'node_modules/google-protobuf/**.js',

            'js/missing-google-crap/message.js',

            'js/missing-google-crap/map.js',

            'js/person.js'

          ],

          compilation_level: 'SIMPLE',

          language_in: 'ECMASCRIPT5',

          create_source_map: 'target/full.js.map'

          

        }

      }

    }

  });

  grunt.registerTask('default', ['closure-compiler']);

}

The javascript is generated via Maven, so here is the command we are using:

	    <plugin>
		   <groupId>com.github.os72</groupId>
		   <artifactId>protoc-jar-maven-plugin</artifactId>
		   <version>3.0.0</version>
		   <executions>
		       <execution>
		           <phase>generate-sources</phase>
		           <goals>
		               <goal>run</goal>
		           </goals>
		           <configuration>
		               <protocVersion>3.0.0</protocVersion> 
		               <inputDirectories>
		                   <include>src/main/resources/proto</include>
		               </inputDirectories>
		               <outputTargets>
		                   <outputTarget>
		                       <type>java</type>
		                       <addSources>none</addSources>
		                       <outputDirectory>src/main/gen</outputDirectory>
		                   </outputTarget>
		                   <outputTarget>
	 							<type>descriptor</type>
	                        	<addSources>none</addSources>
	                        	<outputDirectory>src/main/resources/protoDesc/</outputDirectory>
		                   </outputTarget>
		                   <outputTarget>
		                   		<type>js</type>
		                   		<addSources>none</addSources>
		                   		<outputDirectory>src/main/resources/js/</outputDirectory>
		                   </outputTarget>
		               </outputTargets>                
		           </configuration>
		       </execution>
		   </executions>
		</plugin>

On Wednesday, February 15, 2017 at 9:40:03 AM UTC-9, Adam Cozzette wrote:

If you could share your gruntfile that would be great. I would be interested to know in particular the protoc command used to generate the Javascript. Also it would be good to verify that the generated code for that proto did not somehow get pulled into the protobuf build and end up being part of the actual protobuf library.

On Wed, Feb 15, 2017 at 9:10 AM, <gunnar...@noaa.gov> wrote:

I've got some javascript generated from protoc and I'm trying to compile all those files together with closure-compiler so I can use them in a client app.

I'm using Grunt and closure compiler and can provide my gruntfile and generated javascript if it would help.

My problem is that the closure compiler gives me a namespace error for each type defined in the protobuf.

ERROR - namespace "proto.gov.noaa.alaskafisheries.demoperson.protos.Person" cannot be provided twice
goog.provide('proto.gov.noaa.alaskafisheries.demoperson.protos.Person');
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

And so on for each type defined in the .proto.

So far as I can tell these are only provided once in the generated file, and I've pared down my compilation attempt to that single file + the closure library and protobuf library files.

I've googled the heck out of it and I'm stumped.  I've gotten everything working with https://github.com/dcodeIO/ProtoBuf.js/ but I'd like to be able to use google's generated code rather than bring in another library just for the javascript client.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.

[gunnar...@noaa.gov]

I don't expect that the code got pulled into the protobuf library because I'm pulling the generated javascript files out of a jar created by maven and using them on a box that has been set up solely for doing the javascript side of things.  The protobuf install there is via npm, although I had to add message.js and map.js files to the closure compiler to get it working there.  It is possible I've gone about acquiring protobuf for my javascript build entirely wrong, though.

[Gunnar Gissel - NOAA Federal]

FWIW, switching to commonjs style imports seems to work.

Now I'm wondering how do I get from bytes to objects?  Passing the bytes into the generated constructor doesn't seem to work - I'm still seeing binary values where I expect to see textual values

--

Gunnar Gissel

Programmer

National Marine Fisheries

907-586-7602

[Adam Cozzette]

Ok, that's good you were able to get things working with CommonJS-style imports. To parse the raw bytes into an object you will want to call MyProto.deserializeBinary(bytes).

To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe@googlegroups.com.

[gunnar...@noaa.gov]

Thanks for the help, I appreciate it

I think it's worth noting, for those that follow in my footsteps, that when you do a GET vs a RESTful endpoint you don't get raw bytes back.  The most expedient thing I found (and please set me straight if I'm wrong), was to declare the response type to be a blob then turn the blob into an array buffer, then deserialize that

Like so:

var xhr = new XMLHttpRequest();
xhr.open("GET",'http://localhost:8080/people.proto', true);
xhr.responseType = 'blob';
xhr.onload = function(e){
  if(this.status === 200){
    var arrBuff = null;
    var fileReader = new FileReader();
    fileReader.onload = function(){
      //here we've translated the blob into something our protobuf javascript can understand
      arrBuff = this.result;
      //here we get an object to work with
      var pkg = proto.package.MyPeopleProto.deserializeBinary(arrBuff);
    }
    //here we read the blob
    fileReader.readAsArrayBuffer(this.response);
  }
}

To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.