message forward declaration

[George]

Hi,

It looks to me that the generated result of a single proto file
doesn’t change significantly based on the content of the imported
proto files, but mostly on the fact that it is imported.

Did you have considered replacing the need of actual import with some
kind of message forward declaration?

In my opinion having a single proto file self-sufficient even in case
it actually refers messages from other proto files could significantly
simplify the build process.

What the protobuf community thinks about this?

Thanks,
George

[Jason Hsueh]

There are many things that need to be read from imported .proto files to determine if the .proto is valid, or to produce correct code. e.g.:

- need to differentiate between enum and message imports

- when referencing a qualified type like foo.bar.Baz.Qux, you need to know what components are package specifiers, and which are objects. This changes the forward declaration scheme: if that was package foo.bar; message Bar { message Qux { } }, the declaration is namespace foo { namespace bar { class Bar_Qux; } }, whereas maybe Bar is just a namespace, so you should instead produce namespace foo { namespace bar { namespace Bar { class Qux; } } }

- options like java_multiple_files affects the generated code

- when defining extensions, you need to know whether the extendee accepts extensions, and what range of extension numbers the extendee allows

To use the .proto, you need to build the imported proto files anyway, so even if you could capture all of this in a simple forward-declaration-type scheme, I don't think you would save very much.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To post to this group, send email to prot...@googlegroups.com.
To unsubscribe from this group, send email to protobuf+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.

[George]

Hi,

Sorry for the delayed response I was out a few days.

Determine if the .proto is valid: if we are not importing
another .proto but just having forward message declaration this
doesn’t apply.

I don’t think if we agree that the forward concept is valuable it will
be a problem to find a protocol that will allow specifying all of the
needed information. It doesn’t look so much for me.

> To use the .proto, you need to build the imported proto files anyway, so
even if you could capture all of this in a simple forward-declaration-
type
scheme, I don't think you would save very much.

You will be able to see saving in this relation only if you start
thinking in multiple layers. If we have a single project this feature
obviously doesn’t make so much sense, but imagine you have a library L
that is used from Products P1 and P2. Let’s have some proto files
declared and compiled in L.

With the current functionality if I want to use a message declared in
L from a P1 I will need to deploy not only the language specific
interface say *.h file, but also the proto file itself. This obviously
complicates the building process of P1 to obtain this additional
dependency and to specify the location and so on. Imagine what it will
be if you have a multiple libraries.

Another extra this feature will provide is the possibility I to
provide a different message declaration from P1 and from P2.
For example if I have now
message Addressee {
…
}

message Request {
required Addressee addressee =1;
required bytes command =1;
}

Where command is serialized protobuf message binary with forward
declaration I will be able to do:
forward message Command;
message Request {
required Addressee addressee =1;
required Command command =1;
}
And to define different Command protobuf message in P1 and P2.

Thanks,
George

> > protobuf+u...@googlegroups.com<protobuf%2Bunsubscribe@googlegroups.c om>
> > .

[Henner Zeller]

On Wed, Jan 26, 2011 at 15:16, George <george....@hotmail.com> wrote:
> Hi,
>
> Sorry for the delayed response I was out a few days.
>
> Determine if the .proto is valid: if we are not importing
> another .proto but just having forward message declaration this
> doesn’t apply.
>
> I don’t think if we agree that the forward concept is valuable it will
> be a problem to find a protocol that will allow specifying all of the
> needed information. It doesn’t look so much for me.
>
>> To use the .proto, you need to build the imported proto files anyway, so
> even if you could capture all of this in a simple forward-declaration-
> type
> scheme, I don't think you would save very much.
>
> You will be able to see saving in this relation only if you start
> thinking in multiple layers. If we have a single project this feature
> obviously doesn’t make so much sense, but imagine you have a library L
> that is used from Products P1 and P2. Let’s have some proto files
> declared and compiled in L.
>
> With the current functionality if I want to use a message declared in
> L from a P1 I will need to deploy not only the language specific
> interface say *.h file, but also the proto file itself.

Well, doesn't it make sense to deploy the _source_ file of your
protocol buffer, not the language specific thing it is compiled into
(the *.cc,*.h or *.java) ? After all, this is the build process, and
deploying the *.h/*.cc is more complicated than the source file.

Often it is good to take a step back and think _why_ one would like to
have a specific feature. While in some cases there is a need for
forward declarations (e.g. cyclic dependencies in systems that don't
resolve these on compilation unit level), this is not the problem you
want to solve. Your problem is that you want to cut dependencies by
underspecifying the information not necessary for 'the other user'.

If you are indeed in that situation, that requires P1 and P2 not mess
with (or even know) each others' protocol buffers, then what you
actually mean to use are extensions. That way, P1 and P2 can work on
their part of the protocol buffers without even knowing the details of
the other. http://code.google.com/apis/protocolbuffers/docs/proto.html#extensions

In general it is impractical to have protocol buffers forward
declarations because you wouldn't be able to generate code for most of
the target languages (C++ could be made to work).

> To unsubscribe from this group, send email to protobuf+u...@googlegroups.com.

[George]

I agree it doesn’t make sense to deploy *.java or /.cc files. What I
meant to deploy is *.h + precompiled .lib files or in java case it
will be *.class packaged in a jar or similar.

Anyway, just an idea.

On Jan 26, 3:37 pm, Henner Zeller <henner.zel...@googlemail.com>
wrote:

> the other.http://code.google.com/apis/protocolbuffers/docs/proto.html#extensions

> >> > protobuf+u...@googlegroups.com<protobuf%2Bunsubscr...@googlegroups.c om>