go-protobuf example with self-describing message ?

[Sebastien Binet]

hi there,

any chance somebody has a readily available example in go (of course!)
for the self-describing message technique evocated over there:

https://developers.google.com/protocol-buffers/docs/techniques#self-description

I'd like to add it to my little example:

https://github.com/sbinet/go-pb-example/blob/master/go-pb-example/main.go

(and eventually, if there is enough interest, push it to the official
go-protobuf tutorial...)

-s

[Kyle Lemons]

For what would you need such functionality? I definitely don't see it as
being something that's common enough to be in a tutorial. One of the
advantages of protocol buffers is that both sides know the structure of the
message... In any case, you would not be able to build a Go struct at
run-time that matched the described protobuf, though you could
theoretically use map[string]interface{} in a similar way to what JSON does.

Post by Sebastien Binet
hi there,
any chance somebody has a readily available example in go (of course!)
https://developers.google.com/protocol-buffers/docs/techniques#self-description
https://github.com/sbinet/go-pb-example/blob/master/go-pb-example/main.go
(and eventually, if there is enough interest, push it to the official
go-protobuf tutorial...)
-s

[Sebastien Binet]

Kyle,

Post by Kyle Lemons
For what would you need such functionality?

the plan is to at least be able to generate such self-describing
messages from Go and then consume them in C++ or python.

Post by Kyle Lemons
I definitely don't see it as
being something that's common enough to be in a tutorial. One of the
advantages of protocol buffers is that both sides know the structure of the
message... In any case, you would not be able to build a Go struct at
run-time that matched the described protobuf, though you could theoretically
use map[string]interface{} in a similar way to what JSON does.

there is also the possibility to have a 2-tier process, with 2
processes, one generating the code of the other one :)

my use case is converting data written in a high energy physics -based
file format to/from protobuf files and compare the figure of merit of
these 2 file formats (r/w speed, size, convenience, ...)

-s

[Kyle Lemons]

If you use the self describing proto to which you linked, you can get the
FileDescriptorProto from "protoc -oFILE" and then decoding that file. You
can see the source for
protoc-gen-go<https://code.google.com/p/goprotobuf/source/browse/#hg/protoc-gen-go>if
you want to see how the code is generated or re-wrap it.

Post by Sebastien Binet
Kyle,

Post by Kyle Lemons
For what would you need such functionality?

the plan is to at least be able to generate such self-describing
messages from Go and then consume them in C++ or python.

Post by Kyle Lemons
I definitely don't see it as
being something that's common enough to be in a tutorial. One of the
advantages of protocol buffers is that both sides know the structure of

the

Post by Kyle Lemons
message... In any case, you would not be able to build a Go struct at
run-time that matched the described protobuf, though you could

theoretically

Post by Kyle Lemons
use map[string]interface{} in a similar way to what JSON does.

there is also the possibility to have a 2-tier process, with 2
processes, one generating the code of the other one :)
my use case is converting data written in a high energy physics -based
file format to/from protobuf files and compare the figure of merit of
these 2 file formats (r/w speed, size, convenience, ...)
-s

[Sebastien Binet]

Kyle,

Post by Kyle Lemons
If you use the self describing proto to which you linked, you can get the
FileDescriptorProto from "protoc -oFILE" and then decoding that file. You
can see the source for
protoc-gen-go<https://code.google.com/p/goprotobuf/source/browse/#hg/protoc-gen-go>if
you want to see how the code is generated or re-wrap it.

thanks, that helped me bootstrap my scaffolding.

I noticed that once one enables the filedescriptor generation, one gets
this dubious go-import in the xyz.pb.go file:

import google_protobuf "google/protobuf/descriptor.pb"

which should read:
import google_protobuf "code.google.com/p/goprotobuf/protoc-gen-go/descriptor"


couldn't quickly find where it came from...
did I forget to do something ?

-s

[Kyle Lemons]

Post by Kyle Lemons
Kyle,

Post by Kyle Lemons
If you use the self describing proto to which you linked, you can get the
FileDescriptorProto from "protoc -oFILE" and then decoding that file.

You

Post by Kyle Lemons
can see the source for
protoc-gen-go<

https://code.google.com/p/goprotobuf/source/browse/#hg/protoc-gen-go>if

Post by Kyle Lemons
you want to see how the code is generated or re-wrap it.

thanks, that helped me bootstrap my scaffolding.
I noticed that once one enables the filedescriptor generation, one gets
import google_protobuf "google/protobuf/descriptor.pb"

That, I believe, is based on the import in your self-describing proto file.

Post by Kyle Lemons
import google_protobuf "
code.google.com/p/goprotobuf/protoc-gen-go/descriptor"

Actually, that file is not the canonical one; this is:
http://code.google.com/p/protobuf/source/browse/trunk/src/*
google/protobuf/descriptor*.proto<http://code.google.com/p/protobuf/source/browse/trunk/src/google/protobuf/descriptor.proto>
You'll probably need to edit the import before you check it in, because the
protoc tool probably can't be taught to do it right.

couldn't quickly find where it came from...

Post by Kyle Lemons
did I forget to do something ?
-s
--
#########################################
# Dr. Sebastien Binet
# Laboratoire de l'Accelerateur Lineaire
# Universite Paris-Sud XI
# Batiment 200
# 91898 Orsay
#########################################

--

[Sebastien Binet]

Post by Kyle Lemons

Post by Kyle Lemons
Kyle,

Post by Kyle Lemons
If you use the self describing proto to which you linked, you can get the
FileDescriptorProto from "protoc -oFILE" and then decoding that file.

You

Post by Kyle Lemons
can see the source for
protoc-gen-go<

https://code.google.com/p/goprotobuf/source/browse/#hg/protoc-gen-go>if

Post by Kyle Lemons
you want to see how the code is generated or re-wrap it.

thanks, that helped me bootstrap my scaffolding.
I noticed that once one enables the filedescriptor generation, one gets
import google_protobuf "google/protobuf/descriptor.pb"

That, I believe, is based on the import in your self-describing proto file.

I have this little template:

``` go
const pb_pkg_templ = `package {{.Package}};

import "google/protobuf/descriptor.proto";

message {{.Message}} {
extensions 50000 to max;
{{with .Fields}}
{{range .}} {{.Modifier}} {{.Type}} {{.Name}} = {{.Id}}{{.Attr}};
{{end}}
{{end}}
}

extend google.protobuf.FieldOptions {
optional string root_branch = 50002;
}

message DataHeader {
// Set of .proto files which define the type.
required google.protobuf.FileDescriptorSet proto_files = 1;

// number of entries in the payload message
required int64 NbrEntries = 2;
}
`
```

which looks like what you pointed at...

-s