diff options
author | Russ Cox <rsc@golang.org> | 2020-03-09 23:54:35 -0400 |
---|---|---|
committer | Russ Cox <rsc@golang.org> | 2020-03-17 20:58:37 +0000 |
commit | af5018f64e406aaa646dae066f28de57321ea5ce (patch) | |
tree | 8db7b1f049d83d215fa9abf68851efce7b5ccadb /content/a-new-go-api-for-protocol-buffers.article | |
parent | 86e424fac66fa90ddcb7e8d7febd4c2b07d7c59e (diff) |
content: convert to Markdown-enabled present inputs
Converted blog to Markdown-enabled present (CL 222846)
using present2md (CL 222847).
For golang/go#33955.
Change-Id: Ib39fa1ddd9a46f9c7a62a2ca7b96e117635553e8
Reviewed-on: https://go-review.googlesource.com/c/blog/+/222848
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Diffstat (limited to 'content/a-new-go-api-for-protocol-buffers.article')
-rw-r--r-- | content/a-new-go-api-for-protocol-buffers.article | 79 |
1 files changed, 40 insertions, 39 deletions
diff --git a/content/a-new-go-api-for-protocol-buffers.article b/content/a-new-go-api-for-protocol-buffers.article index f3afe0a..2c5c014 100644 --- a/content/a-new-go-api-for-protocol-buffers.article +++ b/content/a-new-go-api-for-protocol-buffers.article @@ -1,19 +1,20 @@ -A new Go API for Protocol Buffers -02 Mar 2020 +# A new Go API for Protocol Buffers +2 Mar 2020 Tags: protobuf, technical +Summary: We are pleased to announce the release of a major revision of the Go API for [protocol buffers](https://developers.google.com/protocol-buffers), Google's language-neutral data interchange format. Joe Tsai, Damien Neil, and Herbie Ong -* Introduction +## Introduction We are pleased to announce the release of a major revision of the Go API for -[[https://developers.google.com/protocol-buffers][protocol buffers]], +[protocol buffers](https://developers.google.com/protocol-buffers), Google's language-neutral data interchange format. -* Motivations for a new API +## Motivations for a new API The first protocol buffer bindings for Go were -[[https://blog.golang.org/third-party-libraries-goprotobuf-and][announced by Rob Pike]] +[announced by Rob Pike](https://blog.golang.org/third-party-libraries-goprotobuf-and) in March of 2010. Go 1 would not be released for another two years. In the decade since that first release, the package has grown and @@ -21,7 +22,7 @@ developed along with Go. Its users' requirements have grown too. Many people want to write programs that use reflection to examine protocol buffer messages. The -[[https://pkg.go.dev/reflect][`reflect`]] +[`reflect`](https://pkg.go.dev/reflect) package provides a view of Go types and values, but omits information from the protocol buffer type system. For example, we might want to write a function that traverses a log entry and @@ -33,7 +34,7 @@ generated by the protocol buffer compiler, such as a dynamic message type capable of representing messages whose type is not known at compile time. We also observed that a frequent source of problems was that the -[[https://pkg.go.dev/github.com/golang/protobuf/proto?tab=doc#Message][`proto.Message`]] +[`proto.Message`](https://pkg.go.dev/github.com/golang/protobuf/proto?tab=doc#Message) interface, which identifies values of generated message types, does very little to describe the behavior of those types. When users create types that implement that interface (often inadvertently by embedding a message @@ -52,11 +53,11 @@ protobuf module. Today, we're pleased to release that new module. We hope you like it. -* Reflection +## Reflection Reflection is the flagship feature of the new implementation. Similar to how the `reflect` package provides a view of Go types and values, the -[[https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc][`google.golang.org/protobuf/reflect/protoreflect`]] +[`google.golang.org/protobuf/reflect/protoreflect`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc) package provides a view of values according to the protocol buffer type system. @@ -65,7 +66,7 @@ for this post, but let's look at how we might write the log-scrubbing function we mentioned previously. First, we'll write a `.proto` file defining an extension of the -[[https://github.com/protocolbuffers/protobuf/blob/b96241b1b716781f5bc4dc25e1ebb0003dfaba6a/src/google/protobuf/descriptor.proto#L509][`google.protobuf.FieldOptions`]] +[`google.protobuf.FieldOptions`](https://github.com/protocolbuffers/protobuf/blob/b96241b1b716781f5bc4dc25e1ebb0003dfaba6a/src/google/protobuf/descriptor.proto#L509) type so we can annotate fields as containing sensitive information or not. @@ -91,7 +92,7 @@ value and removes all the sensitive fields. } This function accepts a -[[https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc#Message][`proto.Message`]], +[`proto.Message`](https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc#Message), an interface type implemented by all generated message types. This type is an alias for one defined in the `protoreflect` package: @@ -101,7 +102,7 @@ is an alias for one defined in the `protoreflect` package: To avoid filling up the namespace of generated messages, the interface contains only a single method returning a -[[https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Message][`protoreflect.Message`]], +[`protoreflect.Message`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Message), which provides access to the message contents. (Why an alias? Because `protoreflect.Message` has a corresponding @@ -109,7 +110,7 @@ method returning the original `proto.Message`, and we need to avoid an import cycle between the two packages.) The -[[https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Message.Range][`protoreflect.Message.Range`]] +[`protoreflect.Message.Range`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Message.Range) method calls a function for every populated field in a message. m := pb.ProtoReflect() @@ -119,13 +120,13 @@ method calls a function for every populated field in a message. }) The range function is called with a -[[https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#FieldDescriptor][`protoreflect.FieldDescriptor`]] +[`protoreflect.FieldDescriptor`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#FieldDescriptor) describing the protocol buffer type of the field, and a -[[https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Value][`protoreflect.Value`]] +[`protoreflect.Value`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Value) containing the field value. The -[[https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Descriptor.Options][`protoreflect.FieldDescriptor.Options`]] +[`protoreflect.FieldDescriptor.Options`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Descriptor.Options) method returns the field options as a `google.protobuf.FieldOptions` message. @@ -147,9 +148,9 @@ buffer type system, not the Go one. This is also an example of an area where we have simplified the `proto` package API. The original -[[https://pkg.go.dev/github.com/golang/protobuf/proto?tab=doc#GetExtension][`proto.GetExtension`]] +[`proto.GetExtension`](https://pkg.go.dev/github.com/golang/protobuf/proto?tab=doc#GetExtension) returned both a value and an error. The new -[[https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc#GetExtension][`proto.GetExtension`]] +[`proto.GetExtension`](https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc#GetExtension) returns just a value, returning the default value for the field if it is not present. Extension decoding errors are reported at `Unmarshal` time. @@ -176,7 +177,7 @@ A more complete implementation might recursively descend into message-valued fields. We hope that this simple example gives a taste of protocol buffer reflection and its uses. -* Versions +## Versions We call the original version of Go protocol buffers APIv1, and the new one APIv2. Because APIv2 is not backwards compatible with APIv1, @@ -188,11 +189,11 @@ are concrete implementations in Go that both support the `proto2` and `proto3` language versions.) The -[[https://pkg.go.dev/github.com/golang/protobuf?tab=overview][`github.com/golang/protobuf`]] +[`github.com/golang/protobuf`](https://pkg.go.dev/github.com/golang/protobuf?tab=overview) module is APIv1. The -[[https://pkg.go.dev/google.golang.org/protobuf?tab=overview][`google.golang.org/protobuf`]] +[`google.golang.org/protobuf`](https://pkg.go.dev/google.golang.org/protobuf?tab=overview) module is APIv2. We have taken advantage of the need to change the import path to switch to one that is not tied to a specific hosting provider. (We considered `google.golang.org/protobuf/v2`, to make it @@ -205,17 +206,17 @@ version indefinitely. Even within a single program, some parts may use one API while others use another. It is essential, therefore, that we continue to support programs that use APIv1. -- `github.com/golang/protobuf@v1.3.4` is the most recent pre-APIv2 version of APIv1. + - `github.com/golang/protobuf@v1.3.4` is the most recent pre-APIv2 version of APIv1. -- `github.com/golang/protobuf@v1.4.0` is a version of APIv1 implemented in terms of APIv2. - The API is the same, but the underlying implementation is backed by the new one. - This version contains functions to convert between the APIv1 and APIv2 `proto.Message` - interfaces to ease the transition between the two. + - `github.com/golang/protobuf@v1.4.0` is a version of APIv1 implemented in terms of APIv2. + The API is the same, but the underlying implementation is backed by the new one. + This version contains functions to convert between the APIv1 and APIv2 `proto.Message` + interfaces to ease the transition between the two. -- `google.golang.org/protobuf@v1.20.0` is APIv2. - This module depends upon `github.com/golang/protobuf@v1.4.0`, - so any program which uses APIv2 will automatically pick a version of APIv1 - which integrates with it. + - `google.golang.org/protobuf@v1.20.0` is APIv2. + This module depends upon `github.com/golang/protobuf@v1.4.0`, + so any program which uses APIv2 will automatically pick a version of APIv1 + which integrates with it. (Why start at version `v1.20.0`? To provide clarity. We do not anticipate APIv1 to ever reach `v1.20.0`, @@ -232,31 +233,31 @@ minimum version selection means that programs may remain on the old implementation until the maintainers choose to update to the new one (either directly, or by updating a dependency). -* Additional features of note +## Additional features of note The -[[https://pkg.go.dev/google.golang.org/protobuf/encoding/protojson][`google.golang.org/protobuf/encoding/protojson`]] +[`google.golang.org/protobuf/encoding/protojson`](https://pkg.go.dev/google.golang.org/protobuf/encoding/protojson) package converts protocol buffer messages to and from JSON using the -[[https://developers.google.com/protocol-buffers/docs/proto3#json][canonical JSON mapping]], +[canonical JSON mapping](https://developers.google.com/protocol-buffers/docs/proto3#json), and fixes a number of issues with the old `jsonpb` package that were difficult to change without causing problems for existing users. The -[[https://pkg.go.dev/google.golang.org/protobuf/types/dynamicpb][`google.golang.org/protobuf/types/dynamicpb`]] +[`google.golang.org/protobuf/types/dynamicpb`](https://pkg.go.dev/google.golang.org/protobuf/types/dynamicpb) package provides an implementation of `proto.Message` for messages whose protocol buffer type is derived at runtime. The -[[https://pkg.go.dev/google.golang.org/protobuf/testing/protocmp][`google.golang.org/protobuf/testing/protocmp`]] +[`google.golang.org/protobuf/testing/protocmp`](https://pkg.go.dev/google.golang.org/protobuf/testing/protocmp) package provides functions to compare protocol buffer messages with the -[[https://pkg.go.dev/github.com/google/go-cmp/cmp][`github.com/google/cmp`]] +[`github.com/google/cmp`](https://pkg.go.dev/github.com/google/go-cmp/cmp) package. The -[[https://pkg.go.dev/google.golang.org/protobuf/compiler/protogen?tab=doc][`google.golang.org/protobuf/compiler/protogen`]] +[`google.golang.org/protobuf/compiler/protogen`](https://pkg.go.dev/google.golang.org/protobuf/compiler/protogen?tab=doc) package provides support for writing protocol compiler plugins. -* Conclusion +## Conclusion The `google.golang.org/protobuf` module is a major overhaul of Go's support for protocol buffers, providing first-class support |