aboutsummaryrefslogtreecommitdiff
path: root/content/a-new-go-api-for-protocol-buffers.article
diff options
context:
space:
mode:
authorRuss Cox <rsc@golang.org>2020-03-09 23:54:35 -0400
committerRuss Cox <rsc@golang.org>2020-03-17 20:58:37 +0000
commitaf5018f64e406aaa646dae066f28de57321ea5ce (patch)
tree8db7b1f049d83d215fa9abf68851efce7b5ccadb /content/a-new-go-api-for-protocol-buffers.article
parent86e424fac66fa90ddcb7e8d7febd4c2b07d7c59e (diff)
content: convert to Markdown-enabled present inputs
Converted blog to Markdown-enabled present (CL 222846) using present2md (CL 222847). For golang/go#33955. Change-Id: Ib39fa1ddd9a46f9c7a62a2ca7b96e117635553e8 Reviewed-on: https://go-review.googlesource.com/c/blog/+/222848 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org>
Diffstat (limited to 'content/a-new-go-api-for-protocol-buffers.article')
-rw-r--r--content/a-new-go-api-for-protocol-buffers.article79
1 files changed, 40 insertions, 39 deletions
diff --git a/content/a-new-go-api-for-protocol-buffers.article b/content/a-new-go-api-for-protocol-buffers.article
index f3afe0a..2c5c014 100644
--- a/content/a-new-go-api-for-protocol-buffers.article
+++ b/content/a-new-go-api-for-protocol-buffers.article
@@ -1,19 +1,20 @@
-A new Go API for Protocol Buffers
-02 Mar 2020
+# A new Go API for Protocol Buffers
+2 Mar 2020
Tags: protobuf, technical
+Summary: We are pleased to announce the release of a major revision of the Go API for [protocol buffers](https://developers.google.com/protocol-buffers), Google's language-neutral data interchange format.
Joe Tsai, Damien Neil, and Herbie Ong
-* Introduction
+## Introduction
We are pleased to announce the release of a major revision of the Go API for
-[[https://developers.google.com/protocol-buffers][protocol buffers]],
+[protocol buffers](https://developers.google.com/protocol-buffers),
Google's language-neutral data interchange format.
-* Motivations for a new API
+## Motivations for a new API
The first protocol buffer bindings for Go were
-[[https://blog.golang.org/third-party-libraries-goprotobuf-and][announced by Rob Pike]]
+[announced by Rob Pike](https://blog.golang.org/third-party-libraries-goprotobuf-and)
in March of 2010. Go 1 would not be released for another two years.
In the decade since that first release, the package has grown and
@@ -21,7 +22,7 @@ developed along with Go. Its users' requirements have grown too.
Many people want to write programs that use reflection to examine protocol
buffer messages. The
-[[https://pkg.go.dev/reflect][`reflect`]]
+[`reflect`](https://pkg.go.dev/reflect)
package provides a view of Go types and
values, but omits information from the protocol buffer type system. For
example, we might want to write a function that traverses a log entry and
@@ -33,7 +34,7 @@ generated by the protocol buffer compiler, such as a dynamic message type
capable of representing messages whose type is not known at compile time.
We also observed that a frequent source of problems was that the
-[[https://pkg.go.dev/github.com/golang/protobuf/proto?tab=doc#Message][`proto.Message`]]
+[`proto.Message`](https://pkg.go.dev/github.com/golang/protobuf/proto?tab=doc#Message)
interface, which identifies values of generated message types, does very
little to describe the behavior of those types. When users create types
that implement that interface (often inadvertently by embedding a message
@@ -52,11 +53,11 @@ protobuf module.
Today, we're pleased to release that new module. We hope you like it.
-* Reflection
+## Reflection
Reflection is the flagship feature of the new implementation. Similar
to how the `reflect` package provides a view of Go types and values, the
-[[https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc][`google.golang.org/protobuf/reflect/protoreflect`]]
+[`google.golang.org/protobuf/reflect/protoreflect`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc)
package provides a view of values according to the protocol buffer
type system.
@@ -65,7 +66,7 @@ for this post, but let's look at how we might write the log-scrubbing
function we mentioned previously.
First, we'll write a `.proto` file defining an extension of the
-[[https://github.com/protocolbuffers/protobuf/blob/b96241b1b716781f5bc4dc25e1ebb0003dfaba6a/src/google/protobuf/descriptor.proto#L509][`google.protobuf.FieldOptions`]]
+[`google.protobuf.FieldOptions`](https://github.com/protocolbuffers/protobuf/blob/b96241b1b716781f5bc4dc25e1ebb0003dfaba6a/src/google/protobuf/descriptor.proto#L509)
type so we can annotate fields as containing
sensitive information or not.
@@ -91,7 +92,7 @@ value and removes all the sensitive fields.
}
This function accepts a
-[[https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc#Message][`proto.Message`]],
+[`proto.Message`](https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc#Message),
an interface type implemented by all generated message types. This type
is an alias for one defined in the `protoreflect` package:
@@ -101,7 +102,7 @@ is an alias for one defined in the `protoreflect` package:
To avoid filling up the namespace of generated
messages, the interface contains only a single method returning a
-[[https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Message][`protoreflect.Message`]],
+[`protoreflect.Message`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Message),
which provides access to the message contents.
(Why an alias? Because `protoreflect.Message` has a corresponding
@@ -109,7 +110,7 @@ method returning the original `proto.Message`, and we need to avoid an
import cycle between the two packages.)
The
-[[https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Message.Range][`protoreflect.Message.Range`]]
+[`protoreflect.Message.Range`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Message.Range)
method calls a function for every populated field in a message.
m := pb.ProtoReflect()
@@ -119,13 +120,13 @@ method calls a function for every populated field in a message.
})
The range function is called with a
-[[https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#FieldDescriptor][`protoreflect.FieldDescriptor`]]
+[`protoreflect.FieldDescriptor`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#FieldDescriptor)
describing the protocol buffer type of the field, and a
-[[https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Value][`protoreflect.Value`]]
+[`protoreflect.Value`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Value)
containing the field value.
The
-[[https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Descriptor.Options][`protoreflect.FieldDescriptor.Options`]]
+[`protoreflect.FieldDescriptor.Options`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect?tab=doc#Descriptor.Options)
method returns the field options as a `google.protobuf.FieldOptions`
message.
@@ -147,9 +148,9 @@ buffer type system, not the Go one.
This is also an example of an area where we
have simplified the `proto` package API. The original
-[[https://pkg.go.dev/github.com/golang/protobuf/proto?tab=doc#GetExtension][`proto.GetExtension`]]
+[`proto.GetExtension`](https://pkg.go.dev/github.com/golang/protobuf/proto?tab=doc#GetExtension)
returned both a value and an error. The new
-[[https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc#GetExtension][`proto.GetExtension`]]
+[`proto.GetExtension`](https://pkg.go.dev/google.golang.org/protobuf/proto?tab=doc#GetExtension)
returns just a value, returning the default value for the field if it is
not present. Extension decoding errors are reported at `Unmarshal` time.
@@ -176,7 +177,7 @@ A more complete implementation might recursively descend into
message-valued fields. We hope that this simple example gives a
taste of protocol buffer reflection and its uses.
-* Versions
+## Versions
We call the original version of Go protocol buffers APIv1, and the
new one APIv2. Because APIv2 is not backwards compatible with APIv1,
@@ -188,11 +189,11 @@ are concrete implementations in Go that both support the `proto2` and
`proto3` language versions.)
The
-[[https://pkg.go.dev/github.com/golang/protobuf?tab=overview][`github.com/golang/protobuf`]]
+[`github.com/golang/protobuf`](https://pkg.go.dev/github.com/golang/protobuf?tab=overview)
module is APIv1.
The
-[[https://pkg.go.dev/google.golang.org/protobuf?tab=overview][`google.golang.org/protobuf`]]
+[`google.golang.org/protobuf`](https://pkg.go.dev/google.golang.org/protobuf?tab=overview)
module is APIv2. We have taken advantage of the need to change the
import path to switch to one that is not tied to a specific hosting
provider. (We considered `google.golang.org/protobuf/v2`, to make it
@@ -205,17 +206,17 @@ version indefinitely. Even within a single program, some parts may use
one API while others use another. It is essential, therefore, that we
continue to support programs that use APIv1.
-- `github.com/golang/protobuf@v1.3.4` is the most recent pre-APIv2 version of APIv1.
+ - `github.com/golang/protobuf@v1.3.4` is the most recent pre-APIv2 version of APIv1.
-- `github.com/golang/protobuf@v1.4.0` is a version of APIv1 implemented in terms of APIv2.
- The API is the same, but the underlying implementation is backed by the new one.
- This version contains functions to convert between the APIv1 and APIv2 `proto.Message`
- interfaces to ease the transition between the two.
+ - `github.com/golang/protobuf@v1.4.0` is a version of APIv1 implemented in terms of APIv2.
+ The API is the same, but the underlying implementation is backed by the new one.
+ This version contains functions to convert between the APIv1 and APIv2 `proto.Message`
+ interfaces to ease the transition between the two.
-- `google.golang.org/protobuf@v1.20.0` is APIv2.
- This module depends upon `github.com/golang/protobuf@v1.4.0`,
- so any program which uses APIv2 will automatically pick a version of APIv1
- which integrates with it.
+ - `google.golang.org/protobuf@v1.20.0` is APIv2.
+ This module depends upon `github.com/golang/protobuf@v1.4.0`,
+ so any program which uses APIv2 will automatically pick a version of APIv1
+ which integrates with it.
(Why start at version `v1.20.0`? To provide clarity.
We do not anticipate APIv1 to ever reach `v1.20.0`,
@@ -232,31 +233,31 @@ minimum version selection means that programs may remain on the old
implementation until the maintainers choose to update to the new one
(either directly, or by updating a dependency).
-* Additional features of note
+## Additional features of note
The
-[[https://pkg.go.dev/google.golang.org/protobuf/encoding/protojson][`google.golang.org/protobuf/encoding/protojson`]]
+[`google.golang.org/protobuf/encoding/protojson`](https://pkg.go.dev/google.golang.org/protobuf/encoding/protojson)
package converts protocol buffer messages to and from JSON using the
-[[https://developers.google.com/protocol-buffers/docs/proto3#json][canonical JSON mapping]],
+[canonical JSON mapping](https://developers.google.com/protocol-buffers/docs/proto3#json),
and fixes a number of issues with the old `jsonpb` package
that were difficult to change without causing problems for existing users.
The
-[[https://pkg.go.dev/google.golang.org/protobuf/types/dynamicpb][`google.golang.org/protobuf/types/dynamicpb`]]
+[`google.golang.org/protobuf/types/dynamicpb`](https://pkg.go.dev/google.golang.org/protobuf/types/dynamicpb)
package provides an implementation of `proto.Message` for messages whose
protocol buffer type is derived at runtime.
The
-[[https://pkg.go.dev/google.golang.org/protobuf/testing/protocmp][`google.golang.org/protobuf/testing/protocmp`]]
+[`google.golang.org/protobuf/testing/protocmp`](https://pkg.go.dev/google.golang.org/protobuf/testing/protocmp)
package provides functions to compare protocol buffer messages with the
-[[https://pkg.go.dev/github.com/google/go-cmp/cmp][`github.com/google/cmp`]]
+[`github.com/google/cmp`](https://pkg.go.dev/github.com/google/go-cmp/cmp)
package.
The
-[[https://pkg.go.dev/google.golang.org/protobuf/compiler/protogen?tab=doc][`google.golang.org/protobuf/compiler/protogen`]]
+[`google.golang.org/protobuf/compiler/protogen`](https://pkg.go.dev/google.golang.org/protobuf/compiler/protogen?tab=doc)
package provides support for writing protocol compiler plugins.
-* Conclusion
+## Conclusion
The `google.golang.org/protobuf` module is a major overhaul of
Go's support for protocol buffers, providing first-class support