Protocol Buffers

What are Protocol Buffers / Protobuf?

Protocol buffers are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages.

Problem

In proto3, all fields are “optional” (it is not an error if the sender fails to set them). But, fields are no longer “nullable”, there’s no way to tell the difference between a field being explicitly set to its default value vs. not having been set at all.

Illustration

Let’s look at an example with a sample protocol buffer and a golang code snippet.

  • Sample Protocol Buffer
syntax = "proto3";

package exampleproto;

option go_package = "exampleproto";

import "google/protobuf/timestamp.proto";
import "google/protobuf/wrappers.proto";

message ExampleProtoMessage {
  string field1 = 1;
  google.protobuf.StringValue field2 = 2;
}
  • Executor
package main

import (
    "fmt"
    "proto/exampleproto"
    "github.com/golang/protobuf/ptypes/wrappers"
)

func main() {
    ee := executeExperiment()
    fmt.Println(ee)
}

func executeExperiment() *exampleproto.ExampleProtoMessage {
    ep := exampleproto.ExampleProtoMessage{
        Field1:  "12345",
        Field2:   &wrappers.StringValue{Value: "12345"},
    }
    fmt.Println(ep.Field1)
    fmt.Println(ep.Field2)
    return &ep
}

Observation / Solution

In the above example, we have two types of fields in the defined proto.

  • String
  • StringValue (google.protobuf.*)

Results

  • If the value of Field1 is provided as “” (empty-string) then the intercepted value would be “”.
  • If the value of Field1 is not provided at all, again the intercepted value would be “”.

From the above result, it’s clear that we can’t differentiate between the a default vs an unset value.

If we want to differentiate between them, we can make use of google.protobuf.StringValue (.BoolValue, .IntValue)

  • If you look at the executor code, Field2 is of type google.protobuf.StringValue.
  • If we don’t set value of Field2 and try to access it - we get nil instead of “”. Whereas if we set it as “”, we get the value as “”.
Note: We can achieve a similar behavior using OneOf too but OneOf's are not meant to be used like this and is considered hacky.