identifying default vs unset values in Google Protocol Buffers (proto3)

what are Protocol Buffers / Protobuf?

Protocol Buffers are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler; you define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages.

problem

in proto3, all fields are “optional” (it is not an error if the sender fails to set them); but fields are no longer “nullable”, there’s no way to tell the difference between a field being explicitly set to its default value vs. not having been set at all.

illustration

let’s look at an example with a sample Protocol Buffer and a golang code snippet.

• sample Protocol Buffer

syntax = "proto3";

package exampleproto;

option go_package = "exampleproto";

import "google/protobuf/timestamp.proto";
import "google/protobuf/wrappers.proto";

message ExampleProtoMessage {
  string field1 = 1;
  google.protobuf.StringValue field2 = 2;
}

• executor

package main

import (
    "fmt"
    "proto/exampleproto"
    "github.com/golang/protobuf/ptypes/wrappers"
)

func main() {
    ee := executeExperiment()
    fmt.Println(ee)
}

func executeExperiment() *exampleproto.ExampleProtoMessage {
    ep := exampleproto.ExampleProtoMessage{
        Field1:  "12345",
        Field2:   &wrappers.StringValue{Value: "12345"},
    }
    fmt.Println(ep.Field1)
    fmt.Println(ep.Field2)
    return &ep
}

observation / solution

in the above example, we have two types of fields in the defined proto.

• string
• StringValue (google.protobuf.*)

results

• if the value of Field1 is provided as “” (empty-string) then the intercepted value would be “”.
• if the value of Field1 is not provided at all, again the intercepted value would be “”.

from the above result, it’s clear that we can’t differentiate between a default vs an unset value.

if we want to differentiate between them, we can make use of google.protobuf.StringValue (.BoolValue, .IntValue)

• if you look at the executor code, Field2 is of type google.protobuf.StringValue.
• if we don’t set value of Field2 and try to access it - we get nil instead of “”; whereas if we set it as “”, we get the value as “”.

Note: We can achieve a similar behavior using OneOf too but OneOf's are not meant to be used like this and is considered hacky.