ocaml-protoc

A protobuf compiler for OCaml.

• Introduction
• Simple Example
• Build/Install
• Protobuf <-> OCaml mapping
• Compiler Internals
• Protobuf extensions
• Benchmarking

Introduction

ocaml-protoc compiles protobuf message files into OCaml modules. Each message/enum/oneof protobuf type has a corresponding OCaml type along with the following functions:

• encode_<type> : encode the generated type to bytes following protobuf specification
• decode_<type> : decode the generated type from bytes following protobuf specification
• default_<type> : default value honoring protobuf default attributes or protobuf version 3 default rules
• pp_<type> : pretty print of the OCaml type

The compiler relies on a runtime library pbrt which is itself implemented using the same runtime library as ppx_deriving_protobuf for low level encoding/decoding.

OCaml users have now 2 complementary options to choose for protobuf serialization:

• If your application is mainly OCaml then ppx_deriving_protobuf is usually the best tool. You can leverage the OCaml type system as a schema definition and minimum setup is required to support serialization.
• If your application is using multiple languages and you are leveraging Protobuf as a language-independent data specification, then ocaml-protoc is likely a more suitable option. ocaml-protoc guarantees that the generated OCaml types conform to the protobuf specifications.

A simple example

Let's take a similar example as the google one:

message Person {
  required string name = 1;
  required int32 id = 2;
  optional string email = 3;
  repeated string phone = 4;
}

The following OCaml code will get generated after running ocaml-protoc -ml_out ./ example.proto

(** example01.proto Generated Types and Encoding *)

(** {2 Types} *)

type person = {
  name : string;
  id : int32;
  email : string option;
  phone : string list;
}


(** {2 Default values} *)

val default_person : 
  ?name:string ->
  ?id:int32 ->
  ?email:string option ->
  ?phone:string list ->
  unit ->
  person
(** [default_person ()] is the default value for type [person] *)


(** {2 Protobuf Decoding} *)

val decode_person : Pbrt.Decoder.t -> person
(** [decode_person decoder] decodes a [person] value from [decoder] *)


(** {2 Protobuf Encoding} *)

val encode_person : person -> Pbrt.Encoder.t -> unit
(** [encode_person v encoder] encodes [v] with the given [encoder] *)


(** {2 Formatters} *)

val pp_person : Format.formatter -> person -> unit 
(** [pp_person v] formats v] *)

You can then use this OCaml module in your application to populate, serialize, and retrieve person protocol buffer messages. For example:

let () =

  (* Create OCaml value of generated type *) 
  let person = Example_pb.({ 
    name = "John Doe"; 
    id = 1234l;
    email = Some "jdoe@example.com"; 
    phone = ["123-456-7890"];
  }) in 
  
  (* Create a Protobuf encoder and encode value *)
  let encoder = Pbrt.Encoder.create () in 
  Example_pb.encode_person person encoder; 

  (* Output the protobuf message to a file *) 
  let oc = open_out "myfile" in 
  output_bytes oc (Pbrt.Encoder.to_bytes encoder);
  close_out oc

Then later on you can read your message back in:

let () = 
  (* Read bytes from the file *) 
  let bytes = 
    let ic = open_in "myfile" in 
    let len = in_channel_length ic in 
    let bytes = Bytes.create len in 
    really_input ic bytes 0 len; 
    close_in ic; 
    bytes 
  in 
  
  (* Decode the person and Pretty-print it *)
  let person = Example_pb.decode_person (Pbrt.Decoder.of_bytes bytes) in
  Format.fprintf Format.std_formatter "%a" Example_pb.pp_person person

OCaml users will immediately point to the use of int32 type in the generated code which might not be the most convenient choice. One can modify this behavior using custom extensions.

Build-Install

Prerequesite

ocaml-protoc depends on :

• the OCaml compiler distribution (byte code/native compiler and ocamlbuild).
• ppx_deriving_protobuf for the generated code runtime.

Intall from OPAM

opam install ocaml-protoc

Install from source with ocamlfind

mkdir -p tmp/bin
export PREFIX=`pwd`/tmp
make install

Build your program

Here are the steps to build the example above where the source are in src/examples/. We now assume that $PREFIX/bin is in your path.

# Generate the OCaml protobuf module 
ocaml-protoc -ml_out ./ example01.proto

When using findlib:

ocamlfind ocamlopt -linkpkg -package ocaml-protoc \
  -o example01 \
  example01_pb.mli example01_pb.ml example01.ml

You can now run the example

./example01

Protobuf <-> OCaml mapping

see here.

Compiler Internals

see here

Protobuf Extensions

see here

Benchmarking

see here