Skip to main content

Generate Apache Avro

The Avro backend ships as a WASM release bundle from finos/morphir-rust, tagged extension/avro/v<version>. There is no public extension index yet, so you publish the bundle into a local repository and install from it. This guide covers that flow and the contract for a locally built extension. Rename the downloaded <artifact>.release.json to release.json before publishing.

The morphir-avro WASM extension turns public Morphir types and value specifications into Apache Avro schemas or protocols. It accepts Morphir IR v3 and v4. It does not evaluate Morphir values or translate computation.

Install the published extension

Release bundles are published from finos/morphir-rust under the tag extension/avro/v<version>. Each release carries three assets: the .wasm artifact, its .sha256 checksum, and a <artifact>.release.json descriptor. There is no public extension index yet, so you publish the bundle into a local repository and install from it.

Download the assets into one directory, rename the descriptor to release.json, and keep only those three files in the directory:

mkdir -p bundles/avro
gh release download extension/avro/v0.1.1 -R finos/morphir-rust --dir bundles/avro
mv bundles/avro/*.release.json bundles/avro/release.json

Then create a repository, register it, publish the bundle, and install:

morphir extension repository init repositories/local
morphir extension repository add local --directory repositories/local
morphir extension repository publish local --bundle bundles/avro
morphir extension install --repository local morphir-avro
morphir extension list

publish verifies the SHA-256 in release.json against the artifact and the checksum file before it writes anything. install verifies the artifact again, stores the module by content digest, and writes matching lock and catalog state. After that the extension runs offline: the repository directory is only needed to install or update.

Set MORPHIR_HOME to an empty directory first to keep this out of your regular home, and keep the same value when running generation.

Build and install a local extension

Contributors can build the bundle from ecosystem/morphir-rust with its packaging task and publish it the same way. The task writes release.json, the artifact, and the checksum into one directory:

mise run extension:artifact:avro

Publish .morphir/build/extensions/avro with morphir extension repository publish as shown above. These commands do not publish the extension anywhere public.

Run the generator

Once an Avro provider is installed, generate the default JSON schemas with:

morphir generate --target avro --input morphir-ir.json --output generated/avro

Backend options live under [codegen.avro] in morphir.toml:

[codegen]
targets = ["avro"]

[codegen.avro]
representation = "json"
projection = "schemas"
dependencies = "self-contained"
aliases = "inline"
unsupported = "error"
logical_types = true
decimal_precision = 38
decimal_scale = 10

[codegen.avro.type_mappings."acme/customer:customer#customer-id"]
type = "string"
logical_type = "uuid"

Repeat --option <KEY=VALUE> to override that table for one command:

morphir generate --target avro \
--option representation=idl \
--option projection=protocol-public \
--option dependencies=linked

The backend starts with its defaults, then applies [codegen.avro], then applies CLI options in command-line order. The last CLI value for a key wins. The CLI parses a value as JSON when possible. Otherwise it passes a string. For example, logical_types=false is a Boolean and representation=idl is a string. Option names use snake_case.

Options and defaults

OptionAccepted valuesDefault
representationjson, idljson
projectionschemas, protocol-entry-points, protocol-publicschemas
dependenciesself-contained, linkedself-contained
aliasesinline, wrapper-recordinline
unsupportederror, warn-and-skiperror
logical_typesBooleantrue
decimal_precisionPositive integer38
decimal_scaleInteger from zero through the effective precision10
type_mappingsMap from an exact Morphir FQName to a mapping objectEmpty map

Unknown options, wrong JSON types, invalid enum values, and invalid decimal ranges fail with AVRO004. A per-type decimal mapping must use physical type bytes. Its scale must not exceed its effective precision. The allowed physical mapping values are null, boolean, int, long, float, double, bytes, and string.

Choose what to project

The three projection modes answer different questions:

ProjectionTypesMessages
schemasPublic type definitionsNone
protocol-entry-pointsPublic type definitionsDeclared entry points from a v4 Application only
protocol-publicPublic type definitionsEvery public value specification

A function message uses its arguments as request fields and its result type as the response. Libraries and Specs have no declared application entry points, so protocol-entry-points emits a type-only protocol for them. It does not invent messages.

Output files

The representation and projection together determine the suffix:

RepresentationschemasEither protocol mode
jsonOne .avsc per public root typeOne .avpr per Morphir module
idlOne message-free .avdl wrapper per public root typeOne .avdl protocol per Morphir module

Every IDL artifact contains exactly one protocol declaration. A schema-mode wrapper ends in Schemas and can be converted with avro-tools idl2schemata.

Some protocol IDL messages return named types and carry Morphir annotations. Avro Tools 1.12.2 requires avro-tools idl --useJavaCC for those files. The generator places a compatibility comment at the start of each affected .avdl. Primitive-response and message-free schema wrappers continue to work with the default reader.

Morphir IR v3 and v4

The backend accepts baseline version spellings 3, "3.0.0", 4, and "4.0.0". It rejects other major versions and revisions such as 3.1.0 or 4.1.0. Both supported versions normalize into the same body-free model.

Input distributionPublic typesprotocol-entry-pointsprotocol-public
v3 LibraryIncludedType-only protocolPublic value signatures become messages
v4 LibraryIncludedType-only protocolPublic value signatures become messages
v4 SpecsSpecified types includedType-only protocolSpecified values become messages
v4 ApplicationIncludedDeclared entry points become messagesAll public value signatures become messages

Normalization keeps package and module names, public declarations, documentation, source FQNames, dependencies, and v4 entry-point metadata. It drops value bodies.

Type mapping

The default projection is:

Morphir formAvro representation
Boolboolean
Intlong
Floatdouble
Stringstring
Charstring with morphir.type = Char
Unitnull
Maybe aUnion of null and the projection of a
List aArray
Set aArray with morphir.collection-kind = set
Dict String aMap
Record aliasNamed record
Nullary custom typeEnum
Custom type with payload constructorsWrapper record and constructor records
Tuple or closed generic specializationStable generated record or name
Result error valueNamed outer Result record whose value field is a union of named Err and Ok records

The Err record contains the projected error and the Ok record contains the projected success value. Result does not become an Avro protocol error. A Morphir result is ordinary return data, not a transport failure.

When logical_types = true, recognized Morphir types use these Avro pairs:

Morphir conceptAvro physical typeAvro logical type
Local dateintdate
Local timelongtime-micros
Instant or DateTimelongtimestamp-micros
UUIDstringuuid
Decimalbytesdecimal

Use type_mappings for an exact source FQName that needs another physical or logical type:

[codegen.avro.type_mappings."acme/customer:customer#money"]
type = "bytes"
logical_type = "decimal"
precision = 20
scale = 4

Mapping keys use the canonical package:module#local Morphir FQName. They do not match an anonymous or inline type expression. To handle one non-string Dict without mapping every SDK dictionary, give that dictionary shape a named declaration, reference the declaration where it is used, and map the exact FQName of that declaration.

The generator uses UpperCamelCase for Avro types and constructors, lowerCamelCase for fields and messages, and normalized dotted namespaces. It keeps the original Morphir FQName as metadata. Stable synthetic names depend on the projected type, not traversal order. A name collision is an error rather than an implicit rename.

Constants and entry points

protocol-public projects a zero-argument public value as a zero-argument message. The message has morphir.value-kind = constant. It contains no constant value because normalization never reads or evaluates the value body. Functions use morphir.value-kind = function.

Declared application entry points also have morphir.entry-point = true, their entry-point ID, and a lowercase morphir.entry-point-kind of main, command, or handler.

Dependencies and aliases

dependencies = "self-contained" embeds each referenced dependency schema in the generated artifact closure. Use this when each artifact must compile on its own.

dependencies = "linked" emits dependency declarations separately and refers to them by full Avro name. IDL uses deterministic relative imports. A missing linked dependency fails with AVRO006. Linked output is useful when a build already packages shared schemas and wants to avoid duplicate definitions.

aliases = "inline" replaces an alias with its target unless that target is a named record. aliases = "wrapper-record" preserves the alias as a generated record with one value field. Wrapper records give the alias its own Avro name at the cost of an extra record layer.

Unsupported forms and partial output

The backend cannot safely project a function used as data, an open extensible record, an opaque or incomplete type, an unresolved type, unsafe recursion, an unbound generic parameter, or a Dict with a non-string key unless an explicit mapping handles it.

The default unsupported = "error" is strict. Any projection error makes the result unsuccessful and emits no artifacts. warn-and-skip omits the bad public form, emits a deterministic warning at its Morphir FQName, and returns only artifacts that remain independently valid.

CodeMeaning
AVRO001Unsupported Morphir type or form
AVRO002Unbound type parameter
AVRO003Avro name collision
AVRO004Invalid backend option
AVRO005Unsafe or unrepresentable recursion
AVRO006Missing linked dependency

For runtime boundaries and release status, see the accepted WASM extension runtime and Avro backend proposal.