python: zap_py schema DSL (StructBuilder, predefined EVM schemas)#3
Open
abhicris wants to merge 3 commits intoluxfi:mainfrom
Open
python: zap_py schema DSL (StructBuilder, predefined EVM schemas)#3abhicris wants to merge 3 commits intoluxfi:mainfrom
abhicris wants to merge 3 commits intoluxfi:mainfrom
Conversation
added 3 commits
April 27, 2026 17:41
Adds python/zap_py — a pure-stdlib (no deps, py>=3.8) reader and builder that's wire-compatible with the Go reference. Lets non-Go peers (AI agents, ops scripts, FHE clients) speak ZAP without leaving Python. Read: parse() returns a Message backed by a memoryview; .bytes_view() exposes zero-copy slices. Build: Builder/ObjectBuilder/ListBuilder mirror builder.go method for method, including the deferred-write trick for text/bytes so relative offsets match the Go encoder byte for byte. Coverage: header validation, all scalar widths (bool, u8/16/32/64, i8/16/32/64, f32/f64), text, bytes, nested objects, lists of u8/u32/u64/objects/raw bytes, null pointers, flag bits. Bidirectional fixtures prove parity: - python/testdata/gen_fixture.go → emits a Go-built message - python/testdata/gen_python_fixture.py → emits a Python-built one - python/tests/test_roundtrip.py::test_go_fixture_interop reads the Go fixture from Python (skipped unless ZAP_GO_FIXTURE is set) - python_interop_test.go::TestPythonFixture reads the Python fixture from Go (skipped unless ZAP_PYTHON_FIXTURE is set) Both fixtures use identical schemas; output is byte-identical except for the embedded text payload. go.sum gains the missing luxfi/mdns line that `go test ./...` already requires (fixes a tidy-state issue surfaced while wiring up the new Go interop test). Not ported (yet): EVM helpers, MCP bridge, mDNS node, schema DSL. Reader+builder is enough to interop with any Go ZAP service that publishes a fixed schema; higher-level helpers can follow as Python use cases land.
Follow-up to luxfi#1. Adds the EVM helpers called out as not-yet-ported in that PR: fixed-width Address (20), Hash (32), Signature (65), Bloom (256), with hex parsing, zero detection, and reader/builder methods on Object, ObjectBuilder, and List that mirror evm.go. New module: python/zap_py/evm.py - _FixedBytes base + Address / Hash / Signature / Bloom - .from_hex() (0x/0X tolerated), .hex(), .is_zero(), .bytes - bytes-equality so Address == bytes(...) works - ZERO_ADDRESS / ZERO_HASH constants - top-level address_from_hex / hash_from_hex helpers Reader extensions (reader.py): - Object.address / .hash / .signature - Object.address_slice / .hash_slice (zero-copy memoryview) - List.address / .hash for lists of fixed-width elements Builder extensions (builder.py): - ObjectBuilder.set_address / .set_hash / .set_signature - All accept either the typed value or raw bytes; reject mismatched lengths with a clean ValueError Tests (13 new): hex roundtrip (case-insensitive prefix), invalid-length rejection, zero detection, single-field roundtrip, zero-copy slice view, raw-bytes-input acceptance, address-list and hash-list typed access, bytes equality. Cross-language parity: python/testdata/gen_evm_fixture.go emits a Go-built EVM-typed message; tests/test_evm.py::test_evm_go_fixture_interop parses it from Python under ZAP_GO_EVM_FIXTURE.
Closes the "not yet ported" list from luxfi#1 and luxfi#2. Adds the schema DSL that lets Python declare a struct shape, get computed field offsets, and stay byte-compatible with Go-declared schemas. New module: python/zap_py/schema.py - Type IntEnum (matches schema.go's iota ordering) - type_size(t) helper - Field / Struct / Enum / Schema dataclasses - new_schema, new_struct_builder convenience functions - StructBuilder with the full Go API: bool/int8-64/uint8-64/float32-64/text/bytes/list/struct + EVM hooks: address (20), hash (32), signature (65) - Same alignment rules as schema.go: each scalar rounds to its width, EVM types use no alignment, build() final-aligns to 8 - Predefined schemas (1:1 with evm.go): TRANSACTION_SCHEMA, BLOCK_HEADER_SCHEMA, LOG_SCHEMA Tests (11 new): per-type sizes, mixed-width alignment math, no unnecessary padding for same-width chains, field lookup, list-elem type tracking, struct-name carry-through, schema registry, EVM extension offsets, hand-computed Transaction layout (size 192), schema-driven roundtrip with the Builder. Cross-language parity: gen_schema_fixture.go dumps Go's offsets and sizes for the three predefined schemas as JSON. Python loads it back and asserts every field offset and the total size match — both sides compute exactly the same layout for Transaction (192), BlockHeader (280), and Log.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the not-yet-ported list from #1 and #2. Ports the schema DSL so Python code can declare a struct shape, get computed field offsets, and stay byte-compatible with Go-declared schemas.
TypeIntEnum (matchesschema.goiota),type_size()helper.Field,Struct,Enum,Schemadataclasses with a registry (new_schema,add_struct,add_enum).StructBuilderwith the full Go surface — bool/int8–64/uint8–64/float32–64/text/bytes/list/struct — plus the EVM extension methodsaddress/hash/signature. Same alignment rules asschema.go: each scalar rounds to its width, EVM types use no alignment,build()final-aligns to 8.TRANSACTION_SCHEMA,BLOCK_HEADER_SCHEMA,LOG_SCHEMA— declared exactly the wayevm.godeclares them.Cross-language parity
gen_schema_fixture.godumps Go's offsets and sizes for all three predefined schemas as JSON. The Python parity test loads it and asserts every field offset and total size match:go run ./python/testdata/gen_schema_fixture.go > /tmp/zap_schemas.json ZAP_GO_SCHEMA_FIXTURE=/tmp/zap_schemas.json python -m pytest python/tests/test_schema.py::test_go_schema_fixture_parityResult on this branch: byte-identical layout for
Transaction(size 192),BlockHeader(280), andLog. Anything declared withStructBuilderon either side is now interchangeable.Stack
GitHub will rebase/squash cleanly once #1 and #2 land. Until then, the diff is best read as additions on top of
zap-py-evm.Test plan
cd python && .venv/bin/pytest tests/— 35 passed, 3 skipped (interop fixtures gated on env vars)go test ./...— full Go suite still green