feat:rust native modules

[aclauer]

Problem

We need support for Rust native modules to offload performance critical modules.

Closes DIM-XXX

Solution

See native/rust/examples for a Rust native ping pong example

Added NativeModule struct to allow easy publishing and subscribing to topics. NativeModules can use any transport that implements the Transport trait (currently only LCM).

Rust NativeModules also support configuration through stdin instead of cli args. Cli args are kept for backwards compatibility.

Breaking Changes

None

How to Test

For unit tests
cd dimos/native/rust
cargo test

For end to end test with two Rust modules talking over LCM
python3 rust_ping_pong.py

Contributor License Agreement

• I have read and approved the CLA.

[greptile-apps]

Greptile Summary

This PR adds a Rust native module SDK (dimos-native-module) and updates the Python NativeModule coordinator to send topics and config via stdin JSON (in addition to existing CLI args). The architecture is clean — a Transport trait abstracts the wire protocol, NativeModule<T> manages typed routes and a background tokio task, and a thorough unit-test suite covers the config-parsing logic.

• P1 — native_module.py line 181: config_dict if config_dict else None coerces an empty dict {} to null. Any Rust module whose Python config fields are all None will receive JSON null instead of {}, causing serde_json to fail deserializing a struct type — even one with all #[serde(default)] fields. The fix is to drop the conditional and always send config_dict.

Confidence Score: 4/5

Safe to merge after fixing the empty-dict-to-null coercion in native_module.py; remaining findings are P2 style/hardening suggestions.

One P1 logic bug: the config_dict if config_dict else None expression silently sends JSON null instead of {} for Rust modules with all-optional config fields, causing a runtime deserialization failure. The two P2 findings (blocking IO in async, missing recv-error backoff) are real but do not affect correctness for the current usage pattern. Overall structure, trait design, and test coverage are solid.

dimos/core/native_module.py — the config serialization logic at line 181 needs the falsy-dict guard removed.

Important Files Changed

Filename	Overview
dimos/core/native_module.py	Python coordinator that spawns native subprocesses and sends topics/config via stdin JSON; has a P1 logic bug where an empty config dict is coerced to null, which will break Rust modules with all-optional config fields.
native/rust/src/module.rs	Core NativeModule struct with stdin config parsing, typed route dispatch, and a tokio background loop; blocking I/O in from_stdin and no backoff on recv errors are P2 concerns; unit-test coverage is good.
native/rust/src/transport.rs	Clean trait abstraction over the message transport; RPITIT pattern is appropriate for async trait methods in Rust 2021.
native/rust/src/lcm.rs	Thin wrapper delegating to dimos-lcm; correctly maps LCM message to (channel, data) tuple.
native/rust/src/lib.rs	Public re-exports; re-exports LcmOptions so callers avoid a direct dimos-lcm dependency.
native/rust/examples/native_ping.rs	Ping example; publishes Twist at 5 Hz and logs echoes; straightforward use of the NativeModule API.
native/rust/examples/native_pong.rs	Pong example; receives Twist and echoes with config value embedded; correctly uses serde deny_unknown_fields.
native/rust/examples/rust_ping_pong.py	Python orchestrator for the ping-pong end-to-end test; cleanly declares two NativeModule subclasses and wires them with autoconnect.
native/rust/Cargo.toml	Declares dimos-lcm from a git branch; acceptable for a new crate, though a published crate or path dependency would be more stable long-term.

Sequence Diagram

sequenceDiagram
    participant Py as NativeModule (Python)
    participant Proc as Rust Process
    participant BG as Background Task (tokio)
    participant LCM as LCM Bus

    Py->>Proc: spawn(executable + CLI args)
    Py->>Proc: stdin: {"topics": {...}, "config": {...}}
    Proc->>Proc: from_stdin() — parse topics & config
    Proc->>Proc: module.input() / module.output() — register routes
    Proc->>BG: module.spawn() — start select! loop

    loop message loop
        LCM-->>BG: transport.recv() -> (channel, data)
        BG->>BG: match route.topic() -> try_dispatch(data)
        BG-->>Proc: mpsc channel delivers typed message
        Proc->>BG: Output::publish(msg) -> mpsc send
        BG->>LCM: transport.publish(topic, data)
    end

    Py->>Proc: SIGTERM (on stop())
    Proc-->>Py: exit

Loading

Comments Outside Diff (1)

1. dimos/core/native_module.py, line 14-16 (link)

   Module docstring still says "C/C++" only

   The module-level docstring describes NativeModule as "a thin Python Module subclass … for native (C/C++) executables". Now that Rust is a first-class supported language (with its own SDK and examples), the docstring should reflect that.

   Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

_{Reviews (1): Last reviewed commit: "Pre commit" | Re-trigger Greptile}

[leshy]

small tweaks

[leshy]

I don't actually know if this should be in our main repo and how rust manages packages, but you can own these decisions