Skip to content

offbit-ai/reflow

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

488 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Reflow

A graph-driven actor runtime for workflows, media pipelines, and visual systems

License: MIT Rust Version WebAssembly Docs

SDK build status

Python CI offbit-reflow on PyPI Node CI @offbit-ai/reflow on npm Go SDK release ai.offbit:reflow on Maven Central C++ SDK header-only Browser-WASM (in @offbit-ai/reflow)

Documentation | Runtime Crate | Quick Start | SDKs | Examples | API Reference

What is Reflow?

Reflow is a modular workflow execution engine that uses the actor model for concurrent, message-passing computation. It enables graph-authored DAGs for data processing, real-time media, visual tooling, distributed workflows, and optional ML/CV taskpacks.

Key Features

🎭 Actor-Based Architecture - Isolated, concurrent actors communicate via typed messages
πŸ“Š Graph-Authored Workflows - DAG representation with subgraphs, IIPs, history, and undo
πŸ“¦ The Reflow Runtime API - reflow_rt is the public Rust crate for building and running Reflow applications
πŸŽ›οΈ Optional Component Families - GPU, AV, window events, camera, API services, media, and ML are feature gated
🌍 Multi-Language SDKs - first-party bindings for Node.js, Python, Go, JVM (Java + Kotlin), and C++
🌐 Cross-Platform - Native Rust execution plus WebAssembly-oriented runtime surfaces
πŸ”„ Real-Time Processing - Networking, WebSockets, streams, and live graph events
🧠 Media / ML Ready - Typed media packets, mockable inference, LiteRT backend boundary, and taskpack subgraphs

Quick Start

Use Reflow In Rust

For application code, start with the unified runtime crate:

[dependencies]
reflow_rt = "0.1"

The default reflow_rt feature set is intentionally lightweight: runtime plus core utility components. Optional stacks are selected explicitly:

# Runtime + core component catalog only.
reflow_rt = "0.1"

# Add GPU rendering/compute components.
reflow_rt = { version = "0.1", features = ["gpu"] }

# Add typed media packets and graph-driven ML/CV taskpacks.
reflow_rt = { version = "0.1", features = ["media", "ml"] }
use reflow_rt::prelude::*;

fn main() {
    let mut graph = Graph::new("example", false, None);
    graph.add_node("tap", "tpl_passthrough", None);

    let network = Network::with_graph(NetworkConfig::default(), &graph);
    let _ = network;
}

See the reflow_rt README for the runtime API surface and feature map.

Use Reflow from another language

The runtime is the same Rust core; each SDK is a thin native binding plus an idiomatic surface in that language. Pick the published artifact for your runtime, then jump to that SDK's README for quick-start code.

# Node.js β€” npm install pulls the right per-platform .node addon automatically
npm install @offbit-ai/reflow
# Python β€” pre-built wheels for darwin / linux / windows on PyPI
pip install offbit-reflow
# Go β€” install the module, then drop the platform-specific
# libreflow_rt_capi tarball into the module via the bundled script.
# See sdk/go/README.md for full instructions.
go get github.com/offbit-ai/reflow/sdk/go@v0.2.3
// Gradle (Kotlin DSL) β€” JVM SDK is the fat jar; native libs are
// bundled as classpath resources.
dependencies { implementation("ai.offbit:reflow:0.2.4") }
# C++ β€” header-only over libreflow_rt_capi; install the matching
# native lib from a sdk/go/v* GitHub Release and point CMake at it.
add_subdirectory(third_party/reflow/sdk/cpp)
target_link_libraries(myapp PRIVATE reflow::cpp)

The same @offbit-ai/reflow package falls back to a wasm bundle in the browser via the "browser" conditional export β€” Vite, webpack, and esbuild pick it up automatically. Node continues to load the napi .node addon. See sdk/node#browser-target for the runtime surface available there.

SDK Package Docs
Node.js + Browser @offbit-ai/reflow sdk/node/README.md
Python offbit-reflow sdk/python/README.md
Go github.com/offbit-ai/reflow/sdk/go sdk/go/README.md
JVM ai.offbit:reflow sdk/jvm/README.md
C++ header-only at sdk/cpp/ (links libreflow_rt_capi) sdk/cpp/README.md

Optional .rflpack bundles plug additional actor palettes (GPU renderers, ML, browser automation, ~6,700 SaaS API actors) into any SDK at runtime β€” load via loadPack() / load_pack() / LoadPack() / Packs.loadPack().

Architecture Overview

Reflow's architecture is built around three core concepts:

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    Messages    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    Messages    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚     Actor A     │────────────────▢│     Actor B     │────────────────▢│     Actor C     β”‚
β”‚  (JavaScript)   β”‚                 β”‚    (Python)     β”‚                 β”‚     (Rust)      β”‚
β”‚                 β”‚                 β”‚                 β”‚                 β”‚                 β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚                 β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚                 β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚ β”‚Input Ports  β”‚ β”‚                 β”‚ β”‚Input Ports  β”‚ β”‚                 β”‚ β”‚Input Ports  β”‚ β”‚
β”‚ β”‚Output Ports β”‚ β”‚                 β”‚ β”‚Output Ports β”‚ β”‚                 β”‚ β”‚Output Ports β”‚ β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚                 β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚                 β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
  • Actors: Isolated units of computation that process messages
  • Messages: Strongly-typed data passed between actors
  • Graphs: Visual representation of actor connections and data flow

Project Structure

This workspace contains multiple crates that work together:

Core Engine

  • reflow_rt - The public Rust runtime crate for Reflow applications
  • reflow_network - Core actor runtime and message routing
  • reflow_components - Standard library of reusable actors
  • reflow_graph - Graph data model, exports, nodes, connections, and IIPs
  • reflow_actor - Actor traits, contexts, messages, streams, and state
  • actor_macro - Procedural macros for actor creation

Media, Graphics & Assets

  • reflow_assets - Content-addressed asset database conventions
  • reflow_pixel - Pixel/image helpers for media components
  • reflow_sdf - SDF primitives and procedural geometry/codegen support
  • reflow_shader - Shader graph IR and WGSL generation
  • reflow_vector - 2D vector paths, booleans, and rasterization
  • reflow_media_types / reflow_media_codec - Optional typed media and tensor packet contracts
  • reflow_litert / reflow_ml_ops / reflow_cv_ops / reflow_taskpacks - Optional ML/CV stack and taskpack graph builders

Language Runtimes

  • reflow_js - JavaScript/Deno runtime integration
  • reflow_py - Python runtime integration
  • reflow_wasm_actor - WebAssembly runtime and browser support

Execution & Deployment

  • reflow_script - Multi-language script execution
  • reflow_server - HTTP server and API endpoints

Examples & Tools

  • examples/ - Working examples and tutorials
  • docs/ - Comprehensive documentation

Use Cases

Data Processing Pipelines

// ETL pipeline with error handling
source β†’ validate β†’ transform β†’ load β†’ audit

Real-Time Analytics

// Live data processing
websocket β†’ parse β†’ aggregate β†’ alert β†’ dashboard

IoT Data Processing

// Sensor data workflow  
mqtt β†’ decode β†’ filter β†’ analyze β†’ store β†’ notify

Media Processing

// Audio/video pipeline
upload β†’ decode β†’ process β†’ encode β†’ publish

Documentation

πŸ“– Complete Documentation

Getting Started

Architecture & Design

API Documentation

Advanced Topics

Reference

Examples

Explore working examples in the examples/ directory:

WebAssembly Actor

Demonstrates how to create and deploy actors as WebAssembly modules.

Development

Building from Source

# Clone the repository
git clone https://github.com/offbit-ai/reflow.git
cd reflow

# Build all crates
cargo build

# Run tests
cargo test

# Build with optimizations
cargo build --release

WebAssembly Support

# Install wasm-pack
curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh

# Build WebAssembly package
cd crates/reflow_network
wasm-pack build --target web

Development Tools

# Install development dependencies
cargo install cargo-watch
cargo install flamegraph

# Run with hot reloading
cargo watch -x run

# Performance profiling
cargo flamegraph --bin reflow-example

Performance

Reflow is designed for high-performance execution:

  • Zero-copy message passing where possible
  • Parallel actor execution with work-stealing schedulers
  • Memory pooling for frequently allocated objects
  • SIMD optimizations for numeric processing
  • Async I/O throughout the system

Benchmark results on modern hardware:

  • 1M+ messages/second processing throughput
  • Sub-millisecond actor-to-actor latency
  • Linear scaling with CPU core count

Contributing

We welcome contributions! Please see our Contributing Guide for details.

Areas We Need Help With

  • πŸ› Bug Reports - Found an issue? Let us know!
  • πŸ“ Documentation - Help improve our guides and examples
  • πŸ”Œ Components - Build reusable actors for the community
  • 🌐 Language Bindings - Add support for more runtimes
  • ⚑ Performance - Optimization opportunities
  • πŸ§ͺ Testing - Expand our test coverage

Development Process

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Make your changes and add tests
  4. Ensure all tests pass (cargo test)
  5. Commit your changes (git commit -m 'Add amazing feature')
  6. Push to the branch (git push origin feature/amazing-feature)
  7. Open a Pull Request

Community

  • GitHub Discussions - Ask questions and share ideas

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

Built with ❀️ using:

  • Rust - Systems programming language
  • Tokio - Asynchronous runtime

⭐ Star us on GitHub if you find Reflow useful! ⭐

About

Actor-based DAG workflow execution engine for low-code applications

Topics

Resources

License

Unknown and 2 other licenses found

Licenses found

Unknown
LICENSE
Apache-2.0
LICENSE-APACHE
MIT
LICENSE-MIT

Stars

Watchers

Forks

Packages

 
 
 

Contributors