sdk/runanywhere-commons/docs/ARCHITECTURE.md
RunAnywhere Commons (runanywhere-commons) is the shared C++ foundation for the RunAnywhere SDK ecosystem. It provides a unified abstraction layer over multiple ML inference backends, enabling platform SDKs (Swift, Kotlin, Flutter) to access on-device AI capabilities through a consistent C API.
The cross-SDK ownership rules for proto types, public SDK models, and shared
C++ business logic are documented in
docs/CPP_PROTO_OWNERSHIP.md.
Unlike traditional C++ virtual inheritance, all service abstractions use C-style vtables:
// Service interface = struct with ops pointer + implementation handle
typedef struct rac_llm_service {
const rac_llm_service_ops_t* ops; // Function pointers
void* impl; // Backend-specific handle
const char* model_id;
} rac_llm_service_t;
// Operations vtable - each backend provides one
typedef struct rac_llm_service_ops {
rac_result_t (*initialize)(void* impl, const char* model_path);
rac_result_t (*generate)(void* impl, const char* prompt,
const rac_llm_options_t* options,
rac_llm_result_t* out_result);
rac_result_t (*generate_stream)(void* impl, const char* prompt,
const rac_llm_options_t* options,
rac_llm_stream_callback_fn callback,
void* user_data);
rac_result_t (*cancel)(void* impl);
void (*destroy)(void* impl);
} rac_llm_service_ops_t;
Rationale:
Each backend ships a single rac_engine_vtable_t (see
include/rac/plugin/rac_engine_vtable.h) whose metadata.abi_version
must match RAC_PLUGIN_API_VERSION (3u, in
include/rac/plugin/rac_plugin_entry.h). The vtable carries 8 active
per-primitive op-struct slots (LLM / STT / TTS / VAD / embeddings / rerank /
VLM / diffusion) plus 10 reserved slots; primitives the engine does not
serve are left NULL.
// engines/llamacpp/rac_plugin_entry_llamacpp.cpp (per-engine example)
static const rac_engine_vtable_t g_llamacpp_vtable = {
.metadata = {
.abi_version = RAC_PLUGIN_API_VERSION, // 3u
.name = "llamacpp",
.display_name= "llama.cpp",
// ...
.priority = 100,
},
.capability_check = llamacpp_capability_check,
.llm_ops = &g_llamacpp_llm_ops, // populated primitive
.stt_ops = NULL, // not served
// ...
};
// Static build (iOS / WASM / RAC_STATIC_PLUGINS=ON): auto-registers
// at image load time through a constructor.
RAC_STATIC_PLUGIN_REGISTER(llamacpp);
// Shared build: host loads the plugin through the loader.
rac_registry_load_plugin("/path/to/librunanywhere_llamacpp.dylib");
// ... later ...
rac_registry_unload_plugin("llamacpp");
Resolution Flow:
metadata.abi_version is validated against
RAC_PLUGIN_API_VERSION; mismatch returns
RAC_ERROR_ABI_VERSION_MISMATCH.capability_check is invoked once; a non-zero return rejects the
plugin silently (e.g. Metal-only engines on Linux).include/rac/plugin/rac_plugin_entry.h) returns the
highest-priority plugin that serves the requested primitive (plain priority
order via rac_plugin_find) and dispatches through its op-struct.┌─────────────────────────────────────────────────────────────────────────┐
│ Platform SDK Layer │
│ │
│ ┌──────────────────┐ ┌──────────────────┐ ┌────────────────────────┐ │
│ │ Swift SDK │ │ Kotlin SDK │ │ Flutter SDK │ │
│ │ (CRACommons) │ │ (JNI Bridge) │ │ (Dart FFI) │ │
│ └────────┬─────────┘ └────────┬─────────┘ └───────────┬────────────┘ │
└───────────│──────────────────────│───────────────────────│──────────────┘
│ │ │
└──────────────────────┼───────────────────────┘
│
C API (rac_*)
│
┌──────────────────────────────────▼──────────────────────────────────────┐
│ RAC Public API Layer │
│ │
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ ┌───────────┐ │
│ │ rac_llm.h │ │ rac_stt.h │ │ rac_tts.h │ │ rac_vad.h │ │
│ │ LLM Service │ │ STT Service │ │ TTS Service │ │ VAD Svc │ │
│ └───────┬───────┘ └───────┬───────┘ └───────┬───────┘ └─────┬─────┘ │
│ │ │ │ │ │
│ ┌───────▼──────────────────▼──────────────────▼────────────────▼─────┐ │
│ │ Plugin Registry │ │
│ │ ABI-versioned rac_engine_vtable_t (RAC_PLUGIN_API_VERSION = 4u) │ │
│ │ priority-order primitive op-struct dispatch (rac_plugin_find) │ │
│ └────────────────────────────────┬────────────────────────────────────┘ │
└───────────────────────────────────│─────────────────────────────────────┘
│
│ rac_engine_vtable_t dispatch
│
┌───────────────────────────────────▼─────────────────────────────────────┐
│ Backend Layer │
│ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ LlamaCPP │ │ Sherpa │ │ ONNX │ │
│ │ Backend │ │ Backend │ │ Backend │ │
│ │ │ │ │ │ │ │
│ │ • GGUF models │ │ • STT (Sherpa) │ │ • Embeddings │ │
│ │ • Metal GPU │ │ • TTS (Piper) │ │ • ONNX Runtime │ │
│ │ • Streaming │ │ • VAD (Silero) │ │ │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐│
│ │ Platform Backend (Apple only) ││
│ │ • Apple Foundation Models (LLM via Swift callbacks) ││
│ │ • System TTS (AVSpeechSynthesizer via Swift callbacks) ││
│ └─────────────────────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────────────────┘
│
│
┌───────────────────────────────────▼─────────────────────────────────────┐
│ Infrastructure Layer │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────────┐ │
│ │ Logging │ │ Events │ │ Errors │ │ Platform Adapter│ │
│ │ System │ │ System │ │ Handling │ │ (Callbacks) │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────────┘ │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────────────────┐ │
│ │ Module │ │ Model │ │ Telemetry Manager │ │
│ │ Registry │ │ Registry │ │ (Analytics events to SDK) │ │
│ └─────────────┘ └─────────────┘ └─────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────┘
runanywhere-commons/
├── include/rac/ # Public C headers (rac_* prefix)
│ ├── core/ # Core infrastructure
│ │ ├── rac_core.h # Main SDK initialization
│ │ ├── rac_error.h # Error codes (-100 to -999)
│ │ ├── rac_types.h # Basic types, handles, strings
│ │ ├── rac_logger.h # Logging interface
│ │ ├── rac_events.h # Event system
│ │ ├── rac_platform_adapter.h # Platform callbacks
│ │ └── capabilities/
│ │ └── rac_lifecycle.h # Component lifecycle states
│ │
│ ├── features/ # Service interfaces
│ │ ├── llm/ # Large Language Models
│ │ │ ├── rac_llm_service.h # LLM vtable interface
│ │ │ ├── rac_llm_types.h # LLM data structures
│ │ │ └── rac_llm.h # Public API wrapper
│ │ ├── stt/ # Speech-to-Text
│ │ │ ├── rac_stt_service.h # STT vtable interface
│ │ │ ├── rac_stt_types.h # STT data structures
│ │ │ └── rac_stt.h # Public API
│ │ ├── tts/ # Text-to-Speech
│ │ │ ├── rac_tts_service.h # TTS vtable interface
│ │ │ ├── rac_tts_types.h # TTS data structures
│ │ │ └── rac_tts.h # Public API
│ │ ├── vad/ # Voice Activity Detection
│ │ │ ├── rac_vad_service.h # VAD vtable interface
│ │ │ ├── rac_vad_types.h # VAD data structures
│ │ │ └── rac_vad.h # Public API
│ │ ├── voice_agent/ # Complete voice pipeline
│ │ │ └── rac_voice_agent.h # STT+LLM+TTS+VAD orchestration
│ │ └── platform/ # Platform-specific backends
│ │ ├── rac_llm_platform.h # Apple Foundation Models
│ │ └── rac_tts_platform.h # Apple System TTS
│ │
│ ├── infrastructure/ # Support services
│ │ ├── model_management/ # Model registry and lifecycle
│ │ ├── network/ # Network types and endpoints
│ │ ├── device/ # Device management
│ │ ├── storage/ # Storage analysis
│ │ └── telemetry/ # Analytics
│ │
│ └── backends/ # Backend-specific public headers
│ ├── rac_llm_llamacpp.h # LlamaCPP backend API
│ └── rac_embeddings_onnx.h # Generic ONNX embeddings API
│
├── src/ # Implementation files
│ ├── core/ # Core implementations
│ ├── infrastructure/ # Infrastructure implementations
│ │ ├── registry/ # Module & service registries
│ │ ├── model_management/ # Model handling
│ │ ├── network/ # HTTP client, auth
│ │ └── telemetry/ # Telemetry manager
│ ├── features/ # Feature implementations
│ │ ├── llm/ # LLM component & service
│ │ ├── stt/ # STT component & service
│ │ ├── tts/ # TTS component & service
│ │ ├── vad/ # VAD component & energy VAD
│ │ ├── voice_agent/ # Voice agent orchestration
│ │ └── platform/ # Platform backend stubs
│ ├── plugin/ # Engine plugin registry + loader
│ ├── router/ # Engine router (HW profile, hints)
│ └── jni/ # JNI bridge for Android
│
│ # ML engine plugins live at the monorepo root under ../../engines/
│ # (llamacpp, sherpa, onnx, cloud, qhexrt, coreml). Each
│ # ships a rac_plugin_entry_<name>.cpp that publishes a
│ # rac_engine_vtable_t via RAC_STATIC_PLUGIN_REGISTER or a dlopen'd
│ # entry symbol. See ../../engines/.
│
├── cmake/ # CMake modules
├── scripts/ # Build automation
├── third_party/ # Pre-built dependencies
├── dist/ # Build outputs (xcframeworks)
├── CMakeLists.txt # Main CMake configuration
├── VERSION # Project version
└── VERSIONS # Dependency versions
The library must be initialized before use:
// Required: Platform adapter with callbacks
rac_platform_adapter_t adapter = {
.file_exists = my_file_exists,
.file_read = my_file_read,
.log = my_log_callback,
.now_ms = my_get_time_ms,
.user_data = my_context
};
rac_config_t config = {
.platform_adapter = &adapter,
.log_level = RAC_LOG_INFO,
.log_tag = "MyApp"
};
rac_result_t result = rac_init(&config);
Initialization Flow:
(Engine plugins self-register into the plugin registry at image load /
rac_registry_load_plugin time, not during rac_init.)
Historical: v2 had a separate module registry (
rac_module_register/rac_module_list) where backends declared coarse capabilities. It was removed in v3; engine plugins (below) are now the only registration unit, and capabilities are derived from the populatedrac_engine_vtable_tprimitive op-structs.
Engine plugins are the single dispatch primitive in v3. Each plugin
publishes one rac_engine_vtable_t (see
include/rac/plugin/rac_engine_vtable.h) whose ABI is versioned by
RAC_PLUGIN_API_VERSION in include/rac/plugin/rac_plugin_entry.h:
// Static-linked plugin (iOS / WASM / RAC_STATIC_PLUGINS=ON):
// engines/llamacpp/rac_plugin_entry_llamacpp.cpp declares the vtable
// and calls RAC_STATIC_PLUGIN_REGISTER(llamacpp); a constructor wires
// the vtable into the registry at image load time.
// Dynamically-loaded plugin (desktop + server):
rac_result_t rc = rac_registry_load_plugin(
"/path/to/librunanywhere_llamacpp.dylib");
// ... use primitives via their public rac_llm_* / rac_stt_* / ... APIs ...
rac_registry_unload_plugin("llamacpp");
Service creation flows through the matching primitive's public API
(rac_llm_create, rac_stt_create, ...), which the router resolves
against the registered plugins:
rac_handle_t llm;
rac_llm_create("my-model-id", &llm); // router picks a capable plugin
Dispatch Algorithm:
metadata.abi_version is validated against RAC_PLUGIN_API_VERSION;
mismatch returns RAC_ERROR_ABI_VERSION_MISMATCH and the plugin is
rejected.capability_check is invoked once; non-zero quietly filters the
plugin out (e.g. Metal engines on Linux).include/rac/plugin/rac_plugin_entry.h) selects the
highest-priority plugin whose op-struct for the requested primitive is
non-NULL (plain priority order via rac_plugin_find); a name-pinned
engine can be requested via rac_plugin_find_for_engine.Runtimes (include/rac/plugin/rac_runtime_vtable.h) describe the
compute target an engine runs on. Only the built-in CPU runtime fills the
optional session-execution slots; Metal, Core ML, and ONNX Runtime are
capability-only — they advertise device capabilities for routing and leave
session execution to the engine.
Unified logging through platform adapter:
// Logging macros
RAC_LOG_DEBUG("LLM.LlamaCpp", "Loading model: %s", model_path);
RAC_LOG_INFO("ServiceRegistry", "Provider registered: %s", name);
RAC_LOG_WARNING("VAD", "Energy threshold too low: %f", threshold);
RAC_LOG_ERROR("STT", "Transcription failed: %s", rac_error_message(result));
// Implementation routes to platform
void rac_log(rac_log_level_t level, const char* category, const char* message) {
const rac_platform_adapter_t* adapter = rac_get_platform_adapter();
if (adapter && adapter->log) {
adapter->log(level, category, message, adapter->user_data);
}
}
Log Levels:
RAC_LOG_TRACE (0) - Verbose debuggingRAC_LOG_DEBUG (1) - Debug informationRAC_LOG_INFO (2) - General informationRAC_LOG_WARNING (3) - WarningsRAC_LOG_ERROR (4) - ErrorsRAC_LOG_FATAL (5) - Fatal errorsEach capability (LLM, STT, TTS, VAD) follows the same pattern:
// 1. Types header (rac_<cap>_types.h)
typedef struct rac_<cap>_options { ... } rac_<cap>_options_t;
typedef struct rac_<cap>_result { ... } rac_<cap>_result_t;
// 2. Service interface (rac_<cap>_service.h)
typedef struct rac_<cap>_service_ops {
rac_result_t (*initialize)(void* impl, ...);
rac_result_t (*process)(void* impl, ...);
void (*destroy)(void* impl);
} rac_<cap>_service_ops_t;
typedef struct rac_<cap>_service {
const rac_<cap>_service_ops_t* ops;
void* impl;
const char* model_id;
} rac_<cap>_service_t;
// 3. Public API (rac_<cap>.h)
RAC_API rac_result_t rac_<cap>_create(const char* id, rac_handle_t* out);
RAC_API rac_result_t rac_<cap>_process(rac_handle_t h, ...);
RAC_API void rac_<cap>_destroy(rac_handle_t h);
Types:
typedef struct rac_llm_options {
int32_t max_tokens; // Maximum tokens to generate
float temperature; // Sampling temperature (0.0-2.0)
float top_p; // Nucleus sampling threshold
int32_t top_k; // Top-k sampling
const char* system_prompt;
rac_bool_t streaming_enabled;
} rac_llm_options_t;
typedef struct rac_llm_result {
char* text; // Generated text (owned)
int32_t input_tokens; // Prompt tokens
int32_t output_tokens; // Generated tokens
double duration_ms; // Generation time
double tokens_per_second;
double time_to_first_token_ms;
char* thinking_content; // Reasoning (if supported)
} rac_llm_result_t;
Streaming Callback:
typedef rac_bool_t (*rac_llm_stream_callback_fn)(
const char* token, // Generated token
rac_bool_t is_final, // Is this the last token?
void* user_data
);
// Return RAC_FALSE to stop generation
Types:
typedef struct rac_stt_options {
const char* language; // Language code (e.g., "en")
rac_bool_t detect_language;
rac_bool_t enable_timestamps;
int32_t sample_rate; // Audio sample rate
} rac_stt_options_t;
typedef struct rac_stt_result {
char* text; // Transcribed text
float confidence; // 0.0-1.0
const char* language; // Detected language
double duration_ms; // Processing time
// Word timestamps (optional)
} rac_stt_result_t;
Types:
typedef struct rac_tts_options {
const char* voice; // Voice identifier
const char* language; // Language code
float rate; // Speaking rate (0.5-2.0)
float pitch; // Voice pitch (0.5-2.0)
int32_t sample_rate; // Output sample rate
} rac_tts_options_t;
typedef struct rac_tts_result {
void* audio_data; // PCM audio (owned)
size_t audio_size; // Size in bytes
double duration_seconds; // Audio duration
int32_t sample_rate; // Sample rate
} rac_tts_result_t;
Types:
typedef struct rac_vad_result {
rac_bool_t has_speech; // Speech detected
float confidence; // Detection confidence
double speech_start_ms; // Speech start time
double speech_end_ms; // Speech end time
} rac_vad_result_t;
Architecture:
┌─────────────────────────────────────────────────────────────┐
│ rac_llm_llamacpp.h │
│ Public API + Registration │
└────────────────────────────┬────────────────────────────────┘
│
┌────────────────────────────▼────────────────────────────────┐
│ rac_backend_llamacpp_register.cpp │
│ • Registers module with capabilities │
│ • Registers service provider │
│ • Implements can_handle (checks .gguf extension) │
│ • Implements create (creates service with vtable) │
└────────────────────────────┬────────────────────────────────┘
│
┌────────────────────────────▼────────────────────────────────┐
│ llamacpp_backend.cpp │
│ LlamaCppBackend class: │
│ • initialize() - Init llama.cpp backend │
│ • cleanup() - Free resources │
│ LlamaCppTextGeneration class: │
│ • load_model() - Load GGUF model │
│ • generate() - Blocking generation │
│ • generate_stream() - Streaming generation │
│ • cancel() - Abort generation │
└────────────────────────────┬────────────────────────────────┘
│
llama.cpp library
Key Implementation Details:
Model Loading:
llama_model_load_from_file()Text Generation:
common_tokenize()llama_chat_apply_template()Cancellation:
Architecture:
┌─────────────────────────────────────────────────────────────┐
│ rac_plugin_entry_sherpa.h + feature APIs │
│ Public registration and lifecycle/proto inference │
└────────────────────────────┬────────────────────────────────┘
│
┌────────────────────────────▼────────────────────────────────┐
│ rac_backend_sherpa_register.cpp │
│ • Registers Sherpa module │
│ • Registers STT, TTS, VAD providers │
│ • Implements unified engine vtables │
└────────────────────────────┬────────────────────────────────┘
│
┌────────────────────────────▼────────────────────────────────┐
│ sherpa_backend.cpp │
│ Wraps Sherpa-ONNX C API: │
│ • STT: SherpaOnnxOfflineRecognizer │
│ • TTS: SherpaOnnxOfflineTts │
│ • VAD: SherpaOnnxVoiceActivityDetector │
└────────────────────────────┬────────────────────────────────┘
│
Sherpa-ONNX + ONNX Runtime libraries
Supported Models:
Pattern: C++ provides registration and vtable stubs; Swift provides actual implementation via callbacks.
┌─────────────────────────────────────────────────────────────┐
│ Swift SDK (RunAnywhere) │
│ Implements callbacks for Foundation Models + TTS │
└────────────────────────────┬────────────────────────────────┘
│ sets callbacks
┌────────────────────────────▼────────────────────────────────┐
│ rac_llm_platform.h / rac_tts_platform.h │
│ • rac_platform_llm_set_callbacks() │
│ • rac_platform_tts_set_callbacks() │
└────────────────────────────┬────────────────────────────────┘
│
┌────────────────────────────▼────────────────────────────────┐
│ rac_backend_platform_register.cpp │
│ • Registers "platform" module │
│ • Provider calls Swift callbacks via stored pointers │
└─────────────────────────────────────────────────────────────┘
App calls RunAnywhere.generate(prompt)
│
▼
Swift SDK validates state
Calls rac_llm_generate(handle, prompt, options, &result)
│
▼
rac_llm_generate() extracts service from handle
Calls service->ops->generate(service->impl, prompt, options, &result)
│
▼
LlamaCPP vtable generate():
1. Build chat-templated prompt
2. Tokenize prompt
3. Decode prompt tokens
4. Sample generation loop:
- Sample next token
- Check stop conditions
- Accumulate output
5. Populate result struct
│
▼
Return to Swift
Swift maps rac_llm_result_t → LLMGenerationResult
│
▼
Return to App
App calls RunAnywhere.generateStream(prompt)
│
▼
Swift SDK calls rac_llm_generate_stream()
with Swift callback wrapper
│
▼
Backend generation loop:
for each token:
callback(token, is_final, user_data)
│
▼
Swift callback wrapper:
- Maps token to Swift String
- Yields to AsyncStream
│
▼
App receives token via AsyncStream
Audio Input
│
▼
┌───────────────┐
│ VAD │ ──► Speech detected? No → Continue listening
│ (Energy/AI) │
└───────┬───────┘
│ Speech detected
▼
┌───────────────┐
│ STT │ ──► Transcribe audio to text
│ (Sherpa/Cloud)│
└───────┬───────┘
│ Transcription
▼
┌───────────────┐
│ LLM │ ──► Generate response
│ (LlamaCPP) │
└───────┬───────┘
│ Response text
▼
┌───────────────┐
│ TTS │ ──► Synthesize speech
│ (ONNX/System)│
└───────┬───────┘
│
▼
Audio Output
std::mutexstd::mutex// Atomic flag pattern
std::atomic<bool> cancel_requested_{false};
// In generation loop
while (generating) {
if (cancel_requested_.load()) {
break; // Graceful exit
}
// ... sample next token
}
// Cancel API
void cancel() {
cancel_requested_.store(true);
}
OUT parameters with * suffix: Caller owns, must free
rac_llm_result_t result; // Caller allocates struct
rac_llm_generate(..., &result);
// result.text is owned, must free with rac_llm_result_free(&result)
Static strings: Library owns, do not free
const char* msg = rac_error_message(code); // Static, do not free
Handles: Created by library, destroyed by caller
rac_handle_t handle;
rac_llm_create(..., &handle);
// ... use handle ...
rac_llm_destroy(handle); // Required
// Library allocation functions
RAC_API void* rac_alloc(size_t size);
RAC_API void rac_free(void* ptr);
RAC_API char* rac_strdup(const char* str);
// Result free functions
RAC_API void rac_llm_result_free(rac_llm_result_t* result);
RAC_API void rac_stt_result_free(rac_stt_result_t* result);
RAC_API void rac_tts_result_free(rac_tts_result_t* result);
typedef enum rac_event_type {
// LLM Events
RAC_EVENT_LLM_MODEL_LOAD_STARTED = 100,
RAC_EVENT_LLM_MODEL_LOAD_COMPLETED = 101,
RAC_EVENT_LLM_GENERATION_STARTED = 110,
RAC_EVENT_LLM_GENERATION_COMPLETED = 111,
RAC_EVENT_LLM_FIRST_TOKEN = 113,
// STT Events
RAC_EVENT_STT_TRANSCRIPTION_STARTED = 210,
RAC_EVENT_STT_TRANSCRIPTION_COMPLETED = 211,
// TTS Events
RAC_EVENT_TTS_SYNTHESIS_STARTED = 310,
RAC_EVENT_TTS_SYNTHESIS_COMPLETED = 311,
// VAD Events
RAC_EVENT_VAD_SPEECH_STARTED = 402,
RAC_EVENT_VAD_SPEECH_ENDED = 403,
} rac_event_type_t;
C++ Component (e.g., LLM generation)
│
▼
rac_event_emit(type, &data)
│
▼
Platform callback (if registered)
│
▼
Swift EventBridge / Kotlin EventBus
│
▼
App event subscription
// Platform SDK registers callback
rac_result_t rac_events_set_callback(
rac_event_callback_fn callback,
void* user_data
);
// Callback signature
typedef void (*rac_event_callback_fn)(
rac_event_type_t type,
const rac_event_data_t* data,
void* user_data
);
typedef struct rac_platform_adapter {
// File System (Required)
rac_bool_t (*file_exists)(const char* path, void* user_data);
// Logging (Required)
void (*log)(rac_log_level_t level, const char* category,
const char* message, void* user_data);
// Time (Required)
int64_t (*now_ms)(void* user_data);
// Optional
rac_result_t (*file_read)(...);
rac_result_t (*file_write)(...);
rac_result_t (*secure_get)(...); // Keychain
rac_result_t (*secure_set)(...);
rac_result_t (*http_download)(...);
rac_result_t (*extract_archive)(...);
void* user_data; // Passed to all callbacks
} rac_platform_adapter_t;
// SwiftPlatformAdapter.swift
private func createPlatformAdapter() -> rac_platform_adapter_t {
var adapter = rac_platform_adapter_t()
adapter.file_exists = { path, userData in
guard let path = path.map(String.init(cString:)) else { return RAC_FALSE }
return FileManager.default.fileExists(atPath: path) ? RAC_TRUE : RAC_FALSE
}
adapter.log = { level, category, message, userData in
guard let msg = message.map(String.init(cString:)) else { return }
SDKLogger.shared.log(level: LogLevel(rawValue: level), message: msg)
}
adapter.now_ms = { userData in
Int64(Date().timeIntervalSince1970 * 1000)
}
return adapter
}
// Success
#define RAC_SUCCESS ((rac_result_t)0)
// Error ranges
// -100 to -109: Initialization errors
#define RAC_ERROR_NOT_INITIALIZED -100
#define RAC_ERROR_ALREADY_INITIALIZED -101
// -110 to -129: Model errors
#define RAC_ERROR_MODEL_NOT_FOUND -110
#define RAC_ERROR_MODEL_LOAD_FAILED -111
// -130 to -149: Generation errors
#define RAC_ERROR_GENERATION_FAILED -130
#define RAC_ERROR_CONTEXT_TOO_LONG -132
// -400 to -499: Service errors
#define RAC_ERROR_NO_CAPABLE_PROVIDER -422
// Get error message
const char* msg = rac_error_message(result);
// Set detailed error context
rac_error_set_details("Model file not found at: /path/to/model.gguf");
// Get detailed error
const char* details = rac_error_get_details();
rac_result_t my_function() {
rac_result_t result = some_operation();
if (RAC_FAILED(result)) {
rac_error_set_details("Operation failed during my_function");
return result; // Propagate error code
}
return RAC_SUCCESS;
}
Create directory at the monorepo root: engines/<name>/.
Implement backend class:
// engines/<name>/<name>_backend.cpp
class MyBackend {
bool load_model(const std::string& path, const nlohmann::json& config);
Result generate(const std::string& prompt, const Options& options);
};
Implement primitive op-struct(s) — one per capability the engine serves (LLM / STT / TTS / VAD / embeddings / rerank / VLM / diffusion):
// engines/<name>/rac_backend_<name>_register.cpp
static const rac_llm_service_ops_t g_<name>_llm_ops = {
.initialize = ...,
.generate = ...,
.destroy = ...,
};
Publish the unified vtable in the engine's plugin entry TU:
// engines/<name>/rac_plugin_entry_<name>.cpp
#include "rac/plugin/rac_engine_vtable.h"
#include "rac/plugin/rac_plugin_entry.h"
static const rac_engine_vtable_t g_<name>_vtable = {
.metadata = {
.abi_version = RAC_PLUGIN_API_VERSION, // 3u
.name = "<name>",
.display_name = "<Display Name>",
.priority = 100,
},
.capability_check = <name>_capability_check,
.llm_ops = &g_<name>_llm_ops,
// leave other primitive slots NULL
};
RAC_STATIC_PLUGIN_REGISTER(<name>); // static builds auto-wire
Shared builds expose the same entry symbol so hosts can load the
plugin via rac_registry_load_plugin("librunanywhere_<name>.so").
Add to engines CMake: wire the new plugin through
rac_add_engine_plugin(<name> ...) in engines/CMakeLists.txt.
The helper handles static-vs-shared mode based on the top-level
RAC_STATIC_PLUGINS option (forced ON for iOS / WASM).
include/rac/features/<cap>/rac_<cap>_types.hinclude/rac/features/<cap>/rac_<cap>_service.hinclude/rac/features/<cap>/rac_<cap>.hrac_capability_tsrc/features/<cap>/tests/ directoryRAC_BUILD_TESTS=ON# Build with test support
cmake -B build -DRAC_BUILD_TESTS=ON
cmake --build build
ctest --test-dir build
Integration tests run through platform SDKs:
Tests/RunAnywhereTests/sdk/runanywhere-kotlin/src/test/Decision: Pure C API surface with C++ implementation
Rationale:
Decision: C-style vtables instead of C++ virtual inheritance
Rationale:
Decision: Multiple providers can register for same capability with priority
Rationale:
Decision: RACommons + RABackendLLAMACPP + RABackendONNX as separate frameworks
Rationale: