Back to Runanywhere Sdks

RunAnywhere React Native SDK

sdk/runanywhere-react-native/README.md

0.19.723.7 KB
Original Source

RunAnywhere React Native SDK

On-device AI for React Native. Run LLMs, Speech-to-Text, Text-to-Speech, and Voice AI locally with privacy-first, offline-capable inference.

<p align="center"> <a href="#"></a> <a href="#"></a> <a href="#"></a> <a href="#"></a> <a href="LICENSE"></a> </p>

Features

Large Language Models (LLM)

  • On-device text generation with streaming support
  • LlamaCPP backend for GGUF models (Llama 2, Mistral, SmolLM, Qwen, etc.)
  • Metal GPU acceleration on iOS, CPU + NNAPI on Android
  • System prompts and customizable generation parameters
  • Support for thinking/reasoning models
  • Token streaming with real-time callbacks

Speech-to-Text (STT)

  • Real-time and batch audio transcription
  • Multi-language support with Whisper models via ONNX Runtime
  • Word-level timestamps and confidence scores
  • Voice Activity Detection (VAD) integration

Text-to-Speech (TTS)

  • Neural voice synthesis with Piper TTS
  • System voices via platform TTS (AVSpeechSynthesizer / Android TTS)
  • Streaming audio generation for long text
  • Customizable voice, pitch, rate, and volume

Voice Activity Detection (VAD)

  • Energy-based speech detection with Silero VAD
  • Configurable sensitivity thresholds
  • Real-time audio stream processing

Voice Agent Pipeline

  • Full VAD → STT → LLM → TTS orchestration
  • Complete voice conversation flow
  • Push-to-talk and hands-free modes

Infrastructure

  • Automatic model discovery and download with progress tracking
  • Comprehensive event system via EventBus
  • Built-in analytics and telemetry
  • Structured logging with multiple log levels
  • Keychain-persisted device identity (iOS) / EncryptedSharedPreferences (Android)

System Requirements

ComponentMinimumRecommended
React Native0.71+0.74+
iOS15.1+17.0+
AndroidAPI 24 (7.0+)API 28+
Node.js18+20+
Xcode15+16+
Android StudioHedgehog+Latest
RAM3GB6GB+ for 7B models
StorageVariableModels: 200MB–8GB

Apple Silicon devices (M1/M2/M3, A14+) and Android devices with 6GB+ RAM are recommended. Metal GPU acceleration provides 3-5x speedup on iOS.


Multi-Package Architecture

This SDK uses a modular multi-package architecture. Install only the packages you need:

PackageDescriptionRequired
@runanywhere/coreCore SDK infrastructure, public API, events, model registryYes
@runanywhere/llamacppLlamaCPP backend for LLM text generation (GGUF models)For LLM
@runanywhere/onnxONNX Runtime backend for STT/TTS (Whisper, Piper)For Voice

Installation

Full Installation (All Features)

bash
npm install @runanywhere/core @runanywhere/llamacpp @runanywhere/onnx
# or
yarn add @runanywhere/core @runanywhere/llamacpp @runanywhere/onnx

Minimal Installation (LLM Only)

bash
npm install @runanywhere/core @runanywhere/llamacpp

Minimal Installation (STT/TTS Only)

bash
npm install @runanywhere/core @runanywhere/onnx

iOS Setup

bash
cd ios && pod install && cd ..

Android Setup

No additional setup required. Native libraries are automatically downloaded during the Gradle build.


Quick Start

1. Initialize the SDK

typescript
import { RunAnywhere, SDKEnvironment, ModelCategory } from '@runanywhere/core';
import { LlamaCPP } from '@runanywhere/llamacpp';
import { ONNX, ModelArtifactType } from '@runanywhere/onnx';

// Initialize SDK (development mode - no API key needed)
await RunAnywhere.initialize({
  environment: SDKEnvironment.Development,
});

// Register LlamaCpp module and add LLM models
LlamaCPP.register();
await LlamaCPP.addModel({
  id: 'smollm2-360m-q8_0',
  name: 'SmolLM2 360M Q8_0',
  url: 'https://huggingface.co/prithivMLmods/SmolLM2-360M-GGUF/resolve/main/SmolLM2-360M.Q8_0.gguf',
  memoryRequirement: 500_000_000,
});

// Register ONNX module and add STT/TTS models
ONNX.register();
await ONNX.addModel({
  id: 'sherpa-onnx-whisper-tiny.en',
  name: 'Sherpa Whisper Tiny (ONNX)',
  url: 'https://github.com/RunanywhereAI/sherpa-onnx/releases/download/runanywhere-models-v1/sherpa-onnx-whisper-tiny.en.tar.gz',
  modality: ModelCategory.SpeechRecognition,
  artifactType: ModelArtifactType.TarGzArchive,
  memoryRequirement: 75_000_000,
});

console.log('SDK initialized');

2. Download & Load a Model

typescript
// Download model with progress tracking
await RunAnywhere.downloadModel('smollm2-360m-q8_0', (progress) => {
  console.log(`Download: ${(progress.progress * 100).toFixed(1)}%`);
});

// Load model into memory
const modelInfo = await RunAnywhere.getModelInfo('smollm2-360m-q8_0');
if (modelInfo?.localPath) {
  await RunAnywhere.loadModel(modelInfo.localPath);
}

// Check if model is loaded
const isLoaded = await RunAnywhere.isModelLoaded();
console.log('Model loaded:', isLoaded);

3. Generate Text

typescript
// Simple chat
const response = await RunAnywhere.chat('What is the capital of France?');
console.log(response);  // "Paris is the capital of France."

// With options
const result = await RunAnywhere.generate(
  'Explain quantum computing in simple terms',
  {
    maxTokens: 200,
    temperature: 0.7,
    systemPrompt: 'You are a helpful assistant.',
  }
);

console.log('Response:', result.text);
console.log('Speed:', result.performanceMetrics.tokensPerSecond, 'tok/s');
console.log('Latency:', result.latencyMs, 'ms');

4. Streaming Generation

typescript
const streamResult = await RunAnywhere.generateStream(
  'Write a short poem about AI',
  { maxTokens: 150 }
);

// Display tokens in real-time
for await (const token of streamResult.stream) {
  process.stdout.write(token);
}

// Get final metrics
const metrics = await streamResult.result;
console.log('\nSpeed:', metrics.performanceMetrics.tokensPerSecond, 'tok/s');

5. Speech-to-Text

typescript
// Download and load STT model
await RunAnywhere.downloadModel('sherpa-onnx-whisper-tiny.en');
const sttModel = await RunAnywhere.getModelInfo('sherpa-onnx-whisper-tiny.en');
await RunAnywhere.loadSTTModel(sttModel.localPath, 'whisper');

// Transcribe audio file
const result = await RunAnywhere.transcribeFile(audioFilePath, {
  language: 'en',
});

console.log('Transcription:', result.text);
console.log('Confidence:', result.confidence);

6. Text-to-Speech

typescript
// Download and load TTS model
await RunAnywhere.downloadModel('vits-piper-en_US-lessac-medium');
const ttsModel = await RunAnywhere.getModelInfo('vits-piper-en_US-lessac-medium');
await RunAnywhere.loadTTSModel(ttsModel.localPath, 'piper');

// Synthesize speech
const output = await RunAnywhere.synthesize(
  'Hello from the RunAnywhere SDK.',
  { rate: 1.0, pitch: 1.0, volume: 1.0 }
);

// output.audio contains base64-encoded float32 PCM
// output.sampleRate, output.numSamples, output.duration

Architecture Overview

The RunAnywhere SDK follows a modular, provider-based architecture with a shared C++ core:

┌─────────────────────────────────────────────────────────────────┐
│                     Your React Native App                        │
├─────────────────────────────────────────────────────────────────┤
│              @runanywhere/core (TypeScript API)                  │
│  ┌──────────────┐  ┌───────────────┐  ┌──────────────────────┐  │
│  │ RunAnywhere  │  │  EventBus     │  │  ModelRegistry       │  │
│  │ (public API) │  │  (events,     │  │  (model discovery,   │  │
│  │              │  │   callbacks)  │  │   download, storage) │  │
│  └──────────────┘  └───────────────┘  └──────────────────────┘  │
├────────────┬─────────────────────────────────────┬──────────────┤
│            │                                     │              │
│  ┌─────────▼─────────┐             ┌────────────▼────────────┐ │
│  │ @runanywhere/     │             │  @runanywhere/onnx      │ │
│  │    llamacpp       │             │  (STT/TTS/VAD)          │ │
│  │  (LLM/GGUF)       │             │                         │ │
│  └─────────┬─────────┘             └────────────┬────────────┘ │
├────────────┼─────────────────────────────────────┼──────────────┤
│            │          Nitrogen/Nitro JSI         │              │
│            │          (Native Bridge Layer)      │              │
├────────────┼─────────────────────────────────────┼──────────────┤
│  ┌─────────▼──────────────────────────────────────▼───────────┐ │
│  │              runanywhere-commons (C++)                      │ │
│  │  ┌────────────────┐  ┌────────────────┐  ┌───────────────┐ │ │
│  │  │ RACommons      │  │ RABackend      │  │ RABackendONNX │ │ │
│  │  │ (Core Engine)  │  │ LLAMACPP       │  │ (Sherpa-ONNX) │ │ │
│  │  └────────────────┘  └────────────────┘  └───────────────┘ │ │
│  └────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘

Key Components

ComponentDescription
RunAnywhereMain SDK singleton providing all public methods
EventBusEvent subscription system for SDK events (initialization, generation, model, voice)
ModelRegistryManages model metadata, discovery, and download tracking
ServiceContainerDependency injection for internal services
FileSystemCross-platform file operations for model storage
DownloadServiceModel download with progress, resume, and extraction

Native Binaries

FrameworkSizeProvides
RACommons.xcframework / librac_commons.so~2MBCore C++ commons, registries, events
RABackendLLAMACPP.xcframework / librunanywhere_llamacpp.so~15-25MBLLM capability (GGUF models)
RABackendONNX.xcframework / librunanywhere_onnx.so~50-70MBSTT, TTS, VAD (ONNX models)

Configuration

SDK Initialization Parameters

typescript
// Development mode (default) - no API key needed
await RunAnywhere.initialize({
  environment: SDKEnvironment.Development,
});

// Production mode - requires API key
await RunAnywhere.initialize({
  apiKey: '<YOUR_API_KEY>',
  baseURL: 'https://api.runanywhere.ai',
  environment: SDKEnvironment.Production,
});

Environment Modes

EnvironmentDescription
.DevelopmentVerbose logging, local backend, no auth required
.StagingTesting with real services
.ProductionMinimal logging, full authentication, telemetry

Generation Options

typescript
const options: GenerationOptions = {
  maxTokens: 256,              // Maximum tokens to generate
  temperature: 0.7,            // Sampling temperature (0.0–2.0)
  topP: 0.95,                  // Top-p sampling parameter
  stopSequences: ['END'],      // Stop generation at these sequences
  systemPrompt: 'You are a helpful assistant.',
};

Error Handling

The SDK provides structured error handling through SDKError:

typescript
import { SDKError, SDKErrorCode, isSDKError } from '@runanywhere/core';

try {
  const response = await RunAnywhere.generate('Hello!');
} catch (error) {
  if (isSDKError(error)) {
    switch (error.code) {
      case SDKErrorCode.notInitialized:
        console.log('SDK not initialized. Call RunAnywhere.initialize() first.');
        break;
      case SDKErrorCode.modelNotFound:
        console.log('Model not found. Download it first.');
        break;
      case SDKErrorCode.insufficientMemory:
        console.log('Not enough memory. Try a smaller model.');
        break;
      default:
        console.log('Error:', error.message);
    }
  }
}

Error Categories

CategoryDescription
generalGeneral SDK errors
llmLLM generation errors
sttSpeech-to-text errors
ttsText-to-speech errors
vadVoice activity detection errors
voiceAgentVoice pipeline errors
downloadModel download errors
networkNetwork-related errors
authenticationAuth and API key errors

Logging & Observability

Configure Logging

typescript
import { LogLevel, SDKLogger } from '@runanywhere/core';

// Set minimum log level
RunAnywhere.setLogLevel(LogLevel.Debug);  // debug, info, warning, error, fault

// Create a custom logger
const logger = new SDKLogger('MyApp');
logger.info('App started');
logger.debug('Debug info', { modelId: 'llama-2' });

Subscribe to Events

typescript
// Subscribe to generation events
const unsubscribe = RunAnywhere.events.onGeneration((event) => {
  switch (event.type) {
    case 'started':
      console.log('Generation started');
      break;
    case 'tokenGenerated':
      console.log('Token:', event.token);
      break;
    case 'completed':
      console.log('Done:', event.response.text);
      break;
    case 'failed':
      console.error('Error:', event.error);
      break;
  }
});

// Subscribe to model events
RunAnywhere.events.onModel((event) => {
  if (event.type === 'downloadProgress') {
    console.log(`Progress: ${(event.progress * 100).toFixed(1)}%`);
  }
});

// Unsubscribe when done
unsubscribe();

Performance & Best Practices

Model Selection

Model SizeRAM RequiredUse Case
360M–500M (Q8)~500MBFast, lightweight chat
1B–3B (Q4/Q6)1–2GBBalanced quality/speed
7B (Q4)4–5GBHigh quality, slower

Memory Management

typescript
// Unload models when not in use
await RunAnywhere.unloadModel();

// Check storage before downloading
const storageInfo = await RunAnywhere.getStorageInfo();
if (storageInfo.freeSpace > modelSize) {
  // Safe to download
}

// Clean up temporary files
await RunAnywhere.clearCache();
await RunAnywhere.cleanTempFiles();

Best Practices

  1. Prefer streaming for better perceived latency in chat UIs
  2. Unload unused models to free device memory
  3. Handle errors gracefully with user-friendly messages
  4. Test on target devices — performance varies by hardware
  5. Use smaller models for faster iteration during development
  6. Pre-download models during onboarding for better UX

Troubleshooting

Model Download Fails

Symptoms: Download stuck or fails with network error

Solutions:

  1. Check internet connection
  2. Verify sufficient storage (need 2x model size for extraction)
  3. Try on WiFi instead of cellular
  4. Check if model URL is accessible

Out of Memory

Symptoms: App crashes during model loading or inference

Solutions:

  1. Use a smaller model (360M instead of 7B)
  2. Unload unused models first with RunAnywhere.unloadModel()
  3. Close other memory-intensive apps
  4. Test on device with more RAM

Inference Too Slow

Symptoms: Generation takes 10+ seconds per token

Solutions:

  1. Use Apple Silicon device for Metal acceleration (iOS)
  2. Reduce maxTokens for shorter responses
  3. Use quantized models (Q4 instead of Q8)
  4. Check device thermal state

Model Not Found After Download

Symptoms: modelNotFound error even though download completed

Solutions:

  1. Refresh model registry: await RunAnywhere.getAvailableModels()
  2. Check model path in storage
  3. Delete and re-download the model

Native Module Not Available

Symptoms: Native module not available error

Solutions:

  1. Ensure pod install was run for iOS
  2. Rebuild the app: npx react-native run-ios / run-android
  3. Check that all packages are installed correctly
  4. Reset Metro cache: npx react-native start --reset-cache

FAQ

Q: Do I need an internet connection?

A: Only for initial model download. Once downloaded, all inference runs 100% on-device with no network required.

Q: How much storage do models need?

A: Varies by model:

  • Small LLMs (360M–1B): 200MB–1GB
  • Medium LLMs (3B–7B Q4): 2–5GB
  • STT models: 50–200MB
  • TTS voices: 20–100MB

Q: Is user data sent to the cloud?

A: No. All inference happens on-device. Only anonymous analytics (latency, error rates) are collected in production mode, and this can be disabled.

Q: Which devices are supported?

A: iOS 15.1+ (iPhone/iPad) and Android 7.0+ (API 24+). Modern devices with 6GB+ RAM are recommended for larger models.

Q: Can I use custom models?

A: Yes, any GGUF model works with the LlamaCPP backend. ONNX models work for STT/TTS.

Q: What's the difference between chat() and generate()?

A: chat() is a convenience method that returns just the text. generate() returns full metrics (tokens, latency, etc.).


Local Development & Contributing

Contributions are welcome. This section explains how to set up your development environment to build the SDK from source and test your changes with the sample app.

Prerequisites

  • Node.js 18+
  • Xcode 15+ (for iOS builds)
  • Android Studio with NDK (for Android builds)
  • CMake 3.21+

First-Time Setup (Build from Source)

The SDK depends on native C++ libraries from runanywhere-commons. The setup script builds these locally so you can develop and test the SDK end-to-end.

bash
# 1. Clone the repository
git clone https://github.com/RunanywhereAI/runanywhere-sdks.git
cd runanywhere-sdks/sdk/runanywhere-react-native

# 2. Run first-time setup (~15-20 minutes)
./scripts/build-react-native.sh --setup

# 3. Install JavaScript dependencies
yarn install

What the setup script does:

  1. Downloads dependencies (ONNX Runtime, Sherpa-ONNX)
  2. Builds RACommons.xcframework and JNI libraries
  3. Builds RABackendLLAMACPP (LLM backend)
  4. Builds RABackendONNX (STT/TTS/VAD backend)
  5. Copies frameworks to ios/Binaries/ and JNI libs to android/src/main/jniLibs/
  6. Creates .testlocal marker files (enables local library consumption)

Understanding testLocal

The SDK has two modes:

ModeDescription
LocalUses frameworks/JNI libs from package directories (for development)
RemoteDownloads from GitHub releases during pod install/Gradle sync (for end users)

When you run --setup, the script automatically enables local mode via:

  • iOS: .testlocal marker files in ios/ directories
  • Android: RA_TEST_LOCAL=1 environment variable or runanywhere.testLocal=true in gradle.properties

Testing with the React Native Sample App

The recommended way to test SDK changes is with the sample app:

bash
# 1. Ensure SDK is set up (from previous step)

# 2. Navigate to the sample app
cd ../../examples/react-native/RunAnywhereAI

# 3. Install sample app dependencies
npm install

# 4. iOS: Install pods and run
cd ios && pod install && cd ..
npx react-native run-ios

# 5. Android: Run directly
npx react-native run-android

You can open the sample app in VS Code or Cursor for development.

The sample app's package.json uses workspace dependencies to reference the local SDK packages:

Sample App → Local RN SDK Packages → Local Frameworks/JNI libs
                                           ↑
                          Built by build-react-native.sh --setup

Development Workflow

After modifying TypeScript SDK code:

bash
# Type check all packages
yarn typecheck

# Run ESLint
yarn lint

# Build all packages
yarn build

After modifying runanywhere-commons (C++ code):

bash
cd sdk/runanywhere-react-native
./scripts/build-react-native.sh --local --rebuild-commons

Build Script Reference

CommandDescription
--setupFirst-time setup: downloads deps, builds all frameworks, enables local mode
--localUse local frameworks from package directories
--remoteUse remote frameworks from GitHub releases
--rebuild-commonsRebuild runanywhere-commons from source
--iosBuild for iOS only
--androidBuild for Android only
--cleanClean build artifacts before building
--abis=ABISAndroid ABIs to build (default: arm64-v8a)

Code Style

We use ESLint and Prettier for code formatting:

bash
# Run linter
yarn lint

# Auto-fix linting issues
yarn lint:fix

Pull Request Process

  1. Fork the repository
  2. Create a feature branch: git checkout -b feature/my-feature
  3. Make your changes with tests
  4. Ensure type checking passes: yarn typecheck
  5. Run linter: yarn lint
  6. Commit with a descriptive message
  7. Push and open a Pull Request

Reporting Issues

Open an issue on GitHub with:

  • SDK version: RunAnywhere.version
  • Platform (iOS/Android) and OS version
  • Device model
  • React Native version
  • Steps to reproduce
  • Expected vs actual behavior
  • Relevant logs (with sensitive info redacted)

Support


License

MIT License. See LICENSE for details.