services/twitter-services/docs/architecture-20250304.md
Twitter Service is a web automation service based on BrowserBase, providing structured access and interaction capabilities with Twitter data. It employs a layered architecture design that supports multiple adapters for integration with different applications.
┌─────────────────────────────────────────────┐
│ Application/Consumer Layer │
│ │
│ ┌────────────┐ ┌─────────────┐ │
│ │ │ │ │ │
│ │ AIRI Core │ │ Other LLM │ │
│ │ │ │ Applications│ │
│ │ │ │ │ │
│ └──────┬─────┘ └──────┬──────┘ │
└──────────┼─────────────────────┼────────────┘
│ │
┌──────────▼─────────────────────▼────────────┐
│ Adapter Layer │
│ │
│ ┌────────────┐ ┌─────────────┐ │
│ │AIRI Adapter│ │ MCP Adapter │ │
│ │(@server-sdk)│ │ (HTTP/JSON) │ │
│ └──────┬─────┘ └──────┬──────┘ │
└──────────┼─────────────────────┼────────────┘
│ │
┌──────────▼─────────────────────▼────────────┐
│ Core Services Layer │
│ │
│ ┌──────────────────────────────────┐ │
│ │ Twitter Services │ │
│ │ │ │
│ │ ┌────────┐ ┌────────────┐ │ │
│ │ │ Auth │ │ Timeline │ │ │
│ │ │ Service│ │ Service │ │ │
│ │ └────────┘ └────────────┘ │ │
│ │ │ │
│ └──────────────────┬───────────────┘ │
└──────────────────────┼──────────────────────┘
│
┌───────────▼────────────┐
│ Browser Adapter Layer │
│ (BrowserAdapter) │
└───────────┬────────────┘
│
┌───────────▼────────────┐
│ Stagehand │
└───────────┬────────────┘
│
┌───────────▼────────────┐
│ Playwright │
└────────────────────────┘
Provides integration with the AIRI LLM platform, handling event-driven communication.
Implements the Model Context Protocol interface, providing communication based on HTTP. Currently using the official MCP SDK implementation, providing high-performance HTTP server and SSE communication through H3.js.
The MCP adapter exposes several tools and resources:
Additionally, it provides tools for interaction:
The adapter uses internationalized messages (Chinese/English) to provide clear feedback to users about login status and session management.
Using listhen for optimized development experience, including automatic browser opening, real-time logging, and debugging tools.
The Authentication Service has been significantly enhanced to improve reliability and error handling:
The service follows a multi-stage authentication approach:
After successful authentication through any method, sessions are automatically persisted for future use. The system provides clear feedback to users about the current login state and automatically monitors and saves sessions when changes are detected.
Gets and processes Twitter timeline content.
Includes search service, interaction service, user profile service, etc. (not implemented in MVP)
Extracts structured data from HTML.
Controls request frequency to avoid triggering Twitter limits.
Manages authentication session data, providing methods to:
The service has migrated from direct BrowserBase API usage to Stagehand, an AI-powered web browsing framework built on top of Playwright. Stagehand offers three core APIs that simplify browser automation:
Stagehand processes the DOM in chunks to optimize LLM performance and provides fallback vision capabilities for complex page structures. This migration significantly improves code maintainability and automation reliability when interacting with Twitter's interface.
The configuration system has been optimized using the defu library for deep merging configurations, eliminating redundant initialization. The updated configuration structure includes Stagehand-specific settings:
interface Config {
// BrowserBase/Stagehand configuration
browserbase: {
apiKey: string
projectId?: string
endpoint?: string
stagehand?: {
modelName?: string // e.g., "gpt-4o" or "claude-3-5-sonnet-latest"
modelClientOptions?: {
apiKey: string // OpenAI or Anthropic API key
}
}
}
// Browser configuration
browser: BrowserConfig
// Twitter configuration
twitter: {
credentials?: TwitterCredentials
defaultOptions?: {
timeline?: TimelineOptions
search?: SearchOptions
}
}
// Adapter configuration
adapters: {
airi?: {
url?: string
token?: string
enabled: boolean
}
mcp?: {
port?: number
enabled: boolean
}
}
// System configuration
system: {
logLevel: string
concurrency: number
}
}
The system no longer relies on the TWITTER_COOKIES environment variable, as cookies are now managed through the session management system.
# Install dependencies
npm install
# Set environment variables
cp .env.example .env
# Edit .env to add BrowserBase API key and Twitter credentials (optional)
# Development mode startup
npm run dev # Standard mode
npm run dev:mcp # MCP development server mode
import { StagehandAdapter, TwitterService } from 'twitter-services'
async function main() {
// Initialize Stagehand adapter
const browser = new StagehandAdapter(process.env.BROWSERBASE_API_KEY, process.env.BROWSERBASE_PROJECT_ID)
await browser.initialize({
headless: true,
stagehand: {
modelName: 'gpt-4o', // Or 'claude-3-5-sonnet-latest' for Anthropic
modelClientOptions: {
apiKey: process.env.OPENAI_API_KEY // Or process.env.ANTHROPIC_API_KEY
}
}
})
// Create Twitter service
const twitter = new TwitterService(browser)
// Authenticate - will try multi-stage approach
const loggedIn = await twitter.login()
if (loggedIn) {
console.info('Login successful')
// Get timeline using natural language capabilities of Stagehand
const tweets = await twitter.getTimeline({ count: 10 })
console.info(tweets)
}
else {
console.error('Login failed')
}
// Release resources
await browser.close()
}
import { AIRIAdapter, BrowserBaseMCPAdapter, TwitterService } from 'twitter-services'
async function startAIRIModule() {
const browser = new BrowserBaseMCPAdapter(process.env.BROWSERBASE_API_KEY)
await browser.initialize({ headless: true })
const twitter = new TwitterService(browser)
// Create AIRI adapter
const airiAdapter = new AIRIAdapter(twitter, {
url: process.env.AIRI_URL,
token: process.env.AIRI_TOKEN
})
// Start adapter
await airiAdapter.start()
console.info('Twitter service running as AIRI module')
}
// Use MCP SDK to interact with Twitter service
import { Client } from '@modelcontextprotocol/sdk/client/index.js'
import { SSEClientTransport } from '@modelcontextprotocol/sdk/client/sse.js'
async function connectToTwitterService() {
// Create SSE transport
const transport = new SSEClientTransport('http://localhost:8080/sse', 'http://localhost:8080/messages')
// Create client
const client = new Client()
await client.connect(transport)
// Get timeline
const timeline = await client.get('twitter://timeline/10')
console.info('Timeline:', timeline.contents)
// Use simplified login tool without parameters
const loginResult = await client.useTool('login', {})
console.info('Login result:', loginResult.content[0].text)
// Use refresh timeline tool to get latest tweets
const refreshResult = await client.useTool('refresh-timeline', { count: 15, includeReplies: false })
console.info('Refresh result:', refreshResult.content[0].text)
console.info('New tweets:', refreshResult.resources)
// Get user profile information
const profileResult = await client.useTool('get-my-profile', { username: 'twitter' })
console.info('Profile info:', profileResult.content[0].text)
// Use tool to send tweet
const result = await client.useTool('post-tweet', { content: 'Hello from MCP!' })
console.info('Result:', result.content)
return client
}
For example, adding "Get Tweets from a Specific User" functionality:
src/types/twitter.tssrc/core/twitter-service.tsconfigureServer()useVision: true in appropriate operations