Back to Worldmonitor

External APIs Catalog

docs/Docs_To_Review/EXTERNAL_APIS.md

2.5.2340.0 KB
Original Source

External APIs Catalog

Comprehensive reference for every external API consumed by World Monitor. Last updated: 2026-02-19


Table of Contents


1. Overview

World Monitor integrates 38 distinct external API sources (plus ~150 RSS feed domains) to provide a unified real-time intelligence dashboard across geopolitical, financial, military, environmental, humanitarian, and technology domains.

MetricCount
Total external APIs38
Require API key (mandatory)10
Require API key (optional)2
Fully public / no auth26
Free tier sufficient36
Paid / commercial tier needed2
WebSocket sources1
RSS/Atom feed domains~150

Auth breakdown:

  • API key in header/query — ACLED, Finnhub, FRED, Wingbits, AbuseIPDB, NASA FIRMS, Groq, OpenRouter, Cloudflare Radar, EIA
  • Optional API key — GitHub, HDX HAPI
  • No authentication — UCDP, GDELT, NGA MSI, Yahoo Finance, CoinGecko, Polymarket, alternative.me, blockchain.info, OpenSky, Feodo Tracker, URLhaus, C2IntelFeeds, AlienVault OTX, USGS, NOAA, Status Pages, FAA, UNHCR, WorldPop, World Bank, Hacker News, ArXiv, pizzint.watch, RSS feeds, Tech Events
  • URL-based auth — Custom AIS Relay

2. API Key Requirements

#API NameEnv VarRequiredSignup URLTier Needed
1ACLEDACLED_ACCESS_TOKEN, ACLED_EMAILYeshttps://developer.acleddata.com/Free (researcher)
2FinnhubFINNHUB_API_KEYYeshttps://finnhub.io/registerFree
3FREDFRED_API_KEYYeshttps://fred.stlouisfed.org/docs/api/api_key.htmlFree
4NASA FIRMSNASA_FIRMS_API_KEYYeshttps://firms.modaps.eosdis.nasa.gov/api/area/Free (EOSDIS)
5GroqGROQ_API_KEYYeshttps://console.groq.com/Free / Paid
6OpenRouterOPENROUTER_API_KEYYeshttps://openrouter.ai/keysFree (select models)
7Cloudflare RadarCLOUDFLARE_API_TOKENYeshttps://dash.cloudflare.com/profile/api-tokensEnterprise
8AbuseIPDBABUSEIPDB_API_KEYYeshttps://www.abuseipdb.com/account/plansFree (1000/day)
9WingbitsWINGBITS_API_KEYYeshttps://wingbits.com/Commercial
10EIAEIA_API_KEYYeshttps://www.eia.gov/opendata/register.phpFree
11GitHubGITHUB_TOKENOptionalhttps://github.com/settings/tokensFree
12HDX HAPIHDX_APP_IDENTIFIEROptionalhttps://hapi.humdata.org/Free
13AIS RelayWS_RELAY_URLYes¹Self-hostedN/A

¹ The AIS relay is a self-hosted WebSocket server; the env var points to its URL rather than an API key.


3. External APIs by Domain

3.1 Geopolitical Data


1 — ACLED (Armed Conflict Location & Event Data)

FieldValue
Base URLhttps://api.acleddata.com/acled/read
AuthenticationQuery params: key + email
Env VarsACLED_ACCESS_TOKEN, ACLED_EMAIL
Rate LimitsUnspecified; researcher tier has generous limits
Data FormatJSON
WM Endpoints/api/acled, /api/acled-conflict
Frontend ServicesConflictService → ConflictPanel, MapLayer
DegradationReturns empty data array; conflict panels display "no data available"
Tier NeededFree researcher account
QuirksRequires both key and email as separate params. Data lags 1–2 weeks behind real-time events. Pagination via page param.

2 — UCDP (Uppsala Conflict Data Program)

FieldValue
Base URLhttps://ucdpapi.pcr.uu.se/api/
AuthenticationNone
Env Vars
Rate LimitsNo documented limit
Data FormatJSON
WM Endpoints/api/ucdp, /api/ucdp-events
Frontend ServicesConflictService → ConflictPanel
DegradationCached data served with 24h TTL from Upstash/CDN
Tier NeededPublic
QuirksAcademic data source; updates less frequently than ACLED. Supports versioned datasets.

3 — GDELT (Global Database of Events, Language, and Tone)

FieldValue
Base URLhttps://api.gdeltproject.org/api/v2/
AuthenticationNone
Env Vars
Rate LimitsPublic API; no documented limit but aggressive scraping will 429
Data FormatJSON, GeoJSON, CSV (varies by sub-endpoint)
WM Endpoints/api/gdelt-doc, /api/gdelt-geo
Frontend ServicesNewsService → GdeltPanel, GeoHeatmap
DegradationUpstream 502 passed through; panel shows error state
Tier NeededPublic
Quirksgdelt-doc uses the DOC 2.0 API for full-text search; gdelt-geo uses the GEO 2.0 API for geographic heat-mapping. Large result sets can be slow.

4 — NGA MSI (Maritime Safety Information)

FieldValue
Base URLhttps://msi.gs.mil/api/publications/broadcast-warn
AuthenticationNone
Env Vars
Rate LimitsPublic US government endpoint
Data FormatJSON
WM Endpoints/api/nga-warnings
Frontend ServicesMilitaryService → MaritimeWarningsPanel, MapLayer
DegradationPassthrough 502 error; panel shows "service unavailable"
Tier NeededPublic
QuirksUS DoD-hosted; occasionally slow. Returns NAVAREA warnings, HYDROLANT/HYDROPAC notices.

3.2 Markets & Finance


5 — Finnhub

FieldValue
Base URLhttps://finnhub.io/api/v1/
AuthenticationQuery param token or header X-Finnhub-Token
Env VarsFINNHUB_API_KEY
Rate LimitsFree tier: 60 calls/min, 30 API calls/sec
Data FormatJSON
WM Endpoints/api/finnhub, /api/etf-flows
Frontend ServicesMarketService → MarketPanel, ETFFlowsPanel
DegradationReturns unavailable flag; MarketPanel shows stale cached data with timestamp
Tier NeededFree
QuirksWebSocket endpoint available but WM uses REST polling. ETF data requires specific symbol lookups. Free tier lacks some institutional data.

6 — Yahoo Finance (Unofficial)

FieldValue
Base URLhttps://query1.finance.yahoo.com/v8/finance/chart/
AuthenticationNone
Env Vars
Rate LimitsUnofficial API; aggressive rate limiting possible; no SLA
Data FormatJSON
WM Endpoints/api/yahoo-finance, /api/stock-index
Frontend ServicesMarketService → StockIndexPanel, MarketOverview
DegradationCDN cache serves stale data; empty results on sustained outage
Tier NeededPublic (unofficial)
QuirksNo official API — this is an undocumented Yahoo endpoint. May break without notice. Crumb/cookie auth sometimes required by Yahoo; current implementation works without. Consider migrating to official alternative.

7 — CoinGecko

FieldValue
Base URLhttps://api.coingecko.com/api/v3/
AuthenticationNone (free tier); API key for Pro
Env Vars
Rate LimitsFree: 10–30 calls/min (varies)
Data FormatJSON
WM Endpoints/api/coingecko, /api/stablecoin-markets, /api/macro-signals
Frontend ServicesCryptoService → CryptoPanel, StablecoinPanel; MacroService → MacroSignals
DegradationReturns unavailable flag; panels show last-known values
Tier NeededFree
QuirksRate limits fluctuate and are not well-documented. Stablecoin market-cap queries can be slow. Feeds into macro-signals as one of several composite inputs.

8 — FRED (Federal Reserve Economic Data)

FieldValue
Base URLhttps://api.stlouisfed.org/fred/
AuthenticationQuery param api_key
Env VarsFRED_API_KEY
Rate Limits120 requests/min (free tier)
Data FormatJSON or XML (WM uses JSON via file_type=json)
WM Endpoints/api/fred-data, /api/macro-signals
Frontend ServicesMacroService → MacroSignals, EconIndicatorsPanel
DegradationCached data served from Upstash; stale indicator shown
Tier NeededFree
QuirksSeries IDs must be known in advance (e.g. DGS10, T10Y2Y). Data updates on Fed schedule (not real-time).

9 — Gamma (Polymarket)

FieldValue
Base URLhttps://gamma-api.polymarket.com/
AuthenticationNone
Env Vars
Rate LimitsPublic; no documented limit
Data FormatJSON
WM Endpoints/api/polymarket
Frontend ServicesPredictionService → PolymarketPanel
DegradationCDN cache serves stale prediction data
Tier NeededPublic
QuirksGamma is the off-chain API for Polymarket. Market slugs/IDs can change. Filterable by tag for geopolitical/election markets.

10 — alternative.me (Fear & Greed Index)

FieldValue
Base URLhttps://api.alternative.me/fng/
AuthenticationNone
Env Vars
Rate LimitsPublic; lenient
Data FormatJSON
WM Endpoints/api/macro-signals (composite input)
Frontend ServicesMacroService → MacroSignals (fear/greed gauge)
DegradationSignal omitted from aggregate score; composite continues without it
Tier NeededPublic
QuirksReturns crypto-specific Fear & Greed Index (0–100). Single-value endpoint; very lightweight.

11 — blockchain.info (Bitcoin Hash Rate)

FieldValue
Base URLhttps://blockchain.info/
AuthenticationNone
Env Vars
Rate LimitsPublic; moderate (avoid rapid bursts)
Data FormatJSON
WM Endpoints/api/macro-signals (composite input)
Frontend ServicesMacroService → MacroSignals
DegradationHash-rate signal omitted from aggregate; composite score adjusted
Tier NeededPublic
QuirksUsed specifically for BTC network hash-rate as a macro signal. Endpoint: /q/hashrate.

3.3 Military & Security


12 — OpenSky Network

FieldValue
Base URLhttps://opensky-network.org/api/
AuthenticationNone (anonymous); optional Basic auth for higher limits
Env Vars
Rate LimitsAnonymous: 100 requests/day; authenticated: 4000/day
Data FormatJSON
WM Endpoints/api/opensky, /api/theater-posture
Frontend ServicesAviationService → FlightTracker; TheaterService → TheaterPosture
DegradationCDN cache serves stale snapshot; theater posture uses last-known aircraft positions
Tier NeededFree (anonymous sufficient for current usage)
QuirksState vectors update every ~10 seconds but WM polls less frequently. Returns all aircraft in bounding box. theater-posture uses OpenSky as one of multiple inputs. Anonymous rate limit is tight — caching is critical.

13 — Wingbits

FieldValue
Base URLhttps://data.wingbits.com/
AuthenticationAPI key
Env VarsWINGBITS_API_KEY
Rate LimitsCommercial agreement; undisclosed
Data FormatJSON
WM Endpoints/api/wingbits/*, /api/theater-posture
Frontend ServicesAviationService → WingbitsPanel; TheaterService → TheaterPosture
DegradationTheater posture proceeds without Wingbits augmentation; panel shows "source unavailable"
Tier NeededCommercial (paid)
QuirksPremium ADS-B data provider. Multiple sub-endpoints under wingbits/. Augments OpenSky with higher-fidelity data in specific regions.

14 — Custom AIS Relay

FieldValue
Base URLConfigurable via WS_RELAY_URL
AuthenticationURL-based (credentials in URL)
Env VarsWS_RELAY_URL
Rate LimitsSelf-hosted; depends on deployment
Data FormatJSON over WebSocket
WM Endpoints/api/ais-snapshot
Frontend ServicesMaritimeService → VesselTracker, MapLayer
DegradationReturns empty vessel array; maritime layer shows no ship positions
Tier NeededSelf-hosted
QuirksWebSocket relay run via scripts/ais-relay.cjs. Decodes AIS NMEA sentences into JSON. Snapshot endpoint aggregates latest positions from persistent WS connection. See deploy/ for systemd service config.

15 — Feodo Tracker (abuse.ch)

FieldValue
Base URLhttps://feodotracker.abuse.ch/
AuthenticationNone
Env Vars
Rate LimitsPublic
Data FormatCSV
WM Endpoints/api/cyber-threats (aggregated source)
Frontend ServicesCyberService → CyberThreatsPanel
DegradationSource omitted from aggregation; other cyber sources still displayed
Tier NeededPublic
QuirksTracks C2 (command & control) botnet infrastructure. CSV parsed server-side. One of 5 cyber-threat sources aggregated by the endpoint.

16 — URLhaus (abuse.ch)

FieldValue
Base URLhttps://urlhaus.abuse.ch/
AuthenticationNone
Env Vars
Rate LimitsPublic
Data FormatCSV
WM Endpoints/api/cyber-threats (aggregated source)
Frontend ServicesCyberService → CyberThreatsPanel
DegradationSource omitted from aggregation
Tier NeededPublic
QuirksMalicious URL database. Daily CSV dump downloaded and parsed.

17 — C2IntelFeeds

FieldValue
Base URLPublic GitHub repository (CSV files)
AuthenticationNone
Env Vars
Rate LimitsGitHub raw content rate limits apply
Data FormatCSV
WM Endpoints/api/cyber-threats (aggregated source)
Frontend ServicesCyberService → CyberThreatsPanel
DegradationSource omitted from aggregation
Tier NeededPublic
QuirksCommunity-maintained C2 IP/domain feeds. Fetched from GitHub raw URLs.

18 — AlienVault OTX

FieldValue
Base URLhttps://otx.alienvault.com/
AuthenticationNone (public feed)
Env Vars
Rate LimitsPublic
Data FormatJSON
WM Endpoints/api/cyber-threats (aggregated source)
Frontend ServicesCyberService → CyberThreatsPanel
DegradationSource omitted from aggregation
Tier NeededPublic
QuirksOpen Threat Exchange pulses. Used for IoC (indicators of compromise) enrichment.

19 — AbuseIPDB

FieldValue
Base URLhttps://api.abuseipdb.com/api/v2/
AuthenticationHeader: Key
Env VarsABUSEIPDB_API_KEY
Rate LimitsFree: 1000 checks/day; paid tiers higher
Data FormatJSON
WM Endpoints/api/cyber-threats (aggregated source)
Frontend ServicesCyberService → CyberThreatsPanel
DegradationSource omitted from aggregate; other 4 cyber sources still function
Tier NeededFree (1000/day sufficient)
QuirksIP reputation/abuse confidence scoring. Daily limit can be exhausted if scans are broad; WM uses targeted checks only.

3.4 Natural Events


20 — USGS Earthquake Hazards

FieldValue
Base URLhttps://earthquake.usgs.gov/earthquakes/feed/v1.0/
AuthenticationNone
Env Vars
Rate LimitsPublic; updated every 5 min by USGS
Data FormatGeoJSON
WM Endpoints/api/earthquakes
Frontend ServicesSeismicService → EarthquakePanel, MapLayer
DegradationCDN cache serves stale GeoJSON; map shows last-known quakes
Tier NeededPublic
QuirksPre-built feeds by magnitude/time range (e.g. all_day.geojson, significant_month.geojson). No query API — just static feed URLs that USGS regenerates.

21 — NASA FIRMS (Fire Information for Resource Management System)

FieldValue
Base URLhttps://firms.modaps.eosdis.nasa.gov/api/
AuthenticationQuery param MAP_KEY
Env VarsNASA_FIRMS_API_KEY
Rate LimitsFree tier; transaction-based limits
Data FormatCSV (parsed server-side to JSON)
WM Endpoints/api/firms-fires
Frontend ServicesFireService → WildfiresPanel, MapLayer
DegradationCached data served; empty array on sustained outage
Tier NeededFree (EOSDIS Earthdata account)
QuirksVIIRS and MODIS satellite data. Area/country queries. CSV rows can be very large for global queries — WM limits to specific regions or short time windows.

22 — NOAA Climate Monitoring

FieldValue
Base URLVarious NOAA Climate Monitoring endpoints
AuthenticationNone
Env Vars
Rate LimitsPublic US government
Data FormatJSON
WM Endpoints/api/climate-anomalies
Frontend ServicesClimateService → ClimateAnomaliesPanel
DegradationCached data served with 6h TTL
Tier NeededPublic
QuirksGlobal temperature anomaly data. Monthly/annual resolution — not real-time. Multiple NOAA sub-endpoints aggregated.

3.5 AI / ML


23 — Groq

FieldValue
Base URLhttps://api.groq.com/openai/v1/
AuthenticationBearer token in Authorization header
Env VarsGROQ_API_KEY
Rate LimitsFree: varies by model (e.g. 30 req/min for Llama); paid: higher
Data FormatJSON (OpenAI-compatible chat completions)
WM Endpoints/api/groq-summarize, /api/classify-batch, /api/classify-event, /api/country-intel
Frontend ServicesSummaryService, ClassificationService, CountryIntelService
DegradationFalls back to OpenRouter → browser-based Transformers.js pipeline
Tier NeededFree / Paid (free sufficient for moderate usage)
QuirksPrimary LLM provider. OpenAI-compatible API. Ultra-fast inference via custom LPU hardware. Model selection configurable. Fallback chain: Groq → OpenRouter → Transformers.js (in-browser).

24 — OpenRouter

FieldValue
Base URLhttps://openrouter.ai/api/v1/
AuthenticationBearer token in Authorization header
Env VarsOPENROUTER_API_KEY
Rate LimitsVaries by underlying model; free models have lower limits
Data FormatJSON (OpenAI-compatible chat completions)
WM Endpoints/api/openrouter-summarize
Frontend ServicesSummaryService (fallback from Groq)
DegradationFalls back to browser-based Transformers.js
Tier NeededFree (select models only)
QuirksAggregator routing to multiple LLM providers. Used as secondary/fallback LLM. Supports HTTP-Referer and X-Title headers for attribution. Specific free models (e.g. mistralai/mistral-7b-instruct:free) used to avoid cost.

3.6 Infrastructure & Status


25 — Cloudflare Radar

FieldValue
Base URLhttps://api.cloudflare.com/client/v4/radar/
AuthenticationBearer token in Authorization header
Env VarsCLOUDFLARE_API_TOKEN
Rate LimitsEnterprise API; generous limits
Data FormatJSON
WM Endpoints/api/cloudflare-outages
Frontend ServicesInfraService → CloudflareOutagesPanel
DegradationReturns empty outage list; panel shows "no active outages" (may be false negative)
Tier NeededEnterprise (Radar API token required)
QuirksProvides internet outage/anomaly detection globally. Requires Cloudflare account with Radar API access. Token needs radar:read permission.

26 — Status Pages (33 Services)

FieldValue
Base URLVarious: *.statuspage.io, status.* domains
AuthenticationNone
Env Vars
Rate LimitsPublic status pages
Data FormatJSON (Atlassian Statuspage API format)
WM Endpoints/api/service-status
Frontend ServicesInfraService → ServiceStatusPanel
DegradationIndividual services shown as "unknown" status; others continue
Tier NeededPublic
QuirksMonitors 33 major services (AWS, Azure, GCP, GitHub, Cloudflare, Stripe, Twilio, etc.). Each status page polled independently. Circuit breaker per source. Atlassian Statuspage JSON format is standard across most targets.

Monitored services include (non-exhaustive): AWS, Microsoft Azure, Google Cloud, GitHub, Cloudflare, Vercel, Netlify, Fastly, Stripe, Twilio, Datadog, PagerDuty, Slack, Discord, Zoom, Atlassian, HashiCorp, DigitalOcean, Heroku, MongoDB Atlas, Redis Cloud, Supabase, OpenAI, Anthropic, and others.


27 — FAA ASWS (Airport Status Web Service)

FieldValue
Base URLhttps://soa.smext.faa.gov/asws/api/
AuthenticationNone
Env Vars
Rate LimitsPublic US government
Data FormatXML (parsed server-side)
WM Endpoints/api/faa-status
Frontend ServicesAviationService → FAAStatusPanel
DegradationCDN cache serves stale data; panel shows last-known status
Tier NeededPublic
QuirksReturns ground delays, ground stops, closures, and delay info per airport. XML response parsed to JSON. US airports only.

3.7 Humanitarian


28 — UNHCR Population API

FieldValue
Base URLhttps://api.unhcr.org/population/v1/
AuthenticationNone
Env Vars
Rate LimitsPublic
Data FormatJSON
WM Endpoints/api/unhcr-population
Frontend ServicesHumanitarianService → RefugeePanel
DegradationCached data served with 24h TTL
Tier NeededPublic
QuirksRefugee and displaced population statistics. Annual data granularity. Large datasets; WM queries specific country/year combos.

29 — HDX HAPI (Humanitarian API)

FieldValue
Base URLhttps://hapi.humdata.org/api/v2/
AuthenticationOptional app_identifier query param
Env VarsHDX_APP_IDENTIFIER (optional)
Rate LimitsPublic; higher limits with app identifier
Data FormatJSON
WM Endpoints/api/hapi
Frontend ServicesHumanitarianService → HAPIPanel
DegradationCached data served with 6h TTL
Tier NeededFree (identifier optional but recommended)
QuirksOCHA's Humanitarian Data Exchange programmatic API. Covers food security, population, operational presence, etc. Without app_identifier, lower rate limits apply.

30 — WorldPop

FieldValue
Base URLWorldPop raster/API endpoints
AuthenticationNone
Env Vars
Rate LimitsPublic
Data FormatJSON
WM Endpoints/api/worldpop-exposure
Frontend ServicesHumanitarianService → ExposureAnalysis
DegradationCached data served with 7-day TTL
Tier NeededPublic
QuirksPopulation density data for exposure analysis (e.g. "how many people near this earthquake?"). Long cache TTL because population data changes slowly.

31 — World Bank

FieldValue
Base URLhttps://api.worldbank.org/v2/
AuthenticationNone
Env Vars
Rate LimitsPublic
Data FormatJSON (via format=json param)
WM Endpoints/api/worldbank
Frontend ServicesEconService → WorldBankPanel, CountryProfile
DegradationCached data served with 24h TTL
Tier NeededPublic
QuirksDevelopment indicators (GDP, population, etc.). Pagination via page/per_page. Default format is XML — must specify format=json. Annual data; not real-time.

3.8 Content & Research


32 — Hacker News (Firebase API)

FieldValue
Base URLhttps://hacker-news.firebaseio.com/v0/
AuthenticationNone
Env Vars
Rate LimitsPublic Firebase endpoint; generous
Data FormatJSON
WM Endpoints/api/hackernews
Frontend ServicesTechService → HackerNewsPanel
DegradationCDN cache serves stale stories
Tier NeededPublic
QuirksOfficial HN API via Firebase. Each story requires a separate fetch (by ID). WM fetches top N story IDs then batch-fetches details.

33 — GitHub API

FieldValue
Base URLhttps://api.github.com/
AuthenticationOptional Bearer token
Env VarsGITHUB_TOKEN (optional)
Rate LimitsUnauthenticated: 60/hour; Authenticated: 5000/hour
Data FormatJSON
WM Endpoints/api/github-trending, /api/version, /api/download
Frontend ServicesTechService → GitHubTrendingPanel; AppService → VersionCheck
DegradationHTML scrape fallback for trending; version check fails gracefully
Tier NeededFree (token optional but recommended)
QuirksTrending repos: no official API — WM uses search API with date filters as proxy, falls back to HTML scraping github.com/trending. Version endpoint checks latest release tag. Without GITHUB_TOKEN, 60 req/h can be exhausted quickly in development.

34 — ArXiv

FieldValue
Base URLhttps://export.arxiv.org/api/
AuthenticationNone
Env Vars
Rate LimitsPublic; requests should be spaced ≥3 seconds apart
Data FormatXML (Atom feed)
WM Endpoints/api/arxiv
Frontend ServicesResearchService → ArXivPanel
DegradationCDN cache serves stale results
Tier NeededPublic
QuirksAcademic paper search. Atom XML parsed server-side. ArXiv requests that they be polite with rate (3s between requests). Search syntax uses specific field prefixes (ti:, au:, cat:).

35 — EIA (Energy Information Administration)

FieldValue
Base URLhttps://api.eia.gov/v2/
AuthenticationQuery param api_key
Env VarsEIA_API_KEY
Rate LimitsFree tier; undisclosed limits
Data FormatJSON
WM Endpoints/api/eia/* (multiple sub-endpoints)
Frontend ServicesEnergyService → EnergyPanel
DegradationCDN cache serves stale data
Tier NeededFree
QuirksUS energy data: petroleum, natural gas, electricity, coal. V2 API replaces legacy V1. Series IDs follow hierarchical facet structure.

36 — pizzint.watch

FieldValue
Base URLhttps://pizzint.watch/
AuthenticationNone
Env Vars
Rate LimitsPublic
Data FormatJSON
WM Endpoints/api/pizzint/* (multiple sub-endpoints)
Frontend ServicesIntelService → PizzintPanel
DegradationCDN cache serves stale data
Tier NeededPublic
QuirksOSINT aggregation platform. Multiple sub-endpoints proxied through WM edge functions.

37 — RSS Feeds (~150 Domains)

FieldValue
Base URLVarious publisher domains
AuthenticationNone
Env Vars
Rate LimitsVaries per publisher; typically lenient for RSS
Data FormatRSS 2.0 / Atom XML
WM Endpoints/api/rss-proxy
Frontend ServicesNewsService → RSSPanel, NewsFeed
DegradationCircuit breaker per feed; 5-minute cooldown on failure; other feeds unaffected
Tier NeededPublic
QuirksRSS proxy fetches and normalizes feeds from ~150 news sources worldwide. Per-feed circuit breaker prevents one broken feed from affecting others. 5-minute cooldown before retrying a failed feed. Feeds are categorized by region/topic. XML parsed and normalized to common JSON schema.

Feed categories include: Major wire services (AP, Reuters, AFP), regional news (Al Jazeera, BBC, NHK, TASS), defense/security publications, financial news, technology outlets, and specialized OSINT sources.


38 — Tech Event Sources

FieldValue
Base URLVarious scraped sources
AuthenticationNone
Env Vars
Rate LimitsStandard web scraping considerations
Data FormatHTML (scraped and parsed)
WM Endpoints/api/tech-events
Frontend ServicesTechService → TechEventsPanel
DegradationCached data served with 6h TTL
Tier NeededPublic
QuirksScrapes conference/event listings from multiple sources. HTML parsing; fragile if source layouts change. Long cache TTL to reduce scrape frequency.

4. Dependency Chain Diagram

4.1 Overall Architecture

mermaid
graph LR
    subgraph External APIs
        A1[ACLED]
        A2[UCDP]
        A3[GDELT]
        A4[Finnhub]
        A5[CoinGecko]
        A6[USGS]
        A7[Groq]
        A8[OpenRouter]
        A9[Status Pages]
        A10[RSS Feeds]
        A11[Others...]
    end

    subgraph Vercel Edge Functions
        E1[/api/acled]
        E2[/api/ucdp]
        E3[/api/gdelt-doc]
        E4[/api/finnhub]
        E5[/api/coingecko]
        E6[/api/earthquakes]
        E7[/api/groq-summarize]
        E8[/api/openrouter-summarize]
        E9[/api/service-status]
        E10[/api/rss-proxy]
    end

    subgraph Caching Layer
        C1[Upstash Redis]
        C2[Vercel CDN / Edge Cache]
    end

    subgraph Frontend Services
        S1[ConflictService]
        S2[MarketService]
        S3[SeismicService]
        S4[SummaryService]
        S5[InfraService]
        S6[NewsService]
    end

    subgraph UI Components
        U1[ConflictPanel]
        U2[MarketPanel]
        U3[EarthquakePanel]
        U4[SummaryCards]
        U5[ServiceStatusPanel]
        U6[NewsFeed]
    end

    A1 --> E1
    A2 --> E2
    A3 --> E3
    A4 --> E4
    A5 --> E5
    A6 --> E6
    A7 --> E7
    A8 --> E8
    A9 --> E9
    A10 --> E10

    E1 --> C1 --> S1
    E2 --> C1 --> S1
    E3 --> C2 --> S6
    E4 --> C2 --> S2
    E5 --> C2 --> S2
    E6 --> C2 --> S3
    E7 --> C2 --> S4
    E8 --> C2 --> S4
    E9 --> C2 --> S5
    E10 --> C2 --> S6

    S1 --> U1
    S2 --> U2
    S3 --> U3
    S4 --> U4
    S5 --> U5
    S6 --> U6

4.2 AI/LLM Fallback Chain

mermaid
graph TD
    A[Text to Summarize / Classify] --> B{Groq Available?}
    B -->|Yes| C[Groq API
GROQ_API_KEY]
    B -->|No / Error| D{OpenRouter Available?}
    D -->|Yes| E[OpenRouter API
OPENROUTER_API_KEY]
    D -->|No / Error| F[Browser Transformers.js
No API key needed]
    C --> G[Result]
    E --> G
    F --> G

4.3 Cyber Threats Aggregation

mermaid
graph TD
    F1[Feodo Tracker
CSV] --> AGG[/api/cyber-threats
Aggregator]
    F2[URLhaus
CSV] --> AGG
    F3[C2IntelFeeds
CSV] --> AGG
    F4[AlienVault OTX
JSON] --> AGG
    F5[AbuseIPDB
JSON + API Key] --> AGG
    AGG --> CS[CyberService]
    CS --> CP[CyberThreatsPanel]

    style F5 fill:#ff9,stroke:#333

4.4 Macro Signals Composite

mermaid
graph TD
    M1[FRED
Treasury yields, rates] --> MS[/api/macro-signals
Composite Builder]
    M2[CoinGecko
Crypto market cap] --> MS
    M3[alternative.me
Fear & Greed] --> MS
    M4[blockchain.info
BTC hash rate] --> MS
    MS --> MSvc[MacroService]
    MSvc --> MP[MacroSignalsPanel]

4.5 Theater Posture Composite

mermaid
graph TD
    T1[OpenSky
Aircraft positions] --> TP[/api/theater-posture
Analysis]
    T2[Wingbits
ADS-B premium] --> TP
    T3[AIS Relay
Vessel positions] --> TP
    T4[ACLED
Conflict events] --> TP
    TP --> TS[TheaterService]
    TS --> TPnl[TheaterPosturePanel]

    style T2 fill:#ff9,stroke:#333

5. Degradation Matrix

How World Monitor behaves when each external API is unavailable:

#APICache TTLBehavior When DownUser ImpactSeverity
1ACLEDEmpty data arrayConflict panels blankHigh
2UCDP24hStale cached dataData may be outdatedLow
3GDELT502 passthroughPanel shows errorMedium
4NGA MSI502 passthroughMaritime warnings blankMedium
5FinnhubCDNunavailable flag, stale dataMarket data delayedMedium
6Yahoo FinanceCDNStale CDN / empty resultsStock data delayed or missingMedium
7CoinGeckoCDNunavailable flagCrypto data delayedLow
8FREDUpstashStale cached dataMacro indicators delayedLow
9PolymarketCDNStale predictionsPredictions outdatedLow
10alternative.meSignal omittedMacro score slightly less accurateMinimal
11blockchain.infoSignal omittedMacro score slightly less accurateMinimal
12OpenSkyCDNStale aircraft dataFlight positions outdatedMedium
13WingbitsProceeds without augmentationLower-fidelity ADS-B in some regionsLow
14AIS RelayEmpty vessel arrayNo ship trackingHigh
15Feodo TrackerSource omittedCyber panel partialMinimal
16URLhausSource omittedCyber panel partialMinimal
17C2IntelFeedsSource omittedCyber panel partialMinimal
18AlienVault OTXSource omittedCyber panel partialMinimal
19AbuseIPDBSource omittedCyber panel partialMinimal
20USGSCDNStale GeoJSONEarthquake data delayedLow
21NASA FIRMSCDNCached / emptyFire data delayed or missingMedium
22NOAA6hStale cached dataClimate data delayedLow
23GroqFallback → OpenRouter → Transformers.jsSummarization slowerMedium
24OpenRouterFallback → Transformers.jsSummarization slower, lower qualityMedium
25Cloudflare RadarEmpty outage listMay miss internet outagesMedium
26Status PagesIndividual → "unknown"Partial service statusLow
27FAA ASWSCDNStale airport statusAirport delays outdatedLow
28UNHCR24hStale cached dataRefugee data delayedLow
29HDX HAPI6hStale cached dataHumanitarian data delayedLow
30WorldPop7dStale cached dataPopulation estimates unchangedMinimal
31World Bank24hStale cached dataDevelopment indicators delayedLow
32Hacker NewsCDNStale storiesHN feed outdatedMinimal
33GitHubHTML scrape fallback / graceful failTrending may fail; version check skippedLow
34ArXivCDNStale search resultsResearch papers outdatedMinimal
35EIACDNStale energy dataEnergy metrics delayedLow
36pizzint.watchCDNStale intel dataOSINT data delayedLow
37RSS Feeds5m CBCircuit breaker per feedIndividual feeds drop; others continueLow
38Tech Events6hStale cached dataEvent listings outdatedMinimal

Severity legend:

  • High — Core functionality lost, no fallback
  • Medium — Noticeable degradation, partial data or delayed experience
  • Low — Minor impact, cached data fills the gap
  • Minimal — Barely noticeable; one signal among many omitted

6. Cost & Tier Summary

Free APIs (No Payment Required) — 36 APIs

CategoryAPIs
Fully public (no key)UCDP, GDELT, NGA MSI, Yahoo Finance, CoinGecko, Polymarket, alternative.me, blockchain.info, OpenSky, Feodo Tracker, URLhaus, C2IntelFeeds, AlienVault OTX, USGS, NOAA, Status Pages, FAA, UNHCR, WorldPop, World Bank, Hacker News, ArXiv, pizzint.watch, RSS Feeds, Tech Events
Free key requiredACLED (researcher), Finnhub, FRED, NASA FIRMS, AbuseIPDB, Groq (free tier), EIA
Free key optionalGitHub, HDX HAPI
Free with model limitsOpenRouter (select free models)
APICost ModelWhy Paid
WingbitsCommercial subscriptionPremium ADS-B data with higher fidelity
Cloudflare RadarEnterprise (included with CF plan)Radar API requires enterprise-level API token

Monthly Cost Estimate

ItemEstimated Cost
Wingbits commercial tierVaries (contact vendor)
Cloudflare RadarIncluded with Enterprise plan
All other APIs$0 (free tiers)
Upstash Redis (caching)Free tier / ~$10/mo for production
Vercel (hosting + edge functions)Free tier / Pro ~$20/mo

Total API cost for free-tier operation: $0/month (excluding Wingbits and Cloudflare Enterprise, which are optional enhancements).


7. Environment Variable Quick Reference

All environment variables needed for full API coverage:

env
# ── Geopolitical ──────────────────────────────────────
ACLED_ACCESS_TOKEN=           # ACLED API key (researcher account)
ACLED_EMAIL=                  # ACLED registered email

# ── Markets & Finance ────────────────────────────────
FINNHUB_API_KEY=              # Finnhub stock/ETF data
FRED_API_KEY=                 # Federal Reserve Economic Data

# ── Military & Security ──────────────────────────────
WINGBITS_API_KEY=             # Wingbits ADS-B (commercial)
WS_RELAY_URL=                 # AIS WebSocket relay URL
ABUSEIPDB_API_KEY=            # AbuseIPDB threat intel

# ── Natural Events ───────────────────────────────────
NASA_FIRMS_API_KEY=           # NASA FIRMS fire data

# ── AI / ML ──────────────────────────────────────────
GROQ_API_KEY=                 # Groq LLM inference (primary)
OPENROUTER_API_KEY=           # OpenRouter LLM (fallback)

# ── Infrastructure ───────────────────────────────────
CLOUDFLARE_API_TOKEN=         # Cloudflare Radar (enterprise)

# ── Content ──────────────────────────────────────────
GITHUB_TOKEN=                 # GitHub API (optional, higher rate limit)
HDX_APP_IDENTIFIER=           # HDX HAPI (optional, higher rate limit)
EIA_API_KEY=                  # Energy Information Administration

# ── Caching (infrastructure, not external API) ──────
UPSTASH_REDIS_REST_URL=       # Upstash Redis cache URL
UPSTASH_REDIS_REST_TOKEN=     # Upstash Redis cache token

Minimum viable setup (core features work): ACLED_ACCESS_TOKEN, ACLED_EMAIL, FINNHUB_API_KEY, GROQ_API_KEY

Recommended setup (all free features): All env vars above except WINGBITS_API_KEY and CLOUDFLARE_API_TOKEN

Full setup (all features): All env vars populated


This document is auto-referenced from todo_docs.md §3.2. See also: ARCHITECTURE.md, DATA_MODEL.md