Back to Lago

Lago Ingest Connectors

connectors/README.md

1.45.22.7 KB
Original Source

Lago Ingest Connectors

Events format

  • All events should respect the same format as Lago API
json
{
  "event": {
    "external_subscription_id": "string",
    "transaction_id": "unique_transaction_identifier",
    "code": "billable_metric_code",
    // Event Unix Timestamp
    "timestamp": 1620000000,
    "properties": {
      // Should respect the format of your billable metric
      "my_property": "my_value"
    },
    // Optional, used for the dynamic pricing feature, defaulted to 0
    "precise_total_amount_cents": 1000
  }
}

Configuration

Environment Variables

Environment VariableDescriptionRequired
LOG_LEVELLog level for your connector, default: infoNo

SQS Connector

Environment Variables

Environment VariableDescriptionRequired
SQS_ENDPOINTThe endpoint of the SQS serviceYes
SQS_REGIONThe region of the SQS serviceYes
SQS_KEY_IDThe AWS access key idYes
SQS_KEY_SECRETThe AWS secret keyYes
SQS_DLQ_ENDPOINTThe endpoint of the SQS service for the dead letter queueNo
ORGANIZATION_IDLago organization IDYes
KAFKA_BROKERSRedpanda BrokerYes
KAFKA_USERRedpanda UserYes
KAFKA_PASSWORDRedpanda PasswordYes
KAFKA_TOPICRedpanda Topic to send eventsYes

Kinesis Connector

Environment Variables

Environment VariableDescriptionRequired
KINESIS_STREAMThe name of the Kinesis stream to consumeYes
AWS_REGIONThe AWS region for Kinesis and DynamoDBYes
AWS_ROLEThe IAM role ARN to assume for accessing KinesisYes
AWS_ROLE_EXTERNAL_IDThe external ID for the assumed roleYes
DYNAMODB_TABLEThe DynamoDB table name used for checkpointingYes
KAFKA_BROKERSRedpanda BrokerYes
KAFKA_USERRedpanda UserYes
KAFKA_PASSWORDRedpanda PasswordYes
KAFKA_TOPICRedpanda Topic to send eventsYes
KAFKA_TLSEnable TLS for Kafka connection, default: falseNo
KAFKA_BATCH_COUNTNumber of messages to batch before sending, default: 100No
KAFKA_BATCH_BYTE_SIZEMaximum size in bytes for batching, default: 1000000 (1MB)No
KAFKA_BATCH_PERIODTime period for batching, default: 1sNo

HTTP Server

Environment Variables

Environment VariableDescriptionRequired
KAFKA_BROKERSRedpanda BrokerYes
KAFKA_USERRedpanda UserYes
KAFKA_PASSWORDRedpanda PasswordYes
KAFKA_TOPICRedpanda Topic to send eventsYes
KAFKA_TLSEnable TLS for Kafka connection, default: falseNo
KAFKA_BATCH_COUNTNumber of messages to batch before sending, default: 100No
KAFKA_BATCH_BYTE_SIZEMaximum size in bytes for batching, default: 1000000 (1MB)No
KAFKA_BATCH_PERIODTime period for batching, default: 1sNo