OpenObserve Docs
IntegrationDatabase

DynamoDB Integration with OpenObserve

Updated View MarkdownOpen in ChatGPTOpen in Claude

This guide provides step-by-step instructions to capture DynamoDB table changes with Amazon Kinesis and forward them to OpenObserve for real-time monitoring and analysis.

Overview

Applications that rely on Amazon DynamoDB often require real-time insights into table changes. By enabling DynamoDB → Kinesis Data Streams → Kinesis Firehose → OpenObserve, you can ingest and visualize these changes instantly.

Steps to Integrate

Prerequisites
  • AWS account with permissions for DynamoDB, Kinesis Data Streams, and Kinesis Firehose
  • OpenObserve account (Cloud or Self-Hosted)
  • Dynamo DB Table
Step 1: Create Kinesis Data Stream
  1. Open the Kinesis Console.
  2. Follow the instructions and create a new data stream.
    Enable Kinesis Stream
Step 2: Enable Kinesis Data Stream on DynamoDB
  1. Go to the DynamoDB Console → Table → Exports and Streams tab.

  2. Enable Kinesis Data Stream and select your Kinesis data stream.

    Enable Kinesis Stream

Step 3: Create Kinesis Data Firehose Delivery Stream
  1. Open the Kinesis Firehose Console.
  2. Create a new Delivery Stream.
  3. Select Source → Kinesis Data Stream → choose the Kinesis data stream.
  4. Select Destination → HTTP Endpoint.
    • Endpoint URL: your OpenObserve Firehose endpoint
    • Authentication: Access Key (from Data > Data sources > Recommended > Amazon Web Services(AWS) in OpenObserve). The endpoint has the form https://<your-openobserve-domain>/aws/<org_id>/<stream_name>/_kinesis_firehose
  5. (Optional) Configure an S3 bucket for backup of failed records.
  6. Review and Create Firehose Stream.

Kinesis Firehose Config

Step 4: Monitor DynamoDB Changes in OpenObserve

Once the Firehose is active, DynamoDB changes begin flowing to OpenObserve. Go to Logs and select your stream. DynamoDB stream records in the Logs page

Troubleshooting

No logs appearing in OpenObserve?

  • Firehose:

    • Verify the delivery stream is active and error-free.
    • Review the Monitoring tab and look for failed deliveries.
  • OpenObserve:

    • Confirm the HTTP ingestion URL and access key are correct.
    • In the Logs view, select the correct stream and expand the time range to view recent data.
Was this page helpful?

On this page