Effective monitoring of AWS RDS logs is vital to ensuring the reliability, performance, and scalability of your database systems, especially when managing multiple AWS accounts and RDS instances. This guide provides a step-by-step walkthrough for ingesting, analyzing, and visualizing AWS RDS logs to enhance your database monitoring strategy.

Why Monitor AWS RDS Logs?

RDS logs offer invaluable insights into your database's health, performance, and behavior. By leveraging these logs, you can:

• Optimize Query Performance: Analyze slow query logs to fine-tune database performance and reduce latencies.
• Enhance Security Monitoring: Track access logs and detect potential unauthorized activities.
• Diagnose Issues Efficiently: Investigate error logs to quickly identify and resolve database errors.
• Ensure Scalability: Monitor log-based metrics to predict and prepare for workload spikes.
• Centralize Log Analysis: Consolidate logs from multiple AWS accounts into a unified platform for seamless observability.

Prerequisites

Before starting, ensure:

1. You have access to AWS RDS.
2. Running OpenObserve Cluster.
3. Familiarity with AWS Kinesis is recommended.

Step 1: Configure Kinesis Data Stream

CloudWatch streams metrics via Kinesis, which acts as a transport mechanism. Follow these steps:

1. Create a Kinesis Data Stream:
   • Navigate to Kinesis > Data Streams in the AWS Management Console.
   • Click Create data stream and name it appropriately (e.g., RDSlogStream).
   • Set the number of shards based on the expected metric volume.
2. Choose OpenObserve as destination:
   • In the destination configuration, select provide the url of OpenObserve HTTP endpoint.

Refer to this guide for detailed Kinesis configuration steps.

Step 2: Modify RDS Instance to Enable Logs

• Navigate to the Amazon RDS service in the AWS Management Console.
• Select the RDS instance you want to monitor.
  
• Click Modify and ensure the following logs are enabled under the Log Exports section:
  • PostgreSQL Logs
  • Upgrade Logs
• Enable the Apply Immediately option to make changes effective without a scheduled maintenance window.

Step 3: Verify Log Group in CloudWatch

• After enabling log exports, verify that the RDS logs are being sent to the associated CloudWatch Log Group.
• Navigate to CloudWatch > Log Groups and search for the log group corresponding to your RDS instance.

Step 4: Subscribe CloudWatch Log Group to Kinesis Streams

• Go to the CloudWatch Log Group for your RDS instance.
• Create a subscription filter to send log events to an Amazon Kinesis Data Stream:
  • Click Actions > Create Subscription Filter.
  • Select Kinesis Stream as the destination and choose your target stream.
  • Refer to Step1 for the details of the kinesis.

Step 5: Verify Logs in OpenObserve

• Visit OpenObserve dashboard and go to logs to query the ingested data from AWS RDS

Conclusion

By following these steps, you can set up an end-to-end monitoring solution for AWS RDS logs using CloudWatch and OpenObserve. This approach ensures:

• Proactive Database Monitoring
• Enhanced Security and Performance
• Centralized Observability
• Scalable Analysis

Ready to optimize your database performance and simplify your observability workflows? Visit our website to learn more details about how the platform works.