# How to monitor AWS RDS Logs and events with AWS Cloudwatch and Kinesis Data Firehose

> Learn how to set up AWS RDS log ingestion for detailed monitoring and observability. Compare centralized log analysis with OpenObserve vs. traditional methods for optimizing performance, enhancing security, and scaling database insights effectively.

Source: https://openobserve.ai/blog/monitor-aws-rds-cloudwatch-logs/
Published: 2024-12-19
Authors: Chaitanya Sistla
Category: How To
Tags: AWS, Database, Logging

---

Effective monitoring of AWS RDS logs is vital to ensuring the reliability, performance, and scalability of your database systems, especially when managing multiple AWS accounts and RDS instances. This guide provides a step-by-step walkthrough for ingesting, analyzing, and visualizing AWS RDS logs to enhance your database monitoring strategy.

## Why Monitor AWS RDS Logs?

RDS logs offer invaluable insights into your database's health, performance, and behavior. By leveraging these logs, you can:

* **Optimize Query Performance:** Analyze slow query logs to fine-tune database performance and reduce latencies.  
* **Enhance Security Monitoring:** Track access logs and detect potential unauthorized activities.  
* **Diagnose Issues Efficiently:** Investigate error logs to quickly identify and resolve database errors.  
* **Ensure Scalability:** Monitor log-based metrics to predict and prepare for workload spikes.  
* **Centralize Log Analysis:** Consolidate logs from multiple AWS accounts into a unified platform for seamless observability.

### Prerequisites

Before starting, ensure:

1. You have access to AWS RDS.  
2. [Running OpenObserve Cluster](https://cloud.openobserve.ai/).  
3. Familiarity with AWS Kinesis is recommended.

### Step 1: Configure Kinesis Data Stream

CloudWatch streams metrics via Kinesis, which acts as a transport mechanism. Follow these steps:

1. **Create a Kinesis Data Stream:**  
   * Navigate to **Kinesis \> Data Streams** in the AWS Management Console.  
   * Click **Create data stream** and name it appropriately (e.g., `RDSlogStream`).  
   * Set the number of shards based on the expected metric volume.  
2. **Choose OpenObserve as destination:**  
   * In the destination configuration, select provide the url of OpenObserve HTTP endpoint. 
   ![aws rds logs](/assets/kinesis_first_2622a77bb7.png)
   ![aws rds logs](/assets/kinesis_second_dde3c90b6d.png)

Refer to [this guide](https://openobserve.ai/docs/integration/cloud/aws/cloudwatch-logs/) for detailed Kinesis configuration steps.

### Step 2: Modify RDS Instance to Enable Logs

* Navigate to the **Amazon RDS** service in the AWS Management Console.  
* Select the RDS instance you want to monitor.  
![aws rds logs](/assets/rds_first_0cb2dca2f7.png)
* Click **Modify** and ensure the following logs are enabled under the **Log Exports** section:  
  * **PostgreSQL Logs**  
  * **Upgrade Logs**
![aws rds logs](/assets/rds_second_014f796baa.png)  
* Enable the **Apply Immediately** option to make changes effective without a scheduled maintenance window.
![aws rds logs](/assets/rds_third_99d8682476.png)

### Step 3: Verify Log Group in CloudWatch

* After enabling log exports, verify that the RDS logs are being sent to the associated **CloudWatch Log Group**.  
* Navigate to **CloudWatch \> Log Groups** and search for the log group corresponding to your RDS instance.
![aws rds logs](/assets/rds_four_b8a37d220a.png)

### Step 4: Subscribe CloudWatch Log Group to Kinesis Streams

* Go to the **CloudWatch Log Group** for your RDS instance.  
* Create a subscription filter to send log events to an **Amazon Kinesis Data Stream**:  
  * Click **Actions \> Create Subscription Filter**.  
  * Select **Kinesis Stream** as the destination and choose your target stream.  
  * Refer to **Step1** for the details of the kinesis.
  ![aws rds logs](/assets/rds_six_d80576e31e.png)
  ![aws rds logs](/assets/rds_seven_072626091e.png) 

### Step 5: Verify Logs in OpenObserve

* Visit OpenObserve dashboard and go to logs to query the ingested data from AWS RDS
![aws rds logs](/assets/rds_five_b8d39b8a00.png)

## **Conclusion**

| Feature/Aspect | With OpenObserve | Without OpenObserve |
| :---- | :---- | :---- |
| Centralized Monitoring | Logs from multiple AWS accounts consolidated in one view | Logs scattered across individual AWS accounts |
| Proactive Issue Detection | Automated alerts on log anomalies and errors | Manual inspection required to detect issues options. |
| Detailed Query Insights | Analyze slow query logs for performance optimization | Limited to basic metrics without granular details |
| Advanced Security Auditing | Access logs analyzed for unauthorized activities | Minimal visibility into security-related events |

By following these steps, you can set up an end-to-end monitoring solution for AWS RDS logs using CloudWatch and OpenObserve. This approach ensures:

* **Proactive Database Monitoring**  
* **Enhanced Security and Performance**  
* **Centralized Observability**  
* **Scalable Analysis**

Ready to optimize your database performance and simplify your observability workflows? [Visit our website](https://openobserve.ai/) to learn more details about how the platform works.