Integration with Terraform

This guide explains how to forward Terraform logs to OpenObserve using Fluent Bit.

Overview

Terraform doesn’t log to files by default. This integration enables detailed logging by setting environment variables and then uses Fluent Bit to ship those logs to OpenObserve. The setup is lightweight, easy to run locally, and enables structured log analysis and dashboarding for infrastructure-as-code workflows.

Steps to Integrate

Prerequisites

Terraform installed
OpenObserve account (Cloud or Self-Hosted)

Step 1: Enable Logging in Terraform

Set environment variables:: Add the following lines to your shell profile (~/.zshrc, ~/.bashrc, etc.):
```
export TF_LOG=DEBUG
export TF_LOG_PATH="./tf.log"
```
> TF_LOG=DEBUG captures detailed debug-level logs.
> TF_LOG_PATH sets the location where logs are written.
Test log generation:: Run any Terraform command, such as:

terraform apply

Confirm that a file named tf.log is created in your working directory and is populated with Terraform debug logs.

Step 2: Configure Fluent Bit

Install Fluent Bit:
```
brew install fluent-bit
```
On Linux or Windows, install Fluent Bit using platform-specific methods: Fluent Bit installation guide
Create a Fluent Bit config file (e.g., fluent-bit.conf):

[INPUT]
    Name            tail
    Path            ./tf.log
    Tag             terraform
    Mem_Buf_Limit   5MB
    Skip_Long_Lines On
    Read_from_Head  On

[FILTER]
    Name            record_modifier
    Match           terraform
    Record          level DEBUG

[FILTER]
    Name            record_modifier
    Match           terraform
    Record          log_source terraform

[OUTPUT]
    Name            http
    Match           *
    URI             /api/default/terraform/_json
    Host            localhost
    Port            5080
    tls             Off
    Format          json
    Json_date_key   _timestamp
    Json_date_format iso8601
    HTTP_User       <openobserve_user>
    HTTP_Passwd     <openobserve_basic_password>
    compress        gzip

Key Settings:

Path: Location of the Terraform log file
Tag: Identifies the Terraform logs
record_modifier: Adds metadata like level and log_source
HTTP_User and HTTP_Passwd: Your OpenObserve credentials

Step 3: Run Fluent Bit

Start Fluent Bit with your config file:
```
fluent-bit -c /path/to/fluent-bit.conf
```

This will start reading tf.log and forwarding entries to OpenObserve in real time.

Step 4: Validate Logs in OpenObserve

In your Openobserve instance, Go to Logs → select your log stream → Set time range → Click Run Query

Troubleshooting

tf.log not generated?

Ensure both TF_LOG and TF_LOG_PATH are exported in your shell
Run terraform plan or terraform apply to trigger log generation
Verify tf.log exists in the specified path

Fluent Bit not forwarding logs?

Confirm that the Path in [INPUT] matches your actual tf.log location
Run Fluent Bit with -vv to see verbose logs
Ensure OpenObserve is reachable at localhost:5080

No logs in OpenObserve?

Check for correct username/password in the [OUTPUT] section
Confirm the URI path matches your OpenObserve stream:
```
/api/default/terraform/_json
```
Look for errors in the Fluent Bit output

Log fields missing or incorrect?

Check the [FILTER] configuration
log_source and level fields should be injected by record_modifier
Verify your log entries contain structured key-value output