AWS Kinesis Firehose Log Ingestion - AWS Log Streaming
Amazon CloudWatch and many other AWS services can send logs to Amazon Kinesis Firehose for streaming AWS logs to OpenObserve. This enables AWS log ingestion and centralized AWS log management. You can find the complete list in the AWS logs and resource policy reference.
Some of the AWS logs that can be sent to Kinesis firehose are: Amazon Cloudwatch, Amazon Virtual Private Cloud flow logs, AWS Network Firewall logs, AWS WAF logs.
You can configure Kinesis firehose to send logs to OpenObserve Cloud / OpenObserve using below details.
HTTP endpoint: https://api.openobserve.ai/aws/org_name/stream_name/_kinesis_firehose
Access key: vhgjleGFtcGxlLmNvbTo2eUNsSW1HZXV4S3hZanJiAccess key is base64 encoded value of userid:password. e.g. If your user id is root@example.com and password is Complexpass#123 then the base64 encoded value is
echo -n 'root@example.com:Complexpass#123' | base64which will give you
cm9vdEBleGFtcGxlLmNvbTpDb21wbGV4cGFzcyMxMjM=
The Firehose sends this value in the X-Amz-Firehose-Access-Key header. Data > Data sources > Custom > Logs > Kinesis Firehose shows the endpoint and access key for your organization, ready to copy.
Fluentd
Configure Fluentd for log aggregation and centralized log management with OpenObserve, using the HTTP output plugin and JSON formatting.
Splunk HEC
Send logs to OpenObserve from Splunk forwarders and HEC clients over the Splunk HTTP Event Collector protocol, authenticated with a Splunk HEC token.