Skip to content

lambda-promtail - SQS source seems to be not working #15

@Rohlik

Description

@Rohlik

Describe the bug
I created a Lambda that uses Promtail's image to get logs from SQS and push them to the Loki endpoint.
We have a feeling that SQS should be supported based on the source code:
https://github.com/grafana/loki/blob/ffc61fbbf4990ccb00aaed107327fd5ff4d66e82/tools/lambda-promtail/lambda-promtail/main.go#L169
But we are getting the error below from Lambda:

level=error caller=main.go:236 err="error processing event: event bridge event type not supported"

Our pipeline looks like: Log stream > EventBridge > SQS < Lambda with Promtail > Loki
Good to mention is that if we replace SQS with CloudWatch, then it works like a charm.

To Reproduce
Steps to reproduce the behavior:

  1. Create SQS queue
  2. Create Lambda based on the Promtail ECR image and configure it to send logs to a Loki endpoint
  3. Set Lambda trigger against SQS created in step 1
  4. Send some logs to SQS via EventBridge
  5. Errors will appear and nothing will be sent to the Loki

Expected behavior
Logs are correctly processed by Lambda/Promtail and stored in Loki.

Environment:

  • Infrastructure: Loki on EKS, AWS (Lambda, SQS, EventBridge)
  • Deployment tool: Helm, FluxCD

Screenshots, Promtail config, or terminal output
image

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions