⚠️ Archived Repository — The contents of this project have been moved to pgwatch-contrib to have a unified place for everything related to pgwatch that cannot be merged into upstream.

Pgwatch3 RPC Receivers

This repository contains the essential components to build your own Remote Sinks for Pgwatch v3. You can find the basic structure to create a Sink (or Receiver as we call it in this repo) which is basically a RPC Server that the pgwatch RPC Client interacts with.

The primary goal of this repository is to provide you with the building blocks to get started with your own implementations and also to provide some examples of places where measurements from pgwatch can be used. You can find some of our example implementations in the cmd folder.

Checkout PgWatch to get started with this project.

Architecture

The Remote Sinks work using RPC protocol. Some cool advantages of using RPC are:

• PgWatch is not concerned about the actual sink implementation. You can literally do anything with the measurements delivered to you by pgwatch and share messages per function call if required.
• The sink implementations can be easily developed in Go, which has support for most of the storage formats out there and is pretty easy to write and work with.

The RPC receiver is treated as the default sink formats and no special changes are required in your pgwatch setup.

To use a RPC sink you can start pgwatch with the argument: --sink=rpc://<host>:<port>.

Running Sinks

If you are using pgwatch's gRPC sink with Authentication credentials or TLS configured, you'll need to set the following environment variables, to ensure the server works properly.

# if empty, password is ignored during authentication
export PGWATCH_RPC_SERVER_USERNAME="username"

# if empty, username is ignored during authentication
export PGWATCH_RPC_SERVER_PASSWORD="password"

# if not set TLS is not used
export PGWATCH_RPC_SERVER_CERT="/path/to/server.crt"

# if not set TLS is not used
export PGWATCH_RPC_SERVER_KEY="/path/to/server.key"

To start any of the provided receivers you can use:

go generate ./sinks/pb # generate golang code from protobuf 
go run ./cmd/[receiver_dir] [OPTIONS] --port=9999

By default all sinks will listen at 0.0.0.0 with the specified port number.

Now once your receiver is up you can setup pgwatch as follows:

go run ./cmd/pgwatch --sink=grpc://<ip/hostname_of_your_sink>:<port_where_recv_is_listening> [OPTIONS]

Voila! You have seamless integration between pgwatch and your custom sink.
Try out our various implementations to get a feel of how these receivers feel with your custom pgwatch instances.

Developing Custom Sinks

To develop your own custom sinks, refer to this mini tutorial.

You can also look at our example sinks to help with your implementation or extend them for your own use cases:

• CSV Receiver: Store measurements in CSV files.
• Kafka Receiver: Stream measurements using Kafka.
• Parquet Receiver: Store measurements in Parquet files.
• ClickHouse Receiver: Store measurements in OLAP databases like ClickHouse for analytics.
• LLama Receiver: Gain performance insights and recommendations from your measurements using tinyllama.
• S3 Receiver: Store measurements in AWS S3.