Why Tank and Tank vs X

As Jay Kreps remarked in his now seminal “The Log: What every software engineer should know about real-time data's unifying abstraction” post:

Furthermore, the focus on the algorithms obscures the underlying log abstraction systems need. I suspect we will end up focusing more on the log as a commoditized building block irrespective of its implementation in the same way we often talk about a hash table without bothering to get in the details of whether we mean the murmur hash with linear probing or some other variant. The log will become something of a commoditized interface, with many algorithms and implementations competing to provide the best guarantees and optimal performance.

Indeed, this is a different implementation of the log abstraction, specifically designed and engineered for optimal (very high) performance, without sacrificing safety and durability, and being particular resilient to various conditions that can affect the data integrity.

Other than implementing the core log abstraction semantics, it differs significantly from other similar systems (Kafka, Kinesis, Google Pub/Sub, etc). This page will be updated with information describing the differences in current implementation and semantics among the most popular alternatives and Tank.

Please see README.md for some performance metrics (comparing TANK to Kafka).

Measuring Performance using tank-cli

Start by creating a dummy topic ("test").

$> ./tank-cli  -b :1122 -t test create_topic 16

this will create a topic "test" with 16 partitions, by connecting to a Tank node running locally, accepting connections on port 1122

Publishing Messages (Producer To Broker)

$> ./tank-cli  -b :1122 -t test  bm p2b  -s 100 -c 513674 -B 8196   -R

Please refer to ./tank-cli -b :1122 -t test bm p2b -h for the various options. This is a producer to broker test. It will send 513674 messages, in bundles of 8196 messages each, and each message is 100 bytes long. -R disables the compression. You will get back a report with how long it took to do this and get back an ACK from Tank.