RabbitReplication

Replication to many different data stores

RabbitReplication replicates changes from Drizzle to a variety of data stores, these include:

• Drizzle using drizzle-jdbc
• Cassandra, both using it as a key value store, and properly as a column store.
• Project Voldemort
• Memcached
• Infinispan using its REST API
• Redis
• WebSockets

Marshalling data

It is currently possible to marshall the data inserted in two ways:

• As JSON encoded objects, using Jackson
• As serialized Java objects

Extensible

RabbitReplication is also very extensible, which means that it is very easy to add new stores, new transports and new marshallers.

Overview

1. Clients perform DML (INSERT, UPDATE, DELETE etc) on the master drizzle database.
2. Drizzle stores the transaction both in Innodb and in a transaction log.
3. The RabbitReplication master tails the transaction log, and sends the raw transaction to RabbitMQ
4. A slave consumes the transaction from RabbitMQ
5. The transaction is deserialized using protobuf
6. The transaction is transformed into either a set of JDBC prepared statements, java objects or hashmaps
7. The transformed data is marshalled using the configured marshaller if we are using java objects
8. The marshalled, transformed data is persisted in the data store.

Reducing transactions

RabbitReplication can also reduce transactions to enable multi threaded appliers, read about that here

Hudson

Tests are run in Hudson, here.