PHP Classes

PHP MySQL Replication to Kafka Docker: Docker container Move replication events to Kafka

Recommend this page to a friend!
  Info   View files Example   View files View files (12)   DownloadInstall with Composer Download .zip   Reputation   Support forum   Blog (1)    
Ratings Unique User Downloads Download Rankings
Not yet rated by the usersTotal: 46 All time: 10,700 This week: 120Up
Version License PHP version Categories
php-mysql-replicatio 1.0.0MIT/X Consortium ...5PHP 5, Databases, Systems administration
Description 

Author

This package can be used to move replication events to Apache kafka.

It provides the files to setup a Docker container that runs a MySQL replication environment.

The PHP classes provided by the package can use Apache Kafta to exchange MySQL replication events between MySQL server instances.

Innovation Award
PHP Programming Innovation award nominee
February 2021
Number 7


Prize:
Docker is an useful tool that is use to assemble containers that can run with specific software on a server.

When you need to scale an application that uses MySQL, replication is one of the possible solutions that can be used to distribute the database server load among different servers.

Using docker you can assemble a solution for creating multiple MySQL server instances.

This PHP package can be used to manage MySQL replication events in docker contains using Apache kafka.

Manuel Lemos
Picture of Kacper Rowinski
  Performance   Level  
Name: Kacper Rowinski <contact>
Classes: 14 packages by
Country: Poland Poland
Age: 40
All time rank: 91826 in Poland Poland
Week rank: 312 Up5 in Poland Poland Up
Innovation award
Innovation award
Nominee: 8x

Recommendations

Example

#!/usr/bin/env php
<?php

declare(strict_types=1);

require
__DIR__ . '/../vendor/autoload.php';

use
ReplicationToKafka\ReplicationToKafkaCommand;
use
Symfony\Component\Console\Application;

$application = new Application();
$application->add(new ReplicationToKafkaCommand());
/** @noinspection PhpUnhandledExceptionInspection */
$application->run();


Details

php-mysql-replication-to-kafka-docker

Moves mysql replication events to kafka docker.

Env:

REPLICATOR_DEBUG: "false"

print worker action to log

REPLICATOR_USER: "root"

your mysql user (mandatory)

REPLICATOR_HOST: "r2k-mysql"

your mysql host/ip (mandatory)

REPLICATOR_PASSWORD: "root"

your mysql password (mandatory)

REPLICATOR_PORT: "3306"

our mysql host port (default 3306)

REPLICATOR_SLAVE_ID: "100"

script slave id for identification (SHOW SLAVE HOSTS)

REPLICATOR_START_FROM_GTID: ""

GTID marker(s) to start from

REPLICATOR_START_BIN_LOG_FILE_NAME: ""

bin log file name to start from

REPLICATOR_START_BIN_LOG_POSITION: "0"

bin log position to start from

REPLICATOR_START_FROM_MARIA_DB_GTID: ""

MariaDB GTID marker(s) to start from

REPLICATOR_TABLE_CACHE: "1024"

some data are collected from information schema, this data is cached.

REPLICATOR_HEARTBEAT: "0.0"

sets the interval in seconds between replication heartbeats.

REPLICATOR_LISTEN_ON_TABLES: "foo,bar"

only listen on given tables (default all tables)

REPLICATOR_LISTEN_ON_DATABASES: "foo,bar"

only listen on given databases (default all tables)

`REPLICATOR_LISTEN_EVENT_IDS: "73,32"

only listen for given events, look for ids in MySQLReplication\Definitions\ConstEventType::class`

REPLICATOR_IGNORE_EVENT_IDS: "73,32"

ignore given events, look for ids in MySQLReplication\Definitions\ConstEventType::class`

KAFKA_DEBUG: "false"

shows rdkafka events

KAFKA_BROKER: "r2k-kafka:9092"

brokers host:port

KAFKA_TOPIC: "test"

topic name to publish mysql events

KAFKA_RETRIES: "3"

how many tries worker should have before giving up and throw exception

KAFKA_TIMEOUT: "10000"

rdkafa flush timeout ms

Also, you can look and docker-compose.yml in repo.


  Files folder image Files  
File Role Description
Files folder image.github (1 directory)
Files folder imagebin (1 file)
Files folder imagesrc (4 files)
Accessible without login Plain text file composer.json Data Auxiliary data
Accessible without login Plain text file composer.lock Data Auxiliary data
Accessible without login Plain text file docker-compose.yml Data Auxiliary data
Accessible without login Plain text file Dockerfile Data Auxiliary data
Accessible without login Plain text file LICENSE Lic. License text
Accessible without login Plain text file README.md Doc. Documentation

  Files folder image Files  /  .github  
File Role Description
Files folder imageworkflows (1 file)

  Files folder image Files  /  .github  /  workflows  
File Role Description
  Accessible without login Plain text file docker-publish.yml Data Auxiliary data

  Files folder image Files  /  bin  
File Role Description
  Accessible without login Plain text file application.php Example Example script

  Files folder image Files  /  src  
File Role Description
  Plain text file KafkaSubscriber.php Class Class source
  Plain text file ReplicationToKafkaCommand.php Class Class source
  Plain text file ReplicatorResume.php Class Class source
  Plain text file Tools.php Class Class source

 Version Control Unique User Downloads Download Rankings  
 100%
Total:46
This week:0
All time:10,700
This week:120Up