Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
slapos
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Ophélie Gagnard
slapos
Commits
083a62b8
Commit
083a62b8
authored
Jul 24, 2020
by
Alain Takoudjou
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
repman: add support for shardproxy with spider
parent
a8d268d7
Changes
9
Hide whitespace changes
Inline
Side-by-side
Showing
9 changed files
with
267 additions
and
55 deletions
+267
-55
software/repman/buildout.hash.cfg
software/repman/buildout.hash.cfg
+8
-7
software/repman/instance-mariadb.cfg.jinja2.in
software/repman/instance-mariadb.cfg.jinja2.in
+1
-0
software/repman/instance-proxy.cfg.jinja2.in
software/repman/instance-proxy.cfg.jinja2.in
+185
-18
software/repman/instance-repman-input-schema.json
software/repman/instance-repman-input-schema.json
+1
-7
software/repman/instance-repman.cfg.jinja2.in
software/repman/instance-repman.cfg.jinja2.in
+25
-8
software/repman/instance.cfg.in
software/repman/instance.cfg.in
+7
-0
software/repman/templates/cluster-config.toml.in
software/repman/templates/cluster-config.toml.in
+23
-4
software/repman/templates/mariadb_init_root.sql.in
software/repman/templates/mariadb_init_root.sql.in
+6
-2
software/repman/templates/proxy-need-start-stop.sh.in
software/repman/templates/proxy-need-start-stop.sh.in
+11
-9
No files found.
software/repman/buildout.hash.cfg
View file @
083a62b8
...
@@ -14,15 +14,16 @@
...
@@ -14,15 +14,16 @@
# not need these here).
# not need these here).
[instance.cfg]
[instance.cfg]
filename = instance.cfg.in
filename = instance.cfg.in
md5sum =
b6798777834fdc4789591b9a53048e9c
md5sum =
d991276c7f301f5376ee8ea793b9bdfe
[instance-repman.cfg]
[instance-repman.cfg]
_update_hash_filename_ = instance-repman.cfg.jinja2.in
_update_hash_filename_ = instance-repman.cfg.jinja2.in
md5sum =
dc55574a7a7374c7e2d2a8173925b213
md5sum =
5104c1a2539b5d73d80b57abc99d5b25
[instance-proxy.cfg]
[instance-proxy.cfg]
_update_hash_filename_ = instance-proxy.cfg.jinja2.in
_update_hash_filename_ = instance-proxy.cfg.jinja2.in
md5sum = 4a01dc6efaef2caa780024a48339d6a4
md5sum = 34aac08d7b3356466f07bafffa4ae59f
[config-toml.in]
[config-toml.in]
_update_hash_filename_ = templates/config.toml.in
_update_hash_filename_ = templates/config.toml.in
...
@@ -30,7 +31,7 @@ md5sum = 19fe38a342a5c9857e29f78eedb3c46e
...
@@ -30,7 +31,7 @@ md5sum = 19fe38a342a5c9857e29f78eedb3c46e
[config-cluster-toml.in]
[config-cluster-toml.in]
_update_hash_filename_ = templates/cluster-config.toml.in
_update_hash_filename_ = templates/cluster-config.toml.in
md5sum =
079599a2841b5a0d5178bb12c4a30ae
8
md5sum =
f240b2266886b678a338ccdd1f0ba4a
8
[nginx.conf.in]
[nginx.conf.in]
_update_hash_filename_ = templates/nginx.conf.in
_update_hash_filename_ = templates/nginx.conf.in
...
@@ -38,7 +39,7 @@ md5sum = 0eeb24c6aa0760f0d33c4cc2828ddf30
...
@@ -38,7 +39,7 @@ md5sum = 0eeb24c6aa0760f0d33c4cc2828ddf30
[template-mariadb.cfg]
[template-mariadb.cfg]
_update_hash_filename_ = instance-mariadb.cfg.jinja2.in
_update_hash_filename_ = instance-mariadb.cfg.jinja2.in
md5sum =
189ccee60d0fb53e29431a45e0816bc1
md5sum =
3960fceca99190d774162107840c1c15
[template-my-cnf]
[template-my-cnf]
_update_hash_filename_ = templates/my.cnf.in
_update_hash_filename_ = templates/my.cnf.in
...
@@ -50,7 +51,7 @@ md5sum = 9be53e2e92333b93e92556b8a01d9c42
...
@@ -50,7 +51,7 @@ md5sum = 9be53e2e92333b93e92556b8a01d9c42
[mariadb-init-root-sql]
[mariadb-init-root-sql]
_update_hash_filename_ = templates/mariadb_init_root.sql.in
_update_hash_filename_ = templates/mariadb_init_root.sql.in
md5sum =
d927b5d36410bb02717d5ca125525785
md5sum =
f971af55bc1b36e6eecf87a5858b1d19
[init-root-wrapper-in]
[init-root-wrapper-in]
_update_hash_filename_ = templates/init_root_wrapper.in
_update_hash_filename_ = templates/init_root_wrapper.in
...
@@ -70,4 +71,4 @@ md5sum = e9bcee5dc1318fe3acda2663472214f5
...
@@ -70,4 +71,4 @@ md5sum = e9bcee5dc1318fe3acda2663472214f5
[proxy-need-start-stop.sh.in]
[proxy-need-start-stop.sh.in]
_update_hash_filename_ = templates/proxy-need-start-stop.sh.in
_update_hash_filename_ = templates/proxy-need-start-stop.sh.in
md5sum =
455aaf369bf5141758dc57f2c0e67b08
md5sum =
6df22cfc2790a2a922daec5d30fbbd19
software/repman/instance-mariadb.cfg.jinja2.in
View file @
083a62b8
...
@@ -89,6 +89,7 @@ database-list = {{ dumps(database_list) }}
...
@@ -89,6 +89,7 @@ database-list = {{ dumps(database_list) }}
mroonga-mariadb-install-sql = {{ dumps(parameter_dict['mroonga-mariadb-install-sql']) }}
mroonga-mariadb-install-sql = {{ dumps(parameter_dict['mroonga-mariadb-install-sql']) }}
root-user = repman
root-user = repman
heartbeat-user = {{ slapparameter_dict['heartbeat-user'] }}
heartbeat-user = {{ slapparameter_dict['heartbeat-user'] }}
heartbeat-password = {{ slapparameter_dict['heartbeat-password'] }}
[init-script]
[init-script]
< = jinja2-template-executable
< = jinja2-template-executable
...
...
software/repman/instance-proxy.cfg.jinja2.in
View file @
083a62b8
...
@@ -17,6 +17,8 @@
...
@@ -17,6 +17,8 @@
{% set ip = ipv6 -%}
{% set ip = ipv6 -%}
{% endif -%}
{% endif -%}
{% set proxy = slapparameter_dict.get('proxy-type') -%}
{% do cluster_list.append({'name':name, 'host': host, 'port': proxy_admin_port}) -%}
{% do cluster_list.append({'name':name, 'host': host, 'port': proxy_admin_port}) -%}
[directory]
[directory]
...
@@ -33,13 +35,18 @@ promise = ${:etc}/promise
...
@@ -33,13 +35,18 @@ promise = ${:etc}/promise
log = ${:var}/log
log = ${:var}/log
tmp = ${:home}/tmp
tmp = ${:home}/tmp
data = ${:home}/proxy
data = ${:home}/proxy
mariadb-ssl = ${:etc}/mariadb-ssl
config-tmp = ${:tmp}/config
lib = ${:var}/lib
mysql = ${:lib}/mysql
{% import "supervisord_lib" as supervisord_lib with context %}
{% import "supervisord_lib" as supervisord_lib with context %}
{% set proxy
sql_controller = "proxysql
-ctl" -%}
{% set proxy
_controller = "dbproxy
-ctl" -%}
{{ supervisord_lib.supervisord(proxy
sql
_controller, buildout_bin_directory, supervisord_conf, use_service_hash=False) }}
{{ supervisord_lib.supervisord(proxy_controller, buildout_bin_directory, supervisord_conf, use_service_hash=False) }}
{% do part_list.append("supervisord-
proxysql
-ctl") -%}
{% do part_list.append("supervisord-
dbproxy
-ctl") -%}
[download-proxy-config]
[download-proxy
sql
-config]
recipe = slapos.recipe.template:jinja2
recipe = slapos.recipe.template:jinja2
template = inline:#!{{ bash_bin }}
template = inline:#!{{ bash_bin }}
NAME=$1
NAME=$1
...
@@ -68,7 +75,7 @@ stop-on-error = false
...
@@ -68,7 +75,7 @@ stop-on-error = false
config = ${directory:etc}/proxysql.cnf
config = ${directory:etc}/proxysql.cnf
data = ${directory:data}
data = ${directory:data}
command =
command =
${download-proxy-config:rendered} {{ name }} {{ host }} {{ proxy_admin_port }} ${:config}
${download-proxy
sql
-config:rendered} {{ name }} {{ host }} {{ proxy_admin_port }} ${:config}
update-command = ${:command}
update-command = ${:command}
[proxysql-wrapper]
[proxysql-wrapper]
...
@@ -101,7 +108,7 @@ name = proxysql-port-listening.py
...
@@ -101,7 +108,7 @@ name = proxysql-port-listening.py
config-hostname= {{ ip }}
config-hostname= {{ ip }}
config-port = {{ proxy_port }}
config-port = {{ proxy_port }}
{% if not use_ipv6 -%}
{% if not use_ipv6
and proxy == "proxysql"
-%}
[proxysql-ipv6-promise]
[proxysql-ipv6-promise]
<= monitor-promise-base
<= monitor-promise-base
module = check_port_listening
module = check_port_listening
...
@@ -111,15 +118,6 @@ config-port = {{ proxy_port }}
...
@@ -111,15 +118,6 @@ config-port = {{ proxy_port }}
{% do part_list.append("proxysql-ipv6-promise") -%}
{% do part_list.append("proxysql-ipv6-promise") -%}
{% endif -%}
{% endif -%}
{% set service_name = "proxy-" ~ name -%}
{% set proxysql_dict = {"name": service_name, "command": "${proxysql-wrapper:wrapper-path}",
"stopwaitsecs": 60, "environment": [],
"stdout_logfile": "${directory:log}/" ~ service_name ~ ".log",
"stderr_logfile": "${directory:log}/" ~ service_name ~ ".log" } %}
{{ supervisord_lib.supervisord_program(service_name, proxysql_dict) }}
{% do part_list.append("supervisord-" ~ service_name) %}
[slap-configuration]
[slap-configuration]
recipe = slapos.cookbook:slapconfiguration
recipe = slapos.cookbook:slapconfiguration
...
@@ -152,13 +150,14 @@ template = {{ template_proxy_need_stopstart }}
...
@@ -152,13 +150,14 @@ template = {{ template_proxy_need_stopstart }}
mode = 755
mode = 755
cluster-list = {{ dumps(cluster_list) }}
cluster-list = {{ dumps(cluster_list) }}
context =
context =
key proxy
sql_controller {{proxysql
_controller}}-bin:wrapper-path
key proxy
_controller {{proxy
_controller}}-bin:wrapper-path
key get_proxy_config download-proxy-config:rendered
key get_proxy_config download-proxy
sql
-config:rendered
key cluster_list :cluster-list
key cluster_list :cluster-list
raw repman_url {{ repman_url }}
raw repman_url {{ repman_url }}
raw jq_bin {{ jq_bin }}
raw jq_bin {{ jq_bin }}
raw bash_bin {{ bash_bin }}
raw bash_bin {{ bash_bin }}
raw curl_bin {{ curl_bin }}
raw curl_bin {{ curl_bin }}
raw proxy {{ proxy }}
[proxy-need-stop-start]
[proxy-need-stop-start]
recipe = slapos.cookbook:cron.d
recipe = slapos.cookbook:cron.d
...
@@ -167,18 +166,186 @@ name = proxysql-need-stop-start
...
@@ -167,18 +166,186 @@ name = proxysql-need-stop-start
frequency = * * * * *
frequency = * * * * *
command = ${template-proxysql-need-stop-start:rendered}
command = ${template-proxysql-need-stop-start:rendered}
##################### Mariadb Spider Proxy ###################"
[shardproxy-cnf]
recipe = slapos.recipe.template:jinja2
mode = 644
rendered = ${directory:bin}/mysqld_restart
ssl-crt = ${directory:mariadb-ssl}/crt.pem
ssl-key = ${directory:mariadb-ssl}/key.pem
data-directory = ${directory:srv}/mariadb
template = inline:
[mysqld]
socket = ${directory:run}/mariadb.sock
datadir = ${:data-directory}
pid_file = ${directory:run}/mariadb.pid
#plugin_load = ha_mroonga
plugin-dir = {{ parameter_dict['mroonga-mariadb-plugin-dir'] }}
ssl
ssl-cert = ${:ssl-crt}
ssl-key = ${:ssl-key}
ssl-ca = ${certificate-authority:ca-dir}/cacert.pem
[client]
socket = ${directory:run}/mariadb.sock
user = root
# XXX - custom wrapper because we need to remove spider before call mysql_install_db
# XXX - else the script stall
[mysqld]
recipe = slapos.recipe.template:jinja2
rendered = ${directory:bin}/mysqld
mode = 755
mariadb-location = {{ parameter_dict['mariadb-location'] }}
mysql-rc = ${directory:etc}/mysql/rc.d/
spider-conf = ${:mysql-rc}/01_engine_spider.cnf
template =
inline:{% raw %}#!/bin/sh -e
basedir='${:mariadb-location}'
datadir='{{datadir}}'
[ -e "$datadir" ] || {
rm -rvf ${:spider-conf}
rm -vrf "$datadir.new"
"$basedir/scripts/mysql_install_db" \
--defaults-file='{{defaults_file}}' \
--skip-name-resolve \
--auth-root-authentication-method=normal \
--basedir="$basedir" --plugin_dir="$basedir/lib/plugin" \
--datadir="$datadir.new"
mv -v "$datadir.new" "$datadir"
cd ${:mysql-rc} && ln -sf ../engine_spider.cnf 01_engine_spider.cnf
}
{%- if environ is defined %}
{%- for variable in environ.splitlines() %}
{{ variable }} \
{%- endfor %}
{%- endif %}
exec "$basedir/bin/mysqld" --defaults-file='{{defaults_file}}' "$@"
{% endraw %}
context =
key defaults_file install-mysql-config:config
key datadir shardproxy-cnf:data-directory
key environ :environ
environ =
ODBCSYSINI='${directory:etc}'
LD_LIBRARY_PATH=$${LD_LIBRARY_PATH:+$LD_LIBRARY_PATH:}'{{ parameter_dict['unixodbc-location'] }}/lib'
[shardproxy-wrapper]
<= certificate-authority
recipe = slapos.cookbook:certificate_authority.request
key-file = ${shardproxy-cnf:ssl-key}
cert-file = ${shardproxy-cnf:ssl-crt}
executable = ${mysqld:rendered}
wrapper = ${directory:controller}/mariadb-spider
[binary-wrap-base]
recipe = slapos.cookbook:wrapper
# Note: --defaults-file must be the first argument, otherwise wrapped binary
# will reject it.
command-line =
"{{ parameter_dict['mariadb-location'] }}/bin/${:command}"
--defaults-file="${directory:etc}/mysql/my.cnf" --protocol=socket
wrapper-path = ${directory:bin}/${:command}
[binary-wrap-mysql]
<= binary-wrap-base
command = mysql
[binary-wrap-mysql_upgrade]
<= binary-wrap-base
command = mysql_upgrade
extra-args = --skip-write-binlog
[binary-wrap-mysql_tzinfo_to_sql]
<= binary-wrap-base
command-line = "{{ parameter_dict['mariadb-location'] }}/bin/${:command}" --skip-write-binlog
command = mysql_tzinfo_to_sql
# Donwnload mariadb configuration from repman
[mysql-get-config]
recipe = slapos.recipe.template:jinja2
mode = 755
rendered = ${directory:bin}/mysqld-update-config
cluster = {{ name }}
template = inline:#!{{ bash_bin }}
cd ${directory:config-tmp} &&
rm -rf * &&
{{ curl_bin }} -o config.tar.gz {{ repman_url }}/api/clusters/${:cluster}/servers/{{ host }}/{{ proxy_port }}/config
tar -xzf config.tar.gz
cp -r data/.system ${directory:mysql}
rm -rf ${directory:etc}/mysql
cp -r etc/mysql ${directory:etc}
ln -sf ${directory:mysql}/.system ${directory:var}/system
ln -sf ${shardproxy-cnf:rendered} ${directory:etc}/mysql/custom/00_spider-proxy.cnf
[install-mysql-config]
recipe = plone.recipe.command
stop-on-error = true
config = ${directory:etc}/mysql/my.cnf
command = ${mysql-get-config:rendered}
update-command = ${:command}
[init-proxy-script]
recipe = slapos.recipe.template:jinja2
mode = 755
rendered = ${directory:etc}/.init-root.sql
template = {{ parameter_dict['template-mariadb-init-root'] }}
context = section parameter_dict init-proxy-parameters
mode = 600
[update-mysql]
recipe = slapos.cookbook:generic.mysql.wrap_update_mysql
output = ${directory:scripts}/mariadb_update
binary = ${binary-wrap-mysql_upgrade:wrapper-path}
mysql = ${binary-wrap-mysql:wrapper-path}
init-script = ${init-proxy-script:rendered}
mysql_tzinfo_to_sql = ${binary-wrap-mysql_tzinfo_to_sql:wrapper-path}
[init-proxy-parameters]
password = {{ slapparameter_dict['proxy-passwd'] }}
root-user = {{ slapparameter_dict['proxy-user'] }}
heartbeat-user = {{ slapparameter_dict['heartbeat-user'] }}
heartbeat-password = {{ slapparameter_dict['heartbeat-password'] }}
[shardproxy-promise]
<= monitor-promise-base
module = check_command_execute
name = mariadb.py
config-command = "{{ parameter_dict['bin-directory'] }}/is-local-tcp-port-opened" "{{ ip }}" "{{ proxy_port }}"
#############################
#############################
#
#
# Deploy replication-manager instance
# Deploy replication-manager instance
#
#
#############################
#############################
{% set service_name = "proxy-" ~ name -%}
{% set proxydb_dict = {"name": service_name, "command": "${proxysql-wrapper:wrapper-path}",
"stopwaitsecs": 60, "environment": [], "autostart": True,
"stdout_logfile": "${directory:log}/" ~ service_name ~ ".log",
"stderr_logfile": "${directory:log}/" ~ service_name ~ ".log" } %}
{% if proxy == 'proxysql' -%}
{% do part_list.append('proxy-need-stop-start') -%}
{% endif -%}
{% if proxy == 'shardproxy' -%}
{% do part_list.append('proxy-need-stop-start') -%}
{% do part_list.append('update-mysql') -%}
{% do part_list.append('binary-wrap-mysql') -%}
{% do part_list.append('shardproxy-promise') -%}
{% do proxydb_dict.__setitem__("command", "${shardproxy-wrapper:wrapper}") -%}
{% endif -%}
{{ supervisord_lib.supervisord_program(service_name, proxydb_dict) }}
{% do part_list.append("supervisord-" ~ service_name) %}
[buildout]
[buildout]
extends =
extends =
{{ template_monitor }}
{{ template_monitor }}
parts =
parts =
monitor-base
monitor-base
publish-connection-parameter
publish-connection-parameter
proxy-need-stop-start
# Complete parts with sections
# Complete parts with sections
{{ part_list | join('\n ') }}
{{ part_list | join('\n ') }}
...
...
software/repman/instance-repman-input-schema.json
View file @
083a62b8
...
@@ -190,12 +190,6 @@
...
@@ -190,12 +190,6 @@
"type"
:
"string"
,
"type"
:
"string"
,
"default"
:
""
"default"
:
""
},
},
"proxysql-user"
:
{
"title"
:
"Proxysql username"
,
"description"
:
"Proxysql external user, default is 'external'."
,
"type"
:
"string"
,
"default"
:
"external"
},
"proxy-tags"
:
{
"proxy-tags"
:
{
"title"
:
"Proxy tag list"
,
"title"
:
"Proxy tag list"
,
"description"
:
"playbook configuration tags. Default: [
\"
pkg
\"
,
\"
masterslave
\"
,
\"
linux
\"
,
\"
noreadwritesplit
\"
]"
,
"description"
:
"playbook configuration tags. Default: [
\"
pkg
\"
,
\"
masterslave
\"
,
\"
linux
\"
,
\"
noreadwritesplit
\"
]"
,
...
@@ -229,7 +223,7 @@
...
@@ -229,7 +223,7 @@
"default"
:
"proxysql"
,
"default"
:
"proxysql"
,
"enum"
:
[
"enum"
:
[
"proxysql"
,
"proxysql"
,
"s
phinx
"
"s
hardproxy
"
]
]
},
},
"proxy-amount"
:
{
"proxy-amount"
:
{
...
...
software/repman/instance-repman.cfg.jinja2.in
View file @
083a62b8
...
@@ -48,7 +48,8 @@ partition-id = ${slap-connection:partition-id}
...
@@ -48,7 +48,8 @@ partition-id = ${slap-connection:partition-id}
{% set proxy_type = parameter_dict.get('proxy-type', 'proxysql') -%}
{% set proxy_type = parameter_dict.get('proxy-type', 'proxysql') -%}
{% set proxy_port = 6032 + 100*loop.index0 -%}
{% set proxy_port = 6032 + 100*loop.index0 -%}
{% set proxy_config_dict = {'ip': [], 'ipv6': [], 'partition': [],
{% set proxy_config_dict = {'ip': [], 'ipv6': [], 'partition': [],
'port': proxy_port, 'admin-port': proxy_port + 1000} -%}
'port': proxy_port, 'admin-port': proxy_port + 1000,
'user': 'external'} -%}
# Request mariadb instances
# Request mariadb instances
{% set database_list = parameter_dict.get('database-list', [{'name': 'repdb', 'user': 'user', 'password': 'insecure'}]) -%}
{% set database_list = parameter_dict.get('database-list', [{'name': 'repdb', 'user': 'user', 'password': 'insecure'}]) -%}
...
@@ -74,7 +75,8 @@ config-monitor-passwd = ${publish-early:monitor-password}
...
@@ -74,7 +75,8 @@ config-monitor-passwd = ${publish-early:monitor-password}
config-root-password = ${publish-early:db-root-password}
config-root-password = ${publish-early:db-root-password}
config-repman-user = ${repman-parameter:username}
config-repman-user = ${repman-parameter:username}
config-heartbeat-user = ${repman-parameter:heartbeat-user}
config-heartbeat-user = ${repman-parameter:heartbeat-user}
#config-repman-passwd = ${repman-parameter:password}
config-heartbeat-password = ${publish-early:db-root-password}
config-repman-passwd = ${repman-parameter:password}
config-repman-url = ${nginx-parameter:backend-url}
config-repman-url = ${nginx-parameter:backend-url}
config-repman-secure-url = ${nginx-parameter:backend-ssl-url}
config-repman-secure-url = ${nginx-parameter:backend-ssl-url}
config-cluster = {{ name }}
config-cluster = {{ name }}
...
@@ -116,9 +118,14 @@ config-use-ipv6 = True
...
@@ -116,9 +118,14 @@ config-use-ipv6 = True
config-monitor-passwd = ${publish-early:monitor-password}
config-monitor-passwd = ${publish-early:monitor-password}
config-repman-url = ${nginx-parameter:backend-url}
config-repman-url = ${nginx-parameter:backend-url}
config-repman-secure-url = ${nginx-parameter:backend-ssl-url}
config-repman-secure-url = ${nginx-parameter:backend-ssl-url}
config-repman-passwd = ${repman-parameter:password}
config-heartbeat-user = ${repman-parameter:heartbeat-user}
config-heartbeat-password = ${publish-early:db-root-password}
config-proxy-name = {{ name }}
config-proxy-name = {{ name }}
config-
type = {{ parameter_dict.get('proxy-type', "proxysql")
}}
config-
proxy-type = {{ proxy_type
}}
config-proxy-port = {{ proxy_config_dict['port'] }}
config-proxy-port = {{ proxy_config_dict['port'] }}
config-proxy-user = {{ proxy_config_dict['user'] }}
config-proxy-passwd = ${repman-parameter:password}
config-admin-proxy-port = {{ proxy_config_dict['admin-port'] }}
config-admin-proxy-port = {{ proxy_config_dict['admin-port'] }}
config-bootstraped = false
config-bootstraped = false
return =
return =
...
@@ -154,15 +161,25 @@ db-prefered-master = {{ prefered_master }}
...
@@ -154,15 +161,25 @@ db-prefered-master = {{ prefered_master }}
# First database is the prefered master
# First database is the prefered master
db-prefered-master = {{ mariadb_server_list[0] }}
db-prefered-master = {{ mariadb_server_list[0] }}
{% endif -%}
{% endif -%}
proxy-type = {{ proxy_type }}
{% if proxy_type == "proxysql" -%}
{% if proxy_type == "proxysql" -%}
proxysql-servers = {{ proxy_config_dict['ip'] | join(',') }}
proxysql-servers = {{ proxy_config_dict['ip'] | join(',') }}
proxysql-servers-ipv6 = [{{ proxy_config_dict['ipv6'] | join('],[') }}]
proxysql-user = {{ proxy_config_dict['user'] }}
proxysql-partition = {{ proxy_config_dict['partition'] | join(',') }}
proxysql-user = {{ parameter_dict.get("proxysql-user", "external") }}
{% endif -%}
password = ${repman-parameter:password}
proxy-port = {{ proxy_config_dict['port'] }}
proxy-port = {{ proxy_config_dict['port'] }}
proxy-admin-port = {{ proxy_config_dict['admin-port'] }}
proxy-admin-port = {{ proxy_config_dict['admin-port'] }}
{% elif proxy_type == "shardproxy" -%}
{% set server_list = [] -%}
{% for ip in proxy_config_dict['ip'] -%}
{% do server_list.append(ip ~ ':' ~ proxy_config_dict['port']) -%}
{% endfor -%}
shardproxy-copy-grants = {{ parameter_dict.get('shardproxy-copy-grants', True) }}
shardproxy-user = {{ proxy_config_dict['user'] }}
shardproxy-ignore-tables = {{ slapparameter_dict.get("shardproxy-ignore-tables", []) | join(',') }}
shardproxy-servers = {{ server_list | join(',') }}
{% endif -%}
proxy-servers-ipv6 = [{{ proxy_config_dict['ipv6'] | join('],[') }}]
proxy-partition = {{ proxy_config_dict['partition'] | join(',') }}
password = ${repman-parameter:password}
receiver-port-list = {{ receiver_port_list | join(',') }}
receiver-port-list = {{ receiver_port_list | join(',') }}
enabled-tags = {{ slapparameter_dict.get("tag-list", tag_list) | join(',') }}
enabled-tags = {{ slapparameter_dict.get("tag-list", tag_list) | join(',') }}
proxy-tags = {{ parameter_dict.get("proxy-tags", ["pkg", "masterslave", "linux", "noreadwritesplit"]) | join(',') }}
proxy-tags = {{ parameter_dict.get("proxy-tags", ["pkg", "masterslave", "linux", "noreadwritesplit"]) | join(',') }}
...
...
software/repman/instance.cfg.in
View file @
083a62b8
...
@@ -104,6 +104,13 @@ extra-context =
...
@@ -104,6 +104,13 @@ extra-context =
section parameter_dict template-mariadb-parameters
section parameter_dict template-mariadb-parameters
[template-proxydb-parameters]
[template-proxydb-parameters]
mariadb-location = {{ mariadb_location }}
template-mysqld-wrapper = {{ template_mysqld_wrapper }}
template-mysqld-need-start = {{ mysqld_start_template }}
mroonga-mariadb-plugin-dir = {{ mroonga_mariadb_plugin_dir }}
template-mariadb-init-root = {{ template_init_root_sql }}
unixodbc-location = {{ unixodbc_location }}
bin-directory = {{ bin_directory }}
[template-instance-proxydb-cfg]
[template-instance-proxydb-cfg]
<= jinja2-template-base
<= jinja2-template-base
...
...
software/repman/templates/cluster-config.toml.in
View file @
083a62b8
{% macro setbool(value) -%}
{% macro setbool(value) -%}
{% if value in ['true', 'True'] %} true {% else %} false {% endif -%}
{% if value in ['true', 'True'] %} true {% else %} false {% endif -%}
{% endmacro -%}
{% endmacro -%}
{% if parameter_dict['proxy-type'] == 'proxysql' -%}
{% set proxysql = True -%}
{% set shardproxy = False -%}
{% elif parameter_dict['proxy-type'] == 'shardproxy' -%}
{% set proxysql = False -%}
{% set shardproxy = True -%}
{% endif -%}
[{{ parameter_dict['name'] }}]
[{{ parameter_dict['name'] }}]
title = "{{ parameter_dict['name'] }}"
title = "{{ parameter_dict['name'] }}"
monitoring-save-config = true
monitoring-save-config = true
...
@@ -11,16 +18,30 @@ replication-credential = "{{ parameter_dict['db-user'] }}:{{ parameter_dict['db-
...
@@ -11,16 +18,30 @@ replication-credential = "{{ parameter_dict['db-user'] }}:{{ parameter_dict['db-
monitoring-write-heartbeat-credential="{{ parameter_dict['heartbeat-user'] }}:{{ parameter_dict['heartbeat-password'] }}"
monitoring-write-heartbeat-credential="{{ parameter_dict['heartbeat-user'] }}:{{ parameter_dict['heartbeat-password'] }}"
db-servers-connect-timeout = 1
db-servers-connect-timeout = 1
slapos-db-partitions = "{{ parameter_dict['partition-list'] }}"
slapos-db-partitions = "{{ parameter_dict['partition-list'] }}"
slapos-proxysql-partitions = "{{ parameter_dict['proxysql-partition'] }}"
{% if proxysql %}
proxysql = true
proxysql = true
slapos-proxysql-partitions = "{{ parameter_dict['proxy-partition'] }}"
proxysql-port = {{ parameter_dict['proxy-port'] }}
proxysql-port = {{ parameter_dict['proxy-port'] }}
proxysql-servers = "{{ parameter_dict['proxysql-servers'] }}"
proxysql-servers = "{{ parameter_dict['proxysql-servers'] }}"
proxysql-servers-ipv6 = "{{ parameter_dict['proxy
sql
-servers-ipv6'] }}"
proxysql-servers-ipv6 = "{{ parameter_dict['proxy-servers-ipv6'] }}"
proxysql-user = "{{ parameter_dict['proxysql-user'] }}"
proxysql-user = "{{ parameter_dict['proxysql-user'] }}"
proxysql-bootstrap = true
proxysql-bootstrap = true
proxysql-admin-port = {{ parameter_dict['proxy-admin-port'] }}
proxysql-admin-port = {{ parameter_dict['proxy-admin-port'] }}
proxysql-password = "{{ parameter_dict['password'] }}"
proxysql-password = "{{ parameter_dict['password'] }}"
{% elif shardproxy -%}
shardproxy = true
slapos-shardproxy-partitions = "{{ parameter_dict['proxy-partition'] }}"
shardproxy-servers-ipv6 = "{{ parameter_dict['proxy-servers-ipv6'] }}"
shardproxy-copy-grants = {{ setbool(parameter_dict['shardproxy-copy-grants']) }}
shardproxy-credential = "{{ parameter_dict['shardproxy-user'] }}:{{ parameter_dict['password'] }}"
{% if parameter_dict['shardproxy-ignore-tables']-%}
shardproxy-ignore-tables = {{ parameter_dict['shardproxy-ignore-tables'] }}
{% endif -%}
shardproxy-load-system = false
shardproxy-servers = "{{ parameter_dict['shardproxy-servers'] }}"
shardproxy-universal-tables = "replication_manager_schema.bench"
{% endif %}
prov-db-tags = "{{ parameter_dict['enabled-tags'] }}"
prov-db-tags = "{{ parameter_dict['enabled-tags'] }}"
prov-proxy-tags = "{{ parameter_dict['proxy-tags'] }}"
prov-proxy-tags = "{{ parameter_dict['proxy-tags'] }}"
...
@@ -66,5 +87,3 @@ switchover-slave-wait-catch = {{ setbool(parameter_dict['switchover-slave-wait-c
...
@@ -66,5 +87,3 @@ switchover-slave-wait-catch = {{ setbool(parameter_dict['switchover-slave-wait-c
switchover-wait-kill = {{ parameter_dict['switchover-wait-kill'] }}
switchover-wait-kill = {{ parameter_dict['switchover-wait-kill'] }}
switchover-wait-trx = {{ parameter_dict['switchover-wait-trx'] }}
switchover-wait-trx = {{ parameter_dict['switchover-wait-trx'] }}
switchover-wait-write-query = {{ parameter_dict['switchover-wait-write-query'] }}
switchover-wait-write-query = {{ parameter_dict['switchover-wait-write-query'] }}
software/repman/templates/mariadb_init_root.sql.in
View file @
083a62b8
...
@@ -2,17 +2,21 @@
...
@@ -2,17 +2,21 @@
-- or products like mysql-fabric won't work
-- or products like mysql-fabric won't work
SET @@SESSION.SQL_LOG_BIN=0;
SET @@SESSION.SQL_LOG_BIN=0;
{% if parameter_dict.get("root-user") -%}
CREATE USER '{{ parameter_dict["root-user"] }}'@'localhost' IDENTIFIED BY '{{ parameter_dict["password"] }}' ;
CREATE USER '{{ parameter_dict["root-user"] }}'@'localhost' IDENTIFIED BY '{{ parameter_dict["password"] }}' ;
GRANT ALL ON *.* TO '{{ parameter_dict["root-user"] }}'@'localhost' WITH GRANT OPTION ;
GRANT ALL ON *.* TO '{{ parameter_dict["root-user"] }}'@'localhost' WITH GRANT OPTION ;
CREATE USER '{{ parameter_dict["root-user"] }}'@'%' IDENTIFIED BY '{{ parameter_dict["password"] }}' ;
CREATE USER '{{ parameter_dict["root-user"] }}'@'%' IDENTIFIED BY '{{ parameter_dict["password"] }}' ;
GRANT ALL ON *.* TO '{{ parameter_dict["root-user"] }}'@'%' WITH GRANT OPTION ;
GRANT ALL ON *.* TO '{{ parameter_dict["root-user"] }}'@'%' WITH GRANT OPTION ;
CREATE USER '{{ parameter_dict["root-user"] }}'@'::' IDENTIFIED BY '{{ parameter_dict["password"] }}' ;
CREATE USER '{{ parameter_dict["root-user"] }}'@'::' IDENTIFIED BY '{{ parameter_dict["password"] }}' ;
GRANT ALL ON *.* TO '{{ parameter_dict["root-user"] }}'@'::' WITH GRANT OPTION ;
GRANT ALL ON *.* TO '{{ parameter_dict["root-user"] }}'@'::' WITH GRANT OPTION ;
{% endif -%}
CREATE USER '{{ parameter_dict["heartbeat-user"] }}'@'localhost' IDENTIFIED BY '{{ parameter_dict["password"] }}' ;
{% if parameter_dict.get("heartbeat-user") %}
CREATE USER '{{ parameter_dict["heartbeat-user"] }}'@'localhost' IDENTIFIED BY '{{ parameter_dict["heartbeat-password"] }}' ;
GRANT ALL ON *.* TO '{{ parameter_dict["heartbeat-user"] }}'@'localhost' WITH GRANT OPTION ;
GRANT ALL ON *.* TO '{{ parameter_dict["heartbeat-user"] }}'@'localhost' WITH GRANT OPTION ;
CREATE USER '{{ parameter_dict["heartbeat-user"] }}'@'%' IDENTIFIED BY '{{ parameter_dict["password"] }}' ;
CREATE USER '{{ parameter_dict["heartbeat-user"] }}'@'%' IDENTIFIED BY '{{ parameter_dict["
heartbeat-
password"] }}' ;
GRANT ALL ON *.* TO '{{ parameter_dict["heartbeat-user"] }}'@'%' WITH GRANT OPTION ;
GRANT ALL ON *.* TO '{{ parameter_dict["heartbeat-user"] }}'@'%' WITH GRANT OPTION ;
{% endif -%}
DROP DATABASE IF EXISTS test ;
DROP DATABASE IF EXISTS test ;
FLUSH PRIVILEGES ;
FLUSH PRIVILEGES ;
\ No newline at end of file
software/repman/templates/proxy-need-start-stop.sh.in
View file @
083a62b8
...
@@ -8,15 +8,16 @@ check_start_cluster () {
...
@@ -8,15 +8,16 @@ check_start_cluster () {
NAME=$1
NAME=$1
HOST=$2
HOST=$2
PORT=$3
PORT=$3
PROXY=$4
CODE=$(curl -o /dev/null -w "%{http_code}" {{ repman_url }}/api/clusters/$NAME/servers/$HOST/$PORT/need-start)
CODE=$(curl -o /dev/null -w "%{http_code}" {{ repman_url }}/api/clusters/$NAME/servers/$HOST/$PORT/need-start)
if [ $CODE -eq 200 ]; then
if [ $CODE -eq 200 ]; then
echo "$CODE: Starting
proxysql
$HOST:$PORT..."
echo "$CODE: Starting
$PROXY
$HOST:$PORT..."
{{ proxy
sql
_controller }} start proxysql-$NAME
{{ proxy_controller }} start proxysql-$NAME
sleep 1
sleep 1
# check again if the service is still up...
# check again if the service is still up...
{{ proxy
sql_controller }} status proxysql
-$NAME
{{ proxy
_controller }} status $PROXY
-$NAME
fi
fi
}
}
...
@@ -24,21 +25,22 @@ check_stop_cluster () {
...
@@ -24,21 +25,22 @@ check_stop_cluster () {
NAME=$1
NAME=$1
HOST=$2
HOST=$2
PORT=$3
PORT=$3
PROXY=$4
CODE=$(curl -o /dev/null -w "%{http_code}" {{ repman_url }}/api/clusters/$NAME/servers/$HOST/$PORT/need-stop)
CODE=$(curl -o /dev/null -w "%{http_code}" {{ repman_url }}/api/clusters/$NAME/servers/$HOST/$PORT/need-stop)
if [ $CODE -eq 200 ]; then
if [ $CODE -eq 200 ]; then
echo "$CODE: updating
proxysql
config $HOST:$PORT..."
echo "$CODE: updating
$PROXY
config $HOST:$PORT..."
{{ get_proxy_config }} $NAME $HOST $PORT
{{ get_proxy_config }} $NAME $HOST $PORT
echo "$CODE: Stoping
proxysql
$HOST:$PORT..."
echo "$CODE: Stoping
$PROXY
$HOST:$PORT..."
{{ proxy
sql_controller }} stop proxysql
-$NAME
{{ proxy
_controller }} stop $PROXY
-$NAME
sleep 1
sleep 1
# check again if the service is stopped...
# check again if the service is stopped...
{{ proxy
sql_controller }} status proxysql
-$NAME
{{ proxy
_controller }} status $PROXY
-$NAME
fi
fi
}
}
{% for cluster_dict in cluster_list -%}
{% for cluster_dict in cluster_list -%}
check_start_cluster {{ cluster_dict['name'] }} {{ cluster_dict['host'] }} {{ cluster_dict['port'] }}
check_start_cluster {{ cluster_dict['name'] }} {{ cluster_dict['host'] }} {{ cluster_dict['port'] }}
{{ proxy }}
check_stop_cluster {{ cluster_dict['name'] }} {{ cluster_dict['host'] }} {{ cluster_dict['port'] }}
check_stop_cluster {{ cluster_dict['name'] }} {{ cluster_dict['host'] }} {{ cluster_dict['port'] }}
{{ proxy }}
{% endfor -%}
{% endfor -%}
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment