How to change the config.xml file in a ClickHouse Docker image ?

[alexandre...@gmail.com]

Hello,
I want to use Cluster for my ClickHouse database. I suceed in creating two shards on a single PC and I want now to deploy shards on docker.
For this, i create an image of ClickHouse in Docker with the following code :

FROM ubuntu:16.04

ARG repository="deb https://repo.yandex.ru/clickhouse/xenial/ dists/stable/main/binary-amd64/"
ARG version=\*

RUN apt-get update && \
apt-get install -y apt-transport-https tzdata && \
mkdir -p /etc/apt/sources.list.d && \
echo $repository | tee /etc/apt/sources.list.d/clickhouse.list && \
apt-get update && \
apt-get install --allow-unauthenticated -y clickhouse-server-common=$version && \
rm -rf /var/lib/apt/lists/* /var/cache/debconf && \
apt-get clean

RUN sed -i 's,<listen_host>127.0.0.1</listen_host>,<listen_host>::</listen_host>,' /etc/clickhouse-server/config.xml

EXPOSE 9000 8123 9009
VOLUME /var/lib/clickhouse

ENV CLICKHOUSE_CONFIG /etc/clickhouse-server/config.xml

ENTRYPOINT exec /usr/bin/clickhouse-server --config=${CLICKHOUSE_CONFIG}

From this image of ClickHouse, I create two instance of ClickHouse on Docker : clickhouse and clickhouse1.

From the description of ClickHouse website, I declared the two shards (clickhouse and clickhouse1) of ClickHouse like this :

<?xml version="1.0"?>
<yandex>

<!-- Configuration of clusters that could be used in Distributed tables. -->
<remote_servers>
<clickHouseCluster>
<shard>
<replica>
<host>clickhouse</host>
<port>9000</port>
</replica>
</shard>
<shard>
<replica>
<host>clickhouse1</host>
<port>9000</port>
</replica>
</shard>
</clickHouseCluster>
</remote_servers>

<zookeeper-servers>
<node>
<host>kafka</host>
<port>2181</port>
</node>
</zookeeper-servers>
</yandex>

In my docker-compose.yml file, I declare the ClickHouse instance like this :

clickhouse:
image: toto
restart: unless-stopped
ports:
- "8123"
- "9000"
volumes:
- ./clickhouse-server/config.xml:/etc/clickhouse-server/config.xml
volumes_from:
- clickhouse_data

But I have a problem with the volume ./clickhouse-server/config.xml. In fact, ClickHouse uses this file instead of /etc/clickhouse-server/config.xml file to start ClickHouse. However, when I start clickhouse on his port, I have the error : Code: 210. DB::NetException: Connection reset by peer: while reading from socket.
The unique difference I see concerns the permission of the ./clickhouse-server/config.xml file that is : 1000 1000 whereas the others file have the permission : clickhouse clickhouse.

I do not really know how to solve my problem ....

Thank you for your help

[Vitaliy Lyudvichenko]

Using the following docker-compose line:

- ./clickhouse-server/config.xml:/etc/clickhouse-server/config.xml

you completely replace default config in docker container by your local ./clickhouse-server/config.xml file.

Do you want to put above mentioned cluster config in ./clickhouse-server/config.xml ?

If so, you can put your cluster definition (for example cluster.xml) into /etc/clickhouse-server/conf.d/cluster.xml.

Main config /etc/clickhouse-server/config.xml will be merged with /etc/clickhouse-server/conf.d/cluster.xml on server startup.

You will see resulting config in /etc/clickhouse-server/conf.d/config-preprocessed.xml.

You could read more about configuration files generation here: https://clickhouse.yandex/docs/en/single/#configuration-files 

среда, 7 июня 2017 г., 18:31:36 UTC+3 пользователь alexandre...@gmail.com написал:

[alexandre...@gmail.com]

Thank you for your response, you exactly say what i need.

So I create a /etc/clickhouse-server/conf.d/cluster.xml with my configuration.
And in my docker-compose file, I use the following line :

- ./clickhouse-server/conf.d/cluster.xml:/etc/clickhouse-server/conf.d/config.xml

 And this allows me to use clusters in docker.