Kafka Engine, Materialized view and storage

427 views
Skip to first unread message

Loïc Girault

unread,
Dec 1, 2017, 4:12:58 AM12/1/17
to ClickHouse
In Clickhouse's doc for Kafka Engine the following example is given :
CREATE TABLE queue (
  timestamp UInt64,
  level String,
  message String
) ENGINE = Kafka('localhost:9092', 'topic', 'group1', 'JSONEachRow');

CREATE TABLE daily (
  day Date,
  level String,
  total UInt64
) ENGINE = SummingMergeTree(day, (day, level), 8192);

CREATE MATERIALIZED VIEW consumer TO daily
  AS SELECT toDate(toDateTime(timestamp)) AS day, level, count() as total
  FROM queue GROUP BY day, level;

SELECT level, sum(total) FROM daily GROUP BY level;


In addition of the consumer table (`queue`) there is a table and a materialized view. In the  query language section , it is said that "Materialized views store data transformed by the corresponding SELECT query".
So my question is, is the data stored twice ? My intuitition is that obviously not but I'd like to have confirmation.

Mikhail Filimonov

unread,
Dec 4, 2017, 12:36:22 PM12/4/17
to ClickHouse
That materialized view consumer uses daily table as a storage. That syntax is not in documentation yet.

Please check: https://github.com/yandex/ClickHouse/blob/master/CHANGELOG.md
Added the CREATE MATERIALIZED VIEW x TO y query (specifies an existing table for storing the data of a materialized view).

Proabably it was introduced to make it possible to insert materialized view result in some table with aggregation. Like SummingMergeTree in that example. 
Reply all
Reply to author
Forward
0 new messages