Hi guys,
As
we all know, prometheus is a pull-based system, so there is no HTTP API
like `/api/put` in opentsdb, what I try is to write samples by tsdb's
API, full code can be found here:
but the samples appear only in wal directory, not regular blocks directory
$du -h prom
37G prom/wal/checkpoint.001097.tmp
27G prom/wal/checkpoint.000229
721G prom/wal
236K prom/01DWSN9TYMZ6GGNQ93925Z2MQT/chunks
1.5M prom/01DWSN9TYMZ6GGNQ93925Z2MQT
721G prom
input metrics are 10100 series, each series last for 1 hour and have 3600/10(interval)=360 points, generated with following commands
./bin/bulk_data_gen -format opentsdb \
-scale-var 100 \
-timestamp-start "2019-12-02T00:00:00Z" \
-timestamp-end "2019-12-02T01:00:00Z" \
-seed 20191221 > $input &
Ingest is invoked with this command
./bin/bulk_load_prometheus -numWrite 10 -input $input -dbPath $prom_data
I tried different values of MinBlockDuration/MaxBlockDuration, but in vain, data still only exists in wal.
The problem with this issue is wal can be very large, and feel like walk in the wrong direction.
Anyone here have ideas where the problem is? How to ingest samples to prometheus TSDB in batch?
Thanks in advance.