add prometheus to https://github.com/influxdata/influxdb-comparisons

27 views
Skip to first unread message

hello

unread,
Dec 23, 2019, 11:15:09 AM12/23/19
to Prometheus Users
Hi guys,
I want to add prometheus support to influxdb-comparisons, but I was stuck with data ingest.
As we all know, prometheus is a pull-based system, so there is no HTTP API like `/api/put` in opentsdb, what I try is to write samples by tsdb's API, full code can be found here:


but the samples appear only in wal directory, not regular blocks directory


$du -h prom
37G     prom/wal/checkpoint.001097.tmp
27G     prom/wal/checkpoint.000229
721G    prom/wal
236K    prom/01DWSN9TYMZ6GGNQ93925Z2MQT/chunks
1.5M    prom/01DWSN9TYMZ6GGNQ93925Z2MQT
721G    prom


input metrics are 10100 series, each series  last for 1 hour and have 3600/10(interval)=360 points, generated with following commands

./bin/bulk_data_gen -format opentsdb \
-scale-var 100 \
-timestamp-start "2019-12-02T00:00:00Z" \
-timestamp-end "2019-12-02T01:00:00Z" \
-seed 20191221   > $input &



Ingest is invoked with this command 
./bin/bulk_load_prometheus -numWrite 10 -input $input -dbPath $prom_data

I tried different values of MinBlockDuration/MaxBlockDuration, but in vain, data still only exists in wal. 
The problem with this issue is wal can be very large, and feel like walk in the wrong direction. 
Anyone here have ideas where the problem is? How to ingest samples to prometheus TSDB in batch?


Thanks in advance.

Jiacai Liu

unread,
Dec 23, 2019, 9:52:33 PM12/23/19
to Prometheus Users
FYI, I was using prometheus v2.12.0
Reply all
Reply to author
Forward
0 new messages