How to configure Prometheus for a small server?

[Abraham Varricatt]

Hello,

How exactly do you configure Prometheus to keep track of a small digital ocean droplet? (512MB ram, 1 processor, 20GB HDD, ubuntu 14.04). According to the documentation ( https://prometheus.io/docs/operating/configuration/ ) Prometheus is configured via command-line flags and a configuration file.

What would be an appropriate command-line-arg/config for my tiny server? 

The goal, is (using node_exporter) to monitor CPU utilization, network (Tx+Rx) utilization, memory (ram) utilization and disk space. I *think* I've figured out how to install prometheus, node_exporter and grafana to show me graphs on a dashboard, but I'm not sure what config settings to set. The biggest worry is performance - the hdd should not end up getting filled with instrumentation data and it's perfectly fine to only keep track of the previous day (or even an hour's) worth of activity if it would keep the system stable. 

Yours sincerely,

Abraham V.

[Brian Brazil]

https://github.com/RobustPerception/demo_prometheus_ansible is sized to fit on a 1GB droplet, you should be able to get it smaller. You are likely going to have to dedicate most of the droplet to Prometheus as it's so small. so I'd suggest a separate dedicated Prometheus machine to monitor everything instead.

--

Brian Brazil

www.robustperception.io

[Robert Collier]

I have no problem running with mostly default configuration on a 256Mb raspberry pi, monitoring just a few devices (Building prometheus does take ~1h on said machine though).

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND           

10492 promethe  20   0  849m  38m 6788 S   2.3 16.7  47:34.15 prometheus        

25808 promethe  20   0  868m 8824 2624 S   0.0  3.7 140:53.71 node_exporter

$ sudo du -csh /var/lib/prometheus/

111M    /var/lib/prometheus/

111M    total

If you are only going to monitor the one droplet, then you should have no trouble.

--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Robert Collier
r...@ossifrage.net

[Brian Brazil]

On 27 June 2016 at 15:10, Robert Collier <r...@ossifrage.net> wrote:

I have no problem running with mostly default configuration on a 256Mb raspberry pi, monitoring just a few devices (Building prometheus does take ~1h on said machine though).

The default Prometheus configuration is intended for a 4GB machine. Over time it will eat up at least 1GB of RAM, so some tuning will be required.

Brian 

--

Brian Brazil

www.robustperception.io

[Julius Volz]

I guess you want to have the Prometheus server on the same host as the one you want to monitor?

On that tiny 512MB RAM droplet, try setting -storage.local.memory-chunks=100000 (or even less if it still uses too much RAM) and -storage.local.max-chunks-to-persist to around 50% of that or a bit more. If it's only going to monitor a single node_exporter, the data will be tiny actually, and you could even decrease the memory chunks to something like 10000. See also my Prometheus-on-Docker-on-DigitalOcean article which covers exactly the 512MB RAM use case: https://www.digitalocean.com/community/tutorials/how-to-install-prometheus-using-docker-on-ubuntu-14-04

If you only want to keep one day of data, set -storage.local.retention=24h.

Try it with those settings at first, and I don't think you'll run into any issues if one node_exporter is all you're monitoring.

[Abraham Varricatt]

Thank you everyone for the suggestions. The ansible scripts especially. Here's what I've decided to use,

- name: start prometheus background process
  become: yes
  become_user: "{{ prometheus_user }}"
  shell: >
    nohup {{ prometheus_install_path }}/prometheus
    -config.file={{ prometheus_config_path }}/prometheus.yml
    -storage.local.path={{ prometheus_db_path }}
    -storage.local.memory-chunks=10000
    -storage.local.max-chunks-to-persist=6000
    -storage.local.retention=1080h
    -web.console.templates={{ prometheus_install_path }}/consoles
    -web.console.libraries={{ prometheus_install_path }}/console_libraries
    > {{ prometheus_log_path }}/prometheus.log 2>&1 &

It's an ansible task, but I think it indicates what parameters I'm passing to prometheus. The retention I've set for 45 days ( = 1080 hours). Will reduce that if I encounter performance issues. 

-Abraham V.