compaction failed err="reload blocks: head truncate failed: create checkpoint: rename checkpoint directory: rename /prometheus/wal/checkpoint.000001.tmp /prometheus/wal/checkpoint.000001: no such file or directory"

[Daniel HENCHOZ]

Hello,

I'm running prometheus v2.17.2 in a docker container on windows

My .env file looks like :

# PROM_DATADIR=/home/dhenchoz/promdata

PROM_DATADIR=D/DOCKER_PERSISTENT/prometheus_data

PROM_RETENTION=1y

I've set a persitent volume on my windows PC, see docker-compose file :

volumes:

    prometheus_data:

        driver: local

        driver_opts:

            type: 'none'

            o: 'bind'

            device: '${PROM_DATADIR}'

networks:

    back-tier:

        driver: bridge

    front-tier:

        driver: bridge

services:

    prometheus:

        image: prom/prometheus

        container_name: prometheus

        environment:

            - TZ=Europe/Zurich

        volumes:

            - ./prometheus/:/etc/prometheus/

            - prometheus_data:/prometheus

        command:

            - '--config.file=/etc/prometheus/prometheus.yml'

            - '--storage.tsdb.path=/prometheus'

            - '--storage.tsdb.retention.time=${PROM_RETENTION}'

            - '--web.enable-lifecycle'

            - '--web.console.libraries=/usr/share/prometheus/console_libraries'

            - '--web.console.templates=/usr/share/prometheus/consoles'

        ports:

            - 9090

        user: 1000:1000

        networks:

            - back-tier

        restart: always

I can observe the following message in the log :

prometheus | level=info ts=2020-04-29T15:00:11.609Z caller=head.go:662 component=tsdb msg="head GC completed" duration=40.594801ms
prometheus | level=error ts=2020-04-29T15:00:12.430Z caller=db.go:659 component=tsdb msg="compaction failed" err="reload blocks: head truncate failed: create checkpoint: rename checkpoint
directory: rename /prometheus/wal/checkpoint.000001.tmp /prometheus/wal/checkpoint.000001: no such file or directory"

The files on the persistent volume are :

What's wrong ?

Thanks for the help !

Daniel