RavenDB 4.1 - Periodic backup doesn't back up

[Andrej Krivulčík]

I have the following backup configuration set up for three databases (extract from database record document for one of the databases):

    "PeriodicBackups": [
        {
            "TaskId": 231,
            "Disabled": false,
            "Name": "data-full-backup",
            "MentorNode": "A",
            "BackupType": "Backup",
            "FullBackupFrequency": "5 1 * * Sat",
            "IncrementalBackupFrequency": "10 1,13 * * *",
            "LocalSettings": {
                "FolderPath": "F:\\data-full-backup",
                "Disabled": false
            },
            "S3Settings": {
                "Disabled": true
            },
            "GlacierSettings": {
                "Disabled": true
            },
            "AzureSettings": {
                "Disabled": true
            },
            "FtpSettings": {
                "Disabled": true
            }
        }
    ],

However, the backup didn't run in the past 2 weeks (screenshot from node A):

When I trigger the backup manually (clicking on "Backup now"), the backup works correctly. I ran it on one of the three databases and it is completed.

How could I investigate what caused this outage and possibly prevent it from happening in the future? I still have two databases in this state for potential investigation. One of them is very small (only 24 documents), one is larger (32M documents).

Also, showing some kind of indication of backup not having run in the past X days on the dashboard would be useful. In this case, this information is shown only on Databases screen which I don't use often. I noticed this only by checking the disk on the server and noticing no recent backup folders (which is also something I don't do often).

This is on 4.2.0-nightly-20190213-0602.

[eg...@ravendb.net]

Hello,
Thanks for your feedback.
Do you have any notifications in studio about backup task ? 
Can you send us the logs logged when the task meant to be running ?  Do you have any exceptions on that moment ?
Did you try to recreate the task? 

[Andrej Krivulčík]

No relevant notifications in studio, except notifications about backup successfully done 18 days ago.

Also, I needed to restart the RavenDB server (shortly after posting this) and all the backups ran successfully. So unfortunately, I can't provide environment for diagnostics anymore.

The server logs are available only since 2019-03-31, and the only messages in the logs are these two types:

Operations, Server, Raven.Server.RavenServer, Renew check: still have time left to renew the server certificate with thumbprint

and

Operations, A, Raven.Server.Rachis.RachisConsensus, Took way too much time (00:00:00.1769853) to change the state to Candidate in term 506769. (Election timeout:00:00:00.3000000)

Also, today, there were low memory events (cause of the need to restart the server). No other entries in logs.

Given that the backup was triggered after server restart, I assume that there is no point in trying to recreate the task, right?

[Oren Eini (Ayende Rahien)]

Can you see if this has run since then? It has been enough time for it to need to run, so that should give some indication

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
To post to this group, send email to rav...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ravendb/5d94806f-cf90-4e6a-bf34-35f61986223e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Oren Eini CEO   /   Hibernating Rhinos LTD Mobile:  +972-52-548-6969 Sales:  sa...@ravendb.net Skype:  ayenderahien Support:  sup...@ravendb.net

[Andrej Krivulčík]

Yes, the full backup ran successfully during the weekend and the incremental backups run as scheduled so the server seems to be in good condition now.

On Sunday, April 7, 2019 at 11:24:45 AM UTC+2, Oren Eini wrote:

Can you see if this has run since then? It has been enough time for it to need to run, so that should give some indication

On Thu, Apr 4, 2019 at 4:48 PM Andrej Krivulčík <kriv...@gmail.com> wrote:

No relevant notifications in studio, except notifications about backup successfully done 18 days ago.

Also, I needed to restart the RavenDB server (shortly after posting this) and all the backups ran successfully. So unfortunately, I can't provide environment for diagnostics anymore.

The server logs are available only since 2019-03-31, and the only messages in the logs are these two types:

Operations, Server, Raven.Server.RavenServer, Renew check: still have time left to renew the server certificate with thumbprint

and

Operations, A, Raven.Server.Rachis.RachisConsensus, Took way too much time (00:00:00.1769853) to change the state to Candidate in term 506769. (Election timeout:00:00:00.3000000)

Also, today, there were low memory events (cause of the need to restart the server). No other entries in logs.

Given that the backup was triggered after server restart, I assume that there is no point in trying to recreate the task, right?

On Thursday, April 4, 2019 at 1:12:00 PM UTC+2, eg...@ravendb.net wrote:

Hello,
Thanks for your feedback.
Do you have any notifications in studio about backup task ? 
Can you send us the logs logged when the task meant to be running ?  Do you have any exceptions on that moment ?
Did you try to recreate the task? 

 

--