Unable to Restore Backup - The archive entry was compressed using an unsupported compression method

[João Bragança]

I was successfully able to take backups of all my databases on RavenDb 4.1.1 to S3 with the following powershell script:

  $environment = "staging"
  $zoneName = "some-zone"
  $productVersion = "1.2.3"
  $backupName = (Get-Date).ToUniversalTime().ToString('o').Replace(':', '-')


  $(Invoke-RestMethod `
    -Method get `
    -Uri http://localhost:60956/databases).Databases | `
    % {
      Invoke-RestMethod `
        -Method post `
        -Uri "http://localhost:60956/databases/$($_.Name)/smuggler/export" `
        -ContentType "application/x-www-form-urlencoded" `
        -Body "" | `
        & 'C:\Program Files\Amazon\AWSCLI\bin\aws.exe' s3 cp `
          - `
          "s3://$backupsBucket/$environment/$zoneName/backups/ravendb/$productVersion/$($_.Name)/$backupName.ravendbdump" `
          --sse AES256
    }

Now I'm trying to restore with the following:

  $backupKey = "$environment/$zoneName/backups/ravendb/$productVersion/some-database/$backupName.ravendbdump"

  $databaseRecord = @{
    DatabaseName=$databaseName;
  } | ConvertTo-Json


  Invoke-RestMethod `
    -Method Put `
    -Uri http://localhost:60956/admin/databases?name=$databaseName `
    -Body $databaseRecord `
    -ContentType "application/json"


  & 'C:\Program Files\Amazon\AWSCLI\bin\aws.exe' s3 cp "s3://$backupsBucket/$backupKey" - | `
    curl.exe -X POST http://localhost:60956/databases/$databaseName/smuggler/import `
      --form "file=@-"

And it fails with:

{"Url":"/databases/some-database/smuggler/import","Type":"System.IO.InvalidDataException","Message":"The archive entry was compressed using an unsupported compression method.","Error":"System.IO.InvalidDataException: The archive entry was compressed using an unsupported compression method.\r\n at System.IO.Compression.Inflater.Inflate(FlushCode flushCode)\r\n at System.IO.Compression.Inflater.ReadInflateOutput(Byte* bufPtr, Int32 length, FlushCode flushCode, Int32& bytesRead)\r\n at System.IO.Compression.Inflater.InflateVerified(Byte* bufPtr, Int32 length)\r\n at System.IO.Compression.DeflateStream.ReadCore(Span`1 buffer)\r\n at System.IO

Unfortunately the output is truncated as this is executed via AWS SSM command. it seems like the same thing here: https://groups.google.com/d/msg/ravendb/kYW-VhEL9r8/FZrLLonaAgAJ except I don't have any attachments. The content type in S3 is application/octet-stream if that makes any difference.

[Oren Eini (Ayende Rahien)]

This looks like you are actually piping the file over the console? It is possible that this is causing invalid data to come through?

what happens if you manually download the file and try to restore it? 

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Oren Eini CEO   /   Hibernating Rhinos LTD Mobile:  +972-52-548-6969 Sales:  sa...@ravendb.net Skype:  ayenderahien Support:  sup...@ravendb.net

[João Bragança]

Ok I tried that. Now I see

curl.exe -F 'importOptions={"IncludeExpired":true,"RemoveAnalyzers":false,"OperateOnTypes":"DatabaseRecord,Documents,Conflicts,Indexes,RevisionDocuments,Identities,CompareExchange,Counters,Attachments"}' -F "file=@$output" "http://localhost:60956/databases/some-database/smuggler/import"



{"Url":"/databases/some-database/smuggler/import","Type":"System.IO.InvalidDataException","Message":"Invalid token found, expected: Infinity at  (1,4) around: {IncludeExpired:true,RemoveAnalyzers:false,OperateOnTy
pes:DatabaseRecord,Documents,Conflicts,Indexes,RevisionDocuments,Identities,CompareExchange,Counters,Attachments}","Error":"System.IO.InvalidDataException: Invalid token found, expected: Infinity at  (1,4) aroun
d: {IncludeExpired:true,RemoveAnalyzers:false,OperateOnTypes:DatabaseRecord,Documents,Conflicts,Indexes,RevisionDocuments,Identities,CompareExchange,Counters,Attachments}\r\n   at Sparrow.Json.Parsing.UnmanagedJ
sonParser.ThrowException(String message, Exception inner) in C:\\Builds\\RavenDB-Stable-4.1\\src\\Sparrow\\Json\\Parsing\\UnmanagedJsonParser.cs:line 922\r\n   at Sparrow.Json.Parsing.UnmanagedJsonParser.EnsureR
estOfToken(UInt32& pos) in C:\\Builds\\RavenDB-Stable-4.1\\src\\Sparrow\\Json\\Parsing\\UnmanagedJsonParser.cs:line 715\r\n   at Sparrow.Json.Parsing.UnmanagedJsonParser.ReadToken(UInt32& pos, Boolean& couldRead
, Byte[] tokenBuffer, String tokenString, JsonParserTokenContinuation jsonParserTokenContinuation) in C:\\Builds\\RavenDB-Stable-4.1\\src\\Sparrow\\Json\\Parsing\\UnmanagedJsonParser.cs:line 412\r\n   at Sparrow
.Json.Parsing.UnmanagedJsonParser.ReadUnlikely(Byte b, UInt32& pos, Boolean& couldRead) in C:\\Builds\\RavenDB-Stable-4.1\\src\\Sparrow\\Json\\Parsing\\UnmanagedJsonParser.cs:line 340\r\n   at Sparrow.Json.Parsi
ng.UnmanagedJsonParser.Read() in C:\\Builds\\RavenDB-Stable-4.1\\src\\Sparrow\\Json\\Parsing\\UnmanagedJsonParser.cs:line 208\r\n   at Sparrow.Json.BlittableJsonDocumentBuilder.ReadInternal[TWriteStrategy]() in
C:\\Builds\\RavenDB-Stable-4.1\\src\\Sparrow\\Json\\BlittableJsonDocumentBuilder.cs:line 176\r\n   at Sparrow.Json.JsonOperationContext.ParseToMemoryAsync(Stream stream, String documentId, UsageMode mode, Manage
dPinnedBuffer bytes, Nullable`1 token, Int32 maxSize) in C:\\Builds\\RavenDB-Stable-4.1\\src\\Sparrow\\Json\\JsonOperationContext.cs:line 872\r\n   at Raven.Server.Smuggler.Documents.Handlers.SmugglerHandler.<>c
__DisplayClass15_2.<<PostImportAsync>b__1>d.MoveNext() in C:\\Builds\\RavenDB-Stable-4.1\\src\\Raven.Server\\Smuggler\\Documents\\Handlers\\SmugglerHandler.cs:line 661\r\n--- End of stack trace from previous loc
ation where exception was thrown ---\r\n   at Raven.Server.Smuggler.Documents.Handlers.SmugglerHandler.PostImportAsync() in C:\\Builds\\RavenDB-Stable-4.1\\src\\Raven.Server\\Smuggler\\Documents\\Handlers\\Smugg
lerHandler.cs:line 622\r\n   at Raven.Server.Routing.RequestRouter.HandlePath(HttpContext context, String method, String path) in C:\\Builds\\RavenDB-Stable-4.1\\src\\Raven.Server\\Routing\\RequestRouter.cs:line
 124\r\n   at Raven.Server.RavenServerStartup.RequestHandler(HttpContext context) in C:\\Builds\\RavenDB-Stable-4.1\\src\\Raven.Server\\RavenServerStartup.cs:line 162"}* Closing connection 0

[João Bragança]

Well it looks like something weird from the shell maybe:

Write-Host $importOptions

{

    "IncludeExpired":  true,
    "RemoveAnalyzers":  false,
    "OperateOnTypes":  "DatabaseRecord,Documents,Conflicts,Indexes,RevisionDocuments,Identities,CompareExchange,Counters"
}

curl.exe -X POST http://localhost:60956/databases/$databaseName/smuggler/import `
    --form "importOptions=$importOptions" `
    --form "file=@$output"

Invalid token found, expected: Infinity at  (2,7) around: {

    IncludeExpired:  true,
    RemoveAnalyzers:  false,
    OperateOnTypes:  DatabaseRecord,Documents,Conflicts,Indexes,RevisionDocuments,Identities,CompareExchange,Counters}","Error":"System.IO.InvalidDataException: Invalid token found, expected: Infinity at  (2,7) around: {

    IncludeExpired:  true,
    RemoveAnalyzers:  false,
    OperateOnTypes:  DatabaseRecord,Documents,Conflicts,Indexes,RevisionDocuments,Identities,CompareExchange,Counters
}

   at Sparrow.Json.Parsing.UnmanagedJsonParser.ThrowException(String message, Exception inner) in C:\\Builds\\RavenDB-Stable-4.1\\src\\Sparrow\\Json\\Parsing\\UnmanagedJsonParser.cs:line 922
   at Sparrow.Json.Parsing.UnmanagedJsonParser.EnsureRestOfToken(UInt32& pos) in C:\\Builds\\RavenDB-Stable-4.1\\src\\Sparrow\\Json\\Parsing\\UnmanagedJsonParser.cs:line 715
   at Sparrow.Json.Parsing.UnmanagedJsonParser.ReadToken(UInt32& pos, Boolean& couldRead, Byte[] tokenBuffer, String tokenString, JsonParserTokenContinuation jsonParserTokenContinuation) in  C:\\Builds\\RavenDB-Stable-4.1\\src\\Sparrow\\Json\\Parsing\\UnmanagedJsonParser.cs:line 412
   at Sparrow.Json.Parsing.UnmanagedJsonParser.ReadUnlikely(Byte b, UInt32& pos, Boolean& couldRead) in  C:\\Builds\\RavenDB-Stable-4.1\\src\\Sparrow\\Json\\Parsing\\UnmanagedJsonParser.cs:line 340
   at Sparrow.Json.Parsing.UnmanagedJsonParser.Read() in C:\\Builds\\RavenDB-Stable-4.1\\src\\Sparrow\\Json\\Parsing\\UnmanagedJsonParser.cs:line 208

   at Sparrow.Json.BlittableJsonDocumentBuilder.ReadInternal[TWriteStrategy]() in C:\\Builds\\RavenDB-Stable-4.1\\src\\Sparrow\\Json\\BlittableJsonDocumentBuilder.cs:line 176

   at Sparrow.Json.JsonOperationContext.ParseToMemoryAsync(Stream stream, String documentId, UsageMode mode, ManagedPinnedBuffer bytes, Nullable`1 token, Int32 maxSize) inC:\\Builds\\RavenDB-Stable-4.1\\src\\Sparrow\\Json\\JsonOperationContext.cs:line 872

   at Raven.Server.Smuggler.Documents.Handlers.SmugglerHandler.<>c__DisplayClass15_2.<<PostImportAsync>b__1>d.MoveNext() in C:\\Builds\\RavenDB-Stable-4.1\\src\\Raven.Server\\Smuggler\\Documents\\Handlers\\SmugglerHandler.cs:line 661
--- End of stack trace from previous location where exception was thrown ---

   at Raven.Server.Smuggler.Documents.Handlers.SmugglerHandler.PostImportAsync() in C:\\Builds\\RavenDB-Stable-4.1\\src\\Raven.Server\\Smuggler\\Documents\\Handlers\\SmugglerHandler.cs:line 622

   at Raven.Server.Routing.RequestRouter.HandlePath(HttpContext context, String method, String path) in C:\\Builds\\RavenDB-Stable-4.1\\src\\Raven.Server\\Routing\\RequestRouter.cs:line 124

   at Raven.Server.RavenServerStartup.RequestHandler(HttpContext context) in C:\\Builds\\RavenDB-Stable-4.1\\src\\Raven.Server\\RavenServerStartup.cs:line 162"

[Oren Eini (Ayende Rahien)]

Try and see what you are actually sending? I guess you have mixing " in the json and the from there?

[João Bragança]

Hey sorry for the late reply, I figured out what I did wrong.

In some cases, the 'backup' was actually an error document so obviously that couldn't get restored. In other cases, somehow the double quotes around the properties of importOptions were getting removed, even when I escaped them. Powershell is weird. BTW, I tested using just pipes on linux to avoid saving everything to disk first. That works.

Anyway thanks for your help, it works brilliantly!