Error deleting attachments

[ReverentGeek]

I'm trying to write a utility that purges all attachments from a
RavenDB database. I'm running into exceptions on both build 888 and
952. I'm using the /static endpoint to get a list of attachments, and
trying to delete them by key.

On a tenant database using build 952, some of the attachments appeared
to be deleted. However, every time I now request
http://localhost:8080/databases/tenant-name/static, I get the
following exception returned:

System.InvalidOperationException: Nullable object must have a value.
at System.Nullable`1.get_Value()
at
Raven.Storage.Esent.StorageActions.DocumentStorageActions.<GetAttachmentsByReverseUpdateOrder>d__3.MoveNext()
in c:\Builds\RavenDB-Unstable\Raven.Storage.Esent\StorageActions
\DocumentStorageActions.cs:line 110
at System.Linq.Enumerable.<TakeIterator>d__3a`1.MoveNext()
at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
at
Raven.Database.DocumentDatabase.<>c__DisplayClassb6.<GetAttachments>b__b5(IStorageActionsAccessor
actions) in c:\Builds\RavenDB-Unstable\Raven.Database
\DocumentDatabase.cs:line 1085
at Raven.Storage.Esent.TransactionalStorage.ExecuteBatch(Action`1
action) in c:\Builds\RavenDB-Unstable\Raven.Storage.Esent
\TransactionalStorage.cs:line 376
at Raven.Storage.Esent.TransactionalStorage.Batch(Action`1 action)
in c:\Builds\RavenDB-Unstable\Raven.Storage.Esent
\TransactionalStorage.cs:line 337
at Raven.Database.DocumentDatabase.GetAttachments(Int32 start,
Int32 pageSize, Nullable`1 etag) in c:\Builds\RavenDB-Unstable
\Raven.Database\DocumentDatabase.cs:line 1082
at Raven.Database.Server.Responders.Statics.Respond(IHttpContext
context) in c:\Builds\RavenDB-Unstable\Raven.Database\Server\Responders
\Statics.cs:line 26
at Raven.Database.Server.HttpServer.DispatchRequest(IHttpContext
ctx) in c:\Builds\RavenDB-Unstable\Raven.Database\Server
\HttpServer.cs:line 550
at
Raven.Database.Server.HttpServer.HandleActualRequest(IHttpContext ctx)
in c:\Builds\RavenDB-Unstable\Raven.Database\Server\HttpServer.cs:line
316

If I try to retrieve attachment metadata using HeadAttachment(key) on
either the default database or tenant database using build 952, I get
the following exception:

Object reference not set to an instance of an object.
at Raven.Client.Connection.ServerClient.DirectGetAttachment(String
method, String key, String operationUrl) in c:\Builds\RavenDB-Unstable
\Raven.Client.Lightweight\Connection\ServerClient.cs:line 475
at
Raven.Client.Connection.ServerClient.<>c__DisplayClass1c.<HeadAttachment>b__1b(String
operationUrl) in c:\Builds\RavenDB-Unstable\Raven.Client.Lightweight
\Connection\ServerClient.cs:line 457
at Raven.Client.Connection.ServerClient.TryOperation[T](Func`2
operation, String operationUrl, Boolean avoidThrowing, T& result) in c:
\Builds\RavenDB-Unstable\Raven.Client.Lightweight\Connection
\ServerClient.cs:line 223
at Raven.Client.Connection.ServerClient.ExecuteWithReplication[T]
(String method, Func`2 operation) in c:\Builds\RavenDB-Unstable
\Raven.Client.Lightweight\Connection\ServerClient.cs:line 193
at Raven.Client.Connection.ServerClient.HeadAttachment(String key)
in c:\Builds\RavenDB-Unstable\Raven.Client.Lightweight\Connection
\ServerClient.cs:line 457
at RavenPurgeAttachments.Program.PurgeAttachments(String url,
IEnumerable`1 keys) in C:\@Projects\celljournalist\celljournalist
\RavenPurgeAttachments\Program.cs:line 88
at RavenPurgeAttachments.Program.Main(String[] args) in C:\@Projects
\celljournalist\celljournalist\RavenPurgeAttachments\Program.cs:line 70

[Itamar Syn-Hershko]

This happens because we get to an empty attachment. This is wrong and will get fixed, but I want to try and find the root cause for this.

How are you working with attachments? do you issue deletes on a regular basis? can you take a look at the ID of the attachment causing this and tell what operations have been performed on it?

[ReverentGeek]

We do have a way to manually delete attachments that are associated
with a specific document type, but the other attachments, once
created, are not updated or deleted. We are trying to get away from
using attachments in RavenDB, and would like to have a way to purge
these from the database without having to rebuild the database from
scratch.

For the first exception, once the server gets in that state, I can no
longer get a list of attachments. So, I can't get an ID of an
attachment to look at for you.

In the default database, attachments look similar to this:

{
"Size":73296,
"Key":"twm-317db86e-ea14-4750-b630-397fd350188d",
"Metadata":{
"ContentType":"image/jpeg",
"FileName":"C:\\\\Windows\\\\TEMP\\\
\daa1a173-4509-438d-9dcd-165f1debb87d.jpg",
"Content-Type":"application/json; charset=utf-8"
},
"Etag":"00000000-0000-1f00-0000-0000000001d0"
}

In the tenant database, our attachments look similar to this:

{
"Size":73296,
"Key":"twm-317db86e-ea14-4750-b630-397fd350188d",
"Metadata":{
"ContentType":"image/jpeg",
"FileName":"C:\\\\Windows\\\\TEMP\\\
\daa1a173-4509-438d-9dcd-165f1debb87d.jpg",
"Content-Type":"application/json; charset=utf-8",
"Raven-Replication-Version":16576,
"Raven-Replication-Source":"2f5b82eb-9122-4b8e-
a9cd-12b4fde58d51"
},
"Etag":"00000000-0000-1c00-0000-0000000000ea"
}

The difference being that the metadata for the tenant database
attachments includes Raven-Replication-Version and Raven-Replication-
Source.

Thanks,
David

On May 28, 2:25 pm, Itamar Syn-Hershko <ita...@hibernatingrhinos.com>
wrote:

> This happens because we get to an empty attachment. This is wrong and will
> get fixed, but I want to try and find the root cause for this.
>
> How are you working with attachments? do you issue deletes on a regular
> basis? can you take a look at the ID of the attachment causing this and
> tell what operations have been performed on it?
>
> On Mon, May 28, 2012 at 6:38 PM, ReverentGeek <da...@reverentgeek.com>wrote:
>
>
>
>
>
>
>
> > I'm trying to write a utility that purges all attachments from a
> > RavenDB database. I'm running into exceptions on both build 888 and
> > 952. I'm using the /static endpoint to get a list of attachments, and
> > trying to delete them by key.
>
> > On a tenant database using build 952, some of the attachments appeared
> > to be deleted. However, every time I now request
> >http://localhost:8080/databases/tenant-name/static, I get the
> > following exception returned:
>
> > System.InvalidOperationException: Nullable object must have a value.
> >   at System.Nullable`1.get_Value()
> >   at
>

> > Raven.Storage.Esent.StorageActions.DocumentStorageActions.<GetAttachmentsBy ReverseUpdateOrder>d__3.MoveNext()

[Itamar Syn-Hershko]

What is your workflow? can you try mimicking it in a unit-test to see if this behavior repeats itself? I'm suspecting deleting an attachment leaves some left-overs that cause this

And are you saying this only happens on a tenant DB?

[Oren Eini (Ayende Rahien)]

No, this happens when we try to load an empty (0 byte) attachment.

Will be fixed in the next build