Can't Read xml Post to dto

[WWarrior]

Hi,

I'm trying to get the last method of our service finished up and can't
figure out why this won't work.

The call looks something like this: https://www.somedomain.com/nfl/2011/cardinals

We have the following xml file being POSTed to the service:

<?xml version="1.0" encoding="utf-8"?>
<dataLastFiles xmlns="http://www.somedomain.com/2010/v1/datalast">
<dataFile name="feedlist.xml" lastDateTime="2011-07-29 16:47:20Z"></
dataFile>
<dataFile name="injurydata.xml" lastDateTime="2011-07-26
11:00:17Z"></dataFile>
<dataFile name="rosterdata.xml" lastDateTime="2011-07-28
11:03:09Z"></dataFile>
<dataFile name="standingsdata.xml" lastDateTime="2011-07-29
11:00:28Z"></dataFile>
<dataFile name="scheduledata.xml" lastDateTime="2011-07-29
11:06:20Z"></dataFile>
</dataLastFiles>

We are trying to read it with the following custom DTO, by custom I
mean we created it for this purpose. It did not exist until today. The
LeagueName, SeasonYearStart and TeamCode properties come from the uri.

public class dataLastFiles : IEnumerable<dataFile>
{
List<dataFile> innerList = new List<dataFile>();
public String LeagueName { get; set; }
public String SeasonYearStart { get; set; }
public String TeamCode { get; set; }

public dataLastFiles() { }

public void Add(dataFile df)
{
innerList.Add(df);

} // void Add(dataFile df)

public IEnumerator<dataFile> GetEnumerator()
{
return innerList.GetEnumerator();
}

System.Collections.IEnumerator
System.Collections.IEnumerable.GetEnumerator()
{
return innerList.GetEnumerator();
}

} // class dataLastFiles

public class dataFile
{
public String name { get; set; }
public String lastDateTime { get; set; }

public dataFile()
{

}

}

I'm getting a 500 error when running the rest call that should process
this. I don't know how many iterations of this dto I've tried, too
many. I just can't seem to model that xml. We need to be able to read
the xml post. I'm guessing the problem is that ServiceStack cannot
deserialize the xml.

Out of frustration I'm wondering if there is a way, within
ServiceStack, to bypass the automatic dto creation just for this
method (we need the automatic dto creation everywhere else) and just
read the Content of the Request within the method. Is that possible?
If yes, how? For example, is there a way to have the Content of a
Request placed in a String or Stream property of a dto?

I cannot skip or have the post call modified at this time. It already
exists in mobile apps that are deployed. We had this working on a WCF
Rest project, we are trying to switch out that technology and have
decided to use ServiceStack to replace it.

For completeness, I'm not doing an inheritance from List<dataFile>
because we get an error for the wrong number of parameters in the
dataFile class. I'm not sure why... The above looks to me like it
should work, but it does not.

I really want to finish this service and this is the only thing
standing in the way.

Thanks,

\ ^ / i l l

[Demis Bellot]

Unfortunately the Microsoft's XML DataContractSerializer doesn't support serializing attributes so you won't be able to serialize attributed XML markup out of the box.

The only ways around it I see is to serialize it yourself in the service and return a string which will be written to the response as-is.

Or modify the XmlSerializer wrapper used in ServiceStack to inject a different Xml Serialization strategy.

https://github.com/ServiceStack/ServiceStack.Text/blob/master/src/ServiceStack.Text/XmlSerializer.cs

Unfortunately you're running into the edge cases of ServiceStack where you're trying to map service to produce existing markup.

ServiceStack is generally optimized for (and recommended) the other way round where you design your services DTO-first using code-first POCO's.

If you end up modifying the XmlWrapper, feel free to send me a pull request and I'll happily roll the changes into the next version.

Cheers,

--
- Demis

http://twitter.com/demisbellot

http://www.servicestack.net/mythz_blog

[WWarrior]

Hi Demis,

Once again, thank you for the quick, late reply. I wasn't expecting
anything until tomorrow. :)

I totally understand this is an edge case. I wish I could do it the
"right" way but with existing clients out there we still need to
support this case. <shrug>

>>The only ways around it I see is to serialize it yourself in the service and
>>return a string which will be written to the response as-is.

By serialize it myself, can you be more explicit? I'd rather not deal
with the serializer and would prefer to read the request context. Is
there some way to do that? I was hoping for something similar to the
Request.Filter stuff. :)

I thought I saw a posting about having a Body property in a dto, that
would be cool as well. Any suggestions?

If not, then I'll pursue the wrapper approach.

Thanks,

\ ^ / i l l

On Oct 10, 7:57 pm, Demis Bellot <demis.bel...@gmail.com> wrote:
> Unfortunately the Microsoft's XML DataContractSerializer doesn't support
> serializing attributes so you won't be able to serialize attributed XML
> markup out of the box.
>
> The only ways around it I see is to serialize it yourself in the service and
> return a string which will be written to the response as-is.
>
> Or modify the XmlSerializer wrapper used in ServiceStack to inject a

> different Xml Serialization strategy.https://github.com/ServiceStack/ServiceStack.Text/blob/master/src/Ser...

> http://twitter.com/demisbellothttp://www.servicestack.net/mythz_blog- Hide quoted text -
>
> - Show quoted text -

[Demis Bellot]

So basically if you want an XmlSerializer that will support attributes you'll likely need to use XmlSerializer and decorate your DTO's accordingly:

http://msdn.microsoft.com/en-us/library/system.xml.serialization.xmlserializer.aspx

But since ServiceStack doesn't support System.Xml.Serialization.XmlSerializer out of the box you will need to do the serialization yourself and return the serialized result (so effectively ServiceStack just treats it as a dumb string).

You can do this a couple of ways, first in your service you can return a Stream or string with the serialized DTO, e.g something like:

 

public override object OnExecute(RequestDto request) 

{  

  var dto = ...

  using (var ms = new MemoryStream())
  using (var sw = new StreamWriter(ms))
  {
     var serializer = new XmlSerializer(typeof(DataListFilesDtoDecoratedWithXmlSerializerAttributes));
     serialier.Write(sw, dto);
     return new HttpResult(ms, ContentType.Xml); 

  }

}

 

Also you don't need to go the approach of serializing a strong-typed DTO, in fact because the XmlSerializer custom attributes are so hideously ugly and verbose if I was to generate specific markup I would turn to a StringBuilder instead. The limitations of this approach means that this service can only return Xml as ServiceStack doesn't get the raw DTO which it can serialize based on the Request ContentType.

And as you've already guessed you could also do this in the Response Filter but instead of returning a HttpResult you would set the ContentType and serialize directly to the res (IHttpResponse) and close it as you've previously done in the earlier thread.

Cheers,

[WWarrior]

Hi Demis,

Thank you very much for your help!

I'm wondering if we have a miscommunication. The xml listed above is
being sent to the ServiceStack service I'm working on with a Content-
Type of application/xml. I'm trying to read it and can't. All I get is
a 500 error, not even the... this.RequestFilters.Add lambda gets
triggered. I'm not worried about the response, at all, I've learned
enough to handle that, no problem. The problem is that ServiceStack
fails on this call before we get to the RestService based class. It
simply returns a 500, Internal Error. I read somewhere that a
deserialization issue could cause that, that is why I was trying to
create a class that would Map to the Xml. Since that won't work, I'm
just trying to find a way to read the Request and go from there. I've
tried the above mentioned OnExecute, it does not get called, I' also
tried OnPost and Run. None of them are ever called. If I'm guessing
right, I don't think the class is instantiated. Is there a way to just
get the Request into a Service based class where we can do the
processing of the Content? I don't mind doing the reading, etc of the
Stream, but I need the Request, somewhere processable. <sp?> :) The
rest call to the service is an http POST.

Thanks,

\ ^ / i l l

On Oct 10, 9:40 pm, Demis Bellot <demis.bel...@gmail.com> wrote:
> So basically if you want an XmlSerializer that will support attributes

> you'll likely need to use XmlSerializer and decorate your DTO's accordingly:http://msdn.microsoft.com/en-us/library/system.xml.serialization.xmls...

> > >http://twitter.com/demisbellothttp://www.servicestack.net/mythz_blog-...quoted text -

[Demis Bellot]

Yeah reading the payload is going to be a problem since that's done at a very early stage, even before the Request Filters are executed.

You're going to have to do this before the Request gets deserialized and at the moment the only pluggable way to do that is to add a ContentTypeFilter, you can do this like:

this.ContentTypeFilter.Register(ContentType.Xml, MyCustomSerializeFn, MyCustomDeserializeFn);

Although I prefer to group functionality so I would register the ContentType similar to how I'm registering other formats by encapsulating it in a different class, like.

XmlContentFilter.Register(this);

In your XmlContentFilter class you would have all your custom XmlSerializer code, here are some examples of the CSV, HtmlFormat and VCardFormat all using this approach:

https://github.com/ServiceStack/ServiceStack/blob/master/src/ServiceStack/WebHost.EndPoints/Formats/HtmlFormat.cs#L398

https://github.com/ServiceStack/ServiceStack/blob/master/src/ServiceStack/WebHost.EndPoints/Formats/CsvFormat.cs

http://www.servicestack.net/ServiceStack.Northwind/vcard-format.htm

This approach should work but will take some extra effort, you will need to detect the type you want to handle then manually parse it, for all other types deserialize using the existing serializer.

Cheers,

[WWarrior]

Hi Demis,

So I decided to look into the guts of ServiceStack to see how things
were put together and find out if there was a way to solve my use
case. The use case is to get the Content from a request into a DTO for
further processing.I found a solution, though I totally understand
that it goes against the metality of ServiceStack and that you may
want to throw eggs or tomatoes my way. ;)

Then again, if I had this issue, others might as well, and well maybe
you won't hate my approach all together. (he cringes as he types) :)

In my case, I did not care to create a DTO with all of the
XmlSerializer junk. It served no purpose in this case, I decided it
was more useful to get the Raw Content from the httpRequest and go
from there. Below is what I did. I'll gladly submit it to GitHub,
though I imagine you will have some say on this, and that is perfectly
cool. :) This is working beautifully for me and I think it fits ok
within the framework, not perfect, but possibly useful.

I created a new interface called IRestGetRawContentStream. For now I
added it to the ServiceStack.Interfaces ServiceHost folder. It looks
like this:

public interface IRestGetRawContentStream
{
Stream RawContent { get; set; }
}

I then implemented a new DTO that just had my Rest parameters and
RawContent.

Then I went to ServiceStack.WebHost.Endpoints.RestHandler and I
modified GetRequest to manage the new interface. Here is what mine
looks like right now.

private static object GetRequest(IHttpRequest httpReq, IRestPath
restPath)
{
var requestParams = httpReq.GetRequestParams();

object requestDto = null;

if (!string.IsNullOrEmpty(httpReq.ContentType) &&
httpReq.ContentLength > 0)
{
var requestDeserializer =
EndpointHost.AppHost.ContentTypeFilters.GetStreamDeserializer(httpReq.ContentType);
if (requestDeserializer != null)
{
try
{
requestDto = requestDeserializer(restPath.RequestType,
httpReq.InputStream);
}
catch (Exception ex)
{
if
(typeof(IRestGetRawContentStream).IsAssignableFrom(restPath.RequestType))
{
IRestGetRawContentStream holder =
(IRestGetRawContentStream)Activator.CreateInstance(restPath.RequestType);
httpReq.InputStream.Position = 0;
if (holder.RawContent == null)
{
holder.RawContent = new MemoryStream();
}
httpReq.InputStream.CopyTo(holder.RawContent);
holder.RawContent.Position = 0;
requestDto = holder;
}
else
{
throw ex;
}
}
}
}

return restPath.CreateRequest(httpReq.PathInfo, requestParams,
requestDto);
}

My mentality... I needed the raw content and this was the shortest
route I could find to getting it without modifying more than one
method. I don't normally like to rely on errors to implement something
but I also did not want to mess with the innards of ServiceStack and
break something this important. I also did not want to take the
IsAssignableFrom hit for every call. This is an Edge Use Case and so
it seemed better to handle it as an error, normally, in this situation
you just get a 500 error. This way the performance hit only occurs for
the edge case. I chose an interface, as opposed to an attribute, so
there would be a certain place to put the contents.

I am open to pretty much any discussion that would serve this use
case.

Your thoughts, comments, ideas, improvements, virtual tomatoes, etc
are ...

Best,

\ ^ / i l l

That's it! Again, this is working beautifully for me. I need to now
test against Android devices to make sure everything is ok but so far
my Tests all pass. I'm using a Console app to test my service
implementation.

On Oct 10, 9:40 pm, Demis Bellot <demis.bel...@gmail.com> wrote:
> So basically if you want an XmlSerializer that will support attributes

> you'll likely need to use XmlSerializer and decorate your DTO's accordingly:http://msdn.microsoft.com/en-us/library/system.xml.serialization.xmls...

> > >http://twitter.com/demisbellothttp://www.servicestack.net/mythz_blog-...quoted text -

[WWarrior]

Hi Demis,

It looks like you replied while I was replying. I'll look at the
approach you outlined in the morning. I'm done for today/tonight. :)

I would still appreciate your comments on my out of the box approach
from just a bit ago.

Best,

\ ^ / i l l

On Oct 11, 1:21 am, Demis Bellot <demis.bel...@gmail.com> wrote:
> Yeah reading the payload is going to be a problem since that's done at a
> very early stage, even before the Request Filters are executed.
>
> You're going to have to do this before the Request gets deserialized and at
> the moment the only pluggable way to do that is to add a ContentTypeFilter,
> you can do this like:
>
> this.ContentTypeFilter.Register(ContentType.Xml, MyCustomSerializeFn,
> MyCustomDeserializeFn);
>
> Although I prefer to group functionality so I would register the ContentType
> similar to how I'm registering other formats by encapsulating it in a
> different class, like.
>
> XmlContentFilter.Register(this);
>
> In your XmlContentFilter class you would have all your custom XmlSerializer
> code, here are some examples of the CSV, HtmlFormat and VCardFormat all

> using this approach:https://github.com/ServiceStack/ServiceStack/blob/master/src/ServiceS...https://github.com/ServiceStack/ServiceStack/blob/master/src/ServiceS...http://www.servicestack.net/ServiceStack.Northwind/vcard-format.htm

> ...
>
> read more »- Hide quoted text -

[Demis Bellot]

Hi,

Yep very likely, as it looked like you might've spent a while writing your reply :)

Yep thanks for this, as you've experienced ServiceStack needs a better story for handling raw custom payloads.

At one point ServiceStack did have the facility to execute a raw payload (i.e. string) but this was in the early stages of ServiceStack a fair few major refactorings ago - and since it wasn't a commonly used feature, I don't think it's re-hooked up.

At one stage, raw text requests/response got routed through ExecuteText():

https://github.com/ServiceStack/ServiceStack/blob/master/src/ServiceStack/ServiceHost/ServiceController.cs#L326

Unfortunately I've recently moved to the US and I'm still without my development PC (which should arrive < 10 days), So I'll only be able to properly test and debug your approach then. Until then happily go with what works best for you, and I'll look into this and other possible solutions that would work.

Cheers,

[WWarrior]

Hi Demis,

Thanks for the input. I hope your enjoying the new gig. It sounds like
a dream job. :)

As I looked at what I put together this morning, I don't know why I
didn't try to go the inheritance route. I'm gonna go look and see
which classes would need to be inherited to do this as a one off
instead of in the framework. I'll also look at the other two options
you pointed out though I will probably stick with something similar to
what I have, at least for now, I need to move on to other things.

Best,

\ ^ / i l l

On Oct 11, 1:50 am, Demis Bellot <demis.bel...@gmail.com> wrote:
> Hi,
>
> Yep very likely, as it looked like you might've spent a while writing your
> reply :)
>
> Yep thanks for this, as you've experienced ServiceStack needs a better story
> for handling raw custom payloads.
> At one point ServiceStack did have the facility to execute a raw payload
> (i.e. string) but this was in the early stages of ServiceStack a fair few
> major refactorings ago - and since it wasn't a commonly used feature, I
> don't think it's re-hooked up.
>

> At one stage, raw text requests/response got routed through ExecuteText():https://github.com/ServiceStack/ServiceStack/blob/master/src/ServiceS...