c# Web Service performance problems returning a Dataset

[scott F]

I have been evaluating the strengths of web services and as an example test
tried creating a class object that selected and returned a varying amounts
of data and then I took the data and bound it to a datagrid. As an example
I selected 30000 records into a dataset and bound it to a datagrid and when
everything was part of the 1 windows app, this process would take all of 5
seconds. I then took the class object and made it into a web service so that
it would return back a dataset from the web service. It now took 1 minute
for this process to complete. The bottleneck was when the service had to
return back a large dataset . Is this because of the time it takes to render
the data as xml. I dont have a very fast machine, would a faster machine
improve this performance very much or is this just an inherant limitiation
of a web service returning back a large amount of data or is there something
else that has to be done to speed the process up?

TIA

scott

[Alvin Bruney]

The bottleneck occurs when the large amount of data in xml form has to be
moved across the wire. XML by its nature is very heavy, moving it across the
wire will result in performance problems because not only is the data moved
but schema information contained within the dataset is pushed across the
wire as well. If you are looking at ways to improve performance, convert the
dataset to an xml string and the push this across the wire.

rock on

"scott F" <srfc...@hotmail.com> wrote in message
news:#JmuFBr$CHA....@TK2MSFTNGP10.phx.gbl...

[scott F]

After looked at the newsgroups I found many discusions related to this and
one of the was to convert it to xml and convert it back after. Do you have
and example of this? Whould you use, getxm() , then when would be used on
the other side to load it back into the dataset? would you lose any
information from the dataset by doing it this way?

What Im trying to get is to build a very thin windows client that knows
nothing about a database or where it is and calls the web services to get
any data its looking for so its key that I pass datasets to and from the
client and not lose any of the information in the dataset since the client
may send and dataset with updated records back to the service for it to save
back into the database.

Other options I have seen involve compression, 1 I tried was a product
called ASPaccelorator.net but I must be using it wrong since I didnt see any
difference in performance3. I is others but I dont have a lot of examples so
its taking me a while to try.

Another option is remoting but again I dont have a lot of example of that
compared to web services. If you know of a place/places that have
discussions on this and some examples that would be great since web services
provides a great way to seperate the ui from the business and data layers
and I would hate to have to dump it because of the performance issues but
then again if the app is so slow noone will use it then all you have is an
elegant paperweight.

TIA

scott
"Alvin Bruney" <vapo...@hotmail.com> wrote in message
news:#IcgcUx$CHA....@TK2MSFTNGP12.phx.gbl...

[Alvin Bruney]

Scott,
You are on the right track. Just change the return signature of your
webmethod to return a string and do this

return myDataSet.GetXML();

That converts the xml into a string xml and pushes it across the wire.
On the otherside just load the string back into the dataset. You won't
lose any info.
Alvin Bruney

*** Sent via Developersdex http://www.developersdex.com ***
Don't just participate in USENET...get rewarded for it!

[scott F]

I tried that and passing a string back was much faster , it went from 1
minute to 10 seconds. So now on the other side I used .ReadXml to load it
back into the dataset and wow was it SLOW, it took over 2 minutes for that
method to finish processing the 10000 records in the xml file and put it
into the dataset. I think that is the biggest problem and probably what Im
faced with when I did it the original way by passing a dataset. It seems any
time it has do to anything involving taking a xml imput file and loading
into a dataset it will be slow. Unfortunatly thats the only way to do it
with web services and its always going to be slow until they figure out how
to speed up the parsing and loading of xml files into datasets. At any rate,
Im getting a 2.4ghz machine today and will see how fast that method performs
on a super fast machine but I think the performance it always going to be a
problem until they address that issue. Let me know if you think Im off base
with that conclusion anyhow.

scott
"Alvin Bruney" <vapo...@hotmail.com> wrote in message

news:ejSDbe3$CHA....@TK2MSFTNGP12.phx.gbl...

[Alvin Bruney]

I'm in somewhat of the same problem as well. Nicholas has suggested that i
look into xmltextreader class for performance improvements so that is my new
avenue. Maybe you should experiment with different classes to reload the
data and see if that helps.

"scott F" <srfc...@hotmail.com> wrote in message

news:#VKECq3$CHA....@TK2MSFTNGP10.phx.gbl...

[scott F]

I tried it without any luck , this is the code Im using with the execution
times beside each trying to load 10000 records:

string s1=main.loaddataset(); 10 seconds
StringReader strRead=newSringReader(s1); 1 second
XmlTextReader xRead = new XMLTextReader(strRead); 1 second
ds1.ReadXml(xRead); 2 minutes!!!!!
datagrid1.DataSource=ds1; 1 second

I think any way you look at it the ReadXml method is slow and the method
microsoft is using to move an xml document into a dataset is slow and
unfortuanatly I think Im SOL until that is addressed. Im not sure , maybe
the new framework works better but for now I think I may have to shelve the
idea of web services for a while since accessing the data directly in the
windows app only takes 5 seconds to get the datagrid populated but its not
as nice as the web services route .

scott

"Alvin Bruney" <vapo...@hotmail.com> wrote in message

news:eZoeQA4$CHA....@TK2MSFTNGP11.phx.gbl...

[Alvin Bruney]

Scott.
Are you sure you don't have other issues? Granted XML parsing is a tad slow
but your 2 minutes seems on the high end to me. I've clocked a load of a 6mb
xml file with (70,000 rows, 16 columns per record, 9212 records) at 44
seconds. This was being retrieved off of hard disk using ReadXML().
I'm running on a 2g cpu with 1meg of ram, on a 20hd. If your machine is
comparable to mine, then your times should be roughly equivalent, give or
take a few.

"scott F" <srfc...@hotmail.com> wrote in message

news:uiW#JV5$CHA....@TK2MSFTNGP12.phx.gbl...

[scott F]

ya, Im wondering if the ReadXml is cpu intensive and Im running a relitivly
slow machine but will be getting a 2.5ghz machine next week and will see how
it performs on this. Asuming it is cpu intensive, if I did go down the web
services route then would the cpu intensive part be on the client or the
server if I was passing back a dataset that is(I suspect it would be on the
client). Passing back a xml string is obviously client cpu intensive since
it has to run the ReadXml on the client side. Just wondering since the one
of the reasons for doing this partitioning is to not have the client need to
have a lot of heavy resources on there side .

Also just as a comparision , I tried loading the dataset the normal way
without using a web service and I got the dataset loaded and bound to a
datagrid in about 2 seconds with a list of 40000 records.

Do you know much about the remoting option since that is supposed to be
faster and was wondering if xml was a part of it since that seems to be the
big bottleneck with web services and if you know where I could find some
examples of that?

scott
"Alvin Bruney" <vapo...@hotmail.com> wrote in message

news:uqOjq35$CHA....@TK2MSFTNGP10.phx.gbl...

[Alvin Bruney]

>ds1.ReadXml(xRead); 2 minutes!!!!!

You shouldn't be using this line of code. That's painful.
Use the XmlTextReader object to parse through the xml input source and
extract the node data you want. Put the data into appropriate data
structures as you parse. This is the absolute way to proceed.

Nicholas suggested this method and it cut my load time from 45 seconds
to under 1 second for 10,000 records. It's absolutely amazing.

[scott F]

My main goal was to take a dataset and transfer it to and from a web
service, so we know that is painfully slow, and converting it to a xml
string and transmiting that isnt too bad, so once I have the xml string, how
would I do as you say and get it back into a dataset?

scott
"Alvin Bruney" <vapo...@hotmail.com> wrote in message

news:eOh47bDA...@TK2MSFTNGP11.phx.gbl...

[Ravichandran J.V.]

I actually wanted to answer to the OP but I don't know how yours opened.
Anyway, this is for the OP.

I can make one suggestion to improve upon the design of the WS. Instead
of sending a whole dataset object, your client could just send the data
fields as parameters and the web method could translate it into a
meaningful column name for the dataset to act on and return the result
in the form of a string rather than as a whole dataset object.

with regards,

J.V.Ravichandran
- http://www.geocities.com/
jvravichandran
- http://www.411asp.net/func/search?
qry=Ravichandran+J.V.&cob=aspnetpro
- http://www.southasianoutlook.com
- http://www.MSDNAA.Net
- http://www.csharphelp.com
- http://www.poetry.com/Publications/
display.asp?ID=P3966388&BN=999&PN=2
- Or, just search on "J.V.Ravichandran"
at http://www.Google.com