Received: by 10.216.137.196 with SMTP id y46mr1358257wei.7.1335282413735; Tue, 24 Apr 2012 08:46:53 -0700 (PDT) X-BeenThere: mobicents-public@googlegroups.com Received: by 10.216.221.80 with SMTP id q58ls9670943wep.0.gmail; Tue, 24 Apr 2012 08:46:51 -0700 (PDT) Received: by 10.216.136.207 with SMTP id w57mr1358980wei.12.1335282411303; Tue, 24 Apr 2012 08:46:51 -0700 (PDT) Received: by 10.216.136.207 with SMTP id w57mr1358979wei.12.1335282411275; Tue, 24 Apr 2012 08:46:51 -0700 (PDT) Return-Path: Received: from mail-wg0-f44.google.com (mail-wg0-f44.google.com [74.125.82.44]) by gmr-mx.google.com with ESMTPS id ca1si5023050wib.1.2012.04.24.08.46.51 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 24 Apr 2012 08:46:51 -0700 (PDT) Received-SPF: pass (google.com: domain of brains...@gmail.com designates 74.125.82.44 as permitted sender) client-ip=74.125.82.44; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of brains...@gmail.com designates 74.125.82.44 as permitted sender) smtp.mail=brains...@gmail.com; dkim=pass header...@gmail.com Received: by wgbdr13 with SMTP id dr13so816286wgb.1 for ; Tue, 24 Apr 2012 08:46:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=50uuEnxIppjIM55dBy5RRvYTAxq68F/G45bh32zGYzM=; b=ddHHbwzBZn5fXjhjPIp3OU1B1ZbLiNJc3A1x6lnPuJ+kyXdInq+3R5elWzW4Px6zZK KGn34OmL/i0dZoqh5CJNt0RkVQBHlEnN4/tRpKeVMgNfkYab15j3xiwZ42wbTCwFnz0h 8RBrKPymh98XNXfSbTXP590l0YD0TZX6KffT/X/D6fzafQ5F5cRuFHPHFFJOgK2LpOri od5NiibO8eHcWgKQOFf6RuxYRm+Ws0ScKwtc/QW3nbiC+TkJ1nhWGPoWpBXRN9LlotfC jrmLOsI9Bquwh9kHJQ+f/rO3wK1Juw8RAuzB7lofOqhANkXjTL9+zMYbnV6lH25tWA7X d2Zw== Received: by 10.180.107.104 with SMTP id hb8mr32372031wib.8.1335282410726; Tue, 24 Apr 2012 08:46:50 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.163.8 with HTTP; Tue, 24 Apr 2012 08:46:10 -0700 (PDT) In-Reply-To: <8018469.435.1335279154529.JavaMail.geo-discussion-forums@vbdx11> References: <6a3df119-99f1-4fa5-8a7f-d06dc24de...@fo16g2000vbb.googlegroups.com> <4F919AB2.4090...@gmail.com> <8381070b-126f-4878-8702-50634db57...@ca2g2000vbb.googlegroups.com> <4656154.189.1335276673856.JavaMail.geo-discussion-forums@ynje10> <2c9b52e2-2ee5-4ec3-a17d-db87d6fb6...@w7g2000vbg.googlegroups.com> <8018469.435.1335279154529.JavaMail.geo-discussion-forums@vbdx11> From: =?ISO-8859-1?Q?Alexandre_Mendon=E7a?= Date: Tue, 24 Apr 2012 16:46:10 +0100 Message-ID: Subject: Re: [mobicents-public] Re: Diameter CCA performance To: mobicents-public@googlegroups.com Content-Type: multipart/alternative; boundary=e89a8f234ce5c4242904be6ea9a5 --e89a8f234ce5c4242904be6ea9a5 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Yes, if it's one client only, it's one connection established only. Angel, I've also ran several load tests on Diameter and I've been able to reach ~800 requests/second with JAIN SLEE, on a regular 3y old desktop computer. This is related to Base RA, but performance with others should be similar. What are you using to test? -- *Alexandre Mendon=E7a* // JBoss R&D http://ammendonca.blogspot.com/ On Tue, Apr 24, 2012 at 15:52, Robertas Mikaitis wrote: > yes mobicents diameter stack settings was left untouched - default. I > think i was only one TCP/IP connection, since only one link was establish= ed > between mobicents servers, though i might be wrong, mobicents guys shoul= d > know better if creating new DiameterRoClient session opens new TCP/IP lin= k, > a doubt it does that. > > > On Tuesday, April 24, 2012 5:48:18 PM UTC+3, Angel wrote: >> >> Hi Robertas, >> thanks for the information. >> >> Did you tested with the default Mobicents settings or you changed some >> parameters to get better performance? >> If yes, which parameters? >> >> Do you remember how much parallel TCP/IP connections you opened to the >> Mobicents Diameter for these tests? >> How many requests per second do you get for only one TCP/IP >> connection? >> >> Regards, >> Angel >> >> >> >> On Apr 24, 4:11 pm, Robertas Mikaitis wrote: >> > I get good performance with diameter stack(Diameter Ro Ra). I`ve teste= d >> up >> > to 400(250 if Mobicents is configured as cluster with session >> replication) >> > request per second with typical response time < 15ms between 2(3 if >> > clustered) Mobicent servers, where one sends request other(other 2 in >> > clustered mode) responds. I have not needed more then 400 rps, so i di= d >> not >> > tested above it, but i`m sure i could do better. Test was running in >> > VirtualBox virtual machines Windows 2003 Server, host pc: Centos, Inte= l >> i7, >> > 16Gb Ram. While testing other virtual machines was running also. You >> should >> > try different software/hardware configurations. >> > >> > >> > >> > >> > >> > >> > >> > On Monday, April 23, 2012 2:12:05 PM UTC+3, Angel wrote: >> > >> > > Hi Luis, >> > > thinks for the advise. Until now I tested with the Garbage-First >> > > garbage collector (-XX:+UseG1GC). >> > > I switched to CMS garbage collector (-XX:+UseConcMarkSweepGC) and no= w >> > > I got rid of the most slow response times. >> > >> > > I would like to reach your diameter response times. >> > > Can you help me to tune my system for best diameter performance? >> > > Are there some important config parameter, that should be changed fo= r >> > > better diameter performance? >> > > Are there some services or features that are not needed for diameter >> > > and cab be disabled? >> > >> > > Regards, >> > > Angel >> > >> > > On Apr 20, 7:19 pm, Luis Barreiro wrote: >> > > > Hi Angel, >> > >> > > > We do not experience the same in our load tests athttps:// >> > > ci.jboss.org/hudson/view/Mobicents/job/Mobicents-Diameter-1.x... >> > >> > > > We run a similar scenario at 750 requests/second, so you should be >> able >> > > > to get similar results to ours. >> > >> > > > If you analyze the results, you'll see that response time average >> is >> > > > 19ms (with std. deviation of just 8ms). >> > >> > > > The long pauses that you are experiencing may be due to garbage >> > > > collection on the JVM. >> > >> > > > Try to enable the concurrent, low pause, GC with >> > > > '-XX:+UseConcMarkSweepGC' argument either in JAVA_OPTS env variabl= e >> or >> > > > by editing $JBOSS_HOME/bin/run.conf >> > >> > > > Regards, >> > >> > > > Luis Barreiro????? >> > >> > > > On 04/20/2012 04:13 PM, Angel wrote: >> > >> > > > > Hi, >> > > > > I made some performance tests of the Diameter CCA RA 2.6.1. >> > > > > I implemented only the onCreditControlRequest like this: >> > >> > > > > public void onCreditControlRequest(CreditControlRequest >> > > > > creditControlRequest, ActivityContextInterface activityContext) = { >> > > > > if (tracer.isInfoEnabled()) { >> > > > > tracer.info("Credit-Control-Request[272] received: >> > > > > "+creditControlRequest); >> > > > > } >> > > > > CreditControlServerSession creditControlServerSession =3D >> > > > > (CreditControlServerSession) activityContext.getActivity(); >> > > > > CreditControlAnswer answer =3D >> > > > > creditControlServerSession.createCreditControlAnswer(); >> > > > > answer.setResultCode(2001); >> > > > > creditControlServerSession.sendCreditControlAnswer(answer); >> > > > > } >> > >> > > > > I send 800 Diameter requests on one tcp/ip connection. >> > > > > I measured the response times on the client machine with tcpdump= . >> > >> > > > > What I see: I receive about 150-200 fast answers - about 15-18 m= s >> > > > > and than I receive about 10-12 slow answers - about 400-500 ms >> > > > > and than again fast answers and than again slow answers... >> > > > > This repeats by any test. >> > >> > > > > This are the test statistics: >> > > > > Number of messages: 800 >> > > > > average time: 46.258 s >> > > > > minimum time: 14.416 s >> > > > > maximum time: 671.706 s >> > > > > throughput: 38.215/s >> > >> > > > > Can some body confirm this behaviour of the Diameter stack? >> > > > > The overhead of 500ms per request is huge. >> > > > > Until now I'm not able to find why mobicents server have this >> > > > > behaviour, please help! >> > >> > > > > Here is a full list of response times: >> > > > > # - response time >> > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> > > > > 1 18.302 >> > > > > 2 16.414 >> > > > > 3 16.774 >> > > > > 4 16.698 >> > > > > 5 17.308 >> > > > > 6 16.552 >> > > > > 7 15.817 >> > > > > 8 16.695 >> > > > > 9 16.017 >> > > > > 10 15.925 >> > > > > 11 16.441 >> > > > > 12 15.995 >> > > > > 13 15.654 >> > > > > 14 16.186 >> > > > > 15 16.223 >> > > > > 16 18.200 >> > > > > 17 16.788 >> > > > > 18 16.194 >> > > > > 19 17.062 >> > > > > 20 17.996 >> > > > > 21 22.241 >> > > > > 22 22.005 >> > > > > 23 20.659 >> > > > > 24 22.648 >> > > > > 25 21.822 >> > > > > 26 18.615 >> > > > > 27 21.826 >> > > > > 28 42.799 >> > > > > 29 21.094 >> > > > > 30 19.140 >> > > > > 31 41.743 >> > > > > 32 55.097 >> > > > > 33 79.565 >> > > > > 34 637.478 >> > > > > 35 671.706 >> > > > > 36 671.706 >> > > > > 37 608.713 >> > > > > 38 553.160 >> > > > > 39 492.073 >> > > > > 40 553.160 >> > > > > 41 492.073 >> > > > > 42 630.101 >> > > > > 43 373.144 >> > > > > 44 541.256 >> > > > > 45 20.619 >> > > > > 46 18.050 >> > > > > 47 18.325 >> > > > > 48 18.557 >> > > > > 49 18.454 >> > > > > 50 17.904 >> > > > > 51 18.448 >> > > > > 52 18.152 >> > > > > 53 18.165 >> > > > > 54 18.267 >> > > > > 55 18.094 >> > > > > 56 17.727 >> > > > > 57 17.952 >> > > > > 58 18.040 >> > > > > 59 18.058 >> > > > > 60 17.666 >> > > > > 61 18.081 >> > > > > 62 18.050 >> > > > > 63 17.763 >> > > > > 64 17.934 >> > > > > 65 17.651 >> > > > > 66 17.671 >> > > > > 67 18.138 >> > > > > 68 41.887 >> > > > > 69 22.864 >> > > > > 70 18.912 >> > > > > 71 18.942 >> > > > > 72 18.523 >> > > > > 73 18.550 >> > > > > 74 18.705 >> > > > > 75 18.402 >> > > > > 76 17.643 >> > > > > 77 17.630 >> > > > > 78 18.134 >> > > > > 79 17.969 >> > > > > 80 17.822 >> > > > > 81 18.970 >> > > > > 82 18.598 >> > > > > 83 19.349 >> > > > > 84 18.263 >> > > > > 85 17.934 >> > > > > 86 17.879 >> > > > > 87 17.592 >> > > > > 88 17.731 >> > > > > 89 19.530 >> > > > > 90 31.724 >> > > > > 91 19.668 >> > > > > 92 47.105 >> > > > > 93 19.900 >> > > > > 94 22.388 >> > > > > 95 22.246 >> > > > > 96 23.047 >> > > > > 97 21.916 >> > > > > 98 21.800 >> > > > > 99 21.935 >> > > > > 100 22.023 >> > > > > 101 21.673 >> > > > > 102 21.728 >> > > > > 103 21.951 >> > > > > 104 21.973 >> > > > > 105 22.114 >> > > > > 106 22.064 >> > > > > 107 17.458 >> > > > > 108 43.564 >> > > > > 109 16.734 >> > > > > 110 17.978 >> > > > > 111 17.311 >> > > > > 112 16.484 >> > > > > 113 16.728 >> > > > > 114 16.604 >> > > > > 115 15.966 >> > > > > 116 17.015 >> > > > > 117 16.153 >> > > > > 118 15.831 >> > > > > 119 15.887 >> > > > > 120 16.017 >> > > > > 121 15.886 >> > > > > 122 15.762 >> > > > > 123 15.628 >> > > > > 124 15.747 >> > > > > 125 16.403 >> > > > > 126 16.056 >> > > > > 127 15.942 >> > > > > 128 15.651 >> > > > > 129 15.814 >> > > > > 130 15.676 >> > > > > 131 15.637 >> > > > > 132 15.617 >> > > > > 133 15.576 >> > > > > 134 15.542 >> > > > > 135 15.313 >> > > > > 136 15.685 >> > > > > 137 15.453 >> > > > > 138 15.776 >> > > > > 139 15.445 >> > > > > 140 15.030 >> > > > > 141 15.480 >> > > > > 142 17.953 >> > > > > 143 18.210 >> > > > > 144 17.743 >> > > > > 145 17.991 >> > > > > 146 17.764 >> > > > > 147 18.145 >> > > > > 148 18.327 >> > > > > 149 17.764 >> > > > > 150 17.404 >> > > > > 151 17.977 >> > > > > 152 17.859 >> > > > > 153 17.844 >> > > > > 154 18.215 >> > > > > 155 18.536 >> > > > > 156 16.214 >> > > > > 157 15.246 >> > > > > 158 16.121 >> > > > > 159 15.314 >> > > > > 160 16.690 >> > > > > 161 15.374 >> > > > > 162 16.029 >> > > > > 163 15.717 >> > > > > 164 17.630 >> > > > > 165 16.163 >> > > > > 166 16.218 >> > > > > 167 16.712 >> > > > > 168 16.265 >> > > > > 169 16.400 >> > > > > 170 16.087 >> > > > > 171 15.565 >> > > > > 172 16.318 >> > > > > 173 16.249 >> > > > > 174 15.764 >> > > > > 175 16.052 >> > > > > 176 15.599 >> > > > > 177 15.874 >> > > > > 178 15.758 >> > > > > 179 15.812 >> > > > > 180 15.619 >> > > > > 181 15.790 >> > > > > 182 15.657 >> > > > > 183 15.305 >> > > > > 184 15.363 >> > > > > 185 15.221 >> > > > > 186 16.509 >> > > > > 187 16.030 >> > > > > 188 15.914 >> > > > > 189 15.830 >> > > > > 190 15.622 >> > > > > 191 16.066 >> > > > > 192 15.919 >> > > > > 193 17.392 >> > > > > 194 18.002 >> > > > > 195 17.794 >> > > > > 196 17.876 >> > > > > 197 18.539 >> > > > > 198 17.936 >> > > > > 199 18.233 >> > > > > 200 18.222 >> > > > > 201 17.081 >> > > > > 202 18.208 >> > > > > 203 15.874 >> > > > > 204 17.520 >> > > > > 205 18.009 >> > > > > 206 16.699 >> > > > > 207 16.570 >> > > > > 208 15.687 >> > > > > 209 15.594 >> > > > > 210 15.393 >> > > > > 211 15.150 >> > > > > 212 15.697 >> > > > > 213 15.174 >> > > > > 214 15.437 >> > > > > 215 15.197 >> > > > > 216 15.441 >> > > > > 217 15.221 >> > > > > 218 16.466 >> > > > > 219 16.407 >> > > > > 220 17.072 >> > > > > 221 16.832 >> > > > > 222 16.266 >> > > > > 223 16.231 >> > > > > 224 15.798 >> > > > > 225 15.620 >> > > > > 226 16.011 >> > > > > 227 15.901 >> > > > > 228 16.378 >> > > > > 229 15.669 >> > > > > 230 16.269 >> > > > > 231 16.307 >> > > > > 232 41.461 >> > > > > 233 16.147 >> > > > > 234 17.885 >> > > > > 235 16.682 >> > > > > 236 16.399 >> > > > > 237 553.132 >> > > > > 238 511.397 >> > > > > 239 511.397 >> > > > > 240 511.397 >> > > > > 241 437.126 >> > > > > 242 437.126 >> > > > > 243 342.989 >> > > > > 244 437.126 >> > > > > 245 222.916 >> > > > > 246 222.916 >> > > > > 247 384.531 >> > > > > 248 384.531 >> > > > > 249 18.349 >> > > > > 250 16.321 >> > > > > 251 >> > >> > ... >> > >> > read more =BB > > --e89a8f234ce5c4242904be6ea9a5 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Yes, if it&#= 39;s one client only, it's one connection established only.

<= /font>
Angel, = I've also ran several load tests on Diameter and I've been able to = reach ~800 requests/second with JAIN SLEE, on a regular 3y old desktop comp= uter. This is related to Base RA, but performance with others should be sim= ilar.=A0

What are you using to test?
<= br>
--
Alexandre Mendon=E7a // JBoss R&D<= br> http://ammendonca.blogspot.com/<= br>

On Tue, Apr 24, 2012 at 15:52, Robertas = Mikaitis <robism@g= mail.com> wrote:
yes mobicents diameter stack settings was left untouched - default. I think= i was only one TCP/IP connection, since only one link was established betw= een mobicents servers, though i might be wrong, mobicents guys =A0should kn= ow better if creating new DiameterRoClient session opens new TCP/IP link, a= doubt it does that.


On Tuesday, April 24, 2012 5:48:18 PM UTC+3, Ange= l wrote:
Hi Robertas,
thanks for the information.

Did you tested with the default Mobicents settings or you changed some
parameters to get better performance?
If yes, which parameters?

Do you remember how much parallel TCP/IP connections you opened to the
Mobicents Diameter for these tests?
How many requests per second do you get for only one TCP/IP
connection?

Regards,
Angel



On Apr 24, 4:11=A0pm, Robertas Mikaitis <rob...@gmail.com>= wrote:
> I get good performance with diameter stack(Diameter Ro Ra). I`ve t= ested up
> to 400(250 if Mobicents is configured as cluster with session repl= ication)
> request per second with typical response time < 15ms between 2(= 3 if
> clustered) Mobicent servers, where one sends request other(other 2= in
> clustered mode) responds. I have not needed more then 400 rps, so = i did not
> tested above it, but i`m sure i could do better. Test was running = in
> VirtualBox virtual machines Windows 2003 Server, host pc: Centos, = Intel i7,
> 16Gb Ram. While testing other virtual machines was running also. Y= ou should
> try different software/hardware configurations.
>
>
>
>
>
>
>
> On Monday, April 23, 2012 2:12:05 PM UTC+3, Angel wrote:
>
> > Hi Luis,
> > thinks for the advise. Until now I tested with the Garbage-Fi= rst
> > garbage collector (-XX:+UseG1GC).
> > I switched to CMS garbage collector (-XX:+UseConcMarkSweepGC)= and now
> > I got rid of the most slow response times.
>
> > I would like to reach your diameter response times.
> > Can you help me to tune my system for best diameter performan= ce?
> > Are there some important config parameter, that should be cha= nged for
> > better diameter performance?
> > Are there some services or features that are not needed for d= iameter
> > and cab be disabled?
>
> > Regards,
> > Angel
>
> > On Apr 20, 7:19 pm, Luis Barreiro <lbbbarre...@gmail.co= m> wrote:
> > > Hi Angel,
>
> > > We do not experience the same in our load tests athttps:= //
> > ci.jboss.org/hudson/view/Mobicents/j= ob/Mobicents-Diameter-1.x...
>
> > > We run a similar scenario at 750 requests/second, so you= should be able
> > > to get similar results to ours.
>
> > > If you analyze the results, you'll see that response= time average is
> > > 19ms (with std. deviation of just 8ms).
>
> > > The long pauses that you are experiencing may be due to = garbage
> > > collection on the JVM.
>
> > > Try to enable the concurrent, low pause, GC with
> > > '-XX:+UseConcMarkSweepGC' argument either in JAV= A_OPTS env variable or
> > > by editing $JBOSS_HOME/bin/run.conf
>
> > > Regards,
>
> > > Luis Barreiro?????
>
> > > On 04/20/2012 04:13 PM, Angel wrote:
>
> > > > Hi,
> > > > I made some performance tests of the Diameter CCA R= A 2.6.1.
> > > > I implemented only the onCreditControlRequest like = this:
>
> > > > public void onCreditControlRequest(CreditControlReq= uest
> > > > creditControlRequest, ActivityContextInterface acti= vityContext) {
> > > > =A0 =A0if (tracer.isInfoEnabled()) {
> > > > =A0 =A0 =A0tracer.info("Credit-Control-Request[272] received:
> > > > "+creditControlRequest);
> > > > =A0 =A0}
> > > > =A0 =A0CreditControlServerSession creditControlServ= erSession =3D
> > > > (CreditControlServerSession) activityContext.getAct= ivity();
> > > > =A0 =A0CreditControlAnswer answer =3D
> > > > creditControlServerSession.createCreditControlAnswe= r();
> > > > =A0 =A0answer.setResultCode(2001);
> > > > =A0 =A0creditControlServerSession.sendCreditControl= Answer(answer);
> > > > }
>
> > > > I send 800 Diameter requests on one tcp/ip connecti= on.
> > > > I measured the response times on the client machine= with tcpdump.
>
> > > > What I see: I receive about 150-200 fast answers - = about 15-18 ms
> > > > and than I receive about 10-12 slow answers - about= 400-500 ms
> > > > and than again fast answers and than again slow ans= wers...
> > > > This repeats by any test.
>
> > > > This are the test statistics:
> > > > Number of messages: =A0 800
> > > > average time: =A0 =A046.258 s
> > > > minimum time: =A014.416 s
> > > > maximum time: 671.706 s
> > > > throughput: =A0 =A0 =A0 38.215/s
>
> > > > Can some body confirm this behaviour of the Diamete= r stack?
> > > > The overhead of 500ms per request is huge.
> > > > Until now I'm not able to find why mobicents se= rver have this
> > > > behaviour, please help!
>
> > > > Here is a full list of response times:
> > > > =A0 =A0 =A0# - response time
> > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D
> > > > =A0 =A0 =A0 1 =A018.302
> > > > =A0 =A0 =A0 2 =A016.414
> > > > =A0 =A0 =A0 3 =A016.774
> > > > =A0 =A0 =A0 4 =A016.698
> > > > =A0 =A0 =A0 5 =A017.308
> > > > =A0 =A0 =A0 6 =A016.552
> > > > =A0 =A0 =A0 7 =A015.817
> > > > =A0 =A0 =A0 8 =A016.695
> > > > =A0 =A0 =A0 9 =A016.017
> > > > =A0 =A0 =A010 =A015.925
> > > > =A0 =A0 =A011 =A016.441
> > > > =A0 =A0 =A012 =A015.995
> > > > =A0 =A0 =A013 =A015.654
> > > > =A0 =A0 =A014 =A016.186
> > > > =A0 =A0 =A015 =A016.223
> > > > =A0 =A0 =A016 =A018.200
> > > > =A0 =A0 =A017 =A016.788
> > > > =A0 =A0 =A018 =A016.194
> > > > =A0 =A0 =A019 =A017.062
> > > > =A0 =A0 =A020 =A017.996
> > > > =A0 =A0 =A021 =A022.241
> > > > =A0 =A0 =A022 =A022.005
> > > > =A0 =A0 =A023 =A020.659
> > > > =A0 =A0 =A024 =A022.648
> > > > =A0 =A0 =A025 =A021.822
> > > > =A0 =A0 =A026 =A018.615
> > > > =A0 =A0 =A027 =A021.826
> > > > =A0 =A0 =A028 =A042.799
> > > > =A0 =A0 =A029 =A021.094
> > > > =A0 =A0 =A030 =A019.140
> > > > =A0 =A0 =A031 =A041.743
> > > > =A0 =A0 =A032 =A055.097
> > > > =A0 =A0 =A033 =A079.565
> > > > =A0 =A0 =A034 =A0637.478
> > > > =A0 =A0 =A035 =A0671.706
> > > > =A0 =A0 =A036 =A0671.706
> > > > =A0 =A0 =A037 =A0608.713
> > > > =A0 =A0 =A038 =A0553.160
> > > > =A0 =A0 =A039 =A0492.073
> > > > =A0 =A0 =A040 =A0553.160
> > > > =A0 =A0 =A041 =A0492.073
> > > > =A0 =A0 =A042 =A0630.101
> > > > =A0 =A0 =A043 =A0373.144
> > > > =A0 =A0 =A044 =A0541.256
> > > > =A0 =A0 =A045 =A020.619
> > > > =A0 =A0 =A046 =A018.050
> > > > =A0 =A0 =A047 =A018.325
> > > > =A0 =A0 =A048 =A018.557
> > > > =A0 =A0 =A049 =A018.454
> > > > =A0 =A0 =A050 =A017.904
> > > > =A0 =A0 =A051 =A018.448
> > > > =A0 =A0 =A052 =A018.152
> > > > =A0 =A0 =A053 =A018.165
> > > > =A0 =A0 =A054 =A018.267
> > > > =A0 =A0 =A055 =A018.094
> > > > =A0 =A0 =A056 =A017.727
> > > > =A0 =A0 =A057 =A017.952
> > > > =A0 =A0 =A058 =A018.040
> > > > =A0 =A0 =A059 =A018.058
> > > > =A0 =A0 =A060 =A017.666
> > > > =A0 =A0 =A061 =A018.081
> > > > =A0 =A0 =A062 =A018.050
> > > > =A0 =A0 =A063 =A017.763
> > > > =A0 =A0 =A064 =A017.934
> > > > =A0 =A0 =A065 =A017.651
> > > > =A0 =A0 =A066 =A017.671
> > > > =A0 =A0 =A067 =A018.138
> > > > =A0 =A0 =A068 =A041.887
> > > > =A0 =A0 =A069 =A022.864
> > > > =A0 =A0 =A070 =A018.912
> > > > =A0 =A0 =A071 =A018.942
> > > > =A0 =A0 =A072 =A018.523
> > > > =A0 =A0 =A073 =A018.550
> > > > =A0 =A0 =A074 =A018.705
> > > > =A0 =A0 =A075 =A018.402
> > > > =A0 =A0 =A076 =A017.643
> > > > =A0 =A0 =A077 =A017.630
> > > > =A0 =A0 =A078 =A018.134
> > > > =A0 =A0 =A079 =A017.969
> > > > =A0 =A0 =A080 =A017.822
> > > > =A0 =A0 =A081 =A018.970
> > > > =A0 =A0 =A082 =A018.598
> > > > =A0 =A0 =A083 =A019.349
> > > > =A0 =A0 =A084 =A018.263
> > > > =A0 =A0 =A085 =A017.934
> > > > =A0 =A0 =A086 =A017.879
> > > > =A0 =A0 =A087 =A017.592
> > > > =A0 =A0 =A088 =A017.731
> > > > =A0 =A0 =A089 =A019.530
> > > > =A0 =A0 =A090 =A031.724
> > > > =A0 =A0 =A091 =A019.668
> > > > =A0 =A0 =A092 =A047.105
> > > > =A0 =A0 =A093 =A019.900
> > > > =A0 =A0 =A094 =A022.388
> > > > =A0 =A0 =A095 =A022.246
> > > > =A0 =A0 =A096 =A023.047
> > > > =A0 =A0 =A097 =A021.916
> > > > =A0 =A0 =A098 =A021.800
> > > > =A0 =A0 =A099 =A021.935
> > > > =A0 =A0 100 =A022.023
> > > > =A0 =A0 101 =A021.673
> > > > =A0 =A0 102 =A021.728
> > > > =A0 =A0 103 =A021.951
> > > > =A0 =A0 104 =A021.973
> > > > =A0 =A0 105 =A022.114
> > > > =A0 =A0 106 =A022.064
> > > > =A0 =A0 107 =A017.458
> > > > =A0 =A0 108 =A043.564
> > > > =A0 =A0 109 =A016.734
> > > > =A0 =A0 110 =A017.978
> > > > =A0 =A0 111 =A017.311
> > > > =A0 =A0 112 =A016.484
> > > > =A0 =A0 113 =A016.728
> > > > =A0 =A0 114 =A016.604
> > > > =A0 =A0 115 =A015.966
> > > > =A0 =A0 116 =A017.015
> > > > =A0 =A0 117 =A016.153
> > > > =A0 =A0 118 =A015.831
> > > > =A0 =A0 119 =A015.887
> > > > =A0 =A0 120 =A016.017
> > > > =A0 =A0 121 =A015.886
> > > > =A0 =A0 122 =A015.762
> > > > =A0 =A0 123 =A015.628
> > > > =A0 =A0 124 =A015.747
> > > > =A0 =A0 125 =A016.403
> > > > =A0 =A0 126 =A016.056
> > > > =A0 =A0 127 =A015.942
> > > > =A0 =A0 128 =A015.651
> > > > =A0 =A0 129 =A015.814
> > > > =A0 =A0 130 =A015.676
> > > > =A0 =A0 131 =A015.637
> > > > =A0 =A0 132 =A015.617
> > > > =A0 =A0 133 =A015.576
> > > > =A0 =A0 134 =A015.542
> > > > =A0 =A0 135 =A015.313
> > > > =A0 =A0 136 =A015.685
> > > > =A0 =A0 137 =A015.453
> > > > =A0 =A0 138 =A015.776
> > > > =A0 =A0 139 =A015.445
> > > > =A0 =A0 140 =A015.030
> > > > =A0 =A0 141 =A015.480
> > > > =A0 =A0 142 =A017.953
> > > > =A0 =A0 143 =A018.210
> > > > =A0 =A0 144 =A017.743
> > > > =A0 =A0 145 =A017.991
> > > > =A0 =A0 146 =A017.764
> > > > =A0 =A0 147 =A018.145
> > > > =A0 =A0 148 =A018.327
> > > > =A0 =A0 149 =A017.764
> > > > =A0 =A0 150 =A017.404
> > > > =A0 =A0 151 =A017.977
> > > > =A0 =A0 152 =A017.859
> > > > =A0 =A0 153 =A017.844
> > > > =A0 =A0 154 =A018.215
> > > > =A0 =A0 155 =A018.536
> > > > =A0 =A0 156 =A016.214
> > > > =A0 =A0 157 =A015.246
> > > > =A0 =A0 158 =A016.121
> > > > =A0 =A0 159 =A015.314
> > > > =A0 =A0 160 =A016.690
> > > > =A0 =A0 161 =A015.374
> > > > =A0 =A0 162 =A016.029
> > > > =A0 =A0 163 =A015.717
> > > > =A0 =A0 164 =A017.630
> > > > =A0 =A0 165 =A016.163
> > > > =A0 =A0 166 =A016.218
> > > > =A0 =A0 167 =A016.712
> > > > =A0 =A0 168 =A016.265
> > > > =A0 =A0 169 =A016.400
> > > > =A0 =A0 170 =A016.087
> > > > =A0 =A0 171 =A015.565
> > > > =A0 =A0 172 =A016.318
> > > > =A0 =A0 173 =A016.249
> > > > =A0 =A0 174 =A015.764
> > > > =A0 =A0 175 =A016.052
> > > > =A0 =A0 176 =A015.599
> > > > =A0 =A0 177 =A015.874
> > > > =A0 =A0 178 =A015.758
> > > > =A0 =A0 179 =A015.812
> > > > =A0 =A0 180 =A015.619
> > > > =A0 =A0 181 =A015.790
> > > > =A0 =A0 182 =A015.657
> > > > =A0 =A0 183 =A015.305
> > > > =A0 =A0 184 =A015.363
> > > > =A0 =A0 185 =A015.221
> > > > =A0 =A0 186 =A016.509
> > > > =A0 =A0 187 =A016.030
> > > > =A0 =A0 188 =A015.914
> > > > =A0 =A0 189 =A015.830
> > > > =A0 =A0 190 =A015.622
> > > > =A0 =A0 191 =A016.066
> > > > =A0 =A0 192 =A015.919
> > > > =A0 =A0 193 =A017.392
> > > > =A0 =A0 194 =A018.002
> > > > =A0 =A0 195 =A017.794
> > > > =A0 =A0 196 =A017.876
> > > > =A0 =A0 197 =A018.539
> > > > =A0 =A0 198 =A017.936
> > > > =A0 =A0 199 =A018.233
> > > > =A0 =A0 200 =A018.222
> > > > =A0 =A0 201 =A017.081
> > > > =A0 =A0 202 =A018.208
> > > > =A0 =A0 203 =A015.874
> > > > =A0 =A0 204 =A017.520
> > > > =A0 =A0 205 =A018.009
> > > > =A0 =A0 206 =A016.699
> > > > =A0 =A0 207 =A016.570
> > > > =A0 =A0 208 =A015.687
> > > > =A0 =A0 209 =A015.594
> > > > =A0 =A0 210 =A015.393
> > > > =A0 =A0 211 =A015.150
> > > > =A0 =A0 212 =A015.697
> > > > =A0 =A0 213 =A015.174
> > > > =A0 =A0 214 =A015.437
> > > > =A0 =A0 215 =A015.197
> > > > =A0 =A0 216 =A015.441
> > > > =A0 =A0 217 =A015.221
> > > > =A0 =A0 218 =A016.466
> > > > =A0 =A0 219 =A016.407
> > > > =A0 =A0 220 =A017.072
> > > > =A0 =A0 221 =A016.832
> > > > =A0 =A0 222 =A016.266
> > > > =A0 =A0 223 =A016.231
> > > > =A0 =A0 224 =A015.798
> > > > =A0 =A0 225 =A015.620
> > > > =A0 =A0 226 =A016.011
> > > > =A0 =A0 227 =A015.901
> > > > =A0 =A0 228 =A016.378
> > > > =A0 =A0 229 =A015.669
> > > > =A0 =A0 230 =A016.269
> > > > =A0 =A0 231 =A016.307
> > > > =A0 =A0 232 =A041.461
> > > > =A0 =A0 233 =A016.147
> > > > =A0 =A0 234 =A017.885
> > > > =A0 =A0 235 =A016.682
> > > > =A0 =A0 236 =A016.399
> > > > =A0 =A0 237 =A0553.132
> > > > =A0 =A0 238 =A0511.397
> > > > =A0 =A0 239 =A0511.397
> > > > =A0 =A0 240 =A0511.397
> > > > =A0 =A0 241 =A0437.126
> > > > =A0 =A0 242 =A0437.126
> > > > =A0 =A0 243 =A0342.989
> > > > =A0 =A0 244 =A0437.126
> > > > =A0 =A0 245 =A0222.916
> > > > =A0 =A0 246 =A0222.916
> > > > =A0 =A0 247 =A0384.531
> > > > =A0 =A0 248 =A0384.531
> > > > =A0 =A0 249 =A018.349
> > > > =A0 =A0 250 =A016.321
> > > > =A0 =A0 251
>
> ...
>
> read more =BB

--e89a8f234ce5c4242904be6ea9a5--