Slow

[Julio Albuquerque]

Hi.
Use Xamarin Android and Couchbase.Lite.Net (C#).
I have a database (Couchbase Lite .Net) 14,000 documents.
I created a view that returns 152 documents.
I would like to ask how best to work with the result of this query?
After running the query and the QueryEnumerator be with the 152 lines, I'm taking 30 seconds to retrieve the data.
Someone can look where I am going wrong?

var vw = database.GetExistingView (tpTitulo);
             qry = vw.CreateQuery ();
             qry.IndexUpdateMode = IndexUpdateMode.Never;
             lines = qry.Run ();

From here it takes 30 seconds!
 

            foreach (var line in lines) {
                 var telefone = (JArray)line.Document.GetProperty ("fone");
                 objListaTelefonica.Add (new Cadastro () {
                     Nome = line.Document.GetProperty ("nome").ToString (),
                     Razao = line.Document.GetProperty ("razao").ToString (),
                     Logradouro = line.Document.GetProperty ("logradouro").ToString (),
                     Nr = line.Document.GetProperty ("nr").ToString (),
                     Complemento = line.Document.GetProperty ("complemento").ToString (),
                     Bairro = line.Document.GetProperty ("bairro").ToString (),
                     Cidade = line.Document.GetProperty ("cidade").ToString (),
                     Cep = line.Document.GetProperty ("cep").ToString (),
                     Produto = line.Document.GetProperty ("produto").ToString (),
                     Fone = telefone [0].Value<string> ("numero")
                 });
             }

[Jens Alfke]

That time is probably being spent indexing the view. Can you show us the map function the view is using? Most likely the map function is too slow.

—Jens

[Julio Albuquerque]

But it is not in query.Run () that should take? Not in query.Run () that runs the Map Function?
In line foreach (var line in lines) that will run the Map Function?

Now I'm confused!
Below the map function:

                vwrestaurantes.SetMap ((document, emitter) => {
                     if ((document ["blativo"].ToString () == "1") && (document ["type"].ToString () == "E")) {
                         var titulos = (JArray)document ["titulos"];
                         for (int i = 0; i < titulos.Count; i++) {
                             if (ArrayTitulos.Any<string> ((Tools.RemoverAcentos (titulos [i].Value<string> ("descricao").ToLower ()).Trim ()).Equals)) {
                                 emitter (Tools.RemoverAcentos (titulos [i].Value<string> ("descricao").ToLower ()).Trim (), document ["nome"]);
                             }
                         }
                     }
                 }, ver);

[Jens Alfke]

On Jan 26, 2015, at 2:34 PM, Julio Albuquerque <jcezar.al...@gmail.com> wrote:

But it is not in query.Run () that should take? Not in query.Run () that runs the Map Function?
In line foreach (var line in lines) that will run the Map Function?

No. The first thing Query.Run does is check if the index is out of date. If so, it updates the index. This is done by fetching every document that's been modified (or added) since the index was last updated, running it through the map function, and adding the emitted key/value pairs to the index. (Also, obsolete key/value pairs are removed.)

Then it queries the index and returns all of the docs that match your query settings.

Your map function looks reasonable, but it's doing a fair amount of work. I'd suggest profiling that long delay and seeing where the time is being spent. You may be able to optimize the map function.

Actually I just noticed one obvious optimization — you're evaluating this expression twice, and it's probably expensive because it sounds like it's doing Unicode string manipulation:

Tools.RemoverAcentos (titulos [i].Value<string>("descricao").ToLower ()).Trim ()

—Jens

[Julio Albuquerque]

Jens,
The Map function was as short as possible, but still continues at the same time (30 seconds).
see:

                vwrestaurantes.SetMap ((document, emitter) => {
                     if ((document ["blativo"].ToString () == "1") && (document ["type"].ToString () == "E")) {
                         var titulos = (JArray)document ["titulos"];
                         for (int i = 0; i < titulos.Count; i++) {

                             if (titulos [i].Value<string> ("descricao").ToLower ().Equals ("LANCHERIAS")) {
                                 emitter (titulos [i].Value<string> ("descricao").ToLower (), document ["nome"]);
                             }
                         }
                     }
                 }, ver);

This time (30 seconds) is being spent in the code below:

            foreach (var line in lines) {
                 var telefone = (JArray)line.Document.GetProperty ("fone");
                 objListaTelefonica.Add (new Cadastro () {
                     Nome = line.Document.GetProperty ("nome").ToString (),
                     Razao = line.Document.GetProperty ("razao").ToString (),
                     Logradouro = line.Document.GetProperty ("logradouro").ToString (),
                     Nr = line.Document.GetProperty ("nr").ToString (),
                     Complemento = line.Document.GetProperty ("complemento").ToString (),
                     Bairro = line.Document.GetProperty ("bairro").ToString (),
                     Cidade = line.Document.GetProperty ("cidade").ToString (),
                     Cep = line.Document.GetProperty ("cep").ToString (),
                     Produto = line.Document.GetProperty ("produto").ToString (),

                     Fone = telefone [0].Value<string> ("numero").ToString ()
                 });
             }

I do not understand what this code above has to do with the map function, because the query has been executed.

[Jens Alfke]

On Jan 27, 2015, at 10:35 AM, Julio Albuquerque <jcezar.al...@gmail.com> wrote:

This time (30 seconds) is being spent in the code below:

Oh, I see. When you wrote “From here it takes 30 seconds” I thought you meant from the qry.Run() call to the next line.

I can’t tell from the loop what would make it that slow. As I said last time, you should profile the code.

A couple of points on the code:

* You’re calling line.Document over and over again. DRY! You should get the document once and store it in a variable.

* When you design a view you should emit the document properties you need as values. If you dereference the document from a query row, it causes another database lookup to load the document. So your map function should be emitting properties like “nome”, “razao”, etc.

—Jens

[Julio Albuquerque]

Jens,
His tips were brilliant!

Could you help me with one last tip?
How emiter multiple values with the platform I'm using? (Xamarin Studio Android and Couchbase Lite .Net)

[Jens Alfke]

On Jan 27, 2015, at 10:59 AM, Julio Albuquerque <jcezar.al...@gmail.com> wrote:

His tips were brilliant!

Thanks! Does this mean you got the code to run faster already? What did you change?

How emiter multiple values with the platform I'm using? (Xamarin Studio Android and Couchbase Lite .Net)

Wrap the values in an array or dictionary/map, and use that as the 2nd parameter to emit().

Sorry, I can’t show you code because I barely know any C#.

—Jens

[Julio Albuquerque]

Jens,
I could leave with lower response time than 1 second!
I changed the Function Map to issue an object with all the values that need.
But what was really consuming was the way the List <T> was being built, dynamically reallocating memory.
Now I am creating with the constructor Capacity in which fill with QueryEnumerator.Count, as limiting the ability of the List <T>.
Retrieve the document as a JSON object and some other things.
Here's how the code looks:

                vwMercados.SetMap ((document, emitter) => {

                     if ((document ["blativo"].ToString () == "1") && (document ["type"].ToString () == "E")) {
                         var titulos = (JArray)document ["titulos"];
                         for (int i = 0; i < titulos.Count; i++) {

                             if (titulos [i].Value<string> ("descricao").Equals ("MERCADOS E SUPERMERCADOS")) {
                                 var Cad = new Cadastro ();
                                 Cad.Nome = document ["nome"].ToString ();
                                 Cad.Razao = document ["razao"].ToString ();
                                 Cad.Bairro = document ["bairro"].ToString ();
                                 Cad.Cep = document ["cep"].ToString ();
                                 Cad.Cidade = document ["cidade"].ToString ();
                                 Cad.Complemento = document ["complemento"].ToString ();
                                 Cad.Logradouro = document ["logradouro"].ToString ();
                                 Cad.Nr = document ["nr"].ToString ();
                                 Cad.Produto = document ["produto"].ToString ();
                                 Cad.Fone = document ["fone"].ToString ();
                                 emitter (titulos [i].Value<string> ("descricao"), Cad);
                             }
                         }
                     }
                 }, ver);

private void carrega (out List<Cadastro> obj)
         {

             var vw = database.GetExistingView (tpTitulo);
             qry = vw.CreateQuery ();
             qry.IndexUpdateMode = IndexUpdateMode.Never;
             lines = qry.Run ();

 
             var lstRetorno = new List<Cadastro> (lines.Count);
 
             for (int i = 0; i < lines.Count; i++) {
                 var Cad = JObject.Parse (lines.GetRow (i).Value.ToString ());
                 //var telefone = (JArray)Cad.Fone;
                 lstRetorno.Add (new Cadastro () {
                     Nome = Cad.Property ("Nome").Value.ToString (),
                     Razao = Cad.Property ("Razao").Value.ToString (),
                     Logradouro = Cad.Property ("Logradouro").Value.ToString (),
                     Nr = Cad.Property ("Nr").Value.ToString (),
                     Complemento = Cad.Property ("Complemento").Value.ToString (),
                     Bairro = Cad.Property ("Bairro").Value.ToString (),
                     Cidade = Cad.Property ("Cidade").Value.ToString (),
                     Cep = Cad.Property ("Cep").Value.ToString (),
                     Produto = Cad.Property ("Produto").Value.ToString ()
                     //Fone = telefone [0].Value<string> ("numero").ToString ()
                 });
             }
             obj = lstRetorno.GroupBy (s => s.Nome).Select (s => s.First ()).OrderBy (o => o.Nome).ToList<Cadastro> ();
         }

It was very fast!