Looking for insight into why my scripted index isn't working as expected

[Josh Buedel]

Using Raven 3.0 embedded, I'm attempting to convert my index to a scripted index[2] that writes it's reduce results to documents in their own right. If fact, I am replacing this c# code:

         _store.Changes().ForIndex(new RollformingStatisticsIndex().IndexName)

            .Where(rs => rs.Type == IndexChangeTypes.ReduceCompleted).Subscribe(_ =>

         {

            using (var session = _store.OpenSession())

            {

               var rollformingStatistics = session.Query<RollformingStatistics, RollformingStatisticsIndex>().ToList();

               foreach (var rs in rollformingStatistics)

               {

                  session.Store(rs);

               }

               session.SaveChanges();

            }

         });

Q1: How can I tell that the ScriptedIndexResults bundle has been configured properly?  I expected to see some UI for IndexScript on the index in the management console, but I don't. I don't see anything related in Settings->Database Settings->Data either.

My index and delete script look like this (c#):

         IndexScript = @"var stat = LoadDocument(key); if (stat == null) return; PutDocument(key, this);";

         DeleteScript = @"var stat = LoadDocument(key); if(stat == null) return; delete key;";

Q2: Does this script seem correct? I'm not getting any errors (neither exceptions nor logged errors in the admin console). But not seeing results either.  

Q3: Where can I find breakdown of the functions, keywords, etc, available to the javascript in a scripted index? Stuff like what `this`, `key`, `delete`, and other non-standard js stuff is. I've been going off the examples in the scripted index docs[1].

Q4: Is there a better way? The whole reason I am storing my reduction result as documents is so I can both use them, and create another index that reduces them further.  I tried simply passing the reduced type as the TDocument type param of the AbstractScriptedIndexCreationTask but it didn't work. I considered coding that second index such that it does the same reduction plus further reduction. I'd be duplicating logic and the index crunching work though.

Thanks,

Josh

[1] https://ravendb.net/docs/article-page/3.0/csharp/server/bundles/scripted-index-results

[2] Here's my abridged index definition:

namespace AMS.Eclipse.Business.Data.Indexes

{

   public class RollformingStatisticsIndex : AbstractScriptedIndexCreationTask<ProductionEvent, RollformingStatistics>

   {

      public RollformingStatisticsIndex()

      {

         Map = prodEvents => from prodEvent in prodEvents

            select new RollformingStatistics

            { ...

            };

         Reduce = results => from result in results

            group result by new {result.MachineNumber, result.StartShiftCode}

            into g

            select new RollformingStatistics

            { ...

            };

         IndexScript = @"var stat = LoadDocument(key); if (stat == null) return; PutDocument(key, this);";

         DeleteScript = @"var stat = LoadDocument(key); if(stat == null) return; delete key;";

      }

   }

}

[Oren Eini (Ayende Rahien)]

When your db is setup with scripted index, it will show in the index page, like so:

If it is not there, the bundle isn't properly setup.

Q2) I'm guessing that you aren't setup with the bundle, the script looks good.

Q3) We support standard JS function and the lodash library, as well as any global custom functions you define.

Q4) Recursive map/reduce is done in this manner, yes.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Josh Buedel]

I'm at a loss here as to why the bundle is not set up correctly.

I have added "ScriptedIndexResults" to the "Raven/ActiveBundles" appsetting, both via app.config and programmatically (not at the same time). In both cases I see what I expect in my DocumentStore.Configuration.Settings value just before calling Initialize(). 

It looks right to me, yet my edit index page does not look like yours. 

Below is the code I use to set up the Raven DocumentStore singleton every time the app starts.

      public static IDocumentStore GetEmbeddedStore(string applicationDataPath)

      {

         var port = 58082;

         Raven.Database.Server.NonAdminHttp.EnsureCanListenToWhenInNonAdminContext(port);

         var store = new EmbeddableDocumentStore

         {

            DataDirectory = Path.Combine(applicationDataPath, "Data"),

            UseEmbeddedHttpServer = true,

            Conventions =

            {

               FindIdentityProperty = prop => prop.Name == "DocumentID",

               MaxNumberOfRequestsPerSession = 200

            },

            Configuration =

            {

               Port = port,

               MaxPageSize = 10000,

               MaxSecondsForTaskToWaitForDatabaseToLoad = 120 

            },

            DefaultDatabase = "Eclipse"

         };

         store.ActivateBundle("ScriptedIndexResults");

         store.Initialize();

         Business.RavenDbHelper.DeployIndexes(store);

         return store;

      }

      private static void ActivateBundle(this EmbeddableDocumentStore documentStore, string bundleName)

      {

         var settings = documentStore.Configuration.Settings;

         var activeBundles = settings[Constants.ActiveBundles];

         if (string.IsNullOrEmpty(activeBundles))

            settings[Constants.ActiveBundles] = bundleName;

         else if (!activeBundles.Split(';').Contains(bundleName, StringComparer.OrdinalIgnoreCase))

            settings[Constants.ActiveBundles] = activeBundles + ";" + bundleName;

      }

Please point out where I'm going wrong, if you can tell.

Thanks,

Josh

[Oren Eini (Ayende Rahien)]

You are setting this up for the _system_ database_, but use the Elipse Database.

[Josh Buedel]

I am beating my head against the wall here trying to get this to work. I want to enable ScriptedIndexResults bundle on a database named "Eclipse" in an EmbeddableDocumentStore.

My previous initialization code set the EmbeddableDocumentStore.DefaultDatabase property before calling Initialize[3]. This seemed to cause the "Eclipse" database to be created implicitly, if it didn't exist, which was fine. But I never could get that method to also enable "ScriptedIndexResults" bundle.  Is it possible? If so, is that the approach I should be pursuing?

So I switched to a method more like the example in the docs[1]. That is, I call EmbeddableDocumentStore.DatabaseCommand.GlobalAdmin.CreateDatabase explicitly. It works, in so far as it creates my "Eclipse" database and the bundle is enabled (the Studio's index editor shows index script and delete script editors at least).  Yet I then can't seem to change the default database from the system database. Setting the DefaultDatabase member after Initialize seems to have no effect. My indexes get deployed to system and my documents get written there as well. I suppose I could fix the documents by passing "Eclipse" to all OpenSession calls, though I don't see how to target the index deployments to the "Eclipse" database. If this approach is the right one, then how do I target those index deployments? And how would I check for database existence before calling CreateDatabase? Can I enable the bundle programatically on existing databases (as we have a handful of installs already)?

I'd be happy to rtfm if pointed to it. The docs on embedded version[2] don't talk about creating databases or enabling bundles. I feel like I must be missing something simple because it shouldn't be so hard. My complete code thus far is below[4].

Thanks again,

Josh

[1] http://ravendb.net/docs/article-page/3.0/csharp/server/bundles/scripted-index-results

[2] https://ravendb.net/docs/article-page/3.0/csharp/server/installation/embedded

[3] See my first message on this thread.

[4]       

      public static IDocumentStore GetEmbeddedStore(string applicationDataPath)

      {

         var port = 58082;

         Raven.Database.Server.NonAdminHttp.EnsureCanListenToWhenInNonAdminContext(port);

         var store = new EmbeddableDocumentStore

         {

            DataDirectory = Path.Combine(applicationDataPath, "Data"),

            UseEmbeddedHttpServer = true,

            Conventions =

            {

               // This allows us to create an `int Id` on our objects, which can be

               // pretty useful for client side apis.

               FindIdentityProperty = prop => prop.Name == "DocumentID",

               MaxNumberOfRequestsPerSession = 200

            },

            Configuration =

            {

               Port = port,

               MaxPageSize = 10000,

               MaxSecondsForTaskToWaitForDatabaseToLoad = 120 //Without this, index loading sometimes times out.

            },

            // This can't be here, it causes the db to be created implicitly, and without scripted index bundle.

//            DefaultDatabase = "Eclipse"

         };

         store.Initialize();

         store.DatabaseCommands.GlobalAdmin.CreateDatabase(new DatabaseDocument

         {

            Id = "Eclipse",

            Settings =

            {

               {"Raven/ActiveBundles", "ScriptedIndexResults"},

               {"Raven/DataDir", Path.Combine(applicationDataPath, "Eclipse")} // I get an error if I don't include this.

            }

         });

         

         // This seems to have no effect...

         store.DefaultDatabase = "Eclipse";

         // ... and then these indexes get deployed to the system db. All the docs go there too.

         Business.RavenDbHelper.DeployIndexes(store);

         return store;

[Oren Eini (Ayende Rahien)]

Use two document stores.

The first to do the init, which you then dispose.

The second to actual have the default db.