Re: document structure question

[Chris Marisic]

You're getting SelectMany isn't supported from the client, not from the server in this scenario.

    Map = metricentries => from metricentry in metricentries

                                       from metric in metricentry.metrics

                                       select new { metric.host, metric.name, metric.value }

Is fine.

Side note  DatabaseCommands.PutIndex is substantially more complicated than you need. In normal practice you inherit from AbstractIndexCreationTask

  var results =  session.Query<MetricEntry>(@"MetricEntries/Metrics")

                             .Where(me => me.Metrics.Any(m=> m.Host == "r710-08" && m.Name == "Idle Worker Threads")

                              .ToList()

Note my usage of Any to dive deeper into the object graph.


On Thursday, May 23, 2013 9:44:38 AM UTC-4, mag...@gmail.com wrote:

Rather new to document databases.  I'm toying around with storing something currently stored in a SQL database and I'm not sure of the best way to store it as a document.  Any comments will be helpful.

The format I'm trying now is:

{

level_key: 2927270,

sample_dt: 2013-05-21 15:24:20.929,

metrics:

[

{host="ex01dbadm01",

name="CPU User %",

value=95.23},

{host="ex01dbadm02",

name="CPU User %",

value=99.24},

{host="ex01dbadm01",

name="CPU System %",

value=2},

{host="ex01dbadm02",

name="CPU System %",

value=3.0},

{host="ex01dbadm01",

name="CPU IO Wait %",

value=2.2},

{host="ex01dbadm01",

name="Running Oracle Processes",

value=302},

{host="ex01dbadm01",

name="Shared Memory MB",

value=40002.123},

]

}

 

I created two C# classes for this structure:

class Metric

    {

        public string host { get; set; }

        public string name { get; set; }

        public double value {get; set; }

    }

    class MetricEntry

    {

        public int level_key { get; set; }

        public DateTime sample_dt { get; set; }

        public List<Metric> metrics { get; set; }

    }

Relationships:

• Level_key is an an identifier of a test and will later be used to link out to a bunch of static information.
• There are generally hundreds of sample_dts per level_key
• There are generally less than 10 unique hosts per level_key
• There are generally hundreds of metrics per host and sample_dt

Ideally I'd be able to run a query like this to get the value of a metric for a specific host and level_key:

var results = from metricentry in session.Query<MetricEntry>()

                              where metricentry.level_key == 2888656

                              from metric in metricentry.metrics

                              where metric.host == "r710-08" && metric.name == "Idle Worker Threads"

                              select new { metric.host, metric.name, metric.value };

However this results in a a SelectMany not implemented.  Lots of stuff out there are on the web about that with Raven... all say that to do this sort of thing you need to create an index with a Map function, which makes sense to me.  So here is where I get lost. The documentation and examples all assume a high level of familiarity with both Linq and with Map/Reduce style queries and I've got only a very basic conception of both.  I created this index:

static void RavenCreateIndex()

        {

            DocumentStore docStore = new DocumentStore { Url = "http://big-data-2:8080/", DefaultDatabase = "test" };

            docStore.Initialize();

            docStore.DatabaseCommands.PutIndex("MetricEntries/Metrics", new IndexDefinitionBuilder<MetricEntry>

            {

                Map = metricentries => from metricentry in metricentries

                                       from metric in metricentry.metrics

                                       select new { metric.host, metric.name, metric.value }

            });

            docStore.Dispose();

        }

But I'm at a loss for how to actually query the index.  I tried basically using the same query that tosses the SelectMany not implemented exception, but it still tosses it (as I assumed it would).

using (IDocumentSession session = docStore.OpenSession())

            {

                var results = from metricentry in session.Query<MetricEntry>(@"MetricEntries/Metrics")

                              from metric in metricentry.metrics

                              where metric.host == "r710-08" && metric.name == "Idle Worker Threads"

                              select metric;

                foreach (Metric m in results)

                {

                    Console.WriteLine(m.host + "\t" + m.name + "\t" + m.value.ToString());

                }

            }

So I guess two questions here:

1. Am I structuring the document appropriately given the data content?
2. If the answer to #1 is correct, where am I going wrong with the index creation or querying?

Sorry for the long post, but I wanted to make sure I gave enough information to help.  Thanks. 

 

[Chris Marisic]

Note you may be able to define your index as

Map = metricentries => from metricentry in metricentries

                                       select new { metricentry.Metrics, metricentry.level_key, metricentry.sample_dt  }

[mag...@gmail.com]

When I try using your query against the index I get an error that metrics_host isn't indexed.

System.ArgumentException: The field 'metrics_host' is not indexed, cannot query on fields that are not indexed

I feel like I'm missing something fundamental here. 

[Kijana Woodard]

Looks like the index is metric with no "s".

Go to the index in raven studio and check out the fields. You can see what they are being named. You might try naming it explicitly as metrics_host = metric.host.

--
You received this message because you are subscribed to the Google Groups "ravendb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[mag...@gmail.com]

Everywhere I can see metrics has an s on it.  How did it get defined as just metric?  Where do I explicitly name the fields?  In the studio, if I go to: Indexes -> MetricEntries/Metrics -> Terms, I see value, host, name, and __document_id.

[Kijana Woodard]

Oh I was scanning, thought I was looking at your index because of the query comprehension syntax. For whatever reason, most of us use QCS for indexes and fluent for query.

var results = from metricentry in session.Query<MetricEntry>()

                              where metricentry.level_key == 2888656

                              from metric in metricentry.metrics

                              where metric.host == "r710-08" && metric.name == "Idle Worker Threads"

                              select new { metric.host, metric.name, metric.value };

In your query, you're filtering by metric_host which isn't defined in the index. 

[mag...@gmail.com]

I thought including it here like this puts it in the index?

docStore.DatabaseCommands.PutIndex("MetricEntries/Metrics", new IndexDefinitionBuilder<MetricEntry>

            {

                Map = metricentries => from metricentry in metricentries

                                       from metric in metricentry.metrics

                                       select new { metric.name, metric.host, metric.value }

            });

[Kijana Woodard]

ha. yeah. scanned right over that again. so does that come out as "metric_host" in studio?

[Kijana Woodard]

if not, change the index to metric_host = metric.host

I usually am explicit about the Map names despite R# grumblings.

[mag...@gmail.com]

If I go in and explicitly set the names to be "metrics_host" and "metrics_name" in studio, it works.  The why might be interesting to understand at a later date :)

This is returning all metrics that match the host instead of just the metrics that match the metric name.  I think it is actually returning any MetricEntry that has a matching metric.  Since the MetricEntry has all the metrics in it, they all come back.  Unless you know what I'm doing wrong off the top of your head, i'll just play around for a bit to get a better understanding.

var results = session.Query<MetricEntry>(@"MetricEntries/Metrics")

                    .Where(me => me.metrics.Any(m => m.host == "r710-08" && m.name == "Idle Worker Threads")).ToList();