Anonymous functions in Reduce, an exploration in what's possible

[mrmoosehead]

Caveats: this might be better done with a server plugin for the rounding function, but using a hosted raven taht won't support this at the moment. I figured I could put the code inline. 

I could make the code more complicated and do the aggregation calc multiple times where needed, but I figured this would be neater. 

Should this work? I'm getting a server 500 error. 

(The aggregation of the HIGValue is last entry per day per user, averaged over the month, but with a custom rounding function)

XXX, YYY, ZZZ are other object types in the db that need kpi aggregations, but these are simple counts per month.  

namespace xxx.Indexes

{

    public class KPIIndex : AbstractMultiMapIndexCreationTask<KPIIndex.Result>

    {

        public class Result

        {

            public string UserId { get; set; }

            public int xxxCount { get; set; }

public int yyyCount { get; set; }

public int zzzCount { get; set; }

            public int HigValue { get; set; }

            public int HigCounter { get; set; }

            public int Month { get; set; }

            public int Year { get; set; }

            public int YearMonth { get; set; }

        }

      

 

    public override string IndexName

    {

        get

        {

            return "HRProcessKPI";

        }

    }

    public KPIIndex()

    {

        //add maps for xxx, yyy and zzz with the appropraite count of 1 for each instance

        AddMap<XXX>(items => from xxx in items

                                select

                                    new Result

                                        {

                                            

                                            UserId = xxx.SubjectUser.UserId,

                                            xxxCount = 1,

                                            yyyCount = 0,

                                            zzzCount = 0,

                                            HigValue = -10,

                                            HigCounter = 0,

                                            Month = xxx.EventDate.Month,

                                            Year = xxx.EventDate.Year,

                                            YearMonth = (xxx.EventDate.Year * 100) + xxx.EventDate.Month

                                        });

        AddMap<YYY>(items => from yyy in items

                                select

                                    new Result

                                    {

                                        UserId = yyy.SubjectUser.UserId,

                                       xxxCount = 0,

                                            yyyCount = 1,

                                            zzzCount = 0,

                                        HigValue = -10,

                                        HigCounter = 0,

                                        Month = yyy.EventDate.Month,

                                        Year = yyy.EventDate.Year,

                                        YearMonth = (yyy.EventDate.Year * 100) + yyy.EventDate.Month

                                    });

        AddMap<ZZZ>(items => from zzz in items

                                 select

                                     new Result

                                     {

                                         UserId = zzz.Subject.UserId,

                                           xxxCount = 0,

                                            yyyCount = 0,

                                            zzzCount = 1,

                                         HigValue = -10,

                                         HigCounter = 0,

                                         Month = zzz.StartOfProcess.Month,

                                            Year = zzz.StartOfProcess.Year

                                            ,

                                         YearMonth = (zzz.StartOfProcess.Year * 100) + zzz.StartOfProcess.Month

                                     });

        AddMap<HIGEntry>(items => 

            

                                

                                    from hig in items 

                                    group hig by new {hig.Subject.UserId, hig.CaptureDate.Date} into g

                                    let MaxDayDatePerUser = g.Max(h => h.CaptureDate)

                                    from dayhig in g

                                    where dayhig.CaptureDate == MaxDayDatePerUser

                                    select new Result

                                     {

                                         UserId = dayhig.Subject.UserId,

                                         xxxCount = 0,

                                        yyyCount = 0,

                                         zzzCount = 0,

                                         HigValue = dayhig.Value,

                                         HigCounter = 1,

                                         Month = dayhig.CaptureDate.Month,

                                         Year = dayhig.CaptureDate.Year,

                                         YearMonth = (dayhig.CaptureDate.Year * 100) + dayhig.CaptureDate.Month

                                     }

                            );

        

        //the reduce function groups by the month, userid and year and then sums each count

        Reduce = results => from result in results

                            group result by new { result.Month, result.UserId, result.Year } into g

                            select new Result

                            {

                                UserId = g.Key.UserId,

                                xxxCount = g.Sum(x => x.xxxCount),

                                yyyCount = g.Sum(x => x.yyyCount),

                                zzzCount = g.Sum(x => x.zzzCount),

                                HigValue = 

                                    new Func<double, int>(unrounded => (((int) Math.Floor(Math.Abs(unrounded))) +

                                                                                ((Math.Abs(unrounded)%1) >= 0.25 ? 1 : 0)) * ((unrounded < 0) ? -1 : 1)

                                                              

                                    ).Invoke(

                                                    (double)g.Aggregate(0, (a, b) => b.HigValue > -10 ? a + b.HigValue : a) / (double)g.Sum(h => h.HigCounter)

                                                ), 

                                HigCounter = g.Sum(h => h.HigCounter),      

                                Month = g.Key.Month,

                                Year = g.Key.Year,

                                YearMonth = (g.Key.Year * 100) + g.Key.Month

                            };

        }

    }

}

[Oren Eini (Ayende Rahien)]

We don't generally allow to define functions like that, no.
I don't understand what you are doing here, at any rate, and it looks a LOT more complicated than it needs be.

--
You received this message because you are subscribed to the Google Groups "ravendb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[mrmoosehead]

It might be more complicated than required. 

It's essentially an aggregation by month/year/user of certain values. Used the sum thing as count not allowed in indexes 

The aggregation of the hig value is non-trivial. 

[Oren Eini (Ayende Rahien)]

Try explaining this outside of raven, what is it that you are doing?

[mrmoosehead]

returning summarised values for number of XXX, number of YYY, number of ZZZ broken down by user/month/year for using on a graph. Graph will be rendered very often. 

the HIGVAlue is a special timestamped value that may occur 0-n times per day with possible values of -2, -1, 0, 1, 2 

The definition of the monthly aggregation per user is average of the last one recorded per day per user

However, this value is rounded from the quarters, e.g. -0.25 will round to -1, 1.4 will round up to 2, -0.24 will round to 0 etc. 

[Chris Marisic]

I can read the words you've typed, but i have absolutely zero idea what you said.

[mrmoosehead]

:D

Don't worry then. What matters is that I can't achieve what I want to in the index. 

[Chris Marisic]

In general, i recommend doing all math inside documents themselves inside a property with a getter.

Then you can use map reduce with simple aggregate methods Sum, Max, Min. Instead of trying to execute algebra inside the index.

[Kijana Woodard]

@Chris. That's what I think is causing me problems with my index on my other thread. However, I'm doing the math in the index because I'm using scripted patch to update the values, so the getter never would be updated.

[Chris Marisic]

Yeah you would need to load the document and resave the document, which pretty much defeats the purpose of using patch. Although perhaps not necessarily, because syncing the getter, wouldn't need to care about concurrency, you'd just throw the current doc away, load the newest version and save if you ever hit a concurrency exception.

[Kijana Woodard]

I'm considering going with a TaskExecutor. I'm also considering running said TaskExecutor on the app server instead of in the web server, but I didn't want to go to that level yet.