How do second group operation?

[yjinglee]

I post https://groups.google.com/forum/?hl=zh-CN#!topic/re-motion-users/8tlsaQKGLg4,

relinq execute SELECT [q0].[key] AS [Key],[q0].[a0] AS [MaxPrice] FROM (SELECT [t1].[CategoryID] AS [key], MAX([t1].[UnitPrice]) AS [a0] FROM [Products] AS [t1] GROUP BY [t1].[CategoryID]) AS [q0] sql on some db return,

but I think do second group operation for this datas at last return.

[Fabian Schmied]

Hi,

I'm sorry, I don't understand. Could you rephrase your question and
explain your problem in a more detailed way?

Best regards,
Fabian

> --
> You received this message because you are subscribed to the Google Groups
> "re-motion Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/re-motion-users/-/LX7HiHCTUN8J.
> To post to this group, send email to re-moti...@googlegroups.com.
> To unsubscribe from this group, send email to
> re-motion-use...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/re-motion-users?hl=en.

[yjinglee]

I'm Sorry,Fabian 

I mean is 

        public IEnumerable<T> ExecuteCollection<T>(QueryModel queryModel)

        {

            var results = GetResults<T>(queryModel);//this hit db each connections

            var queryDelegate= QueryModelVisitor.GenerateDelegate(queryModel);//resolving resultoperator generate delegate

            if (queryDelegate != null)

                return (IEnumerable<T>)queryDelegate.DynamicInvoke(results.AsQueryable()); //this do in memory all resultoperator Client Side Eval

            return results;

        }

        private IEnumerable<T> GetResults<T>(QueryModel queryModel)

        {

            ArgumentUtility.CheckNotNull("queryModel", queryModel);

            var commandData = GenerateSqlCommand(queryModel);

            var projection = commandData.GetInMemoryProjection<T>().Compile();

            var results = new List<T>();

            Parallel.ForEach(_connectionStrings, conn => results.AddRange(_resultRetriever.GetResults(projection, conn, commandData.CommandText, commandData.Parameters)));

            return results;

        }

in order to shard merge data features

some simple operatorresult , for example, the FirstOrDefault \ First \ SingleOrDefault \ Single very easily accomplished,  but complex operations such as GroupBy operator to achieve memory, I made reference to of NH ProcessNonAggregatingGroupBy class,

can you provide some suggestions

Thanks

[yjinglee]

 public class QueryModelVisitor : QueryModelVisitorBase

    {

        private static readonly ResultOperatorMap ResultOperatorMap;

        public IStreamedDataInfo CurrentEvaluationType { get; private set; }

        public IStreamedDataInfo PreviousEvaluationType { get; private set; }

        public QueryModel QueryModel { get; private set; }

        public IntermediateTree IntermediateTree { get; private set; }

        static QueryModelVisitor()

        {

            ResultOperatorMap = new ResultOperatorMap();

            ResultOperatorMap.Add<FirstResultOperator, ProcessFirst>();

            ResultOperatorMap.Add<SingleResultOperator, ProcessSingle>();

            ResultOperatorMap.Add<GroupResultOperator, ProcessGroupBy>();

        }

        internal QueryModelVisitor(QueryModel queryModel)

        {

            QueryModel = queryModel;

            IntermediateTree = new IntermediateTree();

        }

        private void Visit()

        {

            VisitQueryModel(QueryModel);

        }

        public override void VisitResultOperator(ResultOperatorBase resultOperator, QueryModel queryModel, int index)

        {

            PreviousEvaluationType = CurrentEvaluationType;

            CurrentEvaluationType = resultOperator.GetOutputDataInfo(PreviousEvaluationType);

            ResultOperatorMap.Process(resultOperator, this, IntermediateTree);

        }

        public override void VisitSelectClause(SelectClause selectClause, QueryModel queryModel)

        {

            CurrentEvaluationType = selectClause.GetOutputDataInfo();

            base.VisitSelectClause(selectClause, queryModel);

        }

        public static Delegate GenerateDelegate(QueryModel queryModel)

        {

            var visitor = new QueryModelVisitor(queryModel);

            visitor.Visit();

            ReWrite(visitor);

            return visitor.IntermediateTree.MergeLambdasAndCompile();

        }

        private static void ReWrite(QueryModelVisitor queryModelVisitor)

        {

            if (queryModelVisitor.QueryModel.ResultOperators.Count == 1 && queryModelVisitor.QueryModel.ResultOperators[0] is GroupResultOperator)

            {

                ResultOperatorMap.Process(queryModelVisitor.QueryModel.ResultOperators[0], queryModelVisitor, queryModelVisitor.IntermediateTree);

                return;

            }

            var subQueryExpression = queryModelVisitor.QueryModel.MainFromClause.FromExpression as SubQueryExpression;

            if ((subQueryExpression != null) &&

                (subQueryExpression.QueryModel.ResultOperators.Count == 1) &&

                (subQueryExpression.QueryModel.ResultOperators[0] is GroupResultOperator))

            {

                ResultOperatorMap.Process(subQueryExpression.QueryModel.ResultOperators[0], queryModelVisitor, queryModelVisitor.IntermediateTree);

            }

        }

    }

QueryModelVisitor  reference to NH 

在 2013年1月24日星期四UTC+8下午6时20分36秒，yjinglee写道：

[Fabian Schmied]

Hi,

> in order to shard merge data features
> some simple operatorresult , for example, the FirstOrDefault \ First \
> SingleOrDefault \ Single very easily accomplished, but complex operations
> such as GroupBy operator to achieve memory, I made reference to of NH
> ProcessNonAggregatingGroupBy class,
> can you provide some suggestions

Yes, you're right, it's very complex to merge the results of a DB
query and an in-memory query when a complex result operator like
GroupBy is involved. However, you'll have to implement an algorithm to
do this by yourself - at least I have never done it, so I cannot make
any suggestions. I also do not know NHibernate's
ProcessNonAggregatingGroupBy class.

One tip I can give you is to use the
ReverseResolvingExpressionTreeVisitor to get back in-memory
LambdaExpressions from the expressions held by the
GroupResultOperator, e.g., the KeyExpression, etc.

If you have a specific question about re-linq, I'll be glad to help.

Best regards,
Fabian

> --
> You received this message because you are subscribed to the Google Groups
> "re-motion Users" group.

> To post to this group, send email to re-moti...@googlegroups.com.
> To unsubscribe from this group, send email to
> re-motion-use...@googlegroups.com.

> Visit this group at http://groups.google.com/group/re-motion-users?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[yjinglee]

Hi, how use Dynamic Linq according to select clause

such as

            var select = new List<Basic>().GroupBy(p => p.AdSpaceId).Select(o => new { Id = o.Key, click = o.Sum(g => g.Clicks) });

            var results = select.ToList();

            results.Add(new { Id = (ulong)1, click = (ulong)2 });

            results.Add(new { Id = (ulong)1, click = (ulong)2 });

            results.Add(new { Id = (ulong)2, click = (ulong)2 });

            

            var entityType = results[0].GetType();

            var sourceType = typeof(IEnumerable<>).MakeGenericType(entityType);

            var arg = Expression.Parameter(entityType, "arg");

            var body = Expression.Property(arg, "Id");

            var delegateType = typeof(Func<,>).MakeGenericType(entityType, body.Type);

            var lambda = Expression.Lambda(delegateType, body, arg);

            var source = Expression.Parameter(sourceType, "source");

            var groupByExpression = Expression.Call(typeof(Enumerable), "GroupBy",

                                                    new[] { entityType, body.Type },

                                                    source, lambda);

           

            var groupByLambda = Expression.Lambda(groupByExpression, source).Compile();

            var groups = (IEnumerable<IGrouping<ulong,dynamic>>)groupByLambda.DynamicInvoke(results);

            //how use Dynamic Linq according to Select(o => new { Id = o.Key, click = o.Sum(g => g.Clicks)

            

            var lists = groups.Select(o => new { Id = o.Key, click = o.Sum(g => (ulong)g.click) }).ToList();

            foreach (var list in lists)

            {

                Console.WriteLine(list);

            }

Thanks.

[Fabian Schmied]

Well - very similar how you did it with the GroupBy clause:

var source = Expression.Constant (groups);
var selectorParameter = Expression.Parameter
(ReflectionUtility.TryGetItemTypeOfEnumerable (source.Type), "o");
var resultType = ...; // need to specify the anonymous type's ctor here
var resultTypeConstructor = ...;
var idProperty = resultType.GetProperty ("Id");
var clickProperty = resultType.GetProperty ("click");

var keyExpression = Expression.Property (selectorParameter, "Key");
var sumSelectorParameter = Expression.Parameter
(ReflectionUtility.TryGetItemTypeOfEnumerable
(selectorParameter.Type), "g");
var sumSelector = Expression.Lambda (Expression.Property
(sumSelectorParameter, "Clicks"), sumSelectorParameter);
var sumExpression = Expression.Call (typeof (Enumerable), "Sum",
selectorParameter, sumSelector);
var selector = Expression.Lambda (
Expression.New (resultTypeConstructor, new[] { keyExpression,
sumExpression }, new[] { idProperty, clickProperty }),
selectorParameter);
var selectCall = Expression.Call (typeof (Enumerable), "Select", new[]
{ source.Type }, source, selector);

Or something similar.

But this has nothing to do with re-linq, it's just about Expressions.

Best regards,
Fabian

[yjinglee]

var sumExpression = Expression.Call(typeof(Enumerable), "Sum", selectorParameter, sumSelector); is wrong...

        [Fact]

        public void Basic1Demo()

        {

            var select = new List<Basic>().GroupBy(p => p.AdSpaceId).Select(o => new { Id = o.Key, click = o.Sum(g => g.Clicks) });

            var results = select.ToList();

            results.Add(new { Id = (ulong)1, click = (ulong)2 });

            results.Add(new { Id = (ulong)1, click = (ulong)2 });

            results.Add(new { Id = (ulong)2, click = (ulong)2 });

            

            var entityType = results[0].GetType();

            var sourceType = typeof(IEnumerable<>).MakeGenericType(entityType);

            var arg = Expression.Parameter(entityType, "arg");

            var body = Expression.Property(arg, "Id");

            var delegateType = typeof(Func<,>).MakeGenericType(entityType, body.Type);

            var lambda = Expression.Lambda(delegateType, body, arg);

            var source1 = Expression.Parameter(sourceType, "source");

            var groupByExpression = Expression.Call(typeof(Enumerable), "GroupBy",

                                                    new[] { entityType, body.Type },

                                                    source1, lambda);

            var groupByLambda = Expression.Lambda(groupByExpression,source1).Compile();

            var groups = ((IEnumerable<IGrouping<ulong, dynamic>>)groupByLambda.DynamicInvoke(results));

            var source = Expression.Constant (groups); 

            var selectorParameter = Expression.Parameter (ReflectionUtility.TryGetItemTypeOfIEnumerable (source.Type), "o");

            var resultType = results[0].GetType(); // need to specify the anonymous type's ctor here 

            var resultTypeConstructor = results[0].GetType().GetConstructors()[0];

            var idProperty = resultType.GetProperty ("Id"); 

            var clickProperty = resultType.GetProperty ("click");

            var groupsource = Expression.Constant(groups);

            var keyExpression = Expression.Property(selectorParameter, "Key");

            var sumSelectorParameter = Expression.Parameter(selectorParameter.Type, "g");

            var sumSelector = Expression.Lambda(Expression.Property(sumSelectorParameter, "click"), sumSelectorParameter);

            var sumExpression = Expression.Call(typeof(Enumerable), "Sum", selectorParameter, sumSelector);

            var selector = Expression.Lambda(Expression.New(resultTypeConstructor, new[] { keyExpression, sumExpression }, new[] { idProperty, clickProperty }), selectorParameter);

            var selectCall = Expression.Call(typeof(Enumerable), "Select", new[] { groupsource.Type }, groupsource, selector);

            

            //how use Dynamic Linq according to select above

            var lists = groups.Select(o => new { Id = o.Key, click = o.Sum(g => (ulong)g.click) }).ToList();

            foreach (var list in lists)

            {

                Console.WriteLine(list);

            }

        }

在 2013年1月24日星期四UTC+8下午11时35分41秒，Fabian Schmied写道：

[Fabian Schmied]

> var sumExpression = Expression.Call(typeof(Enumerable), "Sum",
> selectorParameter, sumSelector); is wrong...

Yes, you''re right, I forgot to specify the value for the
"typeArguments" parameter.

> --
> You received this message because you are subscribed to the Google Groups
> "re-motion Users" group.

> To unsubscribe from this group and stop receiving emails from it, send an
> email to re-motion-use...@googlegroups.com.

>
> To post to this group, send email to re-moti...@googlegroups.com.