Re: Rhino ETL: join related data as a field instead of merge rows.

[Jason Meckley]

you will need to query the results from the database and then create an aggregate operations to build the data model.

On Tuesday, July 17, 2012 1:01:38 PM UTC-4, Remco Ros wrote:

Hi,

I'm looking at Rhino ETL to use it for our data export from SqlServer to Solr.

(I know Solr has DataImportHandlers, but we have some specific requirements (like aggregating multiple sources), so we cannot use them).

To post documents to solr, I need to pull data from several tables (ie. Product <-> ProductMaterial <-> Material) (and also: colors/manufacturer/category/and all kinds of other facets.)

and then map it to a class 'ProductIndex' which looks something like

Guid ProductId { get; set; }

IList<Guid> MaterialIds { get; set; } 

IList<string> Materials { get; set; }

I was thinking about implementing a process which pulls all relevant data out of SqlServer, transform it into a list of (Product) objects and post it to Solr.

I've studied some examples about joining, but what JoinOperation does is not really what I want.

I don't want all those table rows merged into one big resultset. The final result of the ETL should be a list of 'Product' classes (with the collection properties filled with id's and names) so that I can post them as one document to solr.

So the real question is: how do I fetch related product data and join it to the final result. Not as merged rows, but as a field of type IList.

in short:

input: flat product and facet data

output: list of class Product with properties for each facet.

Thanks in advance,

Remco