RecordMapper question

[ara...@gmail.com]

Hi, 

This all seems a lot simpler when I started but now that I have wade into the deep end I think I maybe incorrect in some assumptions. 

I have a List<CSVItems> (CSVItems has 30+ members of various types) which I have gotten via parsing a very complicated CSV file using SuperCSV (I do know jOOQ has it's CSV importer, but needed SuperCSV). 

Now I need to map these CSVItems to a table within the jooq generated structure which has about 50 members.   While importing the data I need to apply a bunch of business rules to the data which CRUDs data List members to fit into the table. 

Is RecordMapper the right solution for this?

Aram

[Samir Faci]

You could just create a bean and add any extra logic you need when constructing the object.

See:  http://www.jooq.org/doc/3.8/manual/sql-execution/fetching/pojos/   (I'm mostly using the "immutable" pojos pattern)

I tend to have objects that represent the data in the strutted format I'll be returning as a response (REST services).  You can then read the data in and 

apply any manipulation, validation and such you need when you're constructing the object. 

for example.  I had a use case where an external ID was being read in as a string, validation that it's numeric and greater then 0 occurs in the bean and it's not it throws an exception marking it 

as an invalid Data state.

 Alternatively you can introduce a service that does this.  It's just a mater of preference if you want your pojo to do this work or a related service responsible for transformation from data type A to data type B.

--

Samir Faci

--
You received this message because you are subscribed to the Google Groups "jOOQ User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jooq-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Thank you
Samir Faci

[Aram Mirzadeh]

Thank you for that.  My scenario is a bit different as I already have the List of POJOs in hand since they're being parsed from a file rather than reading them via jOOQ. 

I also want to apply the business rules before persisting the data to the database.  At this point I think one of the generic mappers like ModelMapper maybe a better option for me.

Thank you.

Aram

[Lukas Eder]

I think that jOOQ's importing API is the right API for you to choose:

http://www.jooq.org/doc/latest/manual/sql-execution/importing

But I believe that you should no longer think of your data in terms of CSV, once you pass it to jOOQ. You have already parsed your CSV data into your List<CSVItems> data structure. Now, you can use Java to run business logic upon that data structure. It will be easy to transform that data structure into any of :

- Simple arrays

- jOOQ records

In order to import those via one of the aforementioned APIs.

From what I can tell, you're doing this for only one (or only a few) tables. So I suspect that automatic mapping will not be that much better than manual wiring in this case. But perhaps, I have not yet fully understood your use-case.

Does this help?

Lukas

[Aram Mirzadeh]

Hi Lukas, 

I would have loved to use jOOQ's CSV importer ... sadly the data is a little too finiky and after spending a bit of time trying to get it to import directly into Table I gave up and went back to SuperCSV.  

No you're right as far as doing it manually vs. a mapper is probably not much of a time saver but it's easier to read IMHO.  

Thank you very much for the insight.

Aram

[Lukas Eder]

2016-06-12 9:34 GMT+02:00 Aram Mirzadeh <ara...@gmail.com>:

No you're right as far as doing it manually vs. a mapper is probably not much of a time saver but it's easier to read IMHO.  

Well, there's a tradeoff... Readability and complex business logic. I suspect you'll need to pick one :)

Of course, if you're interested in sharing a bit more about that business logic, perhaps, there's a low hanging fruit where I could help find a simple solution, or perhaps improve the jOOQ APIs... Just let me know if you want to push further on this.

Lukas

[Aram Mirzadeh]

It isn't anything abnormal, just a lot of processing.  BIRT was considered at one point.

The business logic layer consists of reading each item (CSVItem row - a row from a sales entry system)

1 & 2) Normalizing the values in some fields.

3) Finding the monies conversation equivalence of cost and price to a single currency.

5) Applying 200-230 regex expressions to exclude some line items.- concat results of rules in a field.

6) Applying 30-40 regex expressions to to assign "item category"- set result in a field

7) Persist to the database.(all items)

The final step involves building some views using GROUP_CONCAT of the country/item categories of "accepted items" to do some pivot tables in an excel workbook.

Aram

[Lukas Eder]

Hi Aram,

Sorry for the delay, and thanks for sharing your example.

Indeed, it's not out of the ordinary, yet it is a bit complex. As I thought before, this doesn't lend itself to a set of new features in the jOOQ loader API. I think it is best if the loader API just processes records that are already in the final form (give or take some data type conversions from number to string or vice versa).

With jOOQ 3.8, there are now loadArrays(Stream) and loadRecords(Stream) methods, which consume a Java 8 Stream. This could be useful to pass a Stream pipeline directly to the loader, which then executes it to consume individual rows.

But you can of course do everything "outside" of jOOQ as you're doing now.

Lukas

--