High volume message processing in BizTalk 2006 R2, batch, de-batch
bubul
I have a situation where I need to process a heavy file which would look like
<Header></Header>
<Body>
<Detail />
<Detail />
<Detail />
...........
..........
<Detail />
</Body>
< Tailor ></Tailor >
Number of <Detail> node is variable. This may vary from 4000 to 10000,
though which not much data in it. Avg file size in 5-10MB. I could have
easily debatch it and processed. But I have another set of requirement which
says the processed message should be in the same file. The output message
would look like pretty similar to the input with certain extra fields and
some calculations, that would appear in the <tailor>.
One easy solution is looping, but I want to avoid that as number of
iteration would be 2000-100000. Can you suggest any pattern that would fit in
this kind of situation.
Here are the points of worry
1. every <Deatail> need some database operation. If, debatched, how would I
ensure that all messages are processed before writing the consolidated message
2.Some of <Deetail> message may fail. In that case the o/p would not be
generated
2.I need to stop double posting of same <Detail>
It would be great if you can help me in getting the design
Thanks in advance :)