Large HL7 Message

[sue]

Hello,

 

In trying to receive a large HL7 message - about 100,000 characters- most of which are in tye OBX segenet at the end - there are 6 segmnets in all.

Its a type MDM_T02 with html data of the document in the OBX segment at the end.

 

Can Ensemble handle this size of message?

 

We are using the following

 

 set out = pRequest.OutputToString(pRequest.Separators, "")

 

where pRequest is  pRequest As EnsLib.HL7.Message

 

but it only returns up to the end of the TXA segment.

 

How do I define the variable to hold this number of characters in a business operation 

 

Thanks 

Sue

 

 

[Ed de Moel]

On 04-Oct-13 10:15, sue wrote:
> Hello,
> In trying to receive a large HL7 message - about 100,000 characters

In a "standard" configuration, Cache and Ensemble are restricted to
string of about 32,000 characters.
When the configuration is changed to allow "long strings", strings up to
about 3.5 MB can be handled.

Hope that helps,

Ed de Moel.

--
**************************************************************
This electronic mail transmission contains confidential and/or
privileged information intended only for the person(s) named.
Any use, distribution, copying or disclosure by another person
is strictly prohibited.
**************************************************************

[sue]

Thanks Ed,

 

How are the settings changed and any idea what the effects are?

 

Thanks

Sue

 

[Dale du Preez]

Hi Sue,

Ed's right about our default string length limits. If changing the
string configuration isn't an option for you, you can also write the HL7
content to a stream instead of a string, at which point there aren't any
length restrictions (other than how much memory and/or disk you have
available). You can use code like the following to get the HL7 content
into a stream:
Set tStream = ##class(%Stream.TmpCharacter).%New()
Set tStatus = pHL7Message.OutputToLibraryStream(tStream,
pHL7Message.Separators)

Depending on the rest of your code it may or may not be that easy to
switch to using a stream instead of a string.

I hope that helps,
Dale

[Ted Peck]

There are a few points on this topic that I'd like to clarify, since the
topic arises frequently and documents keep getting larger.

- Ensemble has always been able to pass through large HL7 messages and
segments. To retrieve the contents of such a message or segment, use
any of the OutputTo... methods except OutputToString().

- Long strings support enables use of strings up to 4Gb.
OutputToString() will return up to that amount of contents if long
strings are enabled. Long strings are enabled by default in new
installs of Ensemble 2012.2 or later. They are available as an option
since Ensemble 2007.1.

- Even with long strings enabled, until DDP879 which is first available
in our current Field Test versions, Ensemble's other HL7 API's don't
support working with string values longer than 32k.

- When working with the contents of large HL7 segments, it is not safe
to use strings, whether long or short, because if a segment is that long
it means it is containing large documents and those documents could be
larger than 4Gb.

- Therefore we strongly advise customers to always use stream interfaces
when working with the contents of large segments. This will avoid nasty
surprises when an extra-large document happens to come through. This
should not be an especial burden because you need to work with this sort
of segment you have to write special purpose code to do it in any case.
It's just a matter of writing the code the proper way.

- To manipulate stream data in an HL7 segment, use the GetFieldStream...
and StoreFieldStream... methods of the HL7 Segment class. These were
introduced in Ensemble 2012.1 by TAP1311. In earlier versions, the
methods ReadRawDataStream and StoreRawDataStream were available.

Ted

[Ted Peck]

Whoops, for long strings I should have said 3.5Mb, as Ed did, not 4Gb.
I've corrected the numbers in my quoted message below. (The 4Gb number
relates to process memory but that's another topic.)

On 10/4/2013 11:37 AM, Ted Peck wrote:
> There are a few points on this topic that I'd like to clarify, since
> the topic arises frequently and documents keep getting larger.
>
> - Ensemble has always been able to pass through large HL7 messages and
> segments. To retrieve the contents of such a message or segment, use
> any of the OutputTo... methods except OutputToString().
>

> - Long strings support enables use of strings up to 3.5Mb.

> OutputToString() will return up to that amount of contents if long
> strings are enabled. Long strings are enabled by default in new
> installs of Ensemble 2012.2 or later. They are available as an option
> since Ensemble 2007.1.
>
> - Even with long strings enabled, until DDP879 which is first
> available in our current Field Test versions, Ensemble's other HL7
> API's don't support working with string values longer than 32k.
>
> - When working with the contents of large HL7 segments, it is not safe
> to use strings, whether long or short, because if a segment is that
> long it means it is containing large documents and those documents

> could be larger than 3.5Mb.