Defining Segment and Element Terminators

[Quinlan Shanley]

I have a trading partner that sends characters such as , + - ^ : @ within a segment's element. These characters should not denote subfield or element/segment terminator. Is there a way I can define, by route and/or trading partner, to recognize specific characters as element and segment separators? If so, please point me to where I can set this up.

Regards,

Quin Shanley

[henk-jan ebbers]

hi Quin

what editype are you using?
x12, edifact, tradacoms?

kind regards,
henk-jan ebbers

[Quinlan Shanley]

henk-jan,

Thank you for the reply. I am using X12 and transforming to a FlatFile.

Regards,

Quin

[Steven Hammond]

I may not understand what you are wanting, so I apologize if this is not it but we have X12 trading partners with different elements, segment terminators, etc.  We just create a file for that trading partner in /bots/usersys/partners/x12 and fill in the syntax like this:

syntax = {
        'field_sep':'\\',
        'record_sep':'~',
        'sfield_sep':'>',
        'ISA05':'ZZ',
        'ISA07':'12',
        'ISA11':'U',
        'ISA14':'0',
        'ISA15':'P',
        'version':'00501',
        }

Henk-jan is the expert and can better answer your questions, but this works for us.

Steve Hammond

[Quinlan Shanley]

Steve,

I think this is the answer I am looking for. Do you then import that file into a script or do you reference it from somewhere in the BOTS web interface? Thank you for your help.

Regards,

Quin

[Steven Hammond]

I'll *attempt* it but I hope others will steer your right if I get this wrong (ha).

Under the web interface Configuration=>Partner and Groups make note of the PARTNER IDENTIFICATION that you want to deal with (in my case the partern id is simply their ISA ID => 6142786551).  I then go to bots/usersys/partners/x12 and create a file called partern_id.py (e.g., 6142786551.py).  In it is the following:

syntax = {
        'field_sep':'\\',
        'record_sep':'~',
        'sfield_sep':'>',
        'ISA05':'ZZ',
        'ISA07':'12',
        'ISA11':'U',
        'ISA14':'0',
        'ISA15':'P',
        'version':'00501',
        }

In your map you would simply put:

out.ta_info['topartner'] = "partner_id"

In my case for a Big Lots 810 it looks like this:

out.ta_info['topartner'] = '6142786551'

There is much more that you could include in the SYNTAX, but this is the essentials I usually setup.  Henk-Jen could give you the complete setup of what can go in SYNTAX.  I'm still learning myself.  I hope this helps.  Most of our trading partners use a standard but I have a couple that use some really strange separators.

[henk-jan ebbers]

hi Quin,

note that the handling for incoming x12 is very different from the handling of outgoing x12c fiels.

for incoming x12-files: bots will handle the separators correctly (according to x12 specs), no matter what separators are used.
you do not need to indicate anything for this, it just works.
for OUTGOING: see the answer steven gave. as steven indicated, this can be done per partner.

hope this answers your question,
henk-jan

[Quinlan Shanley]

henk-jan,

Maybe something else is happening here? I am very new to BOTS so I do not yet know all the ins-and-outs. Below are the segments of the EDI doc that is giving me trouble and the file errors messages. I also attached the file errors as a screenshot.

MSG^HTTP://WWW-03.IBM.COM/PROCUREMENT/PROWEB.NSF/CONTENTDOCSBYTITLE/!

N1^ST^IBM INVOICE TO CODE:  26P/896^92^992!

N2^BLDG 006 SHIP TO CODE:  5GQ/895!

Errors:

State	Re*	Route	Date/time	#in	InEditype	InMessagetype	FromChannel	FromPartner	ToChannel	ToPartner	OutEditype	OutMessagetype	InFile	OutFile	Error	MappingScript	FromMail	ToMail
	Error		IBM_850_004010	2012-05-28 11:15:53	0	x12		IBM_850_In						ftp://In/850_004010_2.x12		Traceback (most recent call last): InMessageFieldError: line:47 pos:26; expect field, is a subfield; record "N2 BLDG 006 SHIP TO CODE 5GQ/895 ".
	Error		IBM_850_004010	2012-05-28 11:15:53	0	x12		IBM_850_In						ftp://In/850_004010_1.x12		Traceback (most recent call last): InMessageFieldError: line:46 pos:27; expect field, is a subfield; record "N1 ST IBM INVOICE TO CODE 26P/896 (etc)".
	Error		IBM_850_004010	2012-05-28 11:15:53	0	x12		IBM_850_In						ftp://In/850_004010.x12		Traceback (most recent call last): InMessageFieldError: line:10 pos:10; expect field, is a subfield; record "MSG HTTP //WWW-03.IBM.COM/PROCUREME (etc)".

The three documents are copies of the same EDI document. 850_004010.x12 is the original and is not modified. 850_004010_1.x12 has the colon removed from the MSG segment. 850_004010_2.x12 has the colon removed from the MSG and N1 segments.

It seems to colon is creating issues when BOTS is trying to read the document. I don't want to have to remove the colons before reading the documents. Can I make it work without removing the colons?

Thank you,

Quin

[Steven Hammond]

Thanks henk-jen, I forgot to state my solution was OUTBOUND. Bots
indeed handles incoming with no problem. Thanks for a great product!

Steve Hammond

[henk-jan ebbers]

what probably happens in 850_004010.x12 is that the colon is defined as a sub-field separator.
in x12 a rule is that the separators can not be used in the data (yes, this is pretty pathetic....I know. x12 does not have an escape sign...)

looks like same problem occurs in the other files.

these x12 are not valid x12. Compare this to not well-formed xml. The basic syntax rules are violated.
your partner should use another subfield separator.
filtering out the colons will probably not work, as you will also filter out the correct colons.

are you sure this x12 is 'as in production'? (you can send it, I will check if this is indeed a non-valid file)

henk-jan

On 05/28/2012 06:20 PM, Quinlan Shanley wrote:
> henk-jan,
>

> Maybe something else is happening here? I am very new to BOTS so I do not yet know all the ins-and-outs. Below are the segments of the EDI doc that is giving me trouble and the file errors messages.
> I also attached the file errors as a screenshot.
>
> MSG^HTTP://WWW-03.IBM.COM/PROCUREMENT/PROWEB.NSF/CONTENTDOCSBYTITLE/!
> N1^ST^IBM INVOICE TO CODE: 26P/896^92^992!
> N2^BLDG 006 SHIP TO CODE: 5GQ/895!
>
> Errors:
>
> State Re* Route Date/time #in InEditype InMessagetype FromChannel FromPartner ToChannel ToPartner OutEditype OutMessagetype InFile OutFile Error MappingScript FromMail ToMail
>

> *
>
>
> *

>
> Error IBM_850_004010 2012-05-28 11:15:53 0 x12
> IBM_850_In
>
>
>
>

> ftp://In/850_004010_2.x12 <http://localhost:8080/filer/?action=this&idta=10382>

> Traceback (most recent call last): InMessageFieldError: line:47 pos:26; expect field, is a subfield; record "N2 BLDG 006 SHIP TO CODE 5GQ/895 ".
>
>
>

> *
>
>
> *

>
> Error IBM_850_004010 2012-05-28 11:15:53 0 x12
> IBM_850_In
>
>
>
>

> ftp://In/850_004010_1.x12 <http://localhost:8080/filer/?action=this&idta=10380>

> Traceback (most recent call last): InMessageFieldError: line:46 pos:27; expect field, is a subfield; record "N1 ST IBM INVOICE TO CODE 26P/896 (etc)".
>
>
>

> *
>
>
> *

>
> Error IBM_850_004010 2012-05-28 11:15:53 0 x12
> IBM_850_In
>
>
>
>

> ftp://In/850_004010.x12 <http://localhost:8080/filer/?action=this&idta=10378>

[Quinlan Shanley]

henk-jan,

We currently use a VAN (Innovis). The document I've been referring to is a production 850 I download from our VAN's website. We use a software to connect to our mailbox at the VAN. The software is call TrustedLink for Windows (TLW). For some reason, when I get the document from the TLW, the colons are changed to @ symbols. I've included both documents.

In short, the documents I've downloaded from our VAN could have been changed by some preparation process that I don't know about.

Regards,

Quin Shanley

[henk-jan ebbers]

yes, problem is in both files (in the document from TLW all colons have been changed to '@'. the problem of course, stay the same).

contact you edi-partner; they are sending incorrect x12 files.

the problem is fundamental; there is no 'good' (correct ) way to solve it.
if your edi-partner does not want to send correct x12, you will have to monkey-patch it....
possible ways out (but please, first check with your edi-partner to see what is going on!!):
1. check with edi-partner that they do not use sub-field separator (officially called component data separator) in their messages.
if they indeed never use it: write small preprocess script for bots that changes the subfield separator in the ISA header to another value (that is never used in the data ;-)
2. change x12 grammar definition to accommodate for this....

henk-jan

On 05/29/2012 02:05 PM, Quinlan Shanley wrote:
> henk-jan,
>
> We currently use a VAN (Innovis). The document I've been referring to is a production 850 I download from our VAN's website. We use a software to connect to our mailbox at the VAN. The software is
> call TrustedLink for Windows (TLW). For some reason, when I get the document from the TLW, the colons are changed to @ symbols. I've included both documents.
>
> In short, the documents I've downloaded from our VAN could have been changed by some preparation process that I don't know about.
>
> Regards,
> Quin Shanley
>
> On Monday, May 28, 2012 11:49:27 AM UTC-5, eppye wrote:
>
> what probably happens in 850_004010.x12 is that the colon is defined as a sub-field separator.
> in x12 a rule is that the separators can not be used in the data (yes, this is pretty pathetic....I know. x12 does not have an escape sign...)
>
> looks like same problem occurs in the other files.
>
> these x12 are not valid x12. Compare this to not well-formed xml. The basic syntax rules are violated.
> your partner should use another subfield separator.
> filtering out the colons will probably not work, as you will also filter out the correct colons.
>
> are you sure this x12 is 'as in production'? (you can send it, I will check if this is indeed a non-valid file)
>
> henk-jan
>
> On 05/28/2012 06:20 PM, Quinlan Shanley wrote:
> > henk-jan,
> >
> > Maybe something else is happening here? I am very new to BOTS so I do not yet know all the ins-and-outs. Below are the segments of the EDI doc that is giving me trouble and the file errors
> messages.
> > I also attached the file errors as a screenshot.
> >

> > MSG^HTTP://WWW-03.IBM.COM/PROCUREMENT/PROWEB.NSF/CONTENTDOCSBYTITLE/ <HTTP://WWW-03.IBM.COM/PROCUREMENT/PROWEB.NSF/CONTENTDOCSBYTITLE/>!

> > N1^ST^IBM INVOICE TO CODE: 26P/896^92^992!
> > N2^BLDG 006 SHIP TO CODE: 5GQ/895!
> >
> > Errors:
> >
> > State Re* Route Date/time #in InEditype InMessagetype FromChannel FromPartner ToChannel ToPartner
> OutEditype OutMessagetype InFile OutFile Error MappingScript FromMail ToMail
> >
> > *
> >
> >
> > *
> >
> > Error IBM_850_004010 2012-05-28 11:15:53 0 x12
> > IBM_850_In
> >
> >
> >
> >

> > ftp://In/850_004010_2.x12 <ftp://In/850_004010_2.x12> <http://localhost:8080/filer/?action=this&idta=10382 <http://localhost:8080/filer/?action=this&idta=10382>>

> > Traceback (most recent call last): InMessageFieldError: line:47 pos:26; expect field, is a subfield; record "N2 BLDG 006 SHIP TO CODE 5GQ/895 ".
> >
> >
> >
> > *
> >
> >
> > *
> >
> > Error IBM_850_004010 2012-05-28 11:15:53 0 x12
> > IBM_850_In
> >
> >
> >
> >

> > ftp://In/850_004010_1.x12 <ftp://In/850_004010_1.x12> <http://localhost:8080/filer/?action=this&idta=10380 <http://localhost:8080/filer/?action=this&idta=10380>>

> > Traceback (most recent call last): InMessageFieldError: line:46 pos:27; expect field, is a subfield; record "N1 ST IBM INVOICE TO CODE 26P/896 (etc)".
> >
> >
> >
> > *
> >
> >
> > *
> >
> > Error IBM_850_004010 2012-05-28 11:15:53 0 x12
> > IBM_850_In
> >
> >
> >
> >

> > ftp://In/850_004010.x12 <ftp://In/850_004010.x12> <http://localhost:8080/filer/?action=this&idta=10378 <http://localhost:8080/filer/?action=this&idta=10378>>
> > Traceback (most recent call last): InMessageFieldError: line:10 pos:10; expect field, is a subfield; record "MSG HTTP //WWW-03.IBM.COM/PROCUREME <http://WWW-03.IBM.COM/PROCUREME> (etc)".

[Steven Hammond]

Quin, we used TLW years ago (we still use Inovis but use an AS2 connection).  It sounds like a setup issue in TLW.  It’s been a long time, but when you setup the 850 in TLW for that trading partner, what settings do you have for the field separators, segment terminators, etc.  Since it is change field separators and segment terminators it sounds like a profile setup for that trading partner might need to be adjusted.  As I said, I haven’t used TLW in years, but if I remember, each trading partner and their documents could have a profile associated with it for the various EDI envelope structures.  Just a thought.

 

Steve Hammond

Cleburne,TX

[Quinlan Shanley]

Steve,

A question, I thought the idea of using BOTS was to get away from the need to use a VAN? Are you using Inovis because you do not want to manage the AS2 connection in-house?

Quin Shanley

[Steven Hammond]

Oh if only that were so (ha).  We have trading partners that prefer direct connections (e.g., Wal-mart - AS2) then we have smaller folks who still use VANS (for one reason or another don't want to be bothered with having to deal with communications).  I have folks using secure FTP directly.  So, it really depends on the trading partner.  I personally prefer direct communications with our communication server (AS2, FTP, secure FTP, etc), but not everybody can do that so we accommodate to whatever their needs might be.  Many of the smaller ones don't even do their own mapping but have the VAN take care of it for them.  We are not a big shop by no means.  But over the last 10 years we've tried to minimize VAN involvement but it seems there is no getting away from them.  I just upper management to pass the cost along to those partners when they buy the product but require the use of a VAN.  :-)

We replaced our old EDI translator (eBridge) when we went to a new ERP system and the old translator wouldn't work with the new ERP system (new ERP system had a special 3rd party lined up for a very EXPENSIVE piece of software that required too much human intervention) with EDI bots.  I primarily use it for mapping.  We already had pieces written to process the documents (i.e., store them in a MySQL db), print them, email them to CSR, etc.  EDI bots fit right in as a drop-in piece to handle the mapping.  The old translator was very limited in its mapping capabilities where as in EDI bots, I have a lot more control (using python).  The only proprietary piece at this point is the AS2 software (written for Red Hat) and the ERP system.  And we are already discussing replacing the ERP system with an open source alternative!  I'll probably look at an open source piece to replace the AS2 software at some point since it is rather old (maintenance payment is small but we can't run the newer version since they want to create a HUGE software piece and all we need is a communication server).

Sorry to be verbose.  I probably should go on to bed.  :-)

Steve Hammond

[Quinlan Shanley]

Steve,

Just so I understand correctly, if I have a trading partner who only uses a VAN to communicate EDI documents, I am absolutely forced to pay for a VAN service? I guess I (naively) figured that by hosting communication on our side, the VAN would be willing to connect to a 'box' on our side. I only have experience with using a VAN, but I'd like to get my company away from it because of the cost of transmitting the documents.

Regards,

Quin

[BikeMike]

Hi Quinlan,
In most cases, if you are forced to use a VAN then you also must pay
for their service, unfortunately. We are very similar to Steve, we
encourage direct connections with all partners that can manage it and
even offer them technical help to set it up. We do ftp only but I'm
interested in trying out AS2 if a viable free solution is available.

With our "via the VAN" connections we generally pay a per-transaction
price. If possible we get the VAN to just pass on the file they
received with no translation done to it, then we do the translation
ourselves (they still charge though!). This gives us the maximum
control over the process.

Some partners will never move away from VANs because they use them as
a communications gateway to send everything. Then the VAN worries
about how to send it to various partners (EDI, fax, email etc) so I
can see that they have a place in the EDI world... just wish they
weren't so expensive.

Kind Regards,
Mike

[Quinlan Shanley]

Thank you everyone. This has been very useful.

Regards,

Quin Shanley