Manipulating X12 837 4010 files with bots

[ronaldleemhuis]

I've been puzzling over bots and have been gaining familiarity and
confidence. My interest has been in manipulating X12 835, 837 and 997
files for data interchange related to a tiny medical practice in Erie,
PA (USA). Ultimately, I hope to configure ordinary commercial
practice management software to create a properly formatted text file
output which is from the standpoint of the program an "insurance
form."

From the standpoint of bots, this insurance form would be a "fixed"
format input.

With proper settings, it would be possible to map the data to X12 837
format for sending claims to Medicare, Blue Shield or a clearing
house.

The formats 835 and 997 are the corresponding received messages from
the other end.

I started by using known good X12 837 4010 files and set up a
translation to the same format as output. (Not as trivial as it
sounds!)

I had to copy the 837 grammar file to another name. I had to change a
few things including commenting out an undefined FRM field.

In the record definition files, I found that I had to make NM102
conditional rather than mandatory.

I hard coded some ID qualifiers etc. in the grammars.py file.

I also had to remove the LF/CR character insertion in the output
file. (Turns out the clearing house didn't care when they were there,
but Medicare and Blue Shield gave obscure TA1 rejections until I
removed the LR/CR characters from the output file.

Right now, I've got the translation working and creating files
accepted by Medicare, Blue Shield and Anvicare clearing house with the
following exception (Overridden by manual editing for proof):

field GS02 should be the partner ID of the medical practice and is not
in general identical to the transmitter ID in field ISA06. However,
despite all my tweaking, GS02 in the output is changed from the GS02
in my input file so it matches ISA06 in the input file. I'm
struggling with the way to make this GS02 pass through unchanged.

Any ideas, anybody on how to get GS02 to pass through?

I'm also hoping to figure out how to put partner-specific scripts in
the partners subdirectory so I don't have to hard code actual
variables (ID qualifier type for example) into scripts that are used
with different trading partners.

Once I get the X12 837 files to pass through, I'll be glad to create a
plugin for others to use.

Ronald Leemhuis MD
Erie, PA

[henk-jan ebbers]

Hi Ronald,

trying to give some answers below.
if my answers are not clear, ask again.

as I am based in Europe, I know edifact better than x12
(though I am learning quickly)

For me it is interesting to have good support for x12 specific needs in
Bots.
So I really appreciate you feedback.

henk-jan ebbers

ronaldleemhuis wrote:
> I've been puzzling over bots and have been gaining familiarity and
> confidence. My interest has been in manipulating X12 835, 837 and 997
> files for data interchange related to a tiny medical practice in Erie,
> PA (USA). Ultimately, I hope to configure ordinary commercial
> practice management software to create a properly formatted text file
> output which is from the standpoint of the program an "insurance
> form."
>
> >From the standpoint of bots, this insurance form would be a "fixed"
> format input.
>
> With proper settings, it would be possible to map the data to X12 837
> format for sending claims to Medicare, Blue Shield or a clearing
> house.
>
> The formats 835 and 997 are the corresponding received messages from
> the other end.
>
> I started by using known good X12 837 4010 files and set up a
> translation to the same format as output. (Not as trivial as it
> sounds!)
>

you mean 837 -> 837?
(this in itself is simple)

> I had to copy the 837 grammar file to another name. I had to change a
> few things including commenting out an undefined FRM field.
>

I am not sure about the qusality of the grammars for x12....some I
checked thoroughly by hand, others I am not very sure about.
Michael is generating x12 grammars right now, this looks very good, but
not yet finished.
Hope to hear for him this week.

> In the record definition files, I found that I had to make NM102
> conditional rather than mandatory.
>
> I hard coded some ID qualifiers etc. in the grammars.py file.
>

let me know which ones.
adapting grammars.py is sometimes useful but should (in the end) not be
needed.

> I also had to remove the LF/CR character insertion in the output
> file. (Turns out the clearing house didn't care when they were there,
> but Medicare and Blue Shield gave obscure TA1 rejections until I
> removed the LR/CR characters from the output file.
>

is this what you did in the grammars.py?

> Right now, I've got the translation working and creating files
> accepted by Medicare, Blue Shield and Anvicare clearing house with the
> following exception (Overridden by manual editing for proof):
>
> field GS02 should be the partner ID of the medical practice and is not
> in general identical to the transmitter ID in field ISA06. However,
> despite all my tweaking, GS02 in the output is changed from the GS02
> in my input file so it matches ISA06 in the input file. I'm
> struggling with the way to make this GS02 pass through unchanged.
>
> Any ideas, anybody on how to get GS02 to pass through?
>

I will have a look at it tomorrow what is the best solution for this.
At this moment it is not possible, should be accommodated for. (is this
good English?)

> I'm also hoping to figure out how to put partner-specific scripts in
> the partners subdirectory so I don't have to hard code actual
> variables (ID qualifier type for example) into scripts that are used
> with different trading partners.
>

grrrr....partner specific qualifiers!
but that's the way it is.

At this moment bot supports partner-specific syntax.
Eg you send x12 files to partner XYZ:
in bots/usersys/partners/x12 put a file XYZ.py
the syntax in this file will be used for outgoing messages for this
partner (maybe this is also related to what you are doing in grammars.py)

In Bots is quite some functionality for partner specific things.
Problem for me (when setting up translations) is always to keep this
manageable/understandable when are are a lot of partner-specific things.
(I am not talking about the bots source code, but about the
translations); my experience (with other translators ....) is that it
very easy becomes a mess.)

Maybe useful to you:
- you can set up partner-specific translations scripts: message XYZ get
mapped by another mapping script for partner XYZ as for partner VWX
- from a partner specific mapping script you can call other mapping scripts
- in a mapping script you biuld an outgoing messge (using put). It is
possible to change segments, data, qualifiers etc afterwards.
So a possible setup can be:
- use partner-specific mapping scripts
- the partner-specific scripts call a 'central mapping', passing
partner-specific values
- after the central mapping is finished you can do partner specific
adoptions to the outgoing message.

Is this of any help to you?

> Once I get the X12 837 files to pass through, I'll be glad to create a
> plugin for others to use.
>

That would be great!

henk-jan

[ronaldleemhuis]

To get real X12 837 (ver 4010) files translating using x122x12 script,
and functioning for medical claims to Medicare, Blue Shield and
Anvicare clearing house I did this:

1. Copied and renamed 837004010.py to 837004010X098A1.py
2. Commenting out the line containing "FRM" near the bottom of that
file, as it refers to a field not defined in records004010.py.
3. Changed version from 00403 to 00401 near the top of the file.
4. In file records004010.py I changed NM102 from "M" to "C," as my
files did not contain NM102 values.
5. in .\bots\grammar.py in the X12 section, I hard coded ISA05 as
'ZZ,' ISA7 as '33,' ISA11 as 'U.' I also changed the value of
add_crlfafterrecord_sep to nothing i.e. I removed everything from the
quoted value.

ISA7 is 33 for Blue Shield, something else for Medicare. Apparently
anything works for Anvicare clearing house.
6. I changed the value in x12syntax.py from 00403 to 00401.
7. I set up three x122x12 translations with receive ID and transmit
ID for the three applications.

I think these were the only changes I made, and I get working X12 837
medical claim files that actually do the job, with one exception:

GS02 for some reason is not carried through intact. It is replaced
with the value in ISA06. If I edit this value manually, the files
work great.

So at this point, I'm working on figuring out how to get GS02 to carry
through properly. Also, I'm working on getting the ID qualifiers to
be partner-specific.

Again, once I get this working, I'll be glad to make up a plugin for
others to try. This setup actually does not do much other than prove
that the grammar is working correctly. However, it serves as a
template for making small corrections (i.e. converting from version
4010 to the required 5010 without buying new and expensive new
proprietary software). It also serves aas a template for more
advanced modifications.

Ultimately, I want to create text files containing insurance claim
information using the file printing function in Medisoft. One could
then create a grammar for it so that it can translate to X12 837
format. In the near future, we will be required to use version 5010,
but the changes from 4010 are apparently minor.

A really nice open source suite for professional medical insurance
claims would contain the following:

1. Python/bots, naturally -- probably a Windows installation
considering the target market.
2. Ready made insurance form format files for Medisoft, Lytec and/or
other practice management software packages.
3. Plugin to take the text file output from #2 and translate it to X12
5010 format with easily identified naming conventions for the
designated recipient (Medicare, Blue Shield, clearing house etc.)
4. Plugin that translates downloaded X12 997 files to human readable
form, for acknowledgement reports.
5. Plugin that translates X12 835 files to human readable form for the
final reports of payments etc. (For this function, a free downloadable
utilities are available from Medicare.)

Another option would be to use the OBCD driver for the Advantage
database that is part of Medisoft to interact directly with the
database.

Ronald Leemhuis MD

[henk-jan ebbers]

ronaldleemhuis wrote:
> To get real X12 837 (ver 4010) files translating using x122x12 script,
> and functioning for medical claims to Medicare, Blue Shield and
> Anvicare clearing house I did this:
>
> 1. Copied and renamed 837004010.py to 837004010X098A1.py
>

yes.

> 2. Commenting out the line containing "FRM" near the bottom of that
> file, as it refers to a field not defined in records004010.py.
>

oops. maybe you have to comment out some more, like this:
{ID: 'DTP', MIN: 0, MAX: 9},
]},
# {ID: 'LQ', MIN: 0, MAX: 999999, LEVEL: [
# {ID: 'FRM', MIN: 1, MAX: 99},
# ]},
]},

> 3. Changed version from 00403 to 00401 near the top of the file.
>

the x12 envelopes have a different version number as the messages in it.
But if your edi partne want this, this is what you should do.

> 4. In file records004010.py I changed NM102 from "M" to "C," as my
> files did not contain NM102 values.
>

AFAICS it i 'M' in the standards; but thiese things can happen,
companies do not always stick to the standard 100%

> 5. in .\bots\grammar.py in the X12 section, I hard coded ISA05 as
> 'ZZ,' ISA7 as '33,' ISA11 as 'U.' I also changed the value of
> add_crlfafterrecord_sep to nothing i.e. I removed everything from the
> quoted value.
>

Do this in the partner-specific syntax, as explained in my last mail.
You could do it the way you suggest, but:
- another partner might be needing other values here.
- I you would update to antoher version of bots, the grammar.py might be
updated so this will be overwritten.
Net to the partner-specific syntax, you can define this in the message
syntax, so it will be used for all your partners. Probably best way is
to define the non-partner-specific values in the message syntax, and the
partner-specific in the partner files.

> ISA7 is 33 for Blue Shield, something else for Medicare. Apparently
> anything works for Anvicare clearing house.
> 6. I changed the value in x12syntax.py from 00403 to 00401.
> 7. I set up three x122x12 translations with receive ID and transmit
> ID for the three applications.
>
> I think these were the only changes I made, and I get working X12 837
> medical claim files that actually do the job, with one exception:
>
> GS02 for some reason is not carried through intact. It is replaced
> with the value in ISA06. If I edit this value manually, the files
> work great.
>
> So at this point, I'm working on figuring out how to get GS02 to carry
> through properly.

this is not possible now in Bots.
I can make this possible.
But in order to do this right,, I will need some more information.
I assume GS02 is a entity within the ISA06?
is there one GS02 value?
If not, how does this work? Can the ISA06 be derived from the GS02 values?

kind regards,
henk-jan

> Also, I'm working on getting the ID qualifiers to
> be partner-specific.
>

You should have enough information about this now?

> Again, once I get this working, I'll be glad to make up a plugin for
> others to try.

I really appropriate that!

> This setup actually does not do much other than prove
> that the grammar is working correctly. However, it serves as a
> template for making small corrections (i.e. converting from version
> 4010 to the required 5010 without buying new and expensive new
> proprietary software). It also serves aas a template for more
> advanced modifications.
>
> Ultimately, I want to create text files containing insurance claim
> information using the file printing function in Medisoft. One could
> then create a grammar for it so that it can translate to X12 837
> format. In the near future, we will be required to use version 5010,
> but the changes from 4010 are apparently minor.
>
> A really nice open source suite for professional medical insurance
> claims would contain the following:
>
> 1. Python/bots, naturally -- probably a Windows installation
> considering the target market.
> 2. Ready made insurance form format files for Medisoft, Lytec and/or
> other practice management software packages.
> 3. Plugin to take the text file output from #2 and translate it to X12
> 5010 format with easily identified naming conventions for the
> designated recipient (Medicare, Blue Shield, clearing house etc.)
> 4. Plugin that translates downloaded X12 997 files to human readable
> form, for acknowledgement reports.
>

Bots contains functionality for receiving 997's.
I do feel this can be refined though.
Basically I you do not want a acknowledgement for every file, but you
want to be notified when you did NOT receive a 997.

There are 2 ways of handling 997:
1. bots receives 997's and send these to the ERP/medical administration
system.
This works, I think this is in a plugin.
What this lack at the moment is that the edi ID's of edi documents
are passed, which are unknown in the ERP system.
Should be possible to extend this to the documents ID's as used by
the ERP system.
2. Bots handles the 997: this works.
You would want to be notified if a 997 is NOT received for a
documents, eg after 2 days.

> 5. Plugin that translates X12 835 files to human readable form for the
> final reports of payments etc. (For this function, a free downloadable
> utilities are available from Medicare.)
>

wouldn't it be more logical to import this into the ERP system?
(and if you wnat it printed, print this from the ERP system.

btw, an edi file can be converted to html in a translation.
check out the 'edifax' plugin.

kind regards,
henk-jan

[Ronald Leemhuis]

So at this point, I'm working on figuring out how to get GS02 to carry
through properly.  

this is not possible now in Bots.
I can make this possible.
But in order to do this right,, I will need some more information.
I assume GS02 is a entity within the ISA06?
is there one GS02 value? If not, how does this work? Can the ISA06 be derived from the GS02 values?

As I see it, Henk-Jan, GS02 is the second field in the GS section of what appears to be part of the envelope.  I think that the GS section follows the ISA section.  Our medical files use an identification code for the transmitting partner in ISA06.  The identification code (Partner ID?) for the medical practice is in GS02.  In general, the files are transmitted by a third party so these numbers are not the same.  Even if the doctor himself transmits the file, as I do, there are two numbers -- one for the role as transmitter and the other for role as medical practice.

 

Actually, I tried hard coding GS02 in various places as I did for ISA 05 and ISA07, I think, but it kept coming through as equal to ISA06 anyway.

 

I have this set up with Windows (does not have grep).  Otherwise I would grep down into the code to find all places that refer to ISA06 and see where the problem is.

 

Thanks for your advice.

 

I think I'll try to get this working in a bare bones fashion first and then work out all the niceties like partner-specific scripts etc.

 

Ron Leemhuis

[henk-jan ebbers]

Hi Ronald,

Ronald Leemhuis wrote:
>
>
> So at this point, I'm working on figuring out how to get GS02
> to carry
> through properly.
>
> this is not possible now in Bots.
> I can make this possible.
> But in order to do this right,, I will need some more information.
> I assume GS02 is a entity within the ISA06?
> is there one GS02 value? If not, how does this work? Can the ISA06
> be derived from the GS02 values?
>
> As I see it, Henk-Jan, GS02 is the second field in the GS section of
> what appears to be part of the envelope. I think that the GS section
> follows the ISA section. Our medical files use an identification code
> for the transmitting partner in ISA06. The identification code
> (Partner ID?) for the medical practice is in GS02. In general, the
> files are transmitted by a third party so these numbers are not the
> same. Even if the doctor himself transmits the file, as I do, there
> are two numbers -- one for the role as transmitter and the other for
> role as medical practice.
>
> Actually, I tried hard coding GS02 in various places as I did for ISA
> 05 and ISA07, I think, but it kept coming through as equal to ISA06
> anyway.

at this moment in generating the envelope bots fills in GS02 with the
same partnerID as ISA06

in your situation this is different.
so I am trying to find out how to do this best:
- what goes in the ISA06 - the partnerID? or the value for the 3th
party? Is this a fixed value?
- what goes in the GS02 - the partnerID?

Changing this is quite easy - hard part is to find out what to do.

Seems like this 3th party is not the partnerID?

>
> I have this set up with Windows (does not have grep). Otherwise I
> would grep down into the code to find all places that refer to ISA06
> and see where the problem is.

windows has a grep - but it's got another name.
best to google for it.

kind regards,
henk-jan

[ronaldleemhuis]

> so I am trying to find out how to do this best:
> - what goes in the ISA06 - the partnerID? or the value for the 3th
> party? Is this a fixed value?
> - what goes in the GS02 - the partnerID?
>
> Changing this is quite easy - hard part is to find out what to do.
>
> Seems like this 3th party is not the partnerID?

I'm quite happy with how ISA06 is behaving. This is for the partnerID
of the entity transmitting the message. It also happens to be the
login ID we use when dialing up the modem at Blue Shield. This
differs, in general, from the practice partner ID (that goes in GS02)
because one transmitter may be sending messages for multiple medical
practices each with a different ID.

What makes my situation special is that I assume both roles -- I am
both the practice and the third party who transmits it. But the IDs
are different.

All my trading partners require both IDs, i.e. two different numbers
for each trading partner.

If an independent business were sending information for multiple
practices, the ISA06 value would be the same for all, but the GS02
value would reflect a number for each separate practice.

Ronald Leemhuis

[henk-jan ebbers]

so, what are the parties involved: and what are thier relations?

[ronaldleemhuis]

ISA06 is the partnerID of the entity transmitting the insurance claim
file. The partnerID is assigned by the receiver, so it is mutually
defined. In my case, the transmitting entity is my medical practice
in its transmitting role. If a clearing house intermediary were
sending the file, it would be THEIR partnerID.

ISA08 is the partnerID of the entity receiving the insurance claim
file. This can be Medicare, Blue Shield or a medical claims clearing
house. ISA08 is fixed and published for each of the receiving
entities.

GS02 is the partnerID of the medical practice from which the claims
are derived. This partner ID is assigned by each receiver and is
mutually defined. It is different from the value in ISA06 because, in
general, medical practices do not transmit their own files directly to
the receiving entity but rather to an intermediary clearing house. I
do send claims directly to Medicare and Blue Shield to save about
$2000 per year in additional clearing house fees.

A single ISA06 partnerID may transmit files for multiple medical
practices.

Ronald Leemhuis MD

[ronaldleemhuis]

It appears that the ..\bots\envelope.py script is causing GS02 to be
incorrectly populated with what is in ISA06.

Maybe we need a new variable in the mergevars section.

Or maybe the GS02 reference in this section can be removed, and one
could add a line like

'GS02':'practiceID' ,

in the X12 section of the grammar.py script.

Ron Leemhuis

[ronaldleemhuis]

If one goes into the ..\bots\envelope.py script and comments out the
line for GS02 as equal to a certain program variable and instead
inserts

'GS02':'practiceID',

the output file comes out correctly.

Now this is an inelegant ad hoc solution that meets my needs, but the
real solution should equate GS02 to another variable for what is in
the incoming grammar of GS02 field. My solution is inelegant because
it requires typing my specific information into a program script.
(But it works for me!)

Ronald Leemhuis

[henk-jan ebbers]

Ronald,

am I right if:
- ISA06 is a fixed value for you
- GS02 is a fixed value for you

kind regards,
henk-jan

ronaldleemhuis wrote:
> ISA06 is the partnerID of the entity transmitting the insurance claim
> file. The partnerID is assigned by the receiver, so it is mutually
> defined.

or are you suggesting these ID's are different per trading partner??

[Ronald Leemhuis]

[Ronald Leemhuis]

For any given file receiver (1 have three different receivers of my files), I have different values for ISA06, ISA08 and GS02.

 

Also, GS02 is not equal to ISA06.

 

Furthermore, the ID qualifiers ISA05 and ISA07 are different for each of the three receivers.

 

(by the way, my problem with absence of NM102 field in one file to the clearing house was an error on my part.  It is OK to keep NM102 format defined according to official standards because I can fix my files.  It turns out that the clearing house just didn't care whether NM102 was there or not)

 

Ron Leemhuis

[henk-jan ebbers]

OK, would this solution work:
as GS02 and ISA06 you always will have to use partner dependent syntax.
in the partner dependent syntax define (so derived from the topartner):
- ISA05
- ISA06 (if not defined: topartner is usd)
- ISA07
(ISA08 is the topartner you are sending to)
- GS02; if not defined the value for ISA06 will be used
- GS03: the topartner

(this is quite simple to adept in envelope.py)

kind regards,
henk-jan

[Ronald Leemhuis]

OK, would this solution work:
as GS02 and ISA06 you always will have to use partner dependent syntax.
in the partner dependent syntax define (so derived from the topartner):
- ISA05
- ISA06 (if not defined: topartner is usd)
- ISA07
(ISA08 is the topartner you are sending to)
- GS02; if not defined the value for ISA06 will be used
- GS03: the topartner

(this is quite simple to adept in envelope.py)

Conceptually, the scheme is correct.  I don't know how to write such code or where one would place it.

 

Ron Leemhuis 

 

[henk-jan ebbers]

Hi Ronald/Ron,

see the attached zip-file.
is not very well tested, but is a small change.
(this will go in new releases)

henk-jan

[ronaldleemhuis]

I tried the new envelope.py file. I found that my GS02 value did not
carry through properly. GS02 was still set equal to the ISA06 value.

X12 medical claims require:

custom senderID qualifier in ISA05
custom senderID in ISA06
custom receiverID qualifier in ISA07
custom receiverID in ISA08
'U' in ISA11
practiceID in GS02

In addition to developing envelope.py as you are doing, perhaps one
could design an alternate envelope.py script
(envelope_passthrough.py?) that can be substituted after installation
by renaming it or creating a symbolic link. This could simply pass
through all the envelope values from an input file.

I'll try to take a closer look at your new envelope.py file to see why
it isn't working for me yet.

Ron Leemhuis

[henk-jan ebbers]

Ronald,

I tested this, and found it working.

In my configuration I tested this in two ways with GS02:
- I made a partner specif syntax. The topartner is 111111, so I place
file 111111.py in bots/usersys/x12
in that file the value for GS02 is set to 'XXXXXX', and this appears
in the x12 file.
AFAICS this is closest to what you want: the envelope values are
partner-dependent.
- in x12syntax.py (in bots/usersys/x12) I added the GS02 value in the
syntax. This also works.

About your other suggestions:
1. bots contains the possibility so have user defined enveloping.
In this case you'll have to write the whole envelope.
For what you wnat, this does not seems necessary and a bit of overkill.
(form my side, I would rather have a better standard enveloping.
How it is done now adds the functionality you wish; it is in line
with the current configuragion & not more complex.)
2. There is also the possibility so have 'user exits' in an envelope -
there are some, but not at this point.
As said before, I do not think this is needed for what you want.

kind regards,
henk-jan

[ronaldleemhuis]

Nothing I put in a directory bots\usersys\x12 seemed to affect the
value of GS02.

I named a script XXXXX.py where XXXXX was the receiverID in my data
file. I tried putting the entire equivalent of the envelope.py file
there, without success.

I also tried putting a file that contained nothing but

GS02 = 'YYYYY'

for example.

I tried putting the same files in bost\usersys\partners\x12 directory,
also with no success.

I'm sure the problem is just that I'm clueless about what to put where

Ron Leemhuis

[henk-jan ebbers]

Hai Ronald,

first, check if the updated envelope.py is in the right place.

than check if the the frompartner is known in the mapping script. eg in
the mapping script:
print inn.ta_info['frompartner']
it should be known there (else we'll have to tell Bots)

It should be in:

bots\usersys\partners\x12

I made a mistake there, sorry.

Check eg x12syntax.py (in bots/usersys/x12) to see how the syntax is used.

hope this helps,
henk-jan

[ronaldleemhuis]

OK, Henk-Jan, I examined your new envelope.py script and saw that you
added logic that is intended to behave as before if ISA06, GS02, ISA08
and ISA03 are not in the incoming file and to pass them through
unchanged if they are present.

I put the envelope.py file in the bots directory where the old
envelope.py file was located. I put the grammar.py file back to what
it was originally.

Despite all this, GS02 was incorrectly assigned the ISA06 value by
running bots. So, for me, it still does not work exactly right in the
GS02 field.

To test the script, I changed 'frompartner' to 'topartner' in the GS02
line, and that did affect the output. (I changed it back.)

I'm totally confused about how to place and to invoke the partner-
specific scripts and why I should need them for this.

The other thing is that I will probably want to do the same logic for
ISA05, ISA07 and ISA11. These are values that may need to be passed
through because they vary.

Ronald Leemhuis

[henk-jan ebbers]

ronaldleemhuis wrote:
> OK, Henk-Jan, I examined your new envelope.py script and saw that you
> added logic that is intended to behave as before if ISA06, GS02, ISA08
> and ISA03 are not in the incoming file and to pass them through
> unchanged if they are present.
>
> I put the envelope.py file in the bots directory where the old
> envelope.py file was located. I put the grammar.py file back to what
> it was originally.
>
> Despite all this, GS02 was incorrectly assigned the ISA06 value by
> running bots. So, for me, it still does not work exactly right in the
> GS02 field.
>

I am sorry, you must be doing something wrong. It definitely works.

> To test the script, I changed 'frompartner' to 'topartner' in the GS02
> line, and that did affect the output. (I changed it back.)
>
> I'm totally confused about how to place and to invoke the partner-
> specific scripts and why I should need them for this.
>

I really did explain this in a previous mail.
you do not need to invoke anything.
when generating a message or invoice, bots does:
- read the syntax in grammar.py
- reads the grammar for the message or envelope; these values overrule
the esisting values.
- read the grammar for this specific topartner; these values overrule
the esisting values.
Bots only checks for the frompartner in
bopts/usersys/partners/<editype>/<topartnerid>,py
The syntax should be in the form used in other grammars:
syntax = {
<key1>: <value1>,
<key2>: <value2>,
(etc)

}
>
> The other thing is that I will probably want to do the same logic for
> ISA05, ISA07 and ISA11. These are values that may need to be passed
> through because they vary.
>

ISA05, ISA07 can be changed, if needed, per topartner.

[ronaldleemhuis]

I put the receiver-specific syntax for partner 54771 in a file named
54771.py in the partners directory, and it works just as you said it
would, Henk-Jan. Thanks for your patience. It's hard to believe that
something so straightforward could be such a stumbling block for me.

I would like to make a plugin for "converting" X12 837004010X098A1
files to the same output format -- basically a test of concept and a
starting point for modifications, and provide it to the Sourceforge
site.

I hesitate to provide modified grammar and record definition files for
fear of interfering with other plugins. However, I did find an
incompatibility between the two files (837004010X098A1.py and
records004010.py, as the former refers to a field "FRM" not defined in
the latter. Maybe I could define the "FRM" field in the records file
and rename it records004010X098A1.py and then refer to that somewhere.

Ronald Leemhuis

[ronaldleemhuis]

Here is a somewhat treacherous trap when using bots 2.6 in webserver
mode:

If you go to System, then view/edit counters and edit one of the
counters so its value is blank (i.e. not zero or any other number), it
causes the translator to lock up with obscure error messages for the
applicable receiving partner.

Ronald Leemhuis

[henk-jan ebbers]

yes.
this is dangerous.
I'll check; I expected that this need to be a number.

henk-jan

[henk-jan ebbers]

ronaldleemhuis wrote:
> I put the receiver-specific syntax for partner 54771 in a file named
> 54771.py in the partners directory, and it works just as you said it
> would, Henk-Jan. Thanks for your patience. It's hard to believe that
> something so straightforward could be such a stumbling block for me.
>
> I would like to make a plugin for "converting" X12 837004010X098A1
> files to the same output format -- basically a test of concept and a
> starting point for modifications, and provide it to the Sourceforge
> site.
>

OK, if you send it to me I will check it and publish this on the
sourceforge site.
also provide some text for what is in the plugin please.

> I hesitate to provide modified grammar and record definition files for
> fear of interfering with other plugins.

this could arise when the same file is twice in plugins.
When a plugin is read, an existing file (with the same name) is renamed,
never oerwritten.

The problem now for the x12 grammars is that not all segmetns etc are in
it (as you noticed).
Michael is working on getting the grammars better; this problem will be
not important than.

kind regards,
henk-jan

[henk-jan ebbers]

Ronald,

I fixed this: empty value is not accepted anymore.

btw, only users with administrative rights can edit these values.

henk-jan

[ronaldleemhuis]

I have a bots configuration that takes X12 837 version 004010 medical
claim files and runs them through the x122x12.py conversion program
and regenerates the same files in another directory. This is not as
trivial as it sounds, because it demonstrates the functionality needed
to do more useful conversions, such as from a fixed text file to 837,
from version 4010 to version 5010 etc. I'm hoping that somebody can
help figure out grammars for version 5010 and a fixed file format that
could be generated from practice management software "insurance form"
formatting capability. (I was actually thinking of using the 4010
grammar as a template for 5010 and just renaming it in all pertinent
places as a starting point. Then I could send claims to Blue Shield
or Medicare , purporting to be 5010, and see exactly what diagnostic
errors result. It may be that I need to change only a few things
rather than all the official differences to get to a functional 5010.)

Henk-Jan has come up with an update to the envelope.py script that
allows customization of the envelope to meet the requirements of
various receivers by placing some receiver-specific syntax files in
the ..\partners\ directory.

I have created three anonymous test files with fictitious partnerIDs
and fictitious patient names, one each for Medicare, Blue Shield and
for a commercial clearing house.

I have tested the output files with Medicare, Blue Shield and Anvicare
clearing house, and they all pass muster when I modify grammar.py file
so that:

'add_crlfafterrecord_sep' is nothing i.e. '' instead of the lf/cr
characters. While the lf/cr in the output file makes it easier to
read, such a file is not acceptable to Medicare or Blue Shield.

What is the best way to create a plugin with documentation to include
the documentation, infile data, the outfile data, channels, routes,
translations, partnerIDs, Henk-Jan's updated envelope.py and the
change of the 'add_crlfafterrecord_sep' variable in grammar.py? I do
notice that I can copy the whole \python26 tree to the Windows desktop
for backup purposes.

Ronald Leemhuis MD

[henk-jan ebbers]

ronaldleemhuis wrote:
> I have a bots configuration that takes X12 837 version 004010 medical
> claim files and runs them through the x122x12.py conversion program
> and regenerates the same files in another directory. This is not as
> trivial as it sounds, because it demonstrates the functionality needed
> to do more useful conversions, such as from a fixed text file to 837,
> from version 4010 to version 5010 etc. I'm hoping that somebody can
> help figure out grammars for version 5010 and a fixed file format that
> could be generated from practice management software "insurance form"
> formatting capability. (I was actually thinking of using the 4010
> grammar as a template for 5010 and just renaming it in all pertinent
> places as a starting point. Then I could send claims to Blue Shield
> or Medicare , purporting to be 5010, and see exactly what diagnostic
> errors result. It may be that I need to change only a few things
> rather than all the official differences to get to a functional 5010.)
>

Michael is working on generating the x12 grammars; I help with testing.
If this succeeds (and so far there's not reason it won't succeed) this
means the grammars for the 5010 will be available.

> Henk-Jan has come up with an update to the envelope.py script that
> allows customization of the envelope to meet the requirements of
> various receivers by placing some receiver-specific syntax files in
> the ..\partners\ directory.
>
> I have created three anonymous test files with fictitious partnerIDs
> and fictitious patient names, one each for Medicare, Blue Shield and
> for a commercial clearing house.
>
> I have tested the output files with Medicare, Blue Shield and Anvicare
> clearing house, and they all pass muster when I modify grammar.py file
> so that:
>
> 'add_crlfafterrecord_sep' is nothing i.e. '' instead of the lf/cr
> characters. While the lf/cr in the output file makes it easier to
> read, such a file is not acceptable to Medicare or Blue Shield.
>

you can do this better in the syntax section a of a grammar file.
that way with an update of Bots you will not loose this configuration.

> What is the best way to create a plugin with documentation to include
> the documentation, infile data, the outfile data, channels, routes,
> translations, partnerIDs, Henk-Jan's updated envelope.py and the
> change of the 'add_crlfafterrecord_sep' variable in grammar.py? I do
> notice that I can copy the whole \python26 tree to the Windows desktop
> for backup purposes.
>

in bots 1.6.2 there is in GUI/bots-monitor an option: "make
configuration plugin".
the example files in botssys/infile will be included.
the envelope.py will not be in it, though.
2 options:
- place it in the 'root' of the plugin (a plugin is just a plain zip-file.
possible danger is that if I bring out bots 1.6.3 etc, the envelope.py
will also be updated.
- make a seperate plugin for template.py and document that this plugin
has to be installed to use your mainplugin if using <1.6.3
it's not very elegant, but works.

kind regards,
henk-jan
> Ronald Leemhuis MD
>
>

[ronaldleemhuis]

It appears that some of the important changes from X12 837 4010 to X12
837 5010 include the following:

ISA11 must be '^' rather than 'U' -- a newly defined repetition
separator for versions after 4010.

ISA12 must be '00501' rather than '00401.'

GS08 must be '005010X222 ' rather than '004010X098A1.' I'm not sure
about left or right justification and the need for three trailing
spaces. It appears that some receivers parse the ISA and/or GS
sections as though they were fixed length segments.

The entire 'REF*87*004010X098A1~' section should be removed entirely.

The X12 manuals show a long list of official differences between 4010
and 5010 versions of X12 837 files, but I'm tempted to approach the
problem by making only the changes necessary for my medical practice
early on, rather than trying to make everything perfect from the
outset.

Ronald Leemhuis MD

[henk-jan ebbers]

indeed it would be very interesting to read 837 as version 004010 and
write it as 005010, including the differences you noticed above.
if there are version 005010 messages and records this would be easy in
itself.
(Michael is working on generating the x12 grammars; the messages are OK
now; he is working on the records now).

if you have a good relationship with the receiver of the messages, an
approach could be:
send test message in both versions.
they probably convert these message to an inhouse format.
check if the inhouse-file is the same; if so the conversion is OK.

kind regards,
henk-jan

[ronaldleemhuis]

I took an X12 837 4010 claim file and used a text editor to make these
changes to masquerade as a 5010 version of the files. I sent this off
in production mode to the Anvicare (freeclaims.com) clearing house,
and they accepted it without any errors. Not only that, but the
details of the claim went into their system correctly for me to view.

Now this is a simple claim for medical services in a doctor's office,
but it shows that in this situation, we're just about there.

Medicare and Blue Shield appear to have a timetable where they will
accept V 5010 for testing starting on 01/01/2011 and will require V
5010 on 01/01/2012 -- I'm fairly sure of these dates.

Ronald Leemhuis

[ronaldleemhuis]

I have put together a plugin that may serve as a starting point for
converting X12 837 4010 files to X12 837 5010 files for medical
insurance claims. I was able to edit manually a version 4010 file and
send it to the clearing house masquerading as a version 5010 file, and
it was fully accepted. The changes I made were minor.

It is likely that an all-encompassing conversion would be
complicated. However, it may be possible to approach this from the
minimalist side by making the minimum changes necessary and then
dealing with errors and inadequacies as they are discovered. This can
be done at the same time others are working on a more comprensive and
all-encompassing solutions.

Here is, again, what I did:

I changed ISA11 to '^'.
I changed ISA12 from '00401' to '00501'.
I changed GS08 from '004010X098A1' to '005010X222 '. I put in two
spaces at the end but am not sure they are necessary.
I removed the cf/lf at the end of all lines;

I removed the content section between the tilde (~) separators that
contains ~REF*87*.... This is not part of the 5010 standard.

Only the last of the listed changes is concerned with the content of
the message. The others are in the envelope (i.e. ISA and GS)
sections.

The rudimentary plugin does this:

I copied the 837004010X098A1.py grammar to 837005010X222.py with the
understanding that it will be modified to conform to the actual 5010
standards. As a starting point, it will just be 4010 masquerading as
5010.

I copied the passthrough translation x122x12.py to a new script called
x12_837_4010_to_x12_837_5010_basic.py. Ultimately this will be
modified to make the appropirate translations. In my simple case the
first step will be to eliminate the ~REF*87* segment.

I copied records004010.py to a new file named records005010.py.
Again, this can be edited to conform to the new standards.

I changed the partner-specific codes in the /partners/ directory to
accomplish the necessary changes to the envelope.

I did not yet figure out how to modify the translation script to
remove the ~REF*87* segment or to accomplish anything else specific.

I have not yet edited the pertinent files so they reference internally
to the other newly created files, but that should be fairly
straightforward.

I activated the new translation and removed any other translations etc
from the installation I have here.

I've gotten it to run without errors, but for some reason I can't get
it to use the partner-specific scripts for customization, and I
haven't figured out how to do anything useful with the translation
script itself (such as removing the !REF*87* segment.

This is very rudimentary and definitely not ready to be shared
widely. I plan to send a copy to Henk-Jan Ebbers just for his
information. I'll be glad to collaborate with anybody who is
interested.

Ronald Leemhuis MD

[ronaldleemhuis]

I've cleaned up the plugin for 837 4010 to 837 5010 conversion. This
includes renamed grammars and mapping file from the 4010 side. So
far, theseare just renamed and reconfigured.

Right now it makes the ISA anf GS segments correctly for 5010, but the
message content so far is that of 4010 in a 5010 envelope.

To make a messae acctually work I need at the least to to remove the
first REF1 segment. Other changes will certainly be needed too.

Now I'm trying to figure out how to modify x122x12 as renamed to skip
or add specific changes. If I remove REF1 from the grammar I get an
erroe necause the mapping still looks for it.

Ideas?

Ronald Leemhuis

[henk-jan ebbers]

Hi Ronald,

as I expect to have generated grammars for x12 soon, it would be good to
use the generated grammars by that time.

some background information:

internally bots stores messages as 'trees', where the ST segment is the
root.
these trees are independent of editype, message type etc.
that is the reason why a message can be converted eg like you do from
4010 to 5010, but one could also eg use other separator characters.

you now use in the mapping script transform.inn2out(inn,out) which
copies the incoming message tree as the outgoung message tree.

if you want to manipulate the outgoing message tree, yes that is possible:
- put() (and putloop()): adds data to message tree. This is most used in
writing a message tree.
- delete() - this is probably what you are looking for.
- change()

delete and change: see plugin
http://sourceforge.net/projects/bots/files/botsplugins/demo_changedelete_1_5_4.zip/download
this contains good examples.

kind regards,
henk-jan

[Ronald Leemhuis]

Thanks Henk-Jan, for your guidance.  I am reviewing the change_delete script as you recommended.  My first shot, to write a single

 

out.delete({'BOTSID':'REF'},'REF01':'87'})

 

statement was not the answer.  It's going to require some consideration on my part to see what your logic is doing in the change_delete script.

 

On another track, I noticed that the record definition files contain 'M' or 'C' entries for each definition.  (i.e. mandatory, conditional)  I'm wondering whether it would be feasible to allow another entry such as 'P' for prohibited.  That could be configured to nullify the effects of that definition just like commenting it out.  What I am thinking, though, is that successive versions of a configuration file can be created by adding new definitions and removing old ones.  If one uses an all-inclusive template for the definitions in all versions, then those definitions that are active for a given version can be specified by manipulating the M, C or P values.  It could help in finding errors and in comparing versions.

 

Ronald Leemhuis

 

 

[henk-jan ebbers]

Ronald Leemhuis wrote:
> Thanks Henk-Jan, for your guidance. I am reviewing the change_delete
> script as you recommended. My first shot, to write a single
>
> out.delete({'BOTSID':'REF'},'REF01':'87'})

point here is what REF?
take a look at the grammar.
there is a REF at line 14; but there is also a ref at line 33
these REF have different purposes (semantic context).
the one at line 14 is nested directly under ST; so it is a reference for
the whole document (claim)
the REF at line 33 is nested this way: ST-HL-MN1; so it is related to
....(something else, I am not to familiar with the 837 message).

How to address the different REF segments?

the one at line 14:
{'BOTSID':'ST'},{'BOTSID':'REF','REF01':'87'}
The one at line 33:

{'BOTSID':'ST'},{'BOTSID':'HL'},{'BOTSID':'NM1'},{'BOTSID':'REF','REF01':'87'}
(take a good look at the structure of the grammar to see how this works).
(main clue: bots sees an edi message as a tree.. 'ST' is the root)

>
> statement was not the answer. It's going to require some
> consideration on my part to see what your logic is doing in the
> change_delete script.
>
> On another track, I noticed that the record definition files contain
> 'M' or 'C' entries for each definition. (i.e. mandatory,
> conditional) I'm wondering whether it would be feasible to allow
> another entry such as 'P' for prohibited. That could be configured to
> nullify the effects of that definition just like commenting it out.
> What I am thinking, though, is that successive versions of a
> configuration file can be created by adding new definitions and
> removing old ones. If one uses an all-inclusive template for the
> definitions in all versions, then those definitions that are active
> for a given version can be specified by manipulating the M, C or P
> values. It could help in finding errors and in comparing versions.

A grammar is the definition of a message. Not of its use.
If a grammar is only the definition of a message, a grammar can be 'OK':
no more discussions and editing, it can be used by everybody; if the
grammars are available nobody needs to look at it again, it is done,
ready, finished, we can go on.
if this seems trivial: it is not. <rant mode>EDI is still wrestling with
very primitive concepts. The same issue as playing in 1990......in US
the situation is quite dramatic. Usage of a VAN or AS2 is about
required; pricing is ridiculous for what is the simple transfering of
files......I do not see any 'added value'.....technology is basically
pre-internet (say around 1992). Using email (if needed with encryption)
would be more reliable, cheaper & faster to set up. EDI people send a
lot of email messages communication with each other about gettiing the
clumsy edi connections to work.....if edi connection where anywhere as
good as standard email they would be using edi communications for this
wouldn't they?</rant mode>

So let's see how what you want can be realized.
mmmh, the functionality seems to be covered by delete().
Is that right (might not be the fastest/shortest way of doing this)?

the list of differences between the 837 versions is focusing upon the
technical details.
Do you know if the 'functional requirements' are different?
There might be soem real differences in usage.

[Ronald Leemhuis]

You're right, Henk-Jan.  In my medical practice, every 2-3 years, I have to buy some new, expensive, software just to meet a changing X12 standard.  The authorities say it is to increase efficiency, but from my standpoint it is nothing but an expense that takes a system that works and makes it into one that doesn't.

 

I see your point about creating the grammar exactly right once and for all and then leaving it alone.  Sure makes sense at your level.  However, if a tiny medical practice is trying to get something to work -- barely -- then it is tempting to make the mininum change necessary to achieve the result.

 

Before responding to you last time I tried this mapping script, and I got the error that follows.  It's the first REF that has to be removed, the one in which the REF01 value is 87.

 

 

#mapping-script
import bots.transform as transform

def main(inn,out):

 transform.inn2out(inn,out)

out.delete({'BOTSID':'ST'},{'BOTSID':'REF','REF01':'87'})
                                            

 

Response:

 

Traceback (most recent call last): ScriptImportError: Import file: "C:\Python26\lib\site-packages\bots\bots.usersys\mappings\x12.x12_837_4010_2_x12_837_5010_basic", type: "mappings", error: Traceback (most recent call last): NameError: name 'out' is not defined

 

 

I'm really impressed with your bots package.  I have never seen anything else like it.  I like its versatility and its clear and logical construct of channels, routes, mappings, grammars etc.  A less astute developer would have approached this in an ad hoc manner requiring a whole new program for each task.

 

Ronald Leemhuis

[Jim Qiu]

Hi.

 Welcome to join bots.

 I agree with Ronald.

 One grammar set is not perfect for some situation.

 I think it's better if we could have application-scope grammer, for example, different grammar IFTSTAD04BUNNA  for application1 & application2.

But there is no comcept of application in Bots. So the grammar is shared .


Jim

2010/3/15 Ronald Leemhuis <ronaldl...@gmail.com>

[henk-jan ebbers]

Jimmy,

my point here is that the functionality is already there (it can be
achieved in a mapping).
it is no use duplicating the same functionality.

kind regards,
henk-jan

Jim Qiu wrote:
> Hi.
>
> Welcome to join bots.
>
> I agree with Ronald.
>
> One grammar set is not perfect for some situation.
>
> I think it's better if we could have application-scope grammer, for
> example, different grammar IFTSTAD04BUNNA for application1 &
> application2.
>
> But there is no comcept of application in Bots. So the grammar is
> shared .
>
>
> Jim
>
> 2010/3/15 Ronald Leemhuis <ronaldl...@gmail.com

> <mailto:ronaldl...@gmail.com>>

[henk-jan ebbers]

Jim,

at the base there is only one IFTSTAD04BUNNA: the official UN message.
So it i 'logical' to have one grammar for this.

if there are other needs, this might lead to different grammars.
but so far I do not see such need.
I have seen a lot of custom-made grammars based on UN messages. They
delete some segments, delete some elements.
But this never made nay sense to me. The messages produced where exactly
the same.

if you have a good use case for me....

henk-jan

Jim Qiu wrote:
> Hi.
>
> Welcome to join bots.
>
> I agree with Ronald.
>
> One grammar set is not perfect for some situation.
>
> I think it's better if we could have application-scope grammer, for
> example, different grammar IFTSTAD04BUNNA for application1 &
> application2.
>
> But there is no comcept of application in Bots. So the grammar is
> shared .
>
>
> Jim
>
> 2010/3/15 Ronald Leemhuis <ronaldl...@gmail.com

> <mailto:ronaldl...@gmail.com>>

[henk-jan ebbers]

Ronald,

the mapping scipt is python.
python has one unusual thing: instead of 'BEGIN ....END' of brackets
'{....}' it uses indentation for blcoks.
You have:

#mapping-script
import bots.transform as transform
def main(inn,out):
transform.inn2out(inn,out)
out.delete({'BOTSID':'ST'},{'BOTSID':'REF','REF01':'87'})

It should be (watch te indentation:

#mapping-script
import bots.transform as transform
def main(inn,out):
transform.inn2out(inn,out)
out.delete({'BOTSID':'ST'},{'BOTSID':'REF','REF01':'87'})

kind regards,
henk-jan

[Jim Qiu]

Henk-Jan,

 That's a good tip. I will try that.

Jim

2010/3/15 henk-jan ebbers <hjeb...@gmail.com>

[Ronald Leemhuis]

That worked, Henk-Jan.  I can hardly believe it -- I had the right code except for indentation.  I didn't have a clue that indentation even mattered.  So it seems that my very rudimentary plugin now takes my X12 837 4010 files and converts them to X12 837 5010 files that are accepted by my clearing house.

A full version would make ALL the necessary changes to cover all possible transactions etc.  I'm going to play with this to see what it actually does and how trade partners react to it over time.

Ronald Leemhuis MD

[henk-jan ebbers]

yes, the indentation.
bit strange in the beginning.
but after a while you discover it is great: everybody formats the code
in the same way.
this makes mapping scripts etc from others much more readable.

try to avoid tabs, use (4) spaces.
( a lot of editors automatically inset spaces instead of tabs.)

henk-jan

henk-jan