MOD - add fixed recordlength wrapped lines capability to output

[BikeMike]

Hi All,
This plugin adds capability for fixed record length output to Bots
2.0.2. I'm not sure if it will be useful to anyone else but it can
easily be included in the standard code with no effect unless the syntax
is specified to use it.

I needed this for generating files into Movex ERP system. The file
format was csv-like with space field separators, backslash-zero record
separators, wrapped around every 80 characters. Wierd, I know!
This is specified in the grammar syntax as...
'quote_char':'',
'field_sep':' ',
'record_sep':'\\0 ',
'add_crlfafterrecord_sep':'',
'fixed_recordlength':80, # this is my modification

outmessage.py _records2file (line 296 onwards modified)

I'm actually planning to use Bots to bypass the standard Movex input
process incorporating TrustedLink EDI/400, which has a huge licensing
cost and is now over 10 years old.

Also I found a typo preventing an error message from working correctly.
In transform.py at line 337, tabel=ccodeid should be table=ccodeid

--
Kind Regards,
Mike

[hjeb...@gmail.com]

hi Mike,

do you have an example of such a file?

henk-jan

[BikeMike]

Hi Henk-jan,
Here's a sample of the output. It has been mapped from EDIFACT input.
This is still work in progress but looks like it will work fine. The
output is sent by ftp to an AS/400 system where it goes into an 80
character fixed width file, then a Movex (ERP) program inports this file.

I tried to put each record on a new line but the import failed, and
anyway the \BOT and \EOR records are longer than 80. So easiest way is
to just reverse-engineer what the old mapping does!

Kind Regards,
Mike

[BikeMike]

Once orders is working I have to do the reverse for invoices, so will
have to map a file similar to above into EDIFACT INVOICD93AUN.

[henk-jan ebbers]

hi Mike,

this is a funny format!
actually I know it quite well, a lot of edifact uses this (mainframes!).
an US X12 is is even very common.
bots can handle this incoming. but now bots can generate it!
this is so cool!
in edifact/x12 it woudl be look like you have a mainframe (or AS400)

the syntax parameter has name 'fixed_recordlength'. that might be
confusing, as bots already has a fixed format.
the name I know for this format is 'wrapped'. Ok if the parameter is
called something like 'wrap_to_fixed_recordlength'?

btw did I even mention that I worked with Movex for 3/4 of a year?

henk-jan

[BikeMike]

Hi Henk-jan,
Yes I have heard it called "wrapped' format too, so changing the
parameter name is fine. Actually I think shorten it to "wrap_length",
because it's a maximum length rather than a fixed length. You could
still have Cr-Lf record separators as well, although I don't in this
case.

I didn't know you had worked with Movex before. I have been working on
it for around 11 years now, we installed it to be "Y2K ready". It is
mostly being phased out and replaced by SAP next year (which is why I
started looking for a good EDI solution and found Bots). The old EDI
interface into Movex is TrustedLink EDI/400, with mappings provided by
Movex, the rest of the interface is a very convoluted series of
programs, control records and file copying. It's almost like they
wrote 90% of their own EDI interface, then decided the final bit
(mapping) was too hard and bolted EDI/400 on the end of it, but used
only a fraction of it's capabilities. We're still on the same version
we installed back then, Movex 10.6.

I have found that with Bots I can eliminate EDI/400 and all the
intermediate steps, and go straight to the application interface
files. (wish I'd had this solution years ago!) I've since had a look
at the Movex outputs, they are much more conventional "fixed" record
formats. Only the input files are wrapped like this.

One company previously tied closely with us is now separate and will
be sticking with Movex, so this iwill be a simpler solution for them.

Kind Regards,
Mike

[henk-jan ebbers]

hi mike,

added this.

BikeMike wrote:
> Hi Henk-jan,
> Yes I have heard it called "wrapped' format too, so changing the
> parameter name is fine. Actually I think shorten it to "wrap_length",
> because it's a maximum length rather than a fixed length. You could
> still have Cr-Lf record separators as well, although I don't in this
> case.
>
> I didn't know you had worked with Movex before.

yes, but in a 'fashion/apparel' sector, a heavily customized (and
outdated) version of movex.
as I understood they had to customize it because Movex is for the
automotive sector.
add they kept on customizing....

> I have been working on
> it for around 11 years now, we installed it to be "Y2K ready". It is
> mostly being phased out and replaced by SAP next year (which is why I
> started looking for a good EDI solution and found Bots). The old EDI
> interface into Movex is TrustedLink EDI/400, with mappings provided by
> Movex, the rest of the interface is a very convoluted series of
> programs, control records and file copying. It's almost like they
> wrote 90% of their own EDI interface, then decided the final bit
> (mapping) was too hard and bolted EDI/400 on the end of it, but used
> only a fraction of it's capabilities. We're still on the same version
> we installed back then, Movex 10.6.
>

I have seen this sort of setup.
this can happen if they need extra functionality in the edi part....and
edi translator can not do this, or they do not watn to use translatior
for this.
and it will keep growing and growing. that can go pretty fast.

> I have found that with Bots I can eliminate EDI/400 and all the
> intermediate steps, and go straight to the application interface
> files. (wish I'd had this solution years ago!) I've since had a look
> at the Movex outputs, they are much more conventional "fixed" record
> formats. Only the input files are wrapped like this.
>
> One company previously tied closely with us is now separate and will
> be sticking with Movex, so this iwill be a simpler solution for them.
>

nice
> Kind Regards,
> Mike
>
>

[BikeMike]

Hi Henk-jan,
As the final stage of my "wrapped" file processing into Movex, I
wanted to add a single remote command at the end of an ftp session, to
call the AS/400 program which processes the received files. (the
alternative to this is having a scheduled batch process on the AS/400
too, but I think this is better as it runs immediately and only when
needed)

I could not figure out a standard way to do this with a
communicationscript, as they are not called for ftp channels. Also the
session would need to be passed.

To quickly achieve this I added a couple of lines in communication.py,
using the "parameters" field from the channel. It doesn't seem to be
used for anything else that I can find. Whatever is entered there is
sent to the remote server before closing the session. Perhaps you can
suggest a more universal way of scripting this?

def disconnect(self):
# if channel has "parameters" send it as a remote command
# This is used to call a program after sending files to AS/400
if self.channeldict['parameters']:
self.session.sendcmd(self.channeldict['parameters'])
try:
self.session.quit()
except:
self.session.close()

In parameters I enter RCMD CALL PGM(LIBRARY/PROGRAM)

If the program call fails you get a 550 error which Bots reports as a
process error.

Kind regards,
Mike

[henk-jan ebbers]

hi Mike,

in communication.py there are 'exits' for user defined scripting.
A good solution here would be to add a new exit for user defined scripting.

I will take a better look at this tonight.

henk-jan

[BikeMike]

Hi Henk-jan,
I noticed you added my outmessage.py code to the repository but made
some slight changes.
You replaced
s = ''
for record in self.records: # join all records
s = s + self._record2string(record)
with
s = ''.join(self.records)

When I test your version I get an error.
line 301, in _records2file s = ''.join(self.records) TypeError:
sequence item 0: expected string, list found

Kind regards,
Mike

[henk-jan ebbers]

hi mike,
yes, definitely wrong. I will change it.
did not make a test case for this.
I will, probably for edifact.

of all changes for 2.0.3 being able to produce wrapped edifact is not the most important one,
but it is my favourite one!!
(never realized this could be this easy.)

henk-jan

[henk-jan ebbers]

hi Mike,

about the wrapped format:
I think the wrapped format does not play nice with enveloping.
is that right?

henk-jan

[BikeMike]

Yes you are right. I did not need it for an enveloped format, but just
tried it with an edifact output.
This would have to be done in a different way I guess, to make it work
with enveloping. The "wrapping" part would need to be done after the
enveloping.

Kind Regards,
Mike

[hjebbers]

hi Mike,

I want to remove the 'wrapped' thing.
main reason is that this output (if needed at all) is not wrapped fro the envelope.
so AFAICS it is not useful.

Can you agree on this?

kind regards,
henk-jan

[Harold DeWayne]

henk-jan, Mike --

I've seen this handled by a wrap command in the FTP session when sending data to a mainframe... I don't remember the command I used, but once the FTP connnection was established, the wrap was issued, then the data transmitted.  Worked great when I used it.

[BikeMike]

Hi Henk-jan,
My usage for this is not in "real EDI" formats, only for csv. So there is no enveloping to worry about.
I guess the other alternative would be to write a postprocess routescript to do the wrapping.

Kind Regards,
Mike

[henk-jan ebbers]

hi Mike,

OK, did not know (I got the impression that is was was for edifact, but that probably is what I wanted to make of it).

will not remove this.

kind regards,
henk-jan

> --
> You received this message because you are subscribed to the Google Groups "Bots Open Source EDI Translator" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to botsmail+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Angelo de Laat]

Hey,

Is this still integrated in bots 3 ?

What is the syntax because I can not find it in the wiki.

https://code.google.com/p/bots/wiki/GrammarsSyntax 

Kr,

Angelo

[BikeMike]

Hi Angelo,
Yes you're right, it is missing from the wiki... I will update it.
Still there in version 3.

Kind Regards,
Mike