database cleanup

180 views
Skip to first unread message

Dileep ds

unread,
Sep 7, 2010, 10:18:48 AM9/7/10
to bots...@googlegroups.com
Hi,
    I am using bots for 278 X12 messages, there is one scheduler which will call bots-engine three times in a week and it will create an X12 file each time. The system is working
for the last 5 months. Does it require to clean the sqlite db (bots internal db), which having the process details as the database size is increasing..?

--
Thanks and Regards
Dileep.D.S

Steven Hammond

unread,
Sep 7, 2010, 3:23:22 PM9/7/10
to bots...@googlegroups.com

I’m in the process of testing version 2 for production use by our go live date of 11/1/10.  Essentially, we will be using EDI-BOTS to just map some documents:

 

X12

850->FIXED FORMAT (to be imported to the ERP system)

997 GENERATED

FIXED FORMAT->810 (exported from ERP system)

FIXED FORMAT->856 (exported from ERP system)

Misc: print/email documents (830, 812, 824, etc)

 

We already have a communication server to send/receive all the documents (AS2, SFTP, FTP) so really, we just want to use EDI-BOTS to convert the files back and forth.  We want to just drop it in to replace a proprietary component that is no longer compatible with our new ERP system.  Okay, some basic questions:

 

1.      Is version 2 ready for production?

2.      Are there grammars available for X12 versions (4010,4040, 5010).

3.      Are there samples out there available of mapping X12 documents (e.g., 850) to a fixed format.

4.      Can EDI-BOTS communicate with MySQL 5.XX (we already have a database with all our documents in it).

5.      Finally, worst case scenario, we are running out of time – is there anyone available to help create the grammars (e.g., fixed format) and maps?  What is the cost of such a service?  Probably just get the basics done (grammar) and one of the maps for each document and then we could do the rest of the maps.

 

Steve Hammond

Technical Chemical Company

Cleburne, TX 76033

BikeMike

unread,
Sep 7, 2010, 6:35:32 PM9/7/10
to Bots Open Source EDI Translator
On Sep 7, 11:18 pm, Dileep ds <dileep...@gmail.com> wrote:
> Does it require to clean the sqlite db (bots internal
> db), which having the process details as the database size is increasing..?

Hi Dileep,
The bots-engine has a built in cleanup process which deletes the
records after a configured time (default is 30 days). But there was a
bug in version 2.0.1, so make sure you upgrade to 2.0.2,

There is also an issue with SQLite databases that they do not shrink
when records are removed. You can perform a procedure called "vacuum"
the database and this will remove all the empty space and shrink it.
This is not built in to Bots, you need a separate database utility to
do it. I have written a daily housekeeping script to clean up, back up
and vacuum our database. (I will post it later as a plugin).

See my post http://groups.google.com/group/botsmail/browse_thread/thread/d99b14231bd51012#
Unless your database is many hundreds of MB you don't need to worry!

BikeMike

unread,
Sep 7, 2010, 7:22:45 PM9/7/10
to bots...@googlegroups.com
Here's my plugin...

>> usersys\routescripts\daily_housekeeping.py
Routescript to do daily housekeeping...
* 7 day rolling backups
- Configuration backup (plugin)
- Database backup (to zip file)
* Run cleanup
* Vacuum the database (using sqlite3.exe)

Download sqlite3.exe from http://sqlite.org/download.html
Save it as botssys\sqlitedb\sqlite3.exe (same folder as the database)


Kind Regards,
Mike

Plugin - Daily housekeeping 1.zip

BikeMike

unread,
Sep 7, 2010, 7:42:10 PM9/7/10
to Bots Open Source EDI Translator
On Sep 8, 4:23 am, "Steven Hammond" <bcf1...@gmail.com> wrote:
> 1. Is version 2 ready for production?

Hi Steven,
I would say yes, version 2.0.2 is working great for us.
http://groups.google.com/group/botsmail/browse_thread/thread/d99b14231bd51012#

> 4. Can EDI-BOTS communicate with MySQL 5.XX (we already have a database
> with all our documents in it).

Do you mean using MySQL as the Bots back-end database, or reading/
writing EDI data from it. If the latter, you would have to do it with
communication scripts I think. I did try some time ago to use MySQL as
the back-end without success, but we have found SQLite perfectly up to
the task.

> 5.      Finally, worst case scenario, we are running out of time - is there
> anyone available to help create the grammars (e.g., fixed format) and maps?

I don't want to step on henk-jan's toes! but I may be able to help
with this. I don't have any X12 mapping experience but have done some
fixed grammars and mappings to/from edifact and idoc. How well
specified are your fixed formats and mapping requirements?

Kind regards,
Mike

Mellerbeck

unread,
Sep 8, 2010, 12:30:50 PM9/8/10
to Bots Open Source EDI Translator
Yes lots of grammars available :) click grammars for X12
http://sourceforge.net/projects/bots/files/

-Michael

On Sep 7, 12:23 pm, "Steven Hammond" <bcf1...@gmail.com> wrote:
> I'm in the process of testing version 2 for production use by our go live
> date of 11/1/10.  Essentially, we will be using EDI-BOTS to just map some
> documents:
>
> X12
>
> 850->FIXED FORMAT (to be imported to the ERP system)
>
> 997 GENERATED
>
> FIXED FORMAT->810 (exported from ERP system)
>
> FIXED FORMAT->856 (exported from ERP system)
>
> Misc: print/email documents (830, 812, 824, etc)
>
> We already have a communication server to send/receive all the documents
> (AS2, SFTP, FTP) so really, we just want to use EDI-BOTS to convert the
> files back and forth.  We want to just drop it in to replace a proprietary
> component that is no longer compatible with our new ERP system.  Okay, some
> basic questions:
>
> 1.      Is version 2 ready for production?
>
> 2.      Are there grammars available for X12 versions (4010,4040, 5010).
>
> 3.      Are there samples out there available of mapping X12 documents
> (e.g., 850) to a fixed format.
>
> 4.      Can EDI-BOTS communicate with MySQL 5.XX (we already have a database
> with all our documents in it).
>
> 5.      Finally, worst case scenario, we are running out of time - is there

henk-jan ebbers

unread,
Sep 8, 2010, 3:57:09 PM9/8/10
to bots...@googlegroups.com

Steven Hammond wrote:
>
> I�m in the process of testing version 2 for production use by our go

> live date of 11/1/10. Essentially, we will be using EDI-BOTS to just
> map some documents:
>
> X12
>
> 850->FIXED FORMAT (to be imported to the ERP system)
>
> 997 GENERATED
>
> FIXED FORMAT->810 (exported from ERP system)
>
> FIXED FORMAT->856 (exported from ERP system)
>
> Misc: print/email documents (830, 812, 824, etc)
>
> We already have a communication server to send/receive all the
> documents (AS2, SFTP, FTP) so really, we just want to use EDI-BOTS to
> convert the files back and forth. We want to just drop it in to
> replace a proprietary component that is no longer compatible with our
> new ERP system. Okay, some basic questions:
>
> 1. Is version 2 ready for production?
>

yes.
bots consists of 2 parts: the web sever and the engine.
the engine does the actual translations and communications.
the 2.* series have a rewrite of the GUI. Not the engine.
so the critical parts of bots are not very different from the 1.* series.
(I am not saying the GUI in not important, but as an EDI man I think it
is less critical. Just my opinion)


>
> 2. Are there grammars available for X12 versions (4010,4040, 5010).
>

yes, on the web site.


>
> 3. Are there samples out there available of mapping X12 documents
> (e.g., 850) to a fixed format.
>

yes, there is a plugin for this. The trasnlation is quite simple, though.


>
> 4. Can EDI-BOTS communicate with MySQL 5.XX (we already have a
> database with all our documents in it).
>

yes.
>
> 5. Finally, worst case scenario, we are running out of time � is there

> anyone available to help create the grammars (e.g., fixed format) and
> maps? What is the cost of such a service? Probably just get the basics
> done (grammar) and one of the maps for each document and then we could
> do the rest of the maps.
>

yes, see www.ebbersconsult.com

kind regards,
henk-jan ebbers

Reply all
Reply to author
Forward
0 new messages