Problem Echi_Records database

[zakkiarsad]

Dear Jason,
Thanks form your comment. You right, thread was to long and not
related, so my appologize to get a new thread here..
I've allready post my database.log file onto git://gist.github.com/57242.git
or g...@gist.github.com:57242.git
Please check, the problem only inserting on echi_records but its no
problem with echi_log table.

[JasonGoecke]

Need your application.log as well. And not sure how you are linking
your Gists, but I had to go here to see it:

http://gist.github.com/57242

[zakkiarsad]

application.log sample uploaded...
http://github.com/57262
please check again

On Feb 3, 9:07 am, JasonGoecke <jsgoe...@gmail.com> wrote:
> Need your application.log as well. And not sure how you are linking
> your Gists, but I had to go here to see it:
>
> http://gist.github.com/57242
>

[JasonGoecke]

No gist at that link...

On Feb 2, 6:42 pm, zakkiarsad <zakkiar...@gmail.com> wrote:
> application.log sample uploaded...http://github.com/57262

[zakkiarsad]

sorry... http://gist.github.com/57262
there are error on this log file but i dont know how to fix it...

[JasonGoecke]

Apologies, but I really do not have enough information to help in any
way. Not sure where to go from here.

On Feb 2, 7:15 pm, zakkiarsad <zakkiar...@gmail.com> wrote:
> sorry...http://gist.github.com/57262

[zakkiarsad]

Ok... let me start over, but hope you know what is my problem,
fisrt, i use 0.4.1 echi_converter and install successfully.
On application.yml i am use standart12.yml with binnary source file.
i use only 1 binnary data to get trial.
Echi converter start normallly. But when i open database... there are
no data on echi_records event echi_log recive 10 (times) data record.

then i try to open database.log, copy insert into statement and run
manually with mysql management tool...its is succes.
For information i've allready edit my-gist at http://gist.github.com/57262
(database.log and application.log) as you need.

Yes.. there are error on application. After origlogin field!! Here the
example :
in record 1 (No error)
Mon Feb 02 15:22:52 +0700 2009 (8589) firstvdn { type => str & length
=> 8 } value =>
Mon Feb 02 15:22:52 +0700 2009 (8589) origlogid { type => str & length
=> 10 } value => 7871
Mon Feb 02 15:22:52 +0700 2009 (8589) anslogid { type => str & length
=> 10 } value =>

but the other record (with error)
Mon Feb 02 15:22:52 +0700 2009 (8589) firstvdn { type => str & length
=> 8 } value =>
Mon Feb 02 15:22:52 +0700 2009 (8589) origlogid { type => str & length
=> 10 } value =>
Mon Feb 02 15:22:52 +0700 2009 (8589) Error processing ECHI file -
undefined method `unpack' for nil:NilClass
Mon Feb 02 15:22:52 +0700 2009 (8589) Processed file
chr1404.859.220109235207 with 1 records
Mon Feb 02 15:23:52 +0700 2009 (8589) Successfully connected to the
ECHI FTP server

Please help me to solve this problem..
Thanks

[JasonGoecke]

Alright, now that I have been able to see your full file, it looks
like you need to toggle this configuration item in application.yml:

echi_read_extra_byte:

Depending on what you have it set to now, make it the opposite, either
'Y' or 'N'. That should then clear up the problem.

On Feb 2, 8:35 pm, zakkiarsad <zakkiar...@gmail.com> wrote:
> Ok... let me start over, but hope you know what is my problem,
> fisrt, i use 0.4.1 echi_converter and install successfully.
> On application.yml i am use standart12.yml with binnary source file.
> i use only 1 binnary data to get trial.
> Echi converter start normallly. But when i open database... there are
> no data on echi_records event echi_log recive 10 (times) data record.
>
> then i try to open database.log, copy insert into statement and run
> manually with mysql management tool...its is succes.

> For information i've allready edit my-gist athttp://gist.github.com/57262

[Manos Ioannou]

Dear Jason, i d like to ask you about the same issue. We have cms
version 14.x and i have installed echi-converter 0.42 on ubuntu 8.04.
In application.yml i am using extended14.yml and the result is always
the same with extended13 or standard 12.yml. Only echi.logs table is
been updated. No other data is being inserted.
The application.log indicates the following : "Error processing ECHI
file malformed utf8 character"
The database.log indicates a rollback before the echi.logs transaction
Mysql is localhosted default settings
Please help

Thank you in advance
Manos

[JasonGoecke]

One of two things most likely:

1. You need to properly set the utf8 encoding option to config/
database.yml file
2. You need to set the extra_byte paramter properly in the config/
application.yml file

Let me know how it goes.

[Manos Ioannou]

Thanks Jason i ll check immediately
Well now i am going to try with encoding utf8 (database.yml) and
extended_version14.yml(application.yml)
i ll try right now and i will post you my result.

Thank you again for your immediate response
Manos

[Manos Ioannou]

Jason, no change at the moment. The files still can't be imported into
database (echi records table)
i have enabled extra byte (set to Y) and enabled the database.yml
(utf8 encoding )
Do you have any idea ?
It is urgent to find a solution

Thank you in advance!

Manos

On 25 Φεβ, 08:24, JasonGoecke <jsgoe...@gmail.com> wrote:

[Manos Ioannou]

Dear Jason, i have some news on my previous post.
Some of the files are being imported into database some of them not.
The usual error is the malformed utf8 character. From 3900+ records i
can see only 798 records.
I have enabled utf8 (database.yml) & extra byte (application.yml)
Do you have any idea on that ?

Thank you in advance

Manos

On 25 Φεβ, 08:24, JasonGoecke <jsgoe...@gmail.com> wrote:

[Manos Ioannou]

Jason something additional for help :
i have one more message : undefined method 'unpack' for nil:NilClass
That occured once. I will try your eval-file-echi utility and let you
know

Thanks

Manos

On 25 Φεβ, 08:24, JasonGoecke <jsgoe...@gmail.com> wrote:

[Manos Ioannou]

Dear Jason ,
i have uploaded a sample problematic file named chr0501.114, it is
from CMS R14 in extended format. I cannot load it into my mysql using
your utility, i ve tried all available options . Please have a look if
it is possible.

Again thank you very much for your help

Manos

On 25 Φεβ, 08:24, JasonGoecke <jsgoe...@gmail.com> wrote:

[JasonGoecke]

At which record does it break down. Also, please post the log showing
the failure to a gist @ http://gist.github.com and post it here.

[Manos Ioannou]

Jason i ve posted you a part of the log file that indicates the nil
error. In order to help u i will upload a tar archive called
manosfiles.tar with 2 sample problematic files (there are more ), the
application log and the database.log. It seems that some chrxxx files
are being parsed correctly and some not. In the mean time i ve tried
any possible combination between the formats(extended12,13,14
std12,13,14) and the parameters with extra_byte with no change. I
remind you that i am running CMS R14 with extended format for ECHI and
my database is (localhost linux or windows just for testing with no
behavior change)

Thank you again for your response
Manos

On 26 Φεβ, 16:47, JasonGoecke <jsgoe...@gmail.com> wrote:
> At which record does it break down. Also, please post the log showing

> the failure to a gist @http://gist.github.comand post it here.

[JasonGoecke]

If you look here:

===
ruby echi_fileeval.rb chr0030.29
File size: 6418
Version 12
File_number 29
===

The file is a version 12 file format, which ECHI-Converter should be
able to parse. As you may see here on the file chr0030.29 all records
appear to be processed fine until you get to #13:

<====================START RECORD 13 ====================>
callid { type => int & length => 4 } value => 0
echi_fileeval.rb:22:in `dump_binary': undefined method `unpack' for
nil:NilClass (NoMethodError)
from echi_fileeval.rb:96:in `convert_binary_file'
from echi_fileeval.rb:71:in `each'
from echi_fileeval.rb:71:in `convert_binary_file'
from echi_fileeval.rb:118

It appears there is extraneous data at the end of the file that should
not be there. Unfortunately I do not have the time to parse it out and
figure out what is there. This appears to be one for Avaya support
since you think you are on 14, the file is reporting 12 and you are
getting some garbage at the end of some of the files. Something seems
to be amiss.

On Feb 26, 8:06 am, Manos Ioannou <emmanouel.ioan...@gmail.com> wrote:
> Jason i ve posted you a part of the log file that indicates the nil
> error. In order to help u i will upload a tar archive called
> manosfiles.tar with 2 sample problematic files (there are more ), the
> application log and the database.log. It seems that some chrxxx files
> are being parsed correctly and some not. In the mean time i ve tried
> any possible combination between the formats(extended12,13,14
> std12,13,14)  and the parameters with extra_byte  with no change. I
> remind you that i am running CMS R14 with extended format for ECHI and
> my database is (localhost linux or windows just for testing with no
> behavior change)
>
> Thank you again for your response
> Manos
>
> On 26 Φεβ, 16:47, JasonGoecke <jsgoe...@gmail.com> wrote:
>
> > At which record does it break down. Also, please post the log showing

> > the failure to a gist @http://gist.github.comandpost it here.

[Manos Ioannou]

Thank you Jason, i had the same opinion about that. I will contact
Avaya support for the Version mismatch (CMS declares R14 but the ECHI
file declares itself as version 12).
About the extraneous data in the end of the file could it be the ftp
copy script from CMS server to the ftp server responsible ?

thank you
Manos

> > > the failure to a gist @http://gist.github.comandpostit here.

[Manos Ioannou]

Jason i have noticed the following:
From my files (all my chrxxxx data) all the problematic files are
stopped with the error malformed utf8 character after the transferred
field (Boolean field) except the on that stopped with the nil NoMethod
error the who mentioned before by you.
Does it means anything for you ?
In the mean time i ve already opened a ticket for Avaya support
regarding the whole situation and the version mismatch (CMS R14 and
version 12 of echi files).

Best Regards

Manos

On 26 Φεβ, 19:15, JasonGoecke <jsgoe...@gmail.com> wrote:

> > > the failure to a gist @http://gist.github.comandpostit here.