jlogdup stops suddenly without any visible reason

[My_jBase]

Hi,

Do You have any idea why jlogdup process stops many times a day ?

We use jbase.4.0.7.3 on Aix 6.1

There is a simple script running on LIVE server :

#!/usr/bin/ksh

export JBCRELEASEDIR=/usr/jbc

export JBCGLOBALDIR=/usr/jbc

export PATH=$JBCRELEASEDIR/bin:$PATH

export LIBPATH=$JBCRELEASEDIR/lib:$LIBPATH

export SHLIB_PATH=$JBCRELEASEDIR/lib:$SHLIB_PATH

export LD_LIBRARY_PATH=$JBCRELEASEDIR/lib:$LD_LIBRARY_PATH

jlogdup INPUT set=current terminate=wait OUTPUT set=stdout | ssh DRPGLOB /logstore.sh

The logstore.sh on DRPGLOB looks like :

export JBCRELEASEDIR=/usr/jbc

export JBCGLOBALDIR=/usr/jbc

export PATH=$JBCRELEASEDIR/bin:$PATH

export LIBPATH=$JBCRELEASEDIR/lib:$LIBPATH

export SHLIB_PATH=$JBCRELEASEDIR/lib:$SHLIB_PATH

export LD_LIBRARY_PATH=$JBCRELEASEDIR/lib:$LD_LIBRARY_PATH

jlogdup INPUT set=stdin terminate=wait OUTPUT set=database

This process should run whole day to replicate all changes form LIVE logsets to remote database :

 #>ps -ef | grep -v grep | grep jlogdup

    root 57606602 18284560  17 08:33:34 pts/301  1:15 jlogdup INPUT set=current terminate=wait OUTPUT set=stdout

 #>

This is an output of jlogdup in moment of crash:

08:14:51 23 MAY 2013 : STATUS: 

    Begin jlogdup process: From set 'set=stdin terminate=wait' to  set 'set=database'

08:45:06 23 MAY 2013 : STATUS: 

    Termination Statistics: usr 44.78 , sys 83.01 , elapsed 30m15.58

    INPUT : 20623821 records , 0 blocks , 1671927935 record bytes , 0 errors

    OUTPUT: 0 records , 3576232 blocks , 0 bytes , 0 errors

08:45:06 23 MAY 2013 : STATUS: 

    Program terminated. Exit code is 0

08:45:06 23 MAY 2013 : STATUS: 

    Termination Statistics: usr 137.41 , sys 528.32 , elapsed 30m15.28

    INPUT : 20623821 records , 3576232 blocks , 1671927935 record bytes , 0 errors

    OUTPUT: 20623821 records , 0 blocks , 1671927935 bytes , 0 errors

08:45:06 23 MAY 2013 : STATUS: 

    Program terminated. Exit code is 0

If we restart this synchronization from the beginning of logset it will stop in different place. 

My network admin said that it looks like a simple end of the ssh session ( ssh [FIN, ACK] ) sent from LIVE server.

I don't know why because we use a terminate=wait option.

There is no problem with network, there is no problem on aix - do You have any solution ?

Best regards

Piotr

[uiterwyk]

Piotr,

It may be that you are getting a number of errors that exceed the maximum allowed.(10,000 by default)

Add a -l{some_file_name} to both jlogdup scripts and when jlogdup stops, look at these log files for a hint.

I suspect you are getting some sort of error on the receiving side caused by a missing file or permission.

If you have distributed files, that may also be a cause, in which case the easiest cure is to jchmod -L the distributed file stub name and let the partitions do all the logging
(A better method anyway, as otherwise each transaction gets logged twice.)

HTH

Robert Uiterwyk

[My_jBase]

Hi Robert,

Thank You for the answer. I think that if the " -l log_name " is not used the jlogdup writes its output to the default logfile:

/usr/jbc/config/jediLoggerAdminLog

There are no error messages in that file. I can find the same informations attached by me in the first post in the moment of crash of jlogdup - nothing special ...I wonder what is the purpose of timeout=nnn option. Is there any sense to use both options: termianate=wait and timeout=300. 

By the way, I have at least one background process running whole day which writes his activity time to the record in jbase file every second, so every second 

journal sholuld be updated and jlogdup should do the same to remote database. In that case it is impossible to have "complete silence" in databse.

terminate=wait (I)	switch to elder log sets as required and wait for new updates
timeout=nnn (I)	timeout period in seconds for ‘terminate=wait'

I'm afraid that maybe I do not understand how these options work

Piotr

[My_jBase]

Hi,

There is somethink new :

#>more synchronizator.27052013.114139.log

11:41:40 27 MAY 2013 : STATUS: 

    Begin jlogdup process: From set 'set=stdin terminate=wait' to  set 'set=database'

11:41:40 27 MAY 2013 : STATUS: 

    Begin jlogdup process: From set 'set=current terminate=wait' to  set 'set=stdout'

12:29:01 27 MAY 2013 : STATUS: 

    Termination Statistics: usr 66.53 , sys 151.03 , elapsed 47m21.24

    INPUT : 33275375 records , 0 blocks , -1001137788 record bytes , 0 errors

    OUTPUT: 0 records , 5029989 blocks , 0 bytes , 0 errors

12:29:01 27 MAY 2013 : STATUS: 

    Program terminated. Exit code is 0

12:29:01 27 MAY 2013 : ERROR:  For definition set=database

    TRANSACTION violation: jlogdup closing and there are still 9 outstanding updates in a transaction for process id 15663376

12:29:01 27 MAY 2013 : ERROR:  For definition set=database

    TRANSACTION violation: jlogdup closing and there are still 2 outstanding updates in a transaction for process id 60293254

12:29:01 27 MAY 2013 : STATUS: 

    Termination Statistics: usr 209.83 , sys 869.76 , elapsed 47m21.64

    INPUT : 33275375 records , 5029989 blocks , -1001137788 record bytes , 0 errors

    OUTPUT: 33275364 records , 0 blocks , -1001328817 bytes , 2 errors

12:29:01 27 MAY 2013 : STATUS: 

    Program terminated. Exit code is 0

Any ideas ?

Piotr

[uiterwyk]

Piotr,

The /usr/jbc/config/jediLoggerAdminLog IS NOT the same as the one created by the -l option.

Please try the -l option on both machines and see what you get for errors.  I am sure you will see some

Also, I would not use the timeout-nnn option.  I have never used it and have never had a problem

The "TRANSACTION violation: jlogdup closing and there are still 9 outstanding updates in a transaction" is because you are using transaction boundriess: TRANSTART  and  TRANSEND and jlogdup is crashing in the middle of one of these,

Again, use the -l option.  I think it will show your problem

Robert

[My_jBase]

Hello,

This appears to be a problem with the option terminate=wait. After restarting the jlogdup process operates as long as it's rewriting data from logset to the remote database - it stops working as soon as it’s done rewriting the data, It terminates at the moment when it comes to the end, although the journal process is still active.

What should I do ?

Piotr

[Jawwad Khan]

we were facing this problem in jbase5.0.16 release as well Temenos Helodesk provided the jbase release 5.0.32 in which this issue is resolved upto some extend but log corruption is still there which inturn terminate the jlogdup but with indication of corrupt log record. secondly when we were on jbase 5.0.16 we normally switched to the next logset as soon as the jlogdup complete sync of the current log it gave us some resistance till 15 GB log of the next logset.

Piotr

--
--
IMPORTANT: T24/Globus posts are no longer accepted on this forum.
 
To post, send email to jB...@googlegroups.com
To unsubscribe, send email to jBASE-un...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/jBASE?hl=en
 
---
You received this message because you are subscribed to the Google Groups "jBASE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jbase+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[VK]

> looks like a simple end of the ssh session

Just a thought... is your ssh client configured to have keepalive on?

[My_jBase]

Hi,

These are some last lines of jlogdup logfile

10*1*1329824679*1372153745  WRITE   /jbase/jspooler/jspool_log      DEVCONFIG*9627
10*1*1329824897*1372153745  WRITE   /jbase/jspooler/jspool_log      DEVCONFIG*9172
0*0*0*1372153746      EOF                                                   
11:49:06 25 JUN 2013 : STATUS:
    Termination Statistics: usr 1016.14 , sys 3621.59 , elapsed 161m17.05
    INPUT : 35829722 records , 0 blocks , -322285481 record bytes , 0 errors
    OUTPUT: 0 records , 5196925 blocks , 0 bytes , 0 errors
11:49:06 25 JUN 2013 : STATUS:

    Program terminated. Exit code is 0

Can You see EOF ??? - jlogdup was started with terminate=wait

Piotr

[My_jBase]

Hello,

Does anyone use the command jlogdup with parameter terminate=wait with options timeout=nnn and retry=nnn ? What is the purpose of these options ?

I can read in documentation:

terminate=wait (I) - switch to elder log sets as required and wait for new updates  -   it looks OK 

but

timeout=nnn (I) - timeout period in seconds for 'terminate=wait'

retry=nn (I - specifies the interval between retries, when 'terminate=wait'

What is the default value for retry and timeout ?

I don't use them so my jlogdup command looks like :

jlogdup -v -e /journal/TJ.ERROR INPUT set=eldest terminate=wait OUTPUT set=stdout | ssh remote_server /logstore.sh

and it still crashes without any reason.

Regards

Piotr

[Aaqil Khattak]

It is better to switch to Oracle or DB2 jbase is not stable and not suitable for heavy bank.

--

[Jawwad Khan]

We have the same issue with jbase 5016 and after implementing jbase5032 it all gone. Its better to recreate all logs sets at weekend to avoid such issues.

[Charlie Noah]

Hi Aaqil,

Since nobody else has responded to this, I will. Where did you get the idea that Jbase isn't stable? It is very stable and many "heavy" banks are using it, as well as large sites in other industries.

Regards,
Charlie Noah

[Aaqil Khattak]

May b you are right Man i think you forgot with the maintenance issue on jbase people are suffering. Tell me how many heavy banks have jbase can you give me the exact number of branches, any bank more then 200 branches. We have few banks in Pakistan as well running on jbase with less then 100 branches may be running fine but the issues are still there. 

I am talking with reference to scalablity, reliability and if you are comparing with the price then its okay. 

Please update me so i can correct myself with this.

Regards,

Aaqil Khattak

[Charlie Noah]

Temenos acquired Jbase for their T24 banking software in 1999. Although they probably won't reveal how many banks or branches they support, they wouldn't have acquired it if it wasn't solid. Here are some links:
http://www.temenos.com/en-us/
http://www.prospectus.co.uk/multivalue/JBase
http://t24e.com/forum/what-is/what-is-jbase/

Charlie

[Aaqil Khattak]

Dear Charlie,

I know very well about the jBase... My point was clear if you have info let me know.. Jbase is fast suitable for small banks not made for large banks with heavy volume of transactions that was my question and i also said that it will not support huge number of branches as database.... If you know any bank more then 200, 300 branches please let me know so i can correct myself. :) 

If anyone in this group have experience they can also share it.

hope you understand this point if not than then it is up to you.

cheer up

Regards,

Aaqil khattak

[Aaqil Khattak]

And i am not saying jbase is not good every bank on T24 is using jbase in any case only the case is heavy banks more then 400 & 500 branches i know one bank in Pakistan having almost 200 branches running Jbase.

Well for saving money it is best then nothing. But other factors matter.

Regards,

Aaqil Khattak

[Naveed Ahmed Nayyar]

Hi All,

I agree with you Mr. Aaqil. you are rite 100%. i also know about jbase database very well how compatible and support how many branches with T24.

Great Comments! 

Thanks & Regards,
--------------------------------
Naveed Ahmed Nayyar
T24 Technical Consultant

Phone: +923225734761
Email: nahmed...@gmail.com
Temenos Core Banking Solution

[My_jBase]

Dear Gentlemans,

We are using a very old version of jbase ( 4.0.7.3 on Aix 6.1 ). Nevertheless, in my opinion it absolutely fulfills our needs. It is fast and stable enough for our purposes.  In our current situation, the realization of a project like changing the version of jbase would take at least several months, making such a decision virtually impossible, at least for the time being. I have to solve the jlogdup problem without changing the version - it used to work correctly.

Why would creating logsets again help solve this problem?

Best regards
Piotr

[Jawwad Khan]

i can understand the worry you have in regards to jlogdup, have you tried re-installing the jbase on new location. or recreating the logsets.

Secondly after creating new logsets donot use jlogadmin -t command to truncate logset use jlogadmin -lnext command to pass by the logset for truncating data. 

[Jawwad Khan]

secondly we faced this issue when we moved from one server to another with OS version upgrade to higher version. Same is the case with you ?

[My_jBase]

Hello,

Initially I thought that the reason for the problems is the TL of aix, but now I’m affraid that jlogdup was wrong from the beginning, from the first day of useing jlogdup on LIVE environment ( I have checked my mailbox ). 

All 4 logsets has been recreated on clear filesystem.

I didn't reinstall jbase - we are affraid of "big explosion" on LIVE after that :) - but who knows, it could be solution ...

Regards

Piotr