From JDataStore To Interbase

[Mehmet F. Erten]

Since Interbase roadmap shows Interbase will be alive together with JBuilder
in the comming years and JDataStore side is still quiet and I could not get
the "lock held by" situation in JDataStore resolved, I need to migrate into
Interbase from JDataStore yesterday (as soon as possible)

I will appreciate the hints and the tips from the users whom already did
migrate from JDataStore into Interbase.

- Will I have to change JBuilder data types to fit to Interbase after
JDataStore?
- How difficult to copy JDataStore Tables into Interbase tables.

Thanks.

(I do have JBuilder 2005 but still using JBuilderX since some reason I can
not see my design screens with 2005 or 2006 (tested with the trial) which
are perfectly OK with X)

[Steve Shaughnessy]

Hi Mehmet,

I have seen your posts on the lock problem and support contacted me
yesterday about this problem.

The support person said he would forward the test case to me.

I know this must be frustrating for you. When I receive the test case, I
will look at it and get back to you on what the problem is.

If it is a defect in JDataStore, we will fix it and provide a hot fix within
30-45 days.

Devco is committed to maintaining JDataStore.

I cannot provide a product roadmap at this time, but I want you to know that
both JDataStore and NDataStore are considered valuable assets at the highest
levels of Devco.

-Steve

"Mehmet F. Erten" <mehmet...@mblsoft.com> wrote in message
news:44b4...@newsgroups.borland.com...

[Mehmet F. Erten]

Hi,
I was expection a bunch of Programmers in India whom developing JDataStore
but after googling "Devco JDataStore" together I found out who is Steve
Shaughnessy.
I hope you and Jen get it around.

The listed second exception generated from my GUI application and luckily
did not keep the row/table and whole system locked as it happens within my
test run.
My test run does not have any GUI but inserts a row into QueryDataSet and
savechanges in a .bat file loop while an inquiry program displays the same
tables' data.
It works like a charm up to a point. One point the display program hangs and
the lock held by message appears with no way clearing it (database corrupted
with a lock see the first exception)

I used the tabs and tab folder extensively in my GUI presentation; the tab
folders within the tab folders with a lot of QueryDataSet activities.

I worked with maxRowLocks, tableLockTables and
database1.getJdbcConnection().setReadOnly(true) without sucess.

The bmp file shows the application where in the Orders the following (the
second) exception is thrown.
I changed the PO used count QueryDataSet to readonly(yes), hoping that it
will resolve GUI one.

First Exception From Test Run (no GUI plus a GUI (display only some times
Order Detail lock not Header)
C:\TestMillionOHRows>goto :again

C:\TestMillionOHRows>C:\TestMillionOHRows\TestMillionOHRows.exe

Last Number: 990005443

last order: 990005443 new number: 990005444 new surrogate#: S1151528282156

newNumber+ii: 990005444

new order#: 990005444

C:\TestMillionOHRows>goto :again

C:\TestMillionOHRows>C:\TestMillionOHRows\TestMillionOHRows.exe

Last Number: 990005444

last order: 990005444 new number: 990005445 new surrogate#: S1151528286375

newNumber+ii: 990005445

new order#: 990005445

C:\TestMillionOHRows>goto :again

C:\TestMillionOHRows>C:\TestMillionOHRows\TestMillionOHRows.exe

Last Number: 990005445

last order: 990005445 new number: 990005446 new surrogate#: S1151528288609

newNumber+ii: 990005446

new order#: 990005446

See com.borland.dx.sql.dataset.ResolutionException error code: BASE+1

com.borland.dx.sql.dataset.ResolutionException: An error occurred adding a
row t o the "DEFAULT_SCHEMA"."ORDER_HEADER" table

at com.borland.dx.sql.dataset.ResolutionException.a(Unknown Source)

at com.borland.dx.sql.dataset.ResolutionException.b(Unknown Source)

at com.borland.dx.sql.dataset.l.b(Unknown Source)

at com.borland.dx.sql.dataset.QueryResolver.insertRow(Unknown Source)

at com.borland.dx.sql.dataset.ResolutionManager.c(Unknown Source)

at com.borland.dx.sql.dataset.ResolutionManager.d(Unknown Source)

at com.borland.dx.sql.dataset.ResolutionManager.d(Unknown Source)

at com.borland.dx.sql.dataset.ResolutionManager.saveChanges(Unknown Sour

ce)

at com.borland.dx.sql.dataset.ResolutionManager.saveChanges(Unknown Sour

ce)

at com.borland.dx.sql.dataset.Database.saveChanges(Unknown Source)

at com.borland.dx.sql.dataset.QueryDataSet.saveChanges(Unknown Source)

at testmillionohrows.Frame1.jButton1_mouseClicked(Frame1.java:108)

at testmillionohrows.Frame1.<init>(Frame1.java:39)

at testmillionohrows.Application1.<init>(Application1.java:20)

at testmillionohrows.Application1.main(Application1.java:43)

Chained exception:

com.borland.platform.datastore.driver.DriverException: Runtime Error:
19be851:te st:No remarks connection encountered lock timeout for
/DEFAULT_SCHEMA/ORDER_HEADER stream. Lock held by a492cb:test:No remarks
connection

at com.borland.platform.datastore.driver.DriverException.serverException

(Unknown Source)

at com.borland.datastore.driver.StreamReader.request(Unknown Source)

at com.borland.datastore.driver.RDriverStatement.request(Unknown Source)

at com.borland.datastore.driver.RDriverStatement.executeSql(Unknown Sour

ce)

at com.borland.datastore.jdbc.RStatement.executeUpdate(Unknown Source)

at com.borland.dx.sql.dataset.j.c(Unknown Source)

at com.borland.dx.sql.dataset.l.b(Unknown Source)

at com.borland.dx.sql.dataset.QueryResolver.insertRow(Unknown Source)

at com.borland.dx.sql.dataset.ResolutionManager.c(Unknown Source)

at com.borland.dx.sql.dataset.ResolutionManager.d(Unknown Source)

at com.borland.dx.sql.dataset.ResolutionManager.d(Unknown Source)

at com.borland.dx.sql.dataset.ResolutionManager.saveChanges(Unknown Sour

ce)

at com.borland.dx.sql.dataset.ResolutionManager.saveChanges(Unknown Sour

ce)

at com.borland.dx.sql.dataset.Database.saveChanges(Unknown Source)

at com.borland.dx.sql.dataset.QueryDataSet.saveChanges(Unknown Source)

at testmillionohrows.Frame1.jButton1_mouseClicked(Frame1.java:108)

at testmillionohrows.Frame1.<init>(Frame1.java:39)

at testmillionohrows.Application1.<init>(Application1.java:20)

at testmillionohrows.Application1.main(Application1.java:43)

Second Exception from GUI
See com.borland.dx.dataset.DataSetException error code: BASE+62
com.borland.dx.dataset.DataSetException: Execution of query failed.
at com.borland.dx.dataset.DataSetException.a(Unknown Source)
at com.borland.dx.dataset.DataSetException.queryFailed(Unknown
Source)
at com.borland.dx.sql.dataset.QueryProvider.a(Unknown Source)
at com.borland.dx.sql.dataset.JdbcProvider.provideData(Unknown
Source)
at com.borland.dx.dataset.StorageDataSet.refresh(Unknown Source)
at com.borland.dx.sql.dataset.QueryDataSet.refresh(Unknown Source)
at mbl.DataAccess.openRefreshTheTable(DataAccess.java:580)
at mbl.DataAccess.getPOUsedCount(DataAccess.java:442)
at mbl.DataAccess.getDefaultValue(DataAccess.java:1211)
at
mbl.OrderMaintenance.ResidentialOrders.validateDetails(ResidentialOrd
ers.java:10530)
at
mbl.OrderMaintenance.ResidentialOrders.saveDetailLineEtc(ResidentialO
rders.java:8216)
at
mbl.OrderMaintenance.ResidentialOrders.saveDetailHeaderLineEtc(Reside
ntialOrders.java:8483)
at
mbl.OrderMaintenance.ResidentialOrders.saveAll(ResidentialOrders.java
:8301)
at
mbl.OrderMaintenance.ResidentialOrders.orderHeaderSaveButton_mouseCli
cked(ResidentialOrders.java:11018)
at
mbl.OrderMaintenance.ResidentialOrders.orderDetailSaveButton_mouseCli
cked(ResidentialOrders.java:11123)
at
mbl.OrderMaintenance.ResidentialOrders$69.mouseClicked(ResidentialOrd
ers.java:2436)
at java.awt.AWTEventMulticaster.mouseClicked(Unknown Source)
at java.awt.Component.processMouseEvent(Unknown Source)
at javax.swing.JComponent.processMouseEvent(Unknown Source)
at java.awt.Component.processEvent(Unknown Source)
at java.awt.Container.processEvent(Unknown Source)
at java.awt.Component.dispatchEventImpl(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Window.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForHierarchy(Unknown
Source)

at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown
Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.run(Unknown Source)
Chained exception:
com.borland.platform.datastore.driver.DriverException: Runtime Error:
13c468a:mb
lrules:No remarks connection encountered lock timeout for
/DEFAULT_SCHEMA/ORDER_
DETAIL stream. Lock held by ccc588:mblrules:No remarks connection
at
com.borland.platform.datastore.driver.DriverException.serverException
(Unknown Source)
at com.borland.datastore.driver.StreamReader.request(Unknown Source)
at com.borland.datastore.driver.RDriverStatement.request(Unknown
Source)

at com.borland.datastore.driver.RDriverStatement.executeSql(Unknown
Sour
ce)
at com.borland.datastore.jdbc.RStatement.executeQuery(Unknown
Source)
at com.borland.dx.sql.dataset.o.f(Unknown Source)
at com.borland.dx.sql.dataset.QueryProvider.e(Unknown Source)
at com.borland.dx.sql.dataset.JdbcProvider.provideData(Unknown
Source)
at com.borland.dx.dataset.StorageDataSet.refresh(Unknown Source)
at com.borland.dx.sql.dataset.QueryDataSet.refresh(Unknown Source)
at mbl.DataAccess.openRefreshTheTable(DataAccess.java:580)
at mbl.DataAccess.getPOUsedCount(DataAccess.java:442)
at mbl.DataAccess.getDefaultValue(DataAccess.java:1211)
at
mbl.OrderMaintenance.ResidentialOrders.validateDetails(ResidentialOrd
ers.java:10530)
at
mbl.OrderMaintenance.ResidentialOrders.saveDetailLineEtc(ResidentialO
rders.java:8216)
at
mbl.OrderMaintenance.ResidentialOrders.saveDetailHeaderLineEtc(Reside
ntialOrders.java:8483)
at
mbl.OrderMaintenance.ResidentialOrders.saveAll(ResidentialOrders.java
:8301)
at
mbl.OrderMaintenance.ResidentialOrders.orderHeaderSaveButton_mouseCli
cked(ResidentialOrders.java:11018)
at
mbl.OrderMaintenance.ResidentialOrders.orderDetailSaveButton_mouseCli
cked(ResidentialOrders.java:11123)
at
mbl.OrderMaintenance.ResidentialOrders$69.mouseClicked(ResidentialOrd
ers.java:2436)
at java.awt.AWTEventMulticaster.mouseClicked(Unknown Source)
at java.awt.Component.processMouseEvent(Unknown Source)
at javax.swing.JComponent.processMouseEvent(Unknown Source)
at java.awt.Component.processEvent(Unknown Source)
at java.awt.Container.processEvent(Unknown Source)
at java.awt.Component.dispatchEventImpl(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Window.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForHierarchy(Unknown
Source)

at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown
Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.run(Unknown Source)

"Steve Shaughnessy" <sshaug...@b.com> wrote in message
news:44b505ff$1...@newsgroups.borland.com...

[Lori M Olson [TeamB]]

Steve Shaughnessy wrote:

> Devco is committed to maintaining JDataStore.
>
> I cannot provide a product roadmap at this time, but I want you to know that
> both JDataStore and NDataStore are considered valuable assets at the highest
> levels of Devco.
>
> -Steve

Excellent.

--

Regards,

Lori Olson [TeamB]

[Steve Shaughnessy]

I looked at your test case.

The writer starts off with a select that has a where clause for a column
(order) that is not the first column of an index.

Consequently the query optimizer has no choice but to do a full scan of the
table.

As the table gets larger from running the test multiple times, the scan
takes longer.

The reader gets blocked on this long scan. When the table gets big enough
the lock timeout is reached.

Conversely the reader is performing joins without selection criteria. As
the tables get bigger, the query takes longer. You may also run out of
memory since the DataSet is configured to load all rows.

Solutions:

1) Add an index to the order column for the writer and avoid joins on large
tables without selection criteria.
2) Add an index to the order column for the writer and have the reader use a
readonly transaction.

Note that I'm not sure this test is doing what you want it to do. You have
time delays inbetween batch inserts to a cached DataSet. After all inserts
are completed, you call saveChanges to the database which executes the
insert operations to the underlying database. I would have thought you
would want to save the changes to the database before sleeping.

Hope this helps,

-Steve

"Mehmet F. Erten" <mehmet...@mblsoft.com> wrote in message

news:44b5...@newsgroups.borland.com...

[Mehmet F. Erten]

Hi,
My previous problem was the JDataStore temp files as well as LOGA files.
If I don't use Load.ALL the temp file was getting bigger and bigger and
after a while in Windows 2000 Advance Server the temp file did not
refresh/rename/renumber itself but failed the whole system.
To stop the temp file to increase I changed all select statements to
Load.ALL but limited the production search by (>=a to <b) logic. When I
found out LOGA files are freezing the system, than every night I started
programmatically copy the jds file to refresh the LOGA file. (LOGA files
problem looked like not a problem in XP or Linux environment, I did not test
Load.as_needed on this environments)

The reason I am testing the JDataStore is to see if I have hundreds thousand
orders will be OK if not million.
In my test I did insert one row (new Order header) and end the java program
after the saveChanges so there should not be any overhead to LOGA files.

With less than ten thousands orders (a few selected columns) there should
not be memory problem and should not be any temp or LOGA problem.

If the select statement creates the problem, I should get a memory problem
and not a Lock table and hung database.

But the most important issue was after the Lock Held By message I could not
put the table back into a functional stage even after Pack/Refresh.
I will test with where clause to limit the search in the display program to
see if it will help.
Thanks.

"Steve Shaughnessy" <sshaug...@b.com> wrote in message

news:44b5baf5$1...@newsgroups.borland.com...

[Steve]

Can you send me a test case that shows the temp database growing?

Thanks,

-Steve

"Mehmet F. Erten" <mehmet...@mblsoft.com> wrote in message

news:44b6...@newsgroups.borland.com...

[Mehmet F. Erten]

DEFAULT_SCHEMA is not valid in InterBase which is used with JDataStore.
It makes the migration a little longer.
Both Borland family product I don't know which one should comply.

[Mehmet F. Erten]

It took less than a day to modify the JBuilder/Java codes to use Interbase
as well as JDataStore.
I defined a flag which handles one different behavior: set/get Object vs
set/get InputStream.
This is related to Blob column.
InterBase also treats Text type blob as String (which I don't use)

I also use this flag to choose "Load.ALL vs Load.AS_NEEDED.
Although InterBase did not create any problem, it did create sort temp file
(windows\temp) which for my use is counter-productive.
As long as I wait for the this big file creation, does not matter if I see a
page data on the GUI.
I prefer to filter the data myself and show the resultset directly on the
GUI rather than having a big temp file and GUI is talking to it.

If the data types are used to handle the data, also short datatype is
required to be checked (set/get Short together with set/get Int instead of
one set/get Int)
And if a numeric column's length is more than/equal to 18, 0, it requires to
be read with get/set BigDecimal not get/set int. (good practice to define it
Decimal)

But most time I spent while I was waiting for the data conversion from
JDataStore into InterBase.
Once 220000 rows added (with multiple keys) which did not create any problem
except adjusting the run time memory.

Currently my test run is running and within 10 hours it added 100,000 order
header and one million order detail rows.
While through a bat file loop, I am adding the new header/detail rows, I
have 3 full application running to inquire/maintain these orders.
I can inquire 100,000 orders after a little pause without any database
complication such as Lock or Temp file size.
So far no problem.

My InterBase impression is very comforting, It shows itself as a
solid/tested system with a solid performance/support.
Optimistic Lock is a good thing.