Refactoring legacy code to OOP — class/object access 5–7× slower than array
Atul
I am refactoring a legacy Clipper/Harbour console application that processes DBF files and generates reports. The application uses multiple DBFs connected via primary fields. I am implementing hexagonal architecture to separate UI, core logic, and database adapter layers.
Environment:
Harbour version : 3.0.0
Compiler : MinGW
Platform : Windows 11, 64-bit
RDD : DBFNTX
Previously the report processed ~4,000 records in 2 minutes using
direct DBF navigation. After refactoring to OOP class objects the
same processing now takes 14 minutes — a 7× slowdown.
The refactored code has two phases:
Phase 1 — data collection: DBF SEEK/navigation, loads data into
an array of class objects. All DB access is HERE only.
Phase 2 — processing: pure in-memory loop over the object array.
NO DBF access, NO SEEK, NO area switching.
The slowdown is entirely in Phase 2. My class has 140 DATA members.
The loop structure is Header → Detail → Sub-detail:
FOR EACH oHdr IN aHeader // ~1,000 records
nn := AScan(aDetails, {|x| x:code+x:key = oHdr:code+oHdr:key }) // ~3,000 records total
IF nn > 0
oDtl := aDetails[nn]
kk := AScan(aSubs, {|x| x:code+x:key = oDtl:code+oDtl:key }) // ~5,000 records total
IF kk > 0
oDtl := aSubs[kk]
// read properties, compute, output
ENDIF
ENDIF
NEXT
Any guidance from those who have done similar legacy refactoring
projects would be greatly appreciated.
Thank you,
Atul Parate
Francesco Perillo
--
You received this message because you are subscribed to the Google Groups "Harbour Users" group.
Unsubscribe: harbour-user...@googlegroups.com
Web: https://groups.google.com/group/harbour-users
---
You received this message because you are subscribed to the Google Groups "Harbour Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to harbour-user...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/harbour-users/947f4824-e06d-4d4e-aa9c-5ff689cb99f0n%40googlegroups.com.
Francesco Perillo
IF nn > 0
oDtl := aDetails[nn]
Nenad Batoćanin
Hi!
I think AScan does a simple sequential search, so it can't be as fast as index search. Perhaps a replacement for AScan could be made that uses a memory table and a seek search?
Regards, NB
To view this discussion visit https://groups.google.com/d/msgid/harbour-users/CADPHLr9RmwyV7Gca1zGtUyudF0JCNuM%3DW04EvP2VvEX7Errwsw%40mail.gmail.com.
Francesco Perillo
To view this discussion visit https://groups.google.com/d/msgid/harbour-users/008601dcb980%24ecd55a70%24c6800f50%24%40wings.rs.
Nenad Batoćanin
If anyone remembers the CA Visual Objects project... It had a very nice object-oriented approach to working with the database. It looks like this:
oDb := DBServer ("Test")
oDb:SetIndex ("Test_ID")
WHILE ! oDb:Eof()
IF oDb:Id == 0
oDb:Id := RecNo()
oDb:Name := "None"
END IF
oDb:Skip()
END DO
I believe this code is clear to everyone :) The really good thing is that almost identical code could also work with SQL Server. Only the initialization was a little different:
oDb := SQLSelect ("SELECT * FROM Test ORDER BY Id")
To view this discussion visit https://groups.google.com/d/msgid/harbour-users/CADPHLr-OJcJniJq7XKDmy%3D7xfdW_GxBGP4JoYLa9WXG0co32PQ%40mail.gmail.com.
Mario H. Sabado
function MasterDetailSubs
*************************
local xVal,yVal
local aHeader:={{"H1","H1 Header"},;
{"H2","H2 Header"},;
{"H3","H3 Header"} }
local aDetails:={{"H1","H1-Detail 1"},;
{"H1","H1-Detail 2"},;
{"H1","H1-Detail 3"},;
;
{"H2","H2-Detail 1"},;
{"H2","H2-Detail 2"},;
{"H2","H2-Detail 3"},;
;
{"H3","H3-Detail 1"},;
{"H3","H3-Detail 2"},;
{"H3","H3-Detail 3"} }
local aSubs :={ {"H1","H1-Detail 1","H1 D1 Sub 1"},;
{"H1","H1-Detail 1","H1 D1 Sub 2"},;
;
{"H1","H1-Detail 2","H1 D2 Sub 1"},;
{"H1","H1-Detail 2","H1 D2 Sub 2"},;
;
{"H1","H1-Detail 3","H1 D3 Sub 1"},;
{"H1","H1-Detail 3","H1 D3 Sub 2"},;
;
{"H2","H2-Detail 1","H2 D1 Sub 1"},;
{"H2","H2-Detail 1","H2 D1 Sub 2"},;
;
{"H2","H2-Detail 2","H2 D2 Sub 1"},;
{"H2","H2-Detail 2","H2 D2 Sub 2"},;
;
{"H2","H2-Detail 3","H2 D3 Sub 1"},;
{"H2","H2-Detail 3","H2 D3 Sub 2"},;
;
{"H3","H3-Detail 1","H3 D1 Sub 1"},;
{"H3","H3-Detail 1","H3 D1 Sub 2"},;
;
{"H3","H3-Detail 2","H3 D2 Sub 1"},;
{"H3","H3-Detail 2","H3 D2 Sub 2"},;
;
{"H3","H3-Detail 3","H3 D3 Sub 1"},;
{"H3","H3-Detail 3","H3 D3 Sub 2"} }
AEval(aHeader,{|x|Qout( (xVal:=x[1])+"-"+x[2] ), ; //Header
,AEval(aDetails,{|y| iif(xVal==y[1],Qout(yVal:=y[1]+y[2]), ) ; //Detail
, AEval(aSubs,{|z| iif(xVal==z[1] .and. yVal==z[1]+z[2], Qout( yVal+"-"+z[3]) , ) ; //Sub-Detail
}) ;
}) ;
})
return nil
/* SAMPLE OUTPUT
H1-Header 1
H1H1-Detail 1
H1H1-Detail 1-H1 D1 Sub 1
H1H1-Detail 1-H1 D1 Sub 2
H1H1-Detail 2
H1H1-Detail 2-H1 D2 Sub 1
H1H1-Detail 2-H1 D2 Sub 2
H1H1-Detail 3
H1H1-Detail 3-H1 D3 Sub 1
H1H1-Detail 3-H1 D3 Sub 2
H1H1-Detail 3-H1 D3 Sub 1
H1H1-Detail 3-H1 D3 Sub 2
H1H1-Detail 3-H1 D3 Sub 1
H1H1-Detail 3-H1 D3 Sub 2
H1H1-Detail 3-H1 D3 Sub 1
H1H1-Detail 3-H1 D3 Sub 2
H1H1-Detail 3-H1 D3 Sub 1
H1H1-Detail 3-H1 D3 Sub 2
H1H1-Detail 3-H1 D3 Sub 1
H1H1-Detail 3-H1 D3 Sub 2
H1H1-Detail 3-H1 D3 Sub 1
H1H1-Detail 3-H1 D3 Sub 2
H2-Header 2
H2H2-Detail 1
H2H2-Detail 1-H2 D1 Sub 1
H2H2-Detail 1-H2 D1 Sub 2
H2H2-Detail 2
H2H2-Detail 2-H2 D2 Sub 1
H2H2-Detail 2-H2 D2 Sub 2
H2H2-Detail 3
H2H2-Detail 3-H2 D3 Sub 1
H2H2-Detail 3-H2 D3 Sub 2
H2H2-Detail 3-H2 D3 Sub 1
H2H2-Detail 3-H2 D3 Sub 2
H2H2-Detail 3-H2 D3 Sub 1
H2H2-Detail 3-H2 D3 Sub 2
H2H2-Detail 3-H2 D3 Sub 1
H2H2-Detail 3-H2 D3 Sub 2
H3-Header 3
H3H3-Detail 1
H3H3-Detail 1-H3 D1 Sub 1
H3H3-Detail 1-H3 D1 Sub 2
H3H3-Detail 2
H3H3-Detail 2-H3 D2 Sub 1
H3H3-Detail 2-H3 D2 Sub 2
H3H3-Detail 3
H3H3-Detail 3-H3 D3 Sub 1
H3H3-Detail 3-H3 D3 Sub 2
To view this discussion visit https://groups.google.com/d/msgid/harbour-users/00e401dcba52%2417cc93c0%244765bb40%24%40wings.rs.
Atul
Thank you, Francesco Perillo.
You're right about member access being slower.
I tried another approach as shown below:
FOR EACH oHdr IN aHeaders
? "Header : " + oHdr:cHeaderKey + " Customer: " + oHdr:cCustName
FOR EACH oDtl IN oHdr:aDetails
? " Detail : " + oDtl:cDetailKey + " Amt: " + hb_ntos( oDtl:nAmount )
FOR EACH oSub IN oDtl:aSubs
? " Sub : " + oSub:cSubKey + " Sub amount: " + hb_ntos( oSub:nSubAmt )
nGrandTotal += oSub:nSubAmt
NEXT
NEXT
NEXT
This approach is faster than the previous one — it now takes around 3–4 minutes to process the same data. However, I would like to optimize it further if possible.
This could be an alternative to the code + key approach.
The relationship between Header, Detail, and Sub is one-to-many, so I have to loop through all records to calculate the grand total amount.
I also tried using a hash, but it turned out to be slower than the class-based approach.
Regards, Atul
Francesco Perillo
To view this discussion visit https://groups.google.com/d/msgid/harbour-users/6c2f320d-81eb-4a87-876f-c96e2a65f7ebn%40googlegroups.com.
Atul
Francesco Perillo
To view this discussion visit https://groups.google.com/d/msgid/harbour-users/97a698f3-ae3b-4631-9215-adff74996391n%40googlegroups.com.
Atul
That's a great point, Francesco.
The reason is that I am refactoring my code following Hexagonal Architecture, where I want to clearly separate the Model, UI, Core, and Adapter layers.
- Phase 1 is the Adapter layer — responsible for data collection and preparation.
- Phase 2 is the Core / Service layer — responsible for business logic, including all the math and calculations.
- Phase 3 is the UI layer — responsible for displaying the report.
By keeping these layers separate, I can easily replace the Adapter in the future with any other data source or technology, such as SQL, without touching the business logic or the presentation layer.
Thanks, Atul