Google 網路論壇不再支援新的 Usenet 貼文或訂閱項目,但過往內容仍可供查看。

how to add units ?

瀏覽次數:64 次
跳到第一則未讀訊息

A.Martínez

未讀,
2010年6月15日 上午11:21:082010/6/15
收件者:
Hi,

As subject and is it Ascan slow ?

how to add more fastest ?

Using Hash ?

Is it hash a b-tree ?

Example:


FUNCTION CalcSales()
Local aSales:= {}
SELECT Sales
GO TOP
DO WHILE !Eof()
AddUnits(aSales, _FIELD-> Customer, _FIELD-> Product , _FIELD-> Unit)
SKIP
ENDDO
RETURN aSales

STATIC FUNCTION AddUnits(aSales, cCus, cPro, nUni)
Local nPos
nPos:= Ascan(aSales, nil, {|x| a[1] == cCus+ cPro}) //
Slow !
IF nPos == 0
Aadd(aSales, {cCus+ cPro, nUni})
ELSE
aSales[nPos, 2]+= nUni
ENDIF
RETURN NIL

Regards


Ella

未讀,
2010年6月15日 下午3:17:502010/6/15
收件者:
Hello,


IMHO in matter of execution speed there are no general solutions, one
always need to maintain a balance between the concrete hardware
resources and the purpose of the needed application.

In case of arrays and/or hash tables the phisically available RAM
should be able to hold the entire data - using the virtual memory
reserves is wrong, because the continuous swapping slows down the
system.

Example: Windows XP Pro on a PC with 512 MB physical RAM can hold an
array of 10-50 MB data without problems.

On such systems is not a problem to populate an array ( or hash ) with
few hundreds of rows, but when having thousands of rows it's a good
idea to use

ASizeAlloc(...) for arrays, or HAllocate(...) for hash tables

in order to prevent slow-down caused by excessive memory
fragmentation.

Using a hash table is ok if you have up to several thousands of keys,
or if you have a powerful processor with big L2.

Just my two cents,


Ella

Mel Smith

未讀,
2010年6月15日 下午4:22:122010/6/15
收件者:

"Ella" <ella....@xharbour.com> wrote in message
news:fbadbfb9-e64d-4dcd...@5g2000yqz.googlegroups.com...
Hello,


IMHO in matter of execution speed there are no general solutions, one
always need to maintain a balance between the concrete hardware
resources and the purpose of the needed application.

In case of arrays and/or hash tables the phisically available RAM
should be able to hold the entire data - using the virtual memory
reserves is wrong, because the continuous swapping slows down the
system.

Example: Windows XP Pro on a PC with 512 MB physical RAM can hold an
array of 10-50 MB data without problems.

On such systems is not a problem to populate an array ( or hash ) with
few hundreds of rows, but when having thousands of rows it's a good
idea to use

ASizeAlloc(...) for arrays, or HAllocate(...) for hash tables

in order to prevent slow-down caused by excessive memory
fragmentation.

Using a hash table is ok if you have up to several thousands of keys,
or if you have a powerful processor with big L2.

Just my two cents,

Hi Ella:

When building very long strings (e.g., up to 3 megabytes), is there a
size allocation technique that should be used ?

(pls see my posts on string length)

-Mel Smith


Ella

未讀,
2010年6月15日 下午5:09:332010/6/15
收件者:
Mel,

>     When building very long strings (e.g.,  up to 3 megabytes), is there a
> size allocation technique that should be used ?
>
>     (pls see my posts on string length)
>
> -Mel Smith

I can't suggest general solutions, below I explain my point of view
and I list a possible solution.

xHarbour arrays or hashes have aka "tree structure", so they are not
stored in a contiguous memory zones, like it happens in C.

xHarbour strings are stored in contiguous memory zones, and when you
"append" characters to an existing string, usually the old string is
copied in a different memory location.

In this code I allocate a huge string, and with the shown technique I
can update the desired zones whenever I want - you might want to reuse
it many times, and to send out a substring.

Ella

local i, nTotLen
local cCur, cHefty := replicate( "-", 5000000 )

nTotLen := 0

cCur := "starting line"
for i:=1 to len( cCur )
nTotLen++
cHefty[nTotLen] := cCur[i]
next

cCur := "continuing my text"
for i:=1 to len( cCur )
nTotLen++
cHefty[nTotLen] := cCur[i]
next

alert( substr( cHefty, 1, nTotLen ) )

Mel Smith

未讀,
2010年6月15日 下午6:08:522010/6/15
收件者:

"Ella" <ella....@xharbour.com> wrote in message
news:8846cd0a-82d8-459e...@e5g2000yqn.googlegroups.com...

Ella

nTotLen := 0

Ella:

Thank you. I'll try that technique out


dlzc

未讀,
2010年6月15日 下午6:31:312010/6/15
收件者:
Dear Ella:

On Jun 15, 12:17 pm, Ella <ella.st...@xharbour.com> wrote:
> On 15 iun., 18:21, "A.Martínez" <alcis...@arrakis.es> wrote:

> > Using Hash ?
>
> > Is it hash a b-tree ?

Do you know if the hash keys are "indexed", Ella?

Presumably it should be... but I have never used them.

David A. Smith

Ella

未讀,
2010年6月15日 晚上7:06:532010/6/15
收件者:
David,


The hash key IS AN index, so when listing the first, second,... key-
value peers, the keys are in ascendent order.

By default the index depth is 1, IOW we have one "partition".

Please type in HASH in the index of your documentation - the subject
is not simple, but very nice, and I find hashes very handy in
conjunction with ComboBoxes, INI files, and web programming.

Ella

A.Martínez

未讀,
2010年6月16日 凌晨3:43:182010/6/16
收件者:
Ella, Mel

My question initial is:

Is more faster to using hash table pointer than ascan function in large
arrays ?
I think answer is yes... but then, should us always to use hash table
pointer and don´t use ascan() ?

Regards


A.Martínez escribió en mensaje ...

Ella

未讀,
2010年6月16日 凌晨4:19:252010/6/16
收件者:
Dear A.

> My question initial is:
>
> Is more faster to using hash table pointer than ascan function in large
> arrays ?

What do you mean on a "large array" ?

- how many rows, how many columns - for example a hash table with 50
keys, but 800 columns is "small" when searching a key, but "big" when
allocating it on "row by row" basis

- what % of the physical RAM is going to be occupied ( one of the many
speed factors is to keep free about 30% - 50% of the physical RAM )

- what is the main usage of your table ? For example hash tables are
speedy when used for search ( read-only ) and slow when used for
updating

- in case you want to do many searches in the same table, I'd read in
the data in an array, I'd do ASORT on MyKey, and then I'd copy the
table in a hash with MyKey as key

> I think answer is yes... but then, should us always to use hash table
> pointer and don´t use ascan() ?

I'd tried to explain, that each situation needs periodical fine-tuning
by the developer.


Ella

Pavel Tsarenko

未讀,
2010年6月16日 清晨7:21:232010/6/16
收件者:
On 15 июн, 18:21, "A.Martínez" <alcis...@arrakis.es> wrote:
> Hi,
>
> As subject and is it Ascan slow ?
>
> how to add more fastest ?
>

Hi

The first way to solve this problem - write search function in C.
Scanning is much more faster:

ASCANA(aSales, cCus+ cPro, 1)

The source of ASCANA(aArray, xSearch, nIndex[, lEqual] ) functions:


#include "hbapi.h"
#include "hbapiitm.h"

/*
itmCompare(p1, p2)

return:
0 if p1 == p2
-1 if p1 < p2
1 if p1 > p2

*/

int itmCompare( PHB_ITEM pValue, PHB_ITEM pItem, BOOL bExact )
{
int iRet = 1;

if( HB_IS_STRING( pValue ) && HB_IS_STRING( pItem ) )
{
iRet = hb_itemStrCmp( pValue, pItem, bExact );
}
else if( HB_IS_DATE( pValue ) && HB_IS_DATE( pItem ) )
{
#ifdef __XHARBOUR__
if( pItem->item.asDate.value == pValue->item.asDate.value &&
pItem->item.asDate.time == pValue->item.asDate.time )
#else
if( hb_itemGetTD( pItem ) == hb_itemGetTD( pValue ) )
#endif
{
iRet = 0;
}
#ifdef __XHARBOUR__
else if( pValue->item.asDate.value < pItem->item.asDate.value )
#else
else if( hb_itemGetTD( pValue ) < hb_itemGetTD( pItem ) )
#endif
{
iRet = -1;
}
}
else if( HB_IS_NUMINT( pValue ) && HB_IS_NUMINT( pItem ) )
{
HB_LONG l1 = hb_itemGetNInt( pValue );
HB_LONG l2 = hb_itemGetNInt( pItem );
if( l1 == l2 )
{
iRet = 0;
}
else if( l1 < l2 )
{
iRet = -1;
}
}
else if( HB_IS_NUMBER( pValue ) && HB_IS_NUMBER( pItem ) )
{
double d1 = hb_itemGetND( pValue );
double d2 = hb_itemGetND( pItem );
if( d1 == d2 )
{
iRet = 0;
}
else if( d1 < d2 )
{
iRet = -1;
}
}
else if( HB_IS_LOGICAL( pValue ) && HB_IS_LOGICAL( pItem ) )
{
BOOL b1 = hb_itemGetL( pValue );
BOOL b2 = hb_itemGetL( pItem );
if( b1 == b2 )
{
iRet = 0;
}
else if(! b1 )
{
iRet = -1;
}
}

return iRet;
}

static PHB_ITEM get2Array( PHB_ITEM pArray, ULONG ul1, ULONG ul2 )
{
PHB_ITEM pSubArray = hb_arrayGetItemPtr( pArray, ul1 );
return (ul2 ? hb_arrayGetItemPtr( pSubArray, ul2 ) : pSubArray );
}

static ULONG hb_ascana(PHB_ITEM pArray, PHB_ITEM pValue, ULONG
ulIndex, BOOL bExact)
{
ULONG ulLoop, ulLen;
/* PHB_ITEM pSubArray;*/
PHB_ITEM pItem;
BOOL bFound = FALSE;

if( pArray && HB_IS_ARRAY( pArray ) && pValue && ulIndex )
{
ulLen = hb_arrayLen( pArray );
for( ulLoop = 0; ulLoop < ulLen; ulLoop++ )
{
pItem = get2Array( pArray, ulLoop + 1, ulIndex );

if( pItem && ( itmCompare( pItem, pValue, bExact ) == 0) )
{
bFound = TRUE;
break;
}

}
}

return ( bFound ? ulLoop + 1 : 0 );
}

HB_FUNC( ASCANA )
{
hb_retnl( hb_ascana(hb_param( 1, HB_IT_ARRAY ),
hb_param( 2, HB_IT_ANY ),
hb_parnl( 3 ),
( ISLOG(4) ? hb_parl(4) : FALSE ) ) );
}


The second way - scanning in sorted array. If you are interested, I
can post sources of such functions

Best regards, Pavel Tsarenko

A.Martínez

未讀,
2010年6月16日 上午8:14:022010/6/16
收件者:
Pavel

is it ascana() more faster than Hash ?

>The second way - scanning in sorted array. If you are interested, I
>can post sources of such functions

Yes, please, i'm interested !

Regards

druzus

未讀,
2010年6月16日 上午8:45:442010/6/16
收件者:
On 15 Cze, 17:21, "A.Martínez" <alcis...@arrakis.es> wrote:
> Hi,
>
> As subject and is it Ascan slow ?

Yes. In such context is very slow.
This code executes codeblock for each scanned item.
For relatively small tables, i.e with 10000 records the
average number of executed blocks is
10000 * 10000 / 2 = 50000000
The cost of 50 millions of
{|x| a[1] == cCus+ cPro }:EVAL()
calls has to be noticeable and this is the most time
consuming part. It also uses AADD() and AFAIR
xHarbour does not optimize AADD() by default anyhow
in this case the cost of reallocation is minor in
comparison to the cost of EVAL().

> how to add more fastest ?
> Using Hash ?

Yes. Such situations can be greatly improved by
Hashes.

> Is it hash a b-tree ?

This is implementation detail which should
not be important for user. Important is only
the fact that hash array implementation allows
to efficiently use noncontinuous indexes.

> Example:
>
> FUNCTION CalcSales()
> Local aSales:= {}
> SELECT Sales
> GO TOP
> DO WHILE !Eof()
>    AddUnits(aSales, _FIELD-> Customer, _FIELD-> Product , _FIELD-> Unit)
>    SKIP
> ENDDO
> RETURN aSales
>
> STATIC FUNCTION AddUnits(aSales, cCus, cPro, nUni)
> Local nPos
> nPos:= Ascan(aSales, nil, {|x| a[1] == cCus+ cPro})                      //
> Slow !
> IF nPos == 0
>    Aadd(aSales, {cCus+ cPro, nUni})
> ELSE
>    aSales[nPos, 2]+= nUni
> ENDIF
> RETURN NIL

And this is the same code converted (without any runtime
tests so some typos possible) to operate on hash arrays:

FUNCTION CalcSales()
LOCAL hSales := { => }


SELECT Sales
GO TOP
DO WHILE !Eof()

AddUnits( hSales, _FIELD-> Customer, _FIELD-> Product , _FIELD->
Unit )
SKIP
ENDDO
RETURN Hash2Array( hSales )

STATIC FUNCTION AddUnits( hSales, cCus, cPro, nUni )
IF cCus + cPro $ hSales
hSales[ cCus + cPro ] += nUni
ELSE
hSales[ cCus + cPro ] := nUni
ENDIF
RETURN NIL

STATIC FUNCTION Hash2Array( hValue )
LOCAL item, aValue := Array( Len( hValue ) )
FOR EACH item IN aValue
item := { HGetKeyAT( hValue, HB_EnumIndex() ), ;
HGetValueAT( hValue, HB_EnumIndex() ) }
NEXT
RETURN aValue

It should be much faster. If you are using Harbour
then it's possible to make some other improvements
using features which are not available in xHarbour,
like default hash value or extended FOR EACH loops,
i.e.:
FUNCTION CalcSales()
LOCAL hSales := { => }
HB_HDefault( hSales, 0 )
HB_HAutoAdd( hSales, .T. )


SELECT Sales
GO TOP
DO WHILE !Eof()

hSales[ _FIELD-> Customer + _FIELD-> Product ] += _FIELD-> Unit
SKIP
ENDDO
RETURN Hash2Array( hSales )

STATIC FUNCTION Hash2Array( hValue )
LOCAL unit, item, aValue := Array( Len( hValue ) )
FOR EACH unit, item IN hValue, aValue
item := { unit:__enumKey(), unit }
NEXT
RETURN aValue

This code for each new item makes only one hash
array scan instead of two due to default access
value set by HB_HDefault()/HB_HAutoAdd() and
compile time optimization for all <op>=, ++ and --
operators.
Please also note that default hash value is cloned
when new hash item is created. It means that you
can nicely use this functionality for collecting more
complex data, i.e.:

FUNCTION CalcSales()
LOCAL hSales := { => }, aItem
HB_HDefault( hSales, { "", "", 0, 0 } )
HB_HAutoAdd( hSales, .T. )


SELECT Sales
GO TOP
DO WHILE !Eof()

aItem := hSales[ _FIELD-> Customer + _FIELD-> Product ]
aItem[ 1 ] := _FIELD-> Customer
aItem[ 2 ] := _FIELD-> Product
aItem[ 3 ] += _FIELD-> Unit
aItem[ 4 ] += _FIELD-> Unit * _FIELD-> Price
SKIP
ENDDO
RETURN Hash2Array( hSales )

STATIC FUNCTION Hash2Array( hValue )
LOCAL unit, item, aValue := Array( Len( hValue ) )
FOR EACH unit, item IN hValue, aValue
item := unit
NEXT
RETURN aValue

best regards,
Przemek

Pavel Tsarenko

未讀,
2010年6月16日 上午8:56:142010/6/16
收件者:

Parameters of ASCANAS function is the same as ASCANA:

nPos := AScanAS(aArray, xValue, nIndex[, lEqual] )

The return value:
nPos > 0 - array element is found
nPos < 0 - array element is not found and should be inserted before -
nPos index
nPos == 0 - array element should be added into array

The usage of ASCANAS:

nPos := AScanAS(aArray, xValue, 1)

if nPos == 0 .or. -nPos > Len(aArray)

AADD(aArray, xValue)

elseif nPos < 0

nPos := -nPos
#ifdef __XHARBOUR__
AINS(aArray, nPos, xValue, .t.)
#else
HB_AINS(aArray, nPos, xValue, .t.)
#endif

else
...
endif

The source two-dimensional array is sorted on the first element.

The source of ASCANAS:

static LONG hb_ascanas(PHB_ITEM pArray, PHB_ITEM pValue, ULONG
ulIndex, BOOL bExact)
{
ULONG ulLoop = 0;
BOOL bFound = FALSE;

if( pArray && HB_IS_ARRAY( pArray ) && pValue && ulIndex )
{

ULONG ulLen = hb_arrayLen( pArray );
BOOL bEnd = FALSE;
PHB_ITEM pItem;
int iResult;

if( ulLen == 0)
{
ulLoop = 0;
bEnd = TRUE;
}
else
{

pItem = get2Array( pArray, 1, ulIndex );

if( pItem )
{
iResult = itmCompare( pItem, pValue, bExact );
}
else
{
bEnd = TRUE;
}

if( ! bEnd )
{
if( iResult == 0 )
{
ulLoop = 1;
bFound = bEnd = TRUE;
}
else if( iResult > 0 )
{
ulLoop = 1;
bEnd = TRUE;
}
}

if( ! bEnd )
{
if( ulLen > 1 )
{
pItem = get2Array( pArray, ulLen, ulIndex );

if( pItem )
{
iResult = itmCompare( pItem, pValue, bExact );
}
else
{
bEnd = TRUE;
}
}
if( ! bEnd )
{
if( iResult == 0 )
{
ulLoop = ulLen;
bFound = bEnd = TRUE;
}
else if( iResult < 0 )
{
ulLoop = ulLen + 1;
bEnd = TRUE;
}
}
}

}

if( ! bEnd )
{
ULONG ul1 = 1, ul2 = ulLen;

while( TRUE )
{
if( ul1 == ul2 - 1 )
{
ulLoop = ul2;
break;
}
ulLoop = (ul2 + ul1) / 2;

pItem = get2Array( pArray, ulLoop, ulIndex );

if( pItem )
{
iResult = itmCompare( pItem, pValue, bExact );
}
else
{
ulLoop = 0;
break;
}

if( iResult == 0 )
{
bFound = TRUE;
break;
}
else if( iResult > 0 )
{
ul2 = ulLoop;
}
else
{
ul1 = ulLoop;
}
}
}

}

return ( bFound ? ulLoop : - (LONG) ulLoop );
}

HB_FUNC( ASCANAS )
{
hb_retnl( hb_ascanas(hb_param( 1, HB_IT_ARRAY ),


hb_param( 2, HB_IT_ANY ),
hb_parnl( 3 ),
( ISLOG(4) ? hb_parl(4) : FALSE ) ) );
}


Best regards, Pavel Tsarenko

A.Martínez

未讀,
2010年6月16日 中午12:14:122010/6/16
收件者:
Pavel,

Thank you very much...

Do you know... is Ains() a fast operation ?

Regards

Pavel Tsarenko escribió en mensaje
<33150540-9805-4d5b...@u26g2000yqu.googlegroups.com>...

A.Martínez

未讀,
2010年6月16日 中午12:20:242010/6/16
收件者:
przemek,

Thank your very much.... magistral !

I'll try with xharbour hashes scheme.

is bottle neck (cCus+ cPro $ hSales ) operation?


Regards

druzus escribió en mensaje ...

Patrice

未讀,
2010年6月16日 晚上10:06:232010/6/16
收件者:
Hello,

The first traditionnal way speed up your program is to index the base on
the key if index is possible.

FUNCTION CalcSales()
Local aSales:= {}
SELECT Sales

+ index on Customer+Product to tmp
GO TOP
+ cRupt= ""
DO WHILE !Eof()
+ if cRupt != _FIELD-> Customer+ _FIELD-> Product
+ aadd(aSales, { _FIELD-> Customer+ _FIELD-> Product, 0} )
+ cRupt= _FIELD-> Customer+ _FIELD-> Product
+ end
+ atail(aSales)[2] += _FIELD-> Unit
SKIP
ENDDO
RETURN aSales

My second traditionnal way to do it:
If index is not an option, I used to have a sorted array to avoid ascan.
The code could have looked like:

FUNCTION CalcSales()
Local aSales:= {}
SELECT Sales
GO TOP
DO WHILE !Eof()

+ pntr= asearch(aSales, { _FIELD-> Customer+ _FIELD-> Product, 0}, {|_1,
_2| _1[1] < _2[1]})
+ aSales[pntr, 2] += _FIELD-> Unit
SKIP
ENDDO
RETURN aSales

asearch look in the sorted array (by dichotomy) for a match and return the index
in the array, if no match, it insert the record in the array.
The code block is the sorting key

Patrice

Pavel Tsarenko

未讀,
2010年6月17日 凌晨2:44:002010/6/17
收件者:
On 16 июн, 19:14, "A.Martínez" <alcis...@arrakis.es> wrote:
> Pavel,
>
> Thank you very much...
>
> Do you know... is Ains() a fast operation ?
>
> Regards

AINS is slower than AADD, because of elements of array are moved (with
shift by one element) from nPos position to end of array.
But the basic time in a cycle is used in search operation so
performance time aadd/ains is insignificant.
BTW, algorithms with hashes and the sorted arrays are almost identical
on performance time.

Best regards, Pavel Tsarenko

druzus

未讀,
2010年6月17日 凌晨3:52:202010/6/17
收件者:
On 16 Cze, 18:20, "A.Martínez" <alcis...@arrakis.es> wrote:
> I'll try with xharbour hashes scheme.
> is bottle neck (cCus+ cPro $ hSales ) operation?

The cost of
( cCus + cPro $ hSales )
is comparable to the cost of:
hSales[ cCus + cPro ]
In both cases it's necessary to locate item
using some index key. The cost of this operation
depends on internal hash array format. Usually it
is some type of binary search and/or hashed/bucket
sorting. It's _much_ faster then linear scan like in
pure ASCAN() function anyhow it's nice if it
can be reduced to single call:


hSales[ cCus + cPro ] += nUni

instead of two like in:


IF cCus + cPro $ hSales
    hSales[ cCus + cPro ] += nUni
ELSE
    hSales[ cCus + cPro ] := nUni
ENDIF

best regards,
Przemek

druzus

未讀,
2010年6月17日 凌晨4:43:282010/6/17
收件者:
On 15 Cze, 22:22, "Mel Smith" <med_cutout_syn...@aol.com> wrote:
>     When building very long strings (e.g.,  up to 3 megabytes), is there a
> size allocation technique that should be used ?

xHarbour optimize += operator for strings if left side of expression
is
local variable or regukar array item ( a[n] += ...), i.e. try this
code:

#define N_LOOP 1000000
proc main()
local i, t
local l
static s
s := l := ""
t := secondsCPU()
for i := 1 to N_LOOP
l += chr( i )
next
t := secondsCPU() - t
? "LOCAL +=", t, "sec."
t := secondsCPU()
for i := 1 to N_LOOP
s += chr( i )
next
t := secondsCPU() - t
? "STATIC +=", t, "sec."
return

Don't try to interrupt this code and wait for STATIC results.
Here I my results for current xHarbour builds:
LOCAL += 0.43 sec.
STATIC += 185.82 sec.

As you can see working with xHarbour on such code it's
extremely important to operate only on optimized expressions.

Harbour optimize +=, -= just like all other <op>=, ++, --
operators (for strings important are only += and -=) for all
left side expressions except fields, undeclared variables
(because they are potential fields in Clipper) and object
variables. Optimization for object variables is enabled
optionally and is not present by default.
Harbour results for above code:
LOCAL += 0.32 sec.
STATIC += 0.33 sec.

best regards,
Przemek

A.Mart�nez

未讀,
2010年6月17日 凌晨3:38:112010/6/17
收件者:
Patrice,

> The first traditionnal way speed up your program is to index the base on
>the key if index is possible.
>
>FUNCTION CalcSales()
>Local aSales:= {}
>SELECT Sales
>+ index on Customer+Product to tmp
>GO TOP
>+ cRupt= ""
>DO WHILE !Eof()
>+ if cRupt != _FIELD-> Customer+ _FIELD-> Product
>+ aadd(aSales, { _FIELD-> Customer+ _FIELD-> Product, 0} )
>+ cRupt= _FIELD-> Customer+ _FIELD-> Product
>+ end
>+ atail(aSales)[2] += _FIELD-> Unit
> SKIP
>ENDDO
>RETURN aSales

But how to index on shared dbfs because i can't locks dbf using FLock()


> My second traditionnal way to do it:
>If index is not an option, I used to have a sorted array to avoid ascan.
>The code could have looked like:
>
>FUNCTION CalcSales()
>Local aSales:= {}
>SELECT Sales
>GO TOP
>DO WHILE !Eof()
>+ pntr= asearch(aSales, { _FIELD-> Customer+ _FIELD-> Product, 0}, {|_1,
>_2| _1[1] < _2[1]})
>+ aSales[pntr, 2] += _FIELD-> Unit
> SKIP
>ENDDO
>RETURN aSales
>
>asearch look in the sorted array (by dichotomy) for a match and return the
index
>in the array, if no match, it insert the record in the array.
>The code block is the sorting key

Core on second solution is aSearch() function.
Perhaps asearch code is than Pavel Ascanas() function !

Regards

PD. Sorry my poor english.


dlzc

未讀,
2010年6月17日 上午11:01:482010/6/17
收件者:
Dear A.Martinez

On Jun 17, 12:38 am, "A.Mart nez" <alcis...@arrakis.es> wrote:
> Patrice,
...


> > The first traditionnal way speed up your program
> > is to index the base on the key if index is possible.

...


> But how to index on shared dbfs because i can't
> locks dbf using FLock()

If you are using almost any RDD, exclusive access is not required.
Check the online docs for INDEX, where it says: "EXT EXCLUSIVE= With
this option, the index file is created in EXCLUSIVE file access mode.
By default, the index file is created in SHARED mode." Probably
better if you are using something like DBFCDX or such.

You might be able to twist the command SUM to do what you need, to
populate your array / hash. I could not guess how to do that.

Simpler and more transparent (and requires an index on "Customer +
Product") would be to use the TOTAL command like so:
TOTAL ;
ON Customer + Product ;
FIELDS Unit
TO <cTempDBFname>

This makes a copy dbf whose records contain (in this case) the total
sales of each product, by customer, in the field Unit. Even produced
in sort order.

With these caveats from the online docs:
"The target database has the same structure as the source database.
This requires the field length of numeric fields be large enough to
hold total values. If the result is too large, a runtime error is
generated.

Records marked for deletion are not included in the calculation when
SET DELETED is set to ON"

David A. Smith

A.Mart�nez

未讀,
2010年6月17日 中午12:51:332010/6/17
收件者:
dlzc,

>Simpler and more transparent (and requires an index on "Customer +
>Product") would be to use the TOTAL command like so:
>TOTAL ;
> ON Customer + Product ;
> FIELDS Unit
> TO <cTempDBFname>

but, perhaps slowest and requirements solution.

Write to disk is a slow operation
No always is available a index.

regards

dlzc

未讀,
2010年6月17日 下午2:17:402010/6/17
收件者:
Dear A.Martinez:

On Jun 17, 9:51 am, "A.Mart nez" <alcis...@arrakis.es> wrote:
> dlzc,
>
> >Simpler and more transparent (and requires an index on "Customer +
> >Product") would be to use the TOTAL command like so:
> >TOTAL ;
> >   ON Customer + Product ;
> >   FIELDS Unit
> >   TO <cTempDBFname>
>
> but, perhaps slowest and requirements solution.
>
> Write to disk is a slow operation

But is elegant, and easy to maintain in the future. And will it be
*too* slow? Maybe, maybe not, it is optimized code that does not go
through the programmer level of xHarbour, and will access each source
record once. No significant array searches or maintenance, just
"pump". No concerns ever about array or hash "size".

Its only drawbacks are the size of the field "Unit" (in this case),
and you cannot have complex results (a new field that tells you when
in the "month" the units were sold / ordered / shipped on average,
say). And you need to find a place to save the file... ;-)

> No always is available a index.

According to the online documentation, it is always possible to make
it with SHARED access, and once made, can be reused since everyone
updates it (at least on DBFCDX and SQLRDD for examples). If present,
this method may be *faster*, and provide a (possible) permanent
record.

And I understand that you will choose what seems right to you. You do
not have to convince me, since this is your "baby". No response
required, I was just outlining some alternatives.

David A. Smith

A.Martínez

未讀,
2010年6月17日 下午2:27:152010/6/17
收件者:
dlzc,


Thank you very much for your help...

I want to code a decision cube...
A decision cube precises anything order and factors: impossible to create
all neccesary indexes.

My problem is speed... i haven´t precalculates datas...

A example my decision cube (4 deeps)


1.Customer/Product-> 2.Product/Months-> 3.Months/Weeks -> 4.Weeks/Day

Navegation scren is with reports, clicking with mouse

Regards


Mel Smith

未讀,
2010年6月17日 下午3:04:142010/6/17
收件者:
Hi Przemek:

"druzus" <przemysla...@gmail.com> wrote in message
news:699d1ea1-4cfc-41dd...@y4g2000yqy.googlegroups.com...

best regards,
Przemek

I'll read this over carefully and try out your suggestions over the next few
days

Thank you for this look (behind the scenes) !

-Mel Smith


Mel Smith

未讀,
2010年6月17日 下午3:29:242010/6/17
收件者:
Hi Przemek:

I tried your test program on my machine:

I got:

Local += 0.66 sec.
Static += 466.92 sec.

Wow !!!

Thanks again for the guidance. I'm sure not going to use 'Statics' for
string manipulation !

-Mel Smith


druzus

未讀,
2010年6月18日 清晨6:07:262010/6/18
收件者:
On 17 Cze, 21:29, "Mel Smith" <med_cutout_syn...@aol.com> wrote:

Hi,

>     I tried your test program on my machine:
>     I got:
>
>     Local    +=    0.66 sec.
>     Static    +=    466.92 sec.
>
>     Wow !!!
> Thanks again for the guidance.   I'm sure not going to use 'Statics' for
> string manipulation !

STATICs are only example. You will have similar results for GLOBALs
PUBLICs, PRIVATEs, ... In xHarbour for expressions like
<exp> += <exp2>
+= operator is optimized only when <exp> is LOCAL variable
or regular array item. In all other cases it's not optimized and
the performance is like for STATICs in above test.
xBaseScript users should also remember that it internally
does not supprt local so this optimization will not work too
and in such case it's more efficient to switch arrays, i.e.
local a[1]
a[1]:=""
[...]
a[1] += <data>

best regards,
Przemek

0 則新訊息