d3 Basic Programming Question

[Robert S. Lobel]

I have an item that has 3 multi-valued attributes that are each approximately 300 transactions long.  These 3 attributes are linked -- for example, the third transaction in attribute 46 is linked with the third transaction in attribute 47.

Is there an easy way to prune the first 200 transactions from the 3 attributes?

[TonyG Test]

REC<1> = REC<1>[  INDEX(REC<1>,@VM,200) + 1 , 999999 ]

There is some advanced dynamic array manipulation defined in the documentation but I haven't used it.
For example, we often hardcode something like 999999, meaning "there really shouldn't be more than that many characters left in the string. But we also know that's not solid coding, as it'll blow up on us as soon as we get over 1MB of data. So the better technique is VAR[ StartHere, LEN(VAR) ], which implies "we know we're asking for more than what's there but that will certainly get what we need".
The better way would be for some syntactic sugar like VAR[ StartHere,* ] or simply VAR[ StartHere ]. Some systems allow variants of this, but since I haven't researched to know just how global this syntax is, I haven't used it anywhere.

The summary on that is that you might be able to use something other than 999999 there, and for your application you might need to. Please feel free to check the docs and let us know.

HTH
T

[Steve Trimble]

OPEN "THEFILE" TO FL ELSE STOP 201,"THEFILE"

REC.ID = "THEREC"

READ REC FROM FL,REC.ID THEN

   FOR AA = 200 TO 1 STEP -1      ; * AA is the value number to delete

      REC = DELETE(REC,47,AA,0)  ; * delete highest attribute 1st

      REC = DELETE(REC,46,AA,0)  ; * now the 2nd attribute

      REC = DELETE(REC,45,AA,0)  ; * now the 3rd attribute

   NEXT AA

END

WRITE REC ON FL,REC.ID

END

Steve Trimble

Computerized Data Mgmt Inc

(501) 615-8674 office

(501) 772-3450 cell / text

On Tue, Feb 6, 2018 at 2:27 PM, Robert S. Lobel <robl...@gmail.com> wrote:

I have an item that has 3 multi-valued attributes that are each approximately 300 transactions long.  These 3 attributes are linked -- for example, the third transaction in attribute 46 is linked with the third transaction in attribute 47.

Is there an easy way to prune the first 200 transactions from the 3 attributes?

--
You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com
To unsubscribe, email to: mvdbms+unsubscribe@googlegroups.com
For more options, visit http://groups.google.com/group/mvdbms

[Wols Lists]

You'll have to check the exact syntax, but you want the FIELD function.
Iirc (I don't have a working system to check on) it's something like

REC<46> = FIELD( REC<46>, @VM, 201, 9999)

In other words, with a delimiter of @VM, start with the 201st delimited
value, and copy the next 9999. Even better is to replace 9999 with
COUNT( REC<46>, @VM), but I wouldn't do it in the FIELD function itself,
evaluate it first and pass it in.

Tony's idea is good (but obscure, sorry Tony - string manipulation on
arrays works fine but doesn't make it clear what you're doing).

Steve's idea is great (in fact, probably far better and easier) IF
you're only deleting a couple of values. 200? It'll absolutely knacker
performance - that's 600 string copies of a large variable ... not good.

Oh - and if you're dealing with large fields (as in with lots of
values), and you have a lot of fields (up in the forties sounds a bit
like it), look at using DIMensioned arrays. It would speed this up here
a fair chunk!

Cheers,
Wol

[George Gallen]

What about:

X(1)=TRANS<1>

X(2)=TRANS<2>

X(3)=TRANS< 3>

NEWT=""

FOR T=1 TO 3

   FOR Q=201 TO MAXTRANS

        NEWT<T,Q>=X(T)<1,Q>

   NEXT Q

NEXT T

TRANS=NEWT

---

From: mvd...@googlegroups.com <mvd...@googlegroups.com> on behalf of Robert S. Lobel <robl...@gmail.com>
Sent: Tuesday, February 6, 2018 3:27 PM
To: Pick and MultiValue Databases
Subject: [mvdbms] d3 Basic Programming Question

 

I have an item that has 3 multi-valued attributes that are each approximately 300 transactions long.  These 3 attributes are linked -- for example, the third transaction in attribute 46 is linked with the third transaction in attribute 47.

Is there an easy way to prune the first 200 transactions from the 3 attributes?

--

You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+un...@googlegroups.com

[George Gallen]

Forgot

MAXTRANS=DCOUNT(TRANS<1>,CHAR(253))

---

From: mvd...@googlegroups.com <mvd...@googlegroups.com> on behalf of George Gallen <g_ga...@hotmail.com>
Sent: Wednesday, February 7, 2018 9:53 AM
To: mvd...@googlegroups.com
Subject: Re: [mvdbms] d3 Basic Programming Question

 

[George Gallen]

One other typo

NEWT<T,Q-200>=X(T)<1,Q>

---

From: mvd...@googlegroups.com <mvd...@googlegroups.com> on behalf of George Gallen <g_ga...@hotmail.com>

Sent: Wednesday, February 7, 2018 9:54 AM

[George Gallen]

I guess you could also:

FOR T=1 TO 3

    FOR Q=1 TO 200

         DEL TRANS<T,Q>

   NEXT Q

NEXT T

However, I've had issues sometimes with the dynamic array getting totally screwed up using the DEL

but I believe that to be an issue with running a 32bit UV on a 64bit Linux - it shouldn't be a problem...but it is

---

From: mvd...@googlegroups.com <mvd...@googlegroups.com> on behalf of George Gallen <g_ga...@hotmail.com>

Sent: Wednesday, February 7, 2018 9:55 AM

[George Gallen]

yeah...I know  200 to 1 step -1

or

DEL TRANS<T,1> using the 1 to 200

---

From: mvd...@googlegroups.com <mvd...@googlegroups.com> on behalf of George Gallen <g_ga...@hotmail.com>

Sent: Wednesday, February 7, 2018 9:59 AM

To: mvd...@googlegroups.com
Subject: Re: [mvdbms] d3 Basic Programming Question

 

I guess you could also:

[Steve Trimble]

don't think that would work properly

when deleting, you want to start at the end and work backwards.

ex: when you delete value 1, then you have values 2 thru 199 left

so as you up Q (in your case) you start deleting the wrong values

hence then reason to:

AA = 200 to 1 step -1

so AA always is deleting the correct value

Steve Trimble

Computerized Data Mgmt Inc

(501) 615-8674 office

(501) 772-3450 cell / text

Forgot

MAXTRANS=DCOUNT(TRANS<1>,CHAR(253))

To unsubscribe, email to: mvdbms+unsubscribe@googlegroups.com

For more options, visit http://groups.google.com/group/mvdbms

--
You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+unsubscribe@googlegroups.com

For more options, visit http://groups.google.com/group/mvdbms

--
You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+unsubscribe@googlegroups.com

For more options, visit http://groups.google.com/group/mvdbms

--
You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+unsubscribe@googlegroups.com

For more options, visit http://groups.google.com/group/mvdbms

--
You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+unsubscribe@googlegroups.com

[Wols Lists]

On 07/02/18 15:20, Steve Trimble wrote:
> don't think that would work properly
> when deleting, you want to start at the end and work backwards.
> ex: when you delete value 1, then you have values 2 thru 199 left
> so as you up Q (in your case) you start deleting the wrong values
> hence then reason to:
> AA = 200 to 1 step -1
> so AA always is deleting the correct value
>

Now if he'd deleted the first value 200 times ...

Cheers,
Wol

[John Stokka]

If you really want to go forward you can always...

FOR T=1 TO 3

    FOR Q=1 TO 200

         DEL TRANS<3-T+1,200-Q+1>

   NEXT Q

NEXT T

... I know the +1 is kinda goofy but instead of 4-T and 201-Q which introduces two more goofy numbers I like the +1 method...   I'm such a nerd... lol

[Arthur Martz]

Or,  REC<ATTR>=FIELD(REC<ATTR>, @AM,201,9999)

[Jeff K]

Lots of ways to do this... Here's where I would start:

FOR X = 1 TO 200
   DEL REC< 45, 1 >
   DEL REC< 46, 1 >
   DEL REC< 47, 1 >
NEXT X

This reads like the problem description to me: "Delete the first 200 values from these three attributes." If this turned out to be too slow, then I'd try something else.

Into speculation land now... This seems like a need within a larger purge process, which probably runs only periodically, so speed isn't the highest priority. But maybe there are ten million records (or more) to process. Hard to say if clarity or efficiency is more important without knowing the full scope of the problem being addressed.

Tony's solution is probably the fastest, but as mentioned, kind of obscure unless you're steeped in MV Basic and the structure of a dynamic array.

$0.02

[Robert S. Lobel]

First, and BEST!  Thank you, Tony!

[Dan Ell]

That would indeed screw it up.   You would need to delete value 1, not value Q.

 

FOR T=1 TO 3

    FOR Q=1 TO 200

         DEL TRANS<T,1Q>

   NEXT Q

NEXT T

 

 

Dan Ell jBASE Technical Engineer 949-383-2429  |  866‑582‑8447 da...@jbase.com  |  www.jbase.com 9245 Research Drive ,  Irvine ,  92618

[George Gallen]

I like that....

---

From: mvd...@googlegroups.com <mvd...@googlegroups.com> on behalf of Arthur Martz <artsm...@gmail.com>
Sent: Wednesday, February 7, 2018 10:42 AM

To unsubscribe, email to: mvdbms+un...@googlegroups.com

[Tony Gravagno]

I felt compelled to lookup some of that special syntax that I referenced. Here are a couple examples:

REC<1>[  1 , INDEX( REC<1> , @VM , 200 )+1 ] = ""

and this is wild but it's the best answer here... if it works...

REC<1>[ @VM , 200 , -1 ] = ""

Since we're here, for dynamic arrays, I've never seen anyone use this syntax for D3, but it's there...

IF REC<1,*> = "" THEN        ; * Check to see if Any value is empty (without a loop)

IF REC<*,2,*> = "FOO"        ; * Check all attributes to see if any sub-value = "FOO"

While I love all of that syntax, like most of us here I didn't "grow up with it" back in the days of Microdata, Ultimate, ADDS, GA, etc. I think this was in R91 but at the time I was concerned about porting code, so was careful not to use what looked like a platform-specific trick.

I believe that today most or all platforms do support some or all of the above syntax, so I think we should use it, unless there is some compelling reason not to. One reason I might have to worry is that it looks like the above syntax is defined differently in two places in the D3 docs. This is another case of great functionality that some engineers painfully added, and then the docs people never documented it right, and Marketing never told anyone, so all of that work went to waste ... and then sites migrate to QM to get great "new" features that they already had in D3. Hey, if the team keeps fumbling the ball it's no surprise that the other teams are going to win.

T

[chandru murthi]

Now that's  unnecessarily obscure. Why go in reverse order for the *Attributes?*

DEL is the clearest, but beware: UV has a really obscure bug in DEL which screws up the data. Over the years I've given up using it on UV.

Personally, I think the INDEX solution by Tony is the best. And, also I don't worry about 999999 being too low...assuuming we're in a single database attribute, that's comfortable

Chandru

Forgot

MAXTRANS=DCOUNT(TRANS<1>,CHAR(253))

To unsubscribe, email to: mvdbms+un...@googlegroups.com

For more options, visit http://groups.google.com/group/mvdbms

--
You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+un...@googlegroups.com

For more options, visit http://groups.google.com/group/mvdbms

--
You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+un...@googlegroups.com

For more options, visit http://groups.google.com/group/mvdbms

--
You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+un...@googlegroups.com

For more options, visit http://groups.google.com/group/mvdbms

--
You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+un...@googlegroups.com

For more options, visit http://groups.google.com/group/mvdbms

--
You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+un...@googlegroups.com

[chandru murthi]

REC<46> = FIELD( REC<46>, @VM, 201, 9999)

> In other words, with a delimiter of @VM, start with the 201st delimited
value, and copy the next 9999. Even better is to replace 9999 with
COUNT( REC<46>, @VM), but I wouldn't do it in the FIELD function itself,
evaluate it first and pass it in

Can I ask why you would not?
REC<46> = FIELD( REC<46>, @VM, 201, DCOUNT(REC<46>,@VM))
is perfectly fine though overkill as a single database attribute is clearly not going to have, say more than 999999 values. .

Chandru

[Arthur Martz]

When it gets to the end of the string, it quits. Why spend the overhead doing a dcount? 

--

You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com

To unsubscribe, email to: mvdbms+unsubscribe@googlegroups.com

[Steve Trimble]

Chandru wrote:

Now that's  unnecessarily obscure. Why go in reverse order for the *Attributes?*

because deleting the last value of an attribute might 'remove' the attribute as well.

Steve Trimble

Computerized Data Mgmt Inc

(501) 615-8674 office

(501) 772-3450 cell / text

To unsubscribe, email to: mvdbms+unsubscribe@googlegroups.com

[Ian Harper]

I sparingly use this syntax. Unfortunately these aren't widely supported outside of D3 so I don't rely on them.

The field store ( rec[ delimiter, begin, end] ) is great for quick manipulation of data that can't be stored with MV delimiters (like CSV data). The IF statement with wildcards are nice for replacing locates where the position isn't important.

[Wol's lists]

On 07/02/18 20:38, Arthur Martz wrote:
> When it gets to the end of the string, it quits. Why spend the overhead
> doing a dcount?

Because, as we found out with day 10000, just assuming that 9999 (or
even 999999) is big enough, might some day in the future turn out to be
a big mistake.

Yes a dcount will have a small cost. Provided you do it OUTSIDE the
function call (so you can re-use the value three times without needing
to recalculate it), I'd call it "defensive programming", and effort well
spent.

I've been bitten by that sort of thing before, can't you tell :-)

Cheers,
Wol

>
> On Feb 7, 2018 3:35 PM, "chandru murthi" <cmur...@gmail.com

[Arthur Martz]

The field statement works with MV delimiters as well as it does with spaces, commas or any other delimiter. 

--

[chandru murthi]

Eh? What do you mean "MIGHT delete the attribute?". 
Are you saying  X = 'ABC':@am:'DEF':@am:'GHI'  ;  DEL X<2,1>  will result in:
                                'ABC':@am:'GHI'    ?
It most definitely will not.
If you mean it will delete a trailing empty @am if it's the last attribute, yes, but that follows Pick rules.

Chandru

[Steve Trimble]

that's the reason I always start at the bottom and work my way up.

same reason I start at the far end of values or subvalues and work my way to value or subvalue 1

just my preference

seems to work on all flavors of multivalue

Steve Trimble

Computerized Data Mgmt Inc

(501) 615-8674 office

(501) 772-3450 cell / text

To unsubscribe, email to: mvdbms+unsubscribe@googlegroups.com

[Tony Gravagno]

I was asked for a doc reference and found that my statement below was wrong.

Search the D3v10 doc for “substring”.
Under “Substrings” there’s info for string[delimiter, m, n] = substrings
Under “substring expressions”, scroll down to var[delimiter, beg.fld.exp, num.flds.exp] = substr

The supporting text there is different, organized with points in different order. But on careful word-for-word review it's evident that the content is consistent. I just mis-read it the first time through.

Thanks.
T
( Instructions for use of your new Samurai Sword: 1) remove sword from box. 2) fall on sword. )


I wrote:

REC<1>[ @VM , 200 , -1 ] = ""

...