Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Sort duplicates problem

183 views
Skip to first unread message

Howard Brazee

unread,
May 21, 2004, 12:45:48 PM5/21/04
to
We ran a job in the past with the following sort parms:
SORT FIELDS=(04,09,CH,A, SID
01,02,CH,A) INST
SUM FIELDS=NONE ELIMINATE DUPLICATES
DEBUG ABEND ABORT IF ERRORS
END

What I would like to do now is re-run this file and only select those duplicates that were eliminated the first time. (column 3 is different for the duplicates).

Is there a reliable way of getting only the ones that were eliminated in the previous run?

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to list...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Dean Montevago

unread,
May 21, 2004, 12:51:29 PM5/21/04
to
This will write the deleted dup records to the DD, don't know how to only
select them...

SUM FIELDS=NONE,XSUM

//SORTXSUM DD SYSOUT=X

Antonio Hacker

unread,
May 21, 2004, 1:04:50 PM5/21/04
to
Is there a possible problem in that, since EQUALS was not specified, they
may not sort the same way? Unlikely, I'd think...

>SUM FIELDS=NONE,XSUM

_________________________________________________________________
Best Restaurant Giveaway Ever! Vote for your favorites for a chance to win
$1 million! http://local.msn.com/special/giveaway.asp

Swami, Shivprakash , GE Commercial Finance, NonGE

unread,
May 21, 2004, 1:03:02 PM5/21/04
to
If you have ICETOOL ,this should give you desired results.

//S1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN DD DSN=infile..
//LIST1 DD SYSOUT=*
//TOOLIN DD *
OCCUR FROM(IN) LIST(LIST1) ON(1,9,CH) ALLDUPS

Refer link below..

http://www.storage.ibm.com/software/sort/mvs/icetool/online/srtmtocc.html

SUM FIELDS=NONE,XSUM

//SORTXSUM DD SYSOUT=X

----------------------------------------------------------------------

Frank Yaeger

unread,
May 21, 2004, 1:35:36 PM5/21/04
to


Howard Brazee wrote:

>We ran a job in the past with the following sort parms:
> SORT FIELDS=(04,09,CH,A, SID
> 01,02,CH,A) INST
> SUM FIELDS=NONE ELIMINATE DUPLICATES
> DEBUG ABEND ABORT IF ERRORS
> END
>
>What I would like to do now is re-run this file and only select those
duplicates that were eliminated the >first time. (column 3 is different
for the duplicates).
>
>Is there a reliable way of getting only the ones that were eliminated in
the previous run?

SUM FIELDS=NONE keeps non-duplicates and the first record of each set of
duplicates.

You can accomplish the same thing using DFSORT/ICETOOL with:

SELECT FROM(IN) TO(OUT) ON(p,m,f,) FIRST

With SUM FIELDS=NONE, if EQUALS is in effect, the "first" record could be
any one of the duplicates whereas with EQUALS in effect the "first" record
is the first duplicate in the input file. SELECT always turns on EQUALS so
the "first" record is the first duplicate in the file.

If you want to keep the second and subsequent records of each set of
duplicates (that is, the ones that were eliminated by SUM FIELDS=NONE with
EQUALS or by SELECT with FIRST), you can use this DFSORT/ICETOOL statement:

SELECT FROM(IN) DISCARD(OUT) ON(p,m,f) FIRST

If you want to put the FIRST records in one data set and the non-FIRST
records in another data set, you can use this DFSORT/ICETOOL statement:

SELECT FROM(IN) TO(FIRST) DISCARD(NOTFIRST) ON(p,m,f,) FIRST

Frank Yaeger - DFSORT Team (IBM)
Specialties: ICETOOL, OUTFIL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort

Howard Brazee

unread,
May 21, 2004, 2:04:45 PM5/21/04
to

On 21-May-2004, aha...@ibm-main.lst (Antonio Hacker) wrote:

> Is there a possible problem in that, since EQUALS was not specified, they
> may not sort the same way? Unlikely, I'd think...
>
> >SUM FIELDS=NONE,XSUM

********************************* TOP OF DATA
****************************************************
ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED

ICE000I 1 - CONTROL STATEMENTS FOR 5740-SM1, DFSORT REL 14.0 - 12:02 ON FRI MAY
21, 2004 -
SORT FIELDS=(04,09,CH,A, SID

01,02,CH,A) INST

SUM FIELDS=NONE,XSUM PUT DUPS IN XSUM

$

ICE172A E XSUM IS NOT SUPPORTED - USE ICETOOL SELECT IF APPROPRIATE

DEBUG ABEND ABORT IF ERRORS

END

ICE052I 3 END OF DFSORT

I guess I'll try ICETOOL

Howard Brazee

unread,
May 21, 2004, 2:13:52 PM5/21/04
to

On 21-May-2004, shivpraka...@ibm-main.lst (Swami, Shivprakash , GE
Commercial Finance, NonGE) wrote:

> //S1 EXEC PGM=ICETOOL
> //TOOLMSG DD SYSOUT=*
> //DFSMSG DD SYSOUT=*
> //IN DD DSN=infile..
> //LIST1 DD SYSOUT=*
> //TOOLIN DD *
> OCCUR FROM(IN) LIST(LIST1) ON(1,9,CH) ALLDUPS
>
> Refer link below..
>
> http://www.storage.ibm.com/software/sort/mvs/icetool/online/srtmtocc.html

For some reason I keep having trouble getting a handle on icetool. That is a
nice start above, but I will need to modify it a bit, as I had a split-key:


SORT FIELDS=(04,09,CH,A, SID
01,02,CH,A) INST
SUM FIELDS=NONE ELIMINATE DUPLICATES
DEBUG ABEND ABORT IF ERRORS
END

*
/** 03,01,CH,A, EXT

The ESC field is the critical one - any fields after column 12 need to be
ignored.

Howard Brazee

unread,
May 21, 2004, 2:40:43 PM5/21/04
to
---------- Forwarded Message Follows ----------

Path: news.colorado.edu!not-for-mail
From: "Howard Brazee" <how...@ibm-main.lst>
Newsgroups: bit.listserv.ibm-main


Subject: Re: Sort duplicates problem

Date: Fri, 21 May 2004 18:13:52 GMT
Organization: University of Colorado, Boulder
Lines: 28
Message-ID: <c8lgsv$67s$1...@peabody.colorado.edu>
References: <AAAD0CDF97C8CE4ABD0E...@CINMLVEM09.e2k.ad.ge.com>
NNTP-Posting-Host: ums.cusys.edu
X-Trace: peabody.colorado.edu 1085163231 6396 192.12.240.129 (21 May 2004
18:13:51 GMT)
X-Complaints-To: ab...@colorado.edu
NNTP-Posting-Date: 21 May 2004 18:13:51 GMT
X-Newsreader: News Rover 9.2.0 (http://www.NewsRover.com)
Xref: news.colorado.edu bit.listserv.ibm-main:222569

----------------------------------------------------------------------

Howard Brazee

unread,
May 21, 2004, 2:42:23 PM5/21/04
to
---------- Forwarded Message Follows ----------

Path: news.colorado.edu!not-for-mail
From: "Howard Brazee" <how...@ibm-main.lst>
Newsgroups: bit.listserv.ibm-main
Subject: Re: Sort duplicates problem

Date: Fri, 21 May 2004 18:04:45 GMT


Organization: University of Colorado, Boulder

Lines: 33
Message-ID: <c8lgbs$5nh$1...@peabody.colorado.edu>
References: <BAY17-F10YWYG...@hotmail.com>
NNTP-Posting-Host: ums.cusys.edu
X-Trace: peabody.colorado.edu 1085162684 5873 192.12.240.129 (21 May 2004
18:04:44 GMT)
X-Complaints-To: ab...@colorado.edu
NNTP-Posting-Date: 21 May 2004 18:04:44 GMT


X-Newsreader: News Rover 9.2.0 (http://www.NewsRover.com)

Xref: news.colorado.edu bit.listserv.ibm-main:222567


On 21-May-2004, aha...@ibm-main.lst (Antonio Hacker) wrote:

> Is there a possible problem in that, since EQUALS was not specified, they
> may not sort the same way? Unlikely, I'd think...
>
> >SUM FIELDS=NONE,XSUM

********************************* TOP OF DATA
****************************************************
ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED

ICE000I 1 - CONTROL STATEMENTS FOR 5740-SM1, DFSORT REL 14.0 - 12:02 ON FRI MAY
21, 2004 -

SORT FIELDS=(04,09,CH,A, SID

01,02,CH,A) INST

SUM FIELDS=NONE,XSUM PUT DUPS IN XSUM

$

ICE172A E XSUM IS NOT SUPPORTED - USE ICETOOL SELECT IF APPROPRIATE

DEBUG ABEND ABORT IF ERRORS

END

ICE052I 3 END OF DFSORT

I guess I'll try ICETOOL

----------------------------------------------------------------------

Frank Yaeger

unread,
May 21, 2004, 3:01:13 PM5/21/04
to

Howard Brazee wrote:
>I had a split-key:
> SORT FIELDS=(04,09,CH,A,

> 01,02,CH,A)

Note that you can use multiple ON fields as in these DFSORT/ICETOOL
statements:

SELECT FROM(IN) DISCARD(OUT) ON(4,9,CH) ON(1,2,CH) FIRST

SELECT copies the entire record.
OCCUR gives you a "report" of the duplicate values and counts.

For complete information on DFSORT's ICETOOL, see:

http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/ICE1CA00/6.0?DT=20031124143823

Frank Yaeger - DFSORT Team (IBM)
Specialties: ICETOOL, OUTFIL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort

----------------------------------------------------------------------

Frank Yaeger

unread,
May 21, 2004, 3:03:27 PM5/21/04
to

Howard Brazee wrote:

> SUM FIELDS=NONE,XSUM PUT DUPS IN XSUM
> $
>
>ICE172A E XSUM IS NOT SUPPORTED - USE ICETOOL SELECT IF APPROPRIATE

For more information on using DFSORT/ICETOOL's equivalent of XSUM, see:

http://www.storage.ibm.com/software/sort/mvs/tricks/srtmst01.html#t12

Frank Yaeger - DFSORT Team (IBM)
Specialties: ICETOOL, OUTFIL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort

----------------------------------------------------------------------

Howard Brazee

unread,
May 21, 2004, 4:24:34 PM5/21/04
to
On 21-May-2004, yae...@ibm-main.lst (Frank Yaeger) wrote:

> Note that you can use multiple ON fields as in these DFSORT/ICETOOL
> statements:
>
> SELECT FROM(IN) DISCARD(OUT) ON(4,9,CH) ON(1,2,CH) FIRST
>
> SELECT copies the entire record.
> OCCUR gives you a "report" of the duplicate values and counts.


That worked. I saw multiple OUT parms in the documentation, but somehow my
brain and ICETOOL's documentation tend to work differently. (I usually give up
and go back to old fashioned sort - what I know of both utilities is much more
than other people at this shop). So I experimented unsuccessfully.

Thanks.

0 new messages