Hy-tek .hyv File Format

[mpwa...@gmail.com]

I've shared Hy-tek .hyv File Format

Message from mpwa...@gmail.com:

Click to open:

• Hy-tek .hyv File Format

Google Docs makes it easy to create, store and share online documents, spreadsheets and presentations.

[Lisa Diemler]

Hiya,

 

Thank you for the email, it says I do not have permission to open this file.  I will work on it after the weekend, as we have just started swimming, entries are due, yada yada yada.....

Lisa Diemler

--
You received this message because you are subscribed to the Google Groups "SDIF Forum" group.
To post to this group, send email to sdif-...@googlegroups.com.
To unsubscribe from this group, send email to sdif-forum+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/sdif-forum?hl=en.

[Mike Walsh]

I added you as an editor so you should be able to change it now.  If anyone else wants to edit the document, just let me know.

 

Mike

[Lisa Diemler]

Thanks Mike!

[Troy DeLano]

Made some additions/modifications to the file. Also, I have used
the .hyv in my stuff but I thought I read somewhere that the .ev3 is
the newer of the formats.

On Sep 28, 10:33 pm, mpwal...@gmail.com wrote:
> I've shared a document with you:
>
> Hy-tek .hyv File Formathttps://spreadsheets.google.com/ccc?key=0AgBHWDGsX0PUdG9PNzVqaDNPMWps...
>
> It's not an attachment -- it's stored online at Google Docs. To open this  
> document, just click the link above.

[Mike Walsh]

Troy -

I am pretty sure the .ev3 is the older Hy-tek event file format and .hyv is
the newer format. The demo version of Meet Manager will only generate the
.hyv file which is why I think it is the newer. I recall reading that
somewhere in the Hy-tek help but I could be wrong.

Thanks for your additions. I've spent some time recently looking at the
checksum problem on the entries file and I've come to the conclusion that it
really isn't a true checksum. If you look at several rows of data they will
have the same "checksum" which doesn't make sense if it were a true
checksum. Here is an example:

D1M 5Kirkland Andrew
K120701ANDKKIRK 12072001 8 12
E1M 5KirklMM 25A 07 08 0.00 2 0Y 0.00Y
66
E2P 1 5
60
E1M 5KirklMM 25B 07 08 0.00 3 0Y 0.00Y
66
E2P 2 5
60
E1M 5KirklMM 25C 07 08 0.00 4 0Y 0.00Y
66
E2P 3 5
60
E1M 5KirklMM 25D 07 08 0.00 5 0Y 0.00Y
66
E2P 4 5
60

In this case, the checksum field is always 60 or 66. So it seems more like
it is look up table than a true checksum. Anyone else noticed this?

Mike

-----Original Message-----
From: sdif-...@googlegroups.com [mailto:sdif-...@googlegroups.com] On

--

[Bob Goudreau]

I think Troy is correct, and that .ev3 came after the .hyv format.

I base this conclusion on the contents of two different meet events files we
received this past summer. The zip file for the Cary City Meet contained
just one file, a .hyv file. The comment section of the first line of that
file indicated that it was created by TM version 1.4. The zip file for the
Greater Raleigh Meet contained both a .hyv file and an .ev3 file, and the
comments in the former claim that it was created by TM version 2.0. Looking
back at many years of CCM data, this appears to be consistent. Cary Swim
Club (which hosts in the odd-numbered years) upgraded to TM 2.0 many years
ago, and all its zip files contain both .hyv and .ev3 files. Scottish Hills
(even-numbered years) is still running TM 1.x, and their zip files contain
only .hyv files.

Bob

[Troy DeLano]

Exatly the conclusion I was coming to with the checksum. I have found
the C3 line in hy-tek gives me control over the last 96 characters
with the first two "C3" and the next 30 blank. I have been messing
with the data to try and detect a pattern. In the testing I have done
it is even more perplexing!! Just in the early stages though so hope
to gain more insight as I go.

I have pulled some of my comments from some of my parsing code and
have added it to a text file. I have uploaded it to google docs but I
have not really used that service before. For some reason it is
giving a database error from google. As soon as I get it worked out I
will share it with you.

> For more options, visit this group athttp://groups.google.com/group/sdif-forum?hl=en.- Hide quoted text -
>
> - Show quoted text -

[Troy DeLano]

I think I have added everyone...here is the link anyway. Let me know
if you can't access it.

https://docs.google.com/document/edit?id=1WRti1LHLxlLIVyK5wkRDv9kox5Dg92XRQO5W1S0PVYE&hl=en#

> > For more options, visit this group athttp://groups.google.com/group/sdif-forum?hl=en.-Hide quoted text -
>
> > - Show quoted text -- Hide quoted text -

[Bob Goudreau]

Wow, that is a tour de force of file format analysis! Thanks for putting
that incredibly detailed and helpful guide together.

Bob

[Lisa Diemler]

An quick fyi, .hyv is the current file format for Hytek products, prior
version was .hy3. The ev3 format is a concession to Hytek's major
competitor, Easy Meet, so that Hytek exports can somwhat be used in Easy
Meet software.

Lisa

[Lisa Diemler]

Well, maybe not. Old versions of Easy Meet files were .sd3 back prior to
2006, so I could be wrong about the ev3, however for a long time Hytek was
putting out extra files compatible with their major competitor's software.

----- Original Message -----
From: "Bob Goudreau" <BobGo...@nc.rr.com>
To: <sdif-...@googlegroups.com>
Sent: Tuesday, October 12, 2010 12:05 AM

[Lisa Diemler]

Ok, sorry to be posting dreck- please disregard prior posts or delete...

From Team Manager help files:

"Instead of manually setting up a meet and the events for that meet, you can
easily import that information from our MEET MANAGER software. Contact the
meet host and ask for a MM Meet Event file - this file has the file
extension .HYV or it may be zipped and have a file extension of .ZIP. If it
is zipped, it may contain both the "older" .HYV format and then newer .EV3
format, HY-TEK's 2nd generation Meet Event file format used with MEET
MANAGER 2.0. Both the .HYV and .EV3 files contain the meet name, course,
etc. as well as information about each event in that meet including meet
entry qualification times. The newer .EV3 file format contains additional
parameters such as meet venue altitude, sanction number, entry time Since
Date, event divisions to support Varsity/JV type college and high school
meets, and meet and session max entries parameters. "

Lisa

[DistanceSwimmer]

When I use Team Manager Lite it appears that .hyv is the event line up
file (semi-colon separated fields), whereas the .hy3 is the entries
file (column-specific fields). .ev3 (as mentioned) is an updated
version of the .hyv file. Is this no correct?

Bart

On Oct 12, 4:12 am, "Lisa Diemler" <ldiem...@embarqmail.com> wrote:
> An quick fyi, .hyv is the current file format for Hytek products, prior
> version was .hy3.  The ev3 format is a concession to Hytek's major
> competitor, Easy Meet, so that Hytek exports can somwhat be used in Easy
> Meet software.
>
> Lisa
>
>
>
> ----- Original Message -----
> From: "Bob Goudreau" <BobGoudr...@nc.rr.com>
> To: <sdif-...@googlegroups.com>
> Sent: Tuesday, October 12, 2010 12:05 AM
> Subject: RE: [SDIF Forum] Re: Hy-tek .hyv File Format
>
> Wow, that is a tour de force of file format analysis!  Thanks for putting
> that incredibly detailed and helpful guide together.
>
> Bob
>
> -----Original Message-----
> From: sdif-...@googlegroups.com [mailto:sdif-...@googlegroups.com] On
> Behalf Of Troy DeLano
> Sent: Monday, October 11, 2010 11:59 PM
> To: SDIF Forum
> Subject: [SDIF Forum] Re: Hy-tek .hyv File Format
>
> I think I have added everyone...here is the link anyway.  Let me know
> if you can't access it.
>

> https://docs.google.com/document/edit?id=1WRti1LHLxlLIVyK5wkRDv9kox5D...

> athttp://groups.google.com/group/sdif-forum?hl=en.-Hidequoted text -

>
> > > - Show quoted text -- Hide quoted text -
>
> > - Show quoted text -
>
> --
> You received this message because you are subscribed to the Google Groups
> "SDIF Forum" group.
> To post to this group, send email to sdif-...@googlegroups.com.
> To unsubscribe from this group, send email to
> sdif-forum+...@googlegroups.com.

> For more options, visit this group athttp://groups.google.com/group/sdif-forum?hl=en.

[Mike Walsh]

Yes, we determined after the initial e-mail that the .ev3 file is the newer
version of the Hy-tek event file.

[DistanceSwimmer]

My question was more directed at the .hyv vs. .hy3 file formats.

> > athttp://groups.google.com/group/sdif-forum?hl=en.-Hidequotedtext -

[Troy DeLano]

.ev3 and .hyv are the events files wherease .hy3 is the entries/
results file.

> > For more options, visit this group athttp://groups.google.com/group/sdif-forum?hl=en.- Hide quoted text -

[DistanceSwimmer]

For fields 14 & 16 of the .hyv format, they are the qualifying times
in different courses. I am not sure exactly how the file format
decides which field is which as field 10 is always the main course
qualifying time (as in, scy if the meet is scy, lcm if the meet is
lcm). But look at
http://www.mnswim.org/competition/TM_setup_files/2009-10/Meet%20Events-2010%20MN%20Foxj%20Long%20Course%20Championship-29Jul2010-002.zip
for the MN Grand Prix last year and you'll see that field 14 is lcm
qts and field 16 is scm qts.

> > > For more options, visit this group athttp://groups.google.com/group/sdif-forum?hl=en.-Hide quoted text -

[DistanceSwimmer]

Looking at a long course meet with all 3 fields shows:
Field 10: lcm QT
Field 14: scm QT
Field 16: scy QT

As stated before, with a scy meet:
Field 10: scy QT
Field 14: lcm QT
Field 16: scm QT

I also looked at a scm meet that also had lcm QTs:
Field 10: scm QT
Field 14: lcm QT
Field 16: blank

Too early to tell, but it could be a rolling patter of lcm, scm, scy
(or scm, scy, lcm... or scy, lcm, scm) using field 10 first, then
field 14, then field 16, where field 10 is always the course of the
meet.

On Oct 13, 7:34 am, DistanceSwimmer <bwe...@3hc.com> wrote:
> For fields 14 & 16 of the .hyv format, they are the qualifying times
> in different courses.  I am not sure exactly how the file format
> decides which field is which as field 10 is always the main course
> qualifying time (as in, scy if the meet is scy, lcm if the meet is

> lcm).  But look athttp://www.mnswim.org/competition/TM_setup_files/2009-10/Meet%20Event...

> > > > For more options, visit this group athttp://groups.google.com/group/sdif-forum?hl=en.-Hidequoted text -

[DistanceSwimmer]

Field 11 of the .hyv file appears to be a string indicating event
sponsor.

> > > > > For more options, visit this group athttp://groups.google.com/group/sdif-forum?hl=en.-Hidequotedtext -

[DistanceSwimmer]

Field 18 is the division (like Varsity, Junior Varsity, etc) and field
17 is the abbreviation for that field.

Field 18 was previously not indicated. There are 18 fields in the
event record (17 field separators - semicolons, that is).

[Jay]

I ran an analysis on a .hy3 results file and found some potentially
interesting things regarding the two digit checksum (or check digits,
hash, etc.).
The file contained 7096 lines, each ending with said checksum.

The most interesting thing is that not a single line contained a
checksum value of 87. This is the only value that was completely
absent and may be an important piece of the solution.

Probably much less significant, but possibly interesting are some of
the following:

Even Checksums: 4149
Odd Checksums: 2947

The last digit of the checksum was also heavily biased toward certain
numbers:

Last Freq
1 200
2 1011
3 217
4 1311
5 1747
6 66
7 30
8 42
9 753
0 1719

The first digits were much less biased, but did seem to follow a
pattern. If anyone is interested, I could post the distribution
frequency for each of the 100 possibilities if there is any interest.
(really 99 if you exclude the value 87)

On Oct 11, 10:20 pm, "Mike Walsh" <mike_wa...@mindspring.com> wrote:
> Troy -
>
> I am pretty sure the .ev3 is the older Hy-tek event file format and .hyv is
> the newer format.  The demo version of Meet Manager will only generate the
> .hyv file which is why I think it is the newer.  I recall reading that
> somewhere in the Hy-tek help but I could be wrong.
>

> Thanks for your additions.  I've spent some time recently looking at thechecksumproblem on the entries file and I've come to the conclusion that it
> really isn't a truechecksum.  If you look at several rows of data they will
> have the same "checksum" which doesn't make sense if it were a truechecksum.  Here is an example:

>
> D1M    5Kirkland            Andrew
> K120701ANDKKIRK     12072001  8                             12
> E1M    5KirklMM    25A 07 08      0.00  2        0Y    0.00Y
> 66
> E2P                   1  5
> 60
> E1M    5KirklMM    25B 07 08      0.00  3        0Y    0.00Y
> 66
> E2P                   2  5
> 60
> E1M    5KirklMM    25C 07 08      0.00  4        0Y    0.00Y
> 66
> E2P                   3  5
> 60
> E1M    5KirklMM    25D 07 08      0.00  5        0Y    0.00Y
> 66
> E2P                   4  5
> 60
>

> In this case, thechecksumfield is always 60 or 66.  So it seems more like
> it is look up table than a truechecksum.  Anyone else noticed this?

[Mike Walsh]

The more I look at this data the less I think it is a true checksum. A million years ago when I worked for a defense contractor I had to implement a checksum and we looked at 3-4 different ones (XOR, Double-Add-Double, simple addition, etc). One thing I noticed in a lot of the data records is one or both digits of the checksum doesn't change even though something in the record changes, for example, the lane number.

I will try and assemble some examples to demonstrate what I am trying to explain.

Mike

Sent from my iPhone

[Barton Wells]

It is not a traditional checksum.  The two digits are reversed (as in 8-6 is 68, 4-2 is 24).  They take some of the fields and add depending on the alphanumeric in certain positions, and different lines have different formulas.

Correct, not al of the fields affect the "checksum."  The date fields in B1 do not.

The altitude field add a certain number based on how much it is.  I have found this much so far:

0-8 adds 0, 9 - 48 adds 1, 49 - 104 adds 2, 105 - 992 adds 3, 993-999 adds 4,

1000-1092 adds 3, 1093 - 1099 adds 4, 1100-1191 adds 3, 1192-1199 adds 4,1200-1290 adds 3, 1291-1299 adds 4, 1300-1342 adds 3, 1343-1399 adds 4.

Doing some tests on the B2 line:

The base for the B2 line w/ no data is 17 (or 7-1).  Different characters add different amounts, and it depends on what space they are in.

[Sorry for the odd formatting in the following data.  The "checksums" are as shown, but I have a large space between them.  The first field is for Header1 field of line B2].

If the Header 1 section is the following, and there are no other fields, the "checksum" is:

<blank> 7 1

A 8 1

AA 1 2

AAA 3 2

AAAA 6 2

AAAAA 8 2

AAAAAA 1 3

AAAAAAA 2 3

AAAAAAAA 5 3

AAAAAAAAA 7 3

AAAAAAAAAA 0 4

AAAAAAAAAAA 2 4

AAAAAAAAAAAA 5 4

AAAAAAAAAAAAA 6 4

Z 9 1

ZZ 5 2

ZZZ 8 2

ZZZZ 3 3

ZZZZZ 6 3

ZZZZZZ 1 4

ZZZZZZZ 4 4

ZZZZZZZZ 0 5

ZZZZZZZZZ 2 5

ZZZZZZZZZZ 8 5

ZZZZZZZZZZZ 1 6

ZZZZZZZZZZZZ 6 6

ZZZZZZZZZZZZZ 9 6

ZZZZZZZZZZZZZZ 5 7

F 8 1

G 8 1

H 8 1

I 9 1

K 9 1

L 9 1

M 9 1

N 9 1

B 8 1

1 7 1

11 9 1

111 0 2

a 0 2

aa 6 2

aaa 9 2

aaaa 5 3

aaaaa 8 3

aaaaaa 4 4

aaaaaaa 8 4

aaaaaaaa 4 5

aaaaaaaaa 7 5

aaaaaaaaaa 3 6

hhhhhhhhhh 8 6

h 0 2

hh 7 2

bbbbbbbbbb 4 6

bbbbb 9 3

bb 6 2

bbb 9 2

bbbb 5 3

d 0 2

dd 6 2

ddd 0 3

dddd 6 3

dddddddddd 5 6

Originally I thought A-H contributed the same amount and I-Z contributed the same amount, but alas they have slight differences once you have more of them (they are the same in the first position).

For instance A in the Header 1 contributes: [1,3,2,3,2,3,1,3,2,3,2,3,1] (pattern is [1,3,2,3,2,3]) for each position (that is, 1 for the first position, then 3 in the second, etc).  Z contributes [2,6,3,5,3,5,3,6,2,6,3,5,3,6...].  a contributes [3,6,3,6,3,6,4,6,3,6...].  1 contributes [0,2,1...].  I'm guessing there is a formula, not a massive lookup table, but who knows.

It looks hackable, but a long process of creating a ton of different files w/ small changes to see what contributes what.

Bart

[Troy DeLano]

I saw the same thing. I started by using the base ASCII codes. I
also noticed it appears to be on 16 bit boundaries. If I take the C3
row and put a single character in and slide it down the checksum
toggles 01, 10, 01... based on if the character is in the even or odd
position. Not sure but I thought the left of the two byte pair was the
least significant so the order is acutally reversed from what is shown
below. So using the lowest ASCII value and working up I came up with
something like this:

0x2020...10 (double space...actually spaces all across)
0x2021...10 !
0x2022...10 "
0x2023...10 #
0x2024...10 $
0x2025...10 %
0x2026...10 &
0x2027...10 '
0x2028...10 (
0x2029...10 )
0x202A...10 *
0x202B...10 +
0x202C...10 ,
0x202D...10 -
0x202E...10 .
0x202F...10 /
0x2030...10 0
0x2031...10 1
0x2032...10 2
0x2033...10 3
0x2034...20 4
0x2035...20 5
0x2036...20 6
0x2037...20 7
0x2034...20 8
0x2035...20 9
0x2036...20 :
0x2037...20 ;
...
0x2048...20 H
0x2049...30 I
0x204A...30 J
...
0x205D...30 ]
0x205E...40 ^
...
0x2072...40 r
0x2073...50 s
0x207E...50 ~

before 0x20 and after 0x7E it is special characters so I couldn't run
those. I then replace the column of space (0x20) with the exclamation
point (0x21) hoping I would get a continuation of the sequene instead
I got a very similar sequence...except the transitions were at
differnet points...ugh! I also did the test with by adding characers
and found the same results...man great minds think alike. What I did
find that if I then break the sequence up with characters from a
different result group (20 vs 10) it would seem to add up correctly
(or as I would expect).

> > For more options, visit this group athttp://groups.google.com/group/sdif-forum?hl=en.- Hide quoted text -

[Troy DeLano]

If there is something specific you want tried let me know and I can try it on TM  At this point I have contacted some individuals that have "skills" ;) in this type of work so I will let them bang on it for a while,,,I'll keep you posted.

[Mike Walsh]

Ken -

I am about to head out camping and will try and answer what I can my earliest convenience.

I've cc'd the mailing list on your questions, I suspect you'll get a quick summary from one of our members before I get back to my e-mail.

Mike

On Fri, Nov 5, 2010 at 3:25 PM, KenM <kenm...@gmail.com> wrote:

Thanks Mike.

I've read through most of the docs, that I could find, on Hytek file formats and have a few questions that hopefully you can help me with.

1. I have a meet results file that has both an .hy3 and a .cl2 (both contained in a zip file) file. I believe this was from an older version of Meet Manager (pre-2.0). Is there any documentation on the format of these files? I know you've said that CL2 is very similar to SDIF (which is documented) but I've also seen words to the effect that SDIF support doesn't guarantee that a CL2 file will be properly read.

2. Is there any source that lists the various file types that are part of the various swim team software universe (e.g. hy3, hyv, cl2, sdif, ev3, etc.)? This is what I can glean from the various docs I've read (and I could very well have things mixed up):
    hyv - Hytek meet results
    hy3 - Hytek meet results (prior to hyv)
    ev3 - ???
    sdif - USA swimming meet results format
    lots of others that I don't know about ....

3. Are the different Hytek files that are supposed to be for updated versions of the software (e.g. TM 4.0 uses XXX and TM 6.0 uses YYY but is YYY a superset of XXX) supersets of previous formats?

I'm interested in attempting to write some generic parsing code that can handle different file types (these would be read only - don't want to tackle the problems with checksums just yet) so any help you can give me on explaining the swimming file formats universe would be greatly appreciated.

Thanks

Ken

--
Mike Walsh - mpw...@gmail.com

[jwh]

Just a note that I think is related. I decoded the HY3 and CL2 format
checksums already and I suspect the HYV checksum is along the same
lines. See my post at:

http://groups.google.com/group/rec.sport.swimming/browse_thread/thread/4c402f0648552e4a

for details.

Joe

> ...
>
> read more »- Hide quoted text -

[Mike Walsh]

This is awesome! How many test cases have you run through your algorithm?

Mike

Sent from my iPhone

[Troy DeLano]

Darn it.  Now I am going to be up all night auto generating entry files.  lol  ;)

[Troy DeLano]

I just ran a test on 11 .hy3 files (6 entries and 5 results) with a total of just under 26000 lines....looks good.