NIC-format

[notes.tssi.com!tower.tssi.com!h...@tssi.com]

Introduction to NicBase and ChessBase formats (carsten, ch0...@hdc.hha.dk) If you are not familar with bits and bytes, please stop here! I advise you to study the program code more careful than you study this text which I made one night. NicBase format ============== The index file: The index values are 4 bytes word. That is the following: byte0*256*256*256 + byte1*256*256 + byte2*256 + byte3 The index values are the following: start of game 1, start of game 2, start of game 3,,,,,,, The game file: byte0, byte1, byte2 are flags. They tell what information there is in the game. Please study the procedure parse_nic_gamehead in ch_parse. byte3 contains the number of half-moves If the number of half-moves is greater than 127 there is 7 bits extra with unknown information. The information is read in the following order: opening information w_name b_name w_elo b_elo w_titlenumb b_titlenumb place annotator source info year resultnumb round Opening information Reading the opening information is quite complicated and it took me one or two days to parse it! 1) there is two letters each represented by 5 bits(0-31), and therefore it is neccessary to add 64 to the obtained value. 2) two dummy bits 3) a flag consisting of 4 bits, in order to tell the number of keycodes 4) a flag consisting of 2 bits(temp2) 5) dependent on the number of keycodes there are some keycode information temp2 might have the values (0,1,2,3) 0: read no keycode 1: keycode (0..15) 2: keycode is a multiplum of 16 3: keycode (0..99) 6) If there is more than one keycode, the following are easy ones because they are represented by 4 bits (0..15). w_name, b_name The name are represented by a string of 7-bit ASCII values (0..127), and ending with zero. w_elo, b_elo The elos are represented by a 13 bit value each w_titlenumb, b_titlenumb The titles are represented by a 4 bit value each place, annotator, source, info Is in the same text format as w_name and b_name year 8-bit value + 1800 resultnumb 4-bit value round 7-bit value In the bottom of parse_nic_gamehead game.numb_of_w_moves, game.cb_players, and game.cb_source is calculated in order to prepare conversion from NicBase to ChessBase. The moves are not sharing bytes with the gamehead, and therefore it is necessary to increase the bitnumb. The moves ========= The main rule in NicBase, is that, at first, a piecenumb (0..15) is selected, and then a movenumb is selected. E.g., e2e4 is represented as 12-0 because Pe2 is the 12th(13th) piece when you are counting the following way: (Qa1, Nb1, Ng1, Ra1, Rh1, Bc1, Bf1, Ke1, Pa2,Pb2,Pc2,Pd2,Pe2,Pf2,Pg2,Ph2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Notice, NicBase is scanning the board in the following sequence: At first, all Queens, a1,b1,c1,d1,e1,,,,,a2,b2,c2,,,, 2, all Knigths 3, all Rooks 4, all Bishop 5, the King 6, all Pawn, a2,b2,c2,d2,,,,,, Notice, NicBase is only interested in the pieces of the player who is allowed to move! See gen_nic_moves in ch_chess for more information about the generation and sequence of moves. If there only were 8 white pieces on the board, there would only be used three bits in order to select a piece. After the piece has been found the number of legal_moves is calculated, and there is not used any more bits than absolutely necessary. That is: 1 legal move => 0 bits used 2 legal moves => 1 bit used 3 legal moves => 2 bits used 4 legal moves => 2 bits used 5 legal moves => 3 bits used If there is only 13 pieces on the board, NicBase still uses 4 bits for the piecenumb, but piecenumb 13 is now the same as piecenumb 0, and piecenumb 14 is now the same as piecenumb 1. And therefore, it is now possible to save one bit, when the movenumb is going to be found. (See parse_nic_moves in ch_parse). If there is comments or annotations in the game, the first bit in a move is always one. CHESSBASE ========= Indexfile The indexfile consists of a lot of 4-byte values Notice, there is no deletion tag in the Gamefile, so ChessBase is changing the index values instead! byte 0-3 number of games + 1 4-7 start of game[i] + i + 1 (0+1+1=2) 8-11 start of game[i] + i + 1 Gamefile (see parse_cb_gamehead in ch_parse) byte 0 delta_year xor $3f, where delta_year is a shortint (-128..127) year:=1900+delta_year 1 result 2 unknown (often $07) 3 xor $ad (number of half moves + 1) 4 bit 6,7 used in CBUNCODE; bit 0-5 XOR $0f length(players-text) 5 bit 6,7 used in CBUNCODE; bit 0-5 XOR $05 length(source-text) 6 unknown (often $d7) 7 unknown (often $9d) 8 white_elo multiplum of 5 (XOR $df); elo:=1600 + x*5 9 black_elo multiplum of 5 (XOR $f5); elo:=1600 + x*5 10 unknown (often $a7) 11 length(players-text+source-text) xor $dd 12 number of white moves XOR $2f 13 checksum, method of calculation unknown!!!!! 14..x players-text+source-text, with no mark of ending The text is normal 8-bit ASCII, but coded by XOR-values which depends on the number of characters in the total string, and what number a specific character has in the string. See ch_sxor for XOR-values 14..x the moves. Each move is represented by a byte, coded by a XOR-value found in CH_XOR. The XOR-value depends on the total number of moves in the game, and what number a specific move has. The moves(see gen_cb_moves in ch_chess). There is generated a number (0..255) which contains all legal moves. ChessBase is scanning the board the following way: a1,a2,a3,....,b1,b2,b3,...... It is important to notice that ChessBase is not checking whether a king is in check, nor whether castling is allowed due to movement of the king or one of the rooks. A funny move is allowed: If white has a pawn on a5, the pawn is allowed to a5xh6 e.p., if black has not moved any pawns from the 7th to the 5th rank during the last move. The same is the case for a black pawn on a4. Thank God, that I did study some of Larsen's game when I tested this program.