Shogi game data formats
629 views
Skip to first unread message
user923005
Mar 26, 2009, 4:13:54 AM3/26/09
to SHOGI-L
There seems to be lots and lots of different formats for storing shogi
games.
I believe that *.kj2 and *.kj3 are shogi games (for example
persimmon.kj3, Miya2007.kj2, Miya2009.kj2).
There are *.ki2 files which seem to be individual game files.
There are psn files (e.g. the Gnu shogi database project).
My question is:
Is there some way to convert all of these files to one central
format? I want to study them but I want to collect them into a
uniform source before I begin.
What programs read files of the format:
*.ki2
*.kif
*.psn
*.sgf
*.kj2
*.kj3
*.bjd
*.csa
What programs can convert these files into another format?
Is there some format which is more common than the others (e.g. for
Western chess PGN is fairly universal)
games.
I believe that *.kj2 and *.kj3 are shogi games (for example
persimmon.kj3, Miya2007.kj2, Miya2009.kj2).
There are *.ki2 files which seem to be individual game files.
There are psn files (e.g. the Gnu shogi database project).
My question is:
Is there some way to convert all of these files to one central
format? I want to study them but I want to collect them into a
uniform source before I begin.
What programs read files of the format:
*.ki2
*.kif
*.psn
*.sgf
*.kj2
*.kj3
*.bjd
*.csa
What programs can convert these files into another format?
Is there some format which is more common than the others (e.g. for
Western chess PGN is fairly universal)
Fred
Mar 26, 2009, 4:21:37 AM3/26/09
to SHOGI-L
Hi,
so far, I have not really encountered the *.k?n formats you mention.
Everything else (and a lot more) is covered by Bernhard's BCMShogi
(which has its own thread in this group). I use BCM Shogi for format
conversion. Here is the direct link: http://home.arcor.de/Bernhard.Maerz/BCMShogi/index.htm
Warm regards,
Fred
so far, I have not really encountered the *.k?n formats you mention.
Everything else (and a lot more) is covered by Bernhard's BCMShogi
(which has its own thread in this group). I use BCM Shogi for format
conversion. Here is the direct link: http://home.arcor.de/Bernhard.Maerz/BCMShogi/index.htm
Warm regards,
Fred
Yury Shpilev
Mar 26, 2009, 6:51:26 AM3/26/09
to sho...@googlegroups.com
| But BCMShogi not work until Windows XP. Why? Best regards, Yury Shpilev --- Чт, 26.3.09, Fred <fred.spl...@googlemail.com> пишет: От: Fred <fred.spl...@googlemail.com> |
Вы уже с Yahoo!? Испытайте обновленную и улучшенную. Yahoo! Почту!
Yury Shpilev
Mar 26, 2009, 6:53:56 AM3/26/09
to sho...@googlegroups.com
| sorry, not "until" and "under" ^_^ --- Чт, 26.3.09, Yury Shpilev <rus_shog...@yahoo.com> пишет: От: Yury Shpilev <rus_shog...@yahoo.com> |
|
Кому: sho...@googlegroups.com |
Bernhard
Mar 26, 2009, 8:58:30 AM3/26/09
to SHOGI-L
Hi,
you speaking about a thing, every year was started, but didn't had
changed anything.
---------------------------------------------------------
First let's say something about your listed formats:
1) KI2
This format seems not to be very old. It's not so often seen and I
decide not to waste time for implementing it into BCMShogi.
If you can show me (very) LARGE game collections in this format, I
will think again about it.
2) SGF (Smart Game Format)
This is the standard for Go games, but I never have seen any Shogi
game in this format, even there is a Shogi-definition for it.
3) KJ2 and KJ3
This seems own formats of the commercial product Kakinoki Shogi
(http://homepage3.nifty.com/nneo/persimmon_en.html)
I believe, they made there own definition and the won't have interests
in distributing it to other programs.
They like that you buy there programs. ;-)
4) BJD
I didn't found anything about it.
5) PSN, KIF, CSA
This ALL (!!!) are still used widely.
a) KIF is the standard in the japanese world
b) PSN is the standard in the western world
c) CSA is the standard for engines (for example using floodgate)
BCMShogi didn't support KI2 and SGF (and as told also not
KJ2,KJ3,BJD).
---------------------------------------------------------
At the moment, not any of this formats is a good way to use in the
western world.
a) KIF
For not a long time I thought, this format should be used also in the
western world and has advantage to PSN.
But then I recognized, that it's written in Shift_JIS-Code.
This means you can't use for example german Umlaute (ä,ö,ü,...) and
this other chars with lines and dotes above the characters.
Nevertheless for english speaking it can be used.
b) PSN
Here you can't use Kanji and also you have to switch between different
styles (western europe, cyrylic, ...).
Not good.
c) CSA
Absolutely not usable. (No variation tree.)
So for BCMShogi I decide to make a very little extension and support
KIF and PSN also in UNICODE-Code without changing anything from the
format specifiaciton.
For recognization I use the extensions UKIF and UPSN.
(By the way: shogidokoro can also read UKIF, because it uses Microsoft
Studio which makes the conversion automatically).
---------------------------------------------------------
Every year there was a discussion about a successor of the PSN-format.
But every year, nothing happens.
Look for example at this old post:
http://lists.topica.com/lists/shogi/read/message.html?mid=813059285
I said, I believe that NOTHING will be changed in the next 50 years.
And to fbc: what's about your/our bet???
It seems that my definition of "soon" is something other.
Long time later, I decide by myself to write a shogi-viewer/editor:
BCMShogi.
I've implemented many features, for which I need a new file-format to
save all things (for examples this objects you can see in Hidetchis
videos).
I thought about starting a new discussion about creating a new format.
But I'm tired of all this creeping shogi-community. :-(((
Without AT LEAST maybe 10 ambitious people who like to work for the
fame of shogi, it's a dead project, it's frustrating and depressing.
So finaly I decide to create it by myself.
I'm tired and sick of coding, so I like to make the format as easy as
I prefere (this makes it also more probably, that MAYBE sometimes will
also implement it).
I've mad some considerations, what this format will need, what not.
All things have advantages and disadvantages.
I gave it the glorious name: USF (Universal Shogi Format)
(It was a dream of me, but I never believe in it, now.)
Yesterday I made the first implementation of V0.1.
(http://home.arcor.de/Bernhard.Maerz/BCMShogi/Counter.htm)
It's still not important (still no comments, no variation trees), but
the reason I've done it, is the new patternsearch function.
In my tests, it founds for example in 10.000 (!!!) games all games
with Anaguma in about 15 seconds!!!
(The first search unfortunately needs about 30 seconds, but than you
can change the searchpattern and search again and again and again.)
---------------------------------------------------------
@fbc: do you still like to repeat your bet?
Or is there anything else like to bet?
---------------------------------------------------------
BCMShogi works with XP and this is the ONLY operating system I'm
testing (and using).
Still it should work also with Vista (still the fontsize is a little
bit wrong) and also I've heared it will work under Linux (with wine?).
I for myself try to concentrate more in having fun in playing more
shogi and reading shogi literature (instead of dreaming hopeless
dreams).
Bernhard
you speaking about a thing, every year was started, but didn't had
changed anything.
---------------------------------------------------------
First let's say something about your listed formats:
1) KI2
This format seems not to be very old. It's not so often seen and I
decide not to waste time for implementing it into BCMShogi.
If you can show me (very) LARGE game collections in this format, I
will think again about it.
2) SGF (Smart Game Format)
This is the standard for Go games, but I never have seen any Shogi
game in this format, even there is a Shogi-definition for it.
3) KJ2 and KJ3
This seems own formats of the commercial product Kakinoki Shogi
(http://homepage3.nifty.com/nneo/persimmon_en.html)
I believe, they made there own definition and the won't have interests
in distributing it to other programs.
They like that you buy there programs. ;-)
4) BJD
I didn't found anything about it.
5) PSN, KIF, CSA
This ALL (!!!) are still used widely.
a) KIF is the standard in the japanese world
b) PSN is the standard in the western world
c) CSA is the standard for engines (for example using floodgate)
BCMShogi didn't support KI2 and SGF (and as told also not
KJ2,KJ3,BJD).
---------------------------------------------------------
At the moment, not any of this formats is a good way to use in the
western world.
a) KIF
For not a long time I thought, this format should be used also in the
western world and has advantage to PSN.
But then I recognized, that it's written in Shift_JIS-Code.
This means you can't use for example german Umlaute (ä,ö,ü,...) and
this other chars with lines and dotes above the characters.
Nevertheless for english speaking it can be used.
b) PSN
Here you can't use Kanji and also you have to switch between different
styles (western europe, cyrylic, ...).
Not good.
c) CSA
Absolutely not usable. (No variation tree.)
So for BCMShogi I decide to make a very little extension and support
KIF and PSN also in UNICODE-Code without changing anything from the
format specifiaciton.
For recognization I use the extensions UKIF and UPSN.
(By the way: shogidokoro can also read UKIF, because it uses Microsoft
Studio which makes the conversion automatically).
---------------------------------------------------------
Every year there was a discussion about a successor of the PSN-format.
But every year, nothing happens.
Look for example at this old post:
http://lists.topica.com/lists/shogi/read/message.html?mid=813059285
I said, I believe that NOTHING will be changed in the next 50 years.
And to fbc: what's about your/our bet???
It seems that my definition of "soon" is something other.
Long time later, I decide by myself to write a shogi-viewer/editor:
BCMShogi.
I've implemented many features, for which I need a new file-format to
save all things (for examples this objects you can see in Hidetchis
videos).
I thought about starting a new discussion about creating a new format.
But I'm tired of all this creeping shogi-community. :-(((
Without AT LEAST maybe 10 ambitious people who like to work for the
fame of shogi, it's a dead project, it's frustrating and depressing.
So finaly I decide to create it by myself.
I'm tired and sick of coding, so I like to make the format as easy as
I prefere (this makes it also more probably, that MAYBE sometimes will
also implement it).
I've mad some considerations, what this format will need, what not.
All things have advantages and disadvantages.
I gave it the glorious name: USF (Universal Shogi Format)
(It was a dream of me, but I never believe in it, now.)
Yesterday I made the first implementation of V0.1.
(http://home.arcor.de/Bernhard.Maerz/BCMShogi/Counter.htm)
It's still not important (still no comments, no variation trees), but
the reason I've done it, is the new patternsearch function.
In my tests, it founds for example in 10.000 (!!!) games all games
with Anaguma in about 15 seconds!!!
(The first search unfortunately needs about 30 seconds, but than you
can change the searchpattern and search again and again and again.)
---------------------------------------------------------
@fbc: do you still like to repeat your bet?
Or is there anything else like to bet?
---------------------------------------------------------
BCMShogi works with XP and this is the ONLY operating system I'm
testing (and using).
Still it should work also with Vista (still the fontsize is a little
bit wrong) and also I've heared it will work under Linux (with wine?).
I for myself try to concentrate more in having fun in playing more
shogi and reading shogi literature (instead of dreaming hopeless
dreams).
Bernhard
Fred
Mar 26, 2009, 9:40:59 AM3/26/09
to SHOGI-L
Hi everyone,
[Warning: off topic!]
Bernhard wrote:
> I thought about starting a new discussion about creating a new format.
> ...., it's a dead project, it's frustrating and depressing.
> I thought about starting a new discussion about creating a new format.
I certainly hope that this will not be Bernhard's final word, although
I agree with him that such technical discussions have often kind of
petered out in this group (I have thoroughly reviewed some of the
previous threads on game notation formats etc.), a fact I also
consider quite frustrating. Still - I'd be delighted to see Bernhard
continue his groundbreaking work on his software and possibly on a
Shogi database retrieval system.
Warm regards,
Fred
user923005
Mar 26, 2009, 2:39:21 PM3/26/09
to SHOGI-L
On Mar 26, 5:58 am, Bernhard <shogi4...@arcor.de> wrote:
> Hi,
>
> you speaking about a thing, every year was started, but didn't had
> changed anything.
>
> ---------------------------------------------------------
>
> First let's say something about your listed formats:
>
> 1) KI2
> This format seems not to be very old. It's not so often seen and I
> decide not to waste time for implementing it into BCMShogi.
> If you can show me (very) LARGE game collections in this format, I
> will think again about it.
I have 71,398 files, the oldest of which date to 2004.
> Hi,
>
> you speaking about a thing, every year was started, but didn't had
> changed anything.
>
> ---------------------------------------------------------
>
> First let's say something about your listed formats:
>
> 1) KI2
> This format seems not to be very old. It's not so often seen and I
> decide not to waste time for implementing it into BCMShogi.
> If you can show me (very) LARGE game collections in this format, I
> will think again about it.
They are small, so I guess that they are single game files.
> 2) SGF (Smart Game Format)
> This is the standard for Go games, but I never have seen any Shogi
> game in this format, even there is a Shogi-definition for it.
{blush}
of the Shogi world.
I really like your interface. I also like your passion. A few people
like you can drive the planet.
Bernhard
Mar 27, 2009, 5:33:02 AM3/27/09
to SHOGI-L
Hi,
> > 1) KI2
> > ...
Can you tell me a link where to download it, please?
> I guess it is a Go game I accidentally put into my Shogi folder.
> {blush}
Nevertheless Go is also a great game.
Earlier I've played it often, but there is no time for all, so I'm
concentrating at Shogi, now.
Also (for special reason) I didn't like to play go now, but I remember
the days when I played it.
(But I've started again with chess, thinking about learning-technics,
helping one game to improve the other and also there are chess
programs for my PALM with good strength, but not any aceptable Shogi
program for it).
> > Yesterday I made the first implementation of V0.1.
> > (http://home.arcor.de/Bernhard.Maerz/BCMShogi/Counter.htm)
>
> I will take a look.
Still the release with USF-V0.1 ist not released yet, but as I told,
it's nothing worth for you.
And the pattern-search (for which I have started USF) is not
documentated.
(And I don't like to do this work. If there will be anyone writing
documentation ...???)
> I hope that you will become the Steven J. Edwards
> of the Shogi world.
Let's see, I doubt it.
At the moment, I'm not pushing USF.
Maybe if all people will use BCMShogi, writing all games in USF-
format, this situation will become the reason, that other programmers
also will implement USF.
> I really like your interface.
Thanks, but still I have to say it again and again:
the wonderful graphics which are the MAIN reason for the good looking
are not from me.
I took it from the "bona"-page, which are free for non-commercial use.
> I also like your passion. A few people
> like you can drive the planet.
Thanks.
So I should take care of drinking less beer in the future. ;-)
Bernhard
> > 1) KI2
> > ...
>
> I have 71,398 files, the oldest of which date to 2004.
> They are small, so I guess that they are single game files.
oh!!!
> I have 71,398 files, the oldest of which date to 2004.
> They are small, so I guess that they are single game files.
Can you tell me a link where to download it, please?
> I guess it is a Go game I accidentally put into my Shogi folder.
> {blush}
Earlier I've played it often, but there is no time for all, so I'm
concentrating at Shogi, now.
Also (for special reason) I didn't like to play go now, but I remember
the days when I played it.
(But I've started again with chess, thinking about learning-technics,
helping one game to improve the other and also there are chess
programs for my PALM with good strength, but not any aceptable Shogi
program for it).
> > Yesterday I made the first implementation of V0.1.
> > (http://home.arcor.de/Bernhard.Maerz/BCMShogi/Counter.htm)
>
> I will take a look.
it's nothing worth for you.
And the pattern-search (for which I have started USF) is not
documentated.
(And I don't like to do this work. If there will be anyone writing
documentation ...???)
> I hope that you will become the Steven J. Edwards
> of the Shogi world.
At the moment, I'm not pushing USF.
Maybe if all people will use BCMShogi, writing all games in USF-
format, this situation will become the reason, that other programmers
also will implement USF.
> I really like your interface.
the wonderful graphics which are the MAIN reason for the good looking
are not from me.
I took it from the "bona"-page, which are free for non-commercial use.
> I also like your passion. A few people
> like you can drive the planet.
So I should take care of drinking less beer in the future. ;-)
Bernhard
user923005
Mar 27, 2009, 3:43:11 PM3/27/09
to SHOGI-L
On Mar 27, 2:33 am, Bernhard <shogi4...@arcor.de> wrote:
> Hi,
>
> > > 1) KI2
> > > ...
>
> > I have 71,398 files, the oldest of which date to 2004.
> > They are small, so I guess that they are single game files.
>
> oh!!!
>
> Can you tell me a link where to download it, please?
http://cap.connx.com/chess-engines/new-approach/ki2.zip
> Hi,
>
> > > 1) KI2
> > > ...
>
> > I have 71,398 files, the oldest of which date to 2004.
> > They are small, so I guess that they are single game files.
>
> oh!!!
>
> Can you tell me a link where to download it, please?
Bernhard
Mar 29, 2009, 9:23:41 AM3/29/09
to SHOGI-L
Hi,
> > > > 1) KI2
> ...
> http://cap.connx.com/chess-engines/new-approach/ki2.zip
thank you very much.
The next release of BCMShogi will also have KI2-Support (Import only -
Export would be very simple, but I don't like to promote any further
format I didn't like. Still wait for USF ;-) ).
The Implementation was very simple and fast possible.
But with this format I found a few bugs in the current BCMShogi
release in the "short japanese notation" part. So it was nevertheless
a good thing to implement KI2. ;-)
By the way:
a few KI2-files (maybe about 0.1%) couldn't load into BCMShogi.
The reason is one of the following:
1) illegal move
BCMShogi didn't allow illegal moves in most file formats. I could
allow it, but I didn't like it, because this makes error-detection
much harder still almost impossible.
2) using wrong suffix:
two diagonaly silver should use the extension "uppwards" and
"downwards" instead of "from left" and "from right".
I decide not to support this "wrong" files, because it would be too
much work and this happend only 0.01% !!!
By the way: shogidokoro can read such files!
3) simple wrong files by final checking
In the files there is listed the movenumber. Sometimes this is simply
wrong.
BCMShogi will load such files, but tell, that there are problems.
BCM
> > > > 1) KI2
> ...
> http://cap.connx.com/chess-engines/new-approach/ki2.zip
thank you very much.
The next release of BCMShogi will also have KI2-Support (Import only -
Export would be very simple, but I don't like to promote any further
format I didn't like. Still wait for USF ;-) ).
The Implementation was very simple and fast possible.
But with this format I found a few bugs in the current BCMShogi
release in the "short japanese notation" part. So it was nevertheless
a good thing to implement KI2. ;-)
By the way:
a few KI2-files (maybe about 0.1%) couldn't load into BCMShogi.
The reason is one of the following:
1) illegal move
BCMShogi didn't allow illegal moves in most file formats. I could
allow it, but I didn't like it, because this makes error-detection
much harder still almost impossible.
2) using wrong suffix:
two diagonaly silver should use the extension "uppwards" and
"downwards" instead of "from left" and "from right".
I decide not to support this "wrong" files, because it would be too
much work and this happend only 0.01% !!!
By the way: shogidokoro can read such files!
3) simple wrong files by final checking
In the files there is listed the movenumber. Sometimes this is simply
wrong.
BCMShogi will load such files, but tell, that there are problems.
BCM
Reply all
Reply to author
Forward
0 new messages