Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

100 million FICS games

230 views
Skip to first unread message

Marcel Van Kervinck

unread,
Nov 21, 2009, 6:50:00 PM11/21/09
to
Dear all,

Here is a .torrent file for all recorded FICS games from November 1999
onwards.
The games haven been converted into PGN format and verified.

If there are no seeds please send me an e-mail. I attempt to keep at
least
one seed with sufficient bandwidth alive at all times, but it is not
automated.

Torrent file: http://marcelk.net/logics/FICS-games-199911-200910.torrent

Contents looks like this:

FICS-games/
fics-1999-11.pgn.bz2
fics-1999-12.pgn.bz2
fics-2000-01.pgn.bz2
... etc ...
fics-2009-08.pgn.bz2
fics-2009-09.pgn.bz2
fics-2009-10.pgn.bz2

The total number of games is over 100 million.
The total download size is around 15GB, zipped.

http://ficsgames.com/ has a searchable version of this data online.

Marcel van Kervinck
<mar...@bitpit.net>

Offramp

unread,
Nov 21, 2009, 9:17:55 PM11/21/09
to
On 21 Nov, 23:50, Marcel Van Kervinck <kervi...@gmail.com> wrote:
> Dear all,
>
> Here is a .torrent file for all recorded FICS games from November 1999
> onwards.
> The games haven been converted into PGN format and verified.
>
> If there are no seeds please send me an e-mail. I attempt to keep at
> least
> one seed with sufficient bandwidth alive at all times, but it is not
> automated.
>
> Torrent file:http://marcelk.net/logics/FICS-games-199911-200910.torrent
>
> Contents looks like this:
>
>         FICS-games/
>                 fics-1999-11.pgn.bz2
>                 fics-1999-12.pgn.bz2
>                 fics-2000-01.pgn.bz2
>                     ... etc ...
>                 fics-2009-08.pgn.bz2
>                 fics-2009-09.pgn.bz2
>                 fics-2009-10.pgn.bz2
>
> The total number of games is over 100 million.
> The total download size is around 15GB, zipped.
>
> http://ficsgames.com/has a searchable version of this data online.
>
>         Marcel van Kervinck
>         <marc...@bitpit.net>

ALL recorded FICS games? I suppose that must include those lo-fi,
substandard 1-minute bullet games.
I find the idea of possessing a 100,000,000 game database
fascinating... But I would think that the download would take over a
week; wouldn't a cbh file be a ¼ of the size?

SAT W-7

unread,
Nov 22, 2009, 3:56:43 AM11/22/09
to
Out of all the games are there two games just like each other or is
every game different ?

Offramp

unread,
May 24, 2016, 2:50:57 PM5/24/16
to
On Sunday, November 22, 2009 at 8:56:43 AM UTC, SAT W-7 wrote:
> Out of all the games are there two games just like each other or is
> every game different ?

Every game is different. Every game is great.
If a game is wasted God gets quite irate.

Eliyahu

unread,
Jun 3, 2016, 10:09:30 AM6/3/16
to
Bs"d

I switched to Lichess, never looked back.

Offramp

unread,
Jun 3, 2016, 10:16:40 PM6/3/16
to
So have I. Lichess is really good.

Eliyahu

unread,
Jun 4, 2016, 4:20:44 PM6/4/16
to
On Saturday, June 4, 2016 at 5:16:40 AM UTC+3, Offramp wrote:
> So have I. Lichess is really good.

Bs"d

Right. Many more options, all games stored forever, not just 10 like in fics, it is great.

Offramp

unread,
Feb 20, 2017, 1:28:16 PM2/20/17
to
Great value.

Offramp

unread,
Feb 20, 2017, 1:36:54 PM2/20/17
to
I have made an update.

There is a torrent file for all recorded FICS games from November 1999 onwards.
The games haven been converted into PGN format and each one has been played through and the score integrity has been checked.

Contents looks like this:

FICS-games/
fics-1999-11.pgn.bz2
fics-1999-12.pgn.bz2
fics-2000-01.pgn.bz2
... etc ...
fics-2009-08.pgn.bz2
fics-2009-09.pgn.bz2
fics-2009-10.pgn.bz2
... etc ...
fics-2016-12.pgn.bz2
fics-2017-01.pgn.bz2
fics-2017-02.pgn.bz2

The total number of games is just over 1.74 billion.

The total download size is about 165 Gb, zipped. Unzipped... I am not sure.

raylopez99

unread,
Feb 20, 2017, 8:19:21 PM2/20/17
to
On Monday, February 20, 2017 at 1:36:54 PM UTC-5, Offramp wrote:
> I have made an update.
.
>
> http://ficsgames.com/ has a searchable version of this data online.

This is tremendous. Now if you can please "Auto-check" the games at say five seconds a move on the latest Fritz and report back on what the most likely blunders were, what move the blunder occurred, how many pieces were on the board (the middlegame seems more prone to blunders but endgames can be tough too) and so on it would be a boon to chess researchers everywhere.

RL

Offramp

unread,
Feb 22, 2017, 9:07:35 AM2/22/17
to
That was a good idea. I have three laptops and I and doing a blundercheck with Stockfish 8 at 15 seconds per move.

Setting that up allowed me to see exactly how many games there were in this interesting collection.

It is much more than I thought: 2,518,172,389. That shows how much FICS usage has increased in recent years.

Andy Walker

unread,
Feb 22, 2017, 12:31:41 PM2/22/17
to
On 22/02/17 14:07, Offramp wrote:
> That was a good idea. I have three laptops and I and doing a
> blundercheck with Stockfish 8 at 15 seconds per move.

OK; if games last 30 moves, 60 ply, that's 15 minutes
per game, or, with three invocations of Stockfish, 12 games per
hour on average, ...

> Setting that up allowed me to see exactly how many games there were
> in this interesting collection.
> It is much more than I thought: 2,518,172,389.

..., totalling ~210,000,000 hours, ~9,000,000 days, call
it 25,000 years. I hope the collection is *very* interesting ...!
You need to save a factor of 1t least 10,000 somewhere. If you buy
another 297 laptops, find another nine numpties with similar set-ups,
and allow Stockfish only 1.5s/m, you're about there.

--
Andy Walker,
Nottingham.

raylopez99

unread,
Feb 22, 2017, 12:39:45 PM2/22/17
to
Great! I'll check back with you Offramp in 30 years and see how you're doing...

RL

raylopez99

unread,
Feb 22, 2017, 12:41:55 PM2/22/17
to
Quantum computers will solve this in seconds. Please do keep up with the literature, even when retired...

RL

Andy Walker

unread,
Feb 22, 2017, 3:02:13 PM2/22/17
to
On 22/02/17 17:41, raylopez99 wrote:
>> ..., totalling ~210,000,000 hours, ~9,000,000 days, call
>> it 25,000 years. I hope the collection is *very* interesting ...!
>> You need to save a factor of at least 10,000 somewhere. If you buy
>> another 297 laptops, find another nine numpties with similar set-ups,
>> and allow Stockfish only 1.5s/m, you're about there.
> Quantum computers will solve this in seconds.

Perhaps they will, though they're a tad short of that sort of
power today and for the foreseeable future. I'd guess it will be more
than 2.5 years before Offramp, or you, can afford a quantum computer
to replace his current three laptops, let alone to do substantially
better at analysing chess positions.

By contrast, crowd-funding or otherwise acquiring ~10,000x
Offramp's current computing power is practicable, though unlikely,
even today. After all, that is roughly how the 2 billion games
were created in the first place.

> Please do keep up with
> the literature, even when retired...

Perhaps you should read the scientific literature rather than
tabloid hype?

--
Andy Walker,
Nottingham.

Offramp

unread,
Feb 22, 2017, 4:18:00 PM2/22/17
to
You are right; it is going to take about 30 years at this rate. I have found three more laptops (that reduces it to 15 years), and I have reduced the tome from 15 seconds /move to 1.5 (as someone suggested), so that would make just 18 months! That isn't too long. If the programme finds some duplicates then I could cut the time down even further.

Andy Walker

unread,
Feb 22, 2017, 5:21:41 PM2/22/17
to
On 22/02/17 21:17, Offramp wrote:
> If the programme finds some duplicates then I could cut the time down
> even further.

If you sort the games, then you need to analyse each game only
from the point where it deviates from its predecessor. Given how many
will start with one of the standard opening lines, I'd guess that the
majority of games will first deviate more than 20 ply in, saving perhaps
1/3 of the analysis.

Bucket sort would be particularly suitable for this problem,
esp as you can be analysing moves within the bucket while continuing
the sort.

--
Andy Walker,
Nottingham.

raylopez99

unread,
Feb 23, 2017, 11:40:45 PM2/23/17
to
Oh, you got me professor... So I guess Offramp can hire, on Amazon.com, X thousands of computers and analyze the games that way? I guess so...

RL

raylopez99

unread,
Mar 12, 2017, 6:32:52 PM3/12/17
to
This week's Economist has a special section on how quantum computers have gone mainstream. Chess tree to be solved soon, to xyz moves, depending on what kind of quantum computer you have (the more qubits you have--which is hardware related--the more moves you can search; I can see in the future the entire chess tree to say 1000 moves deep being completed exhaustively searched with a top-of-the-line quantum computer).

Science fiction says Andy Walker, but the poor chap, being retired and retarded, doesn't keep up with the literature!

RL

raylopez99

unread,
Mar 12, 2017, 6:37:26 PM3/12/17
to
Now take this to the logical extreme: one quantum computer, model Ace1, can search up to 1000 moves deep. The other quantum computer, improved model Ace2, can search 1001 moves deep. Ace 1 makes an innocent knight move Nh3 to the edge of the board, in an equal position, and Ace2 announces mate in 1001! The next move, after being made by Ace2, results in Ace1 resigning since it now sees, within its 1000 move event horizon, that it is lost. The human spectators would of course not have a clue the game was lost until say 995 moves were played, then they would see the mate in five.

The stuff of science faction!

RL

Rainer

unread,
Mar 13, 2017, 6:07:22 AM3/13/17
to
Rubbish. Quantum computers are able to solve a limited subset of
computational problems efficiently. Playing chess is none of these problems.

Quantum computers will contribute as much to the solution of chess as
you--nothing.

Cheers,
Rainer

Offramp

unread,
Mar 13, 2017, 11:57:08 AM3/13/17
to
Based on information received from recent postings I

***I HAVE SUSPENDED THE FRITZ 5.1 BLUNDERCHECK
***OF THE 3,327,268,818
***GAMES

which are downloadable from my site.

It is obvious that in a few years the entire database will be analysable within a few seconds.

raylopez99

unread,
Mar 13, 2017, 2:46:06 PM3/13/17
to
Stupidity rains down on us. You, not a programmer like me, know nothing --NOTHING--about what you speak of. If you know how RLC analog computers work, and how they can solve second-order differential equations if you set them up correctly, you'd understand that programming a general purpose quantum computer is the same. You program the quantum computer to assume the state of the chess tree, maybe with one or more qubits representing legal moves (how many legal moves are there in chess per position? That would be your maximum number of qubits). They already have a dedicated quantum computer to solve the Traveling Salesman problem, which is an O(n!) complex problem (do you even know what Big-Oh notation means? Doubtful), and the chess tree is no more difficult than that.

And oh--big Oh--I have solved such problems and even more difficult problems in my professional capacity. As part of a team, but I have done it.

RL

Rainer

unread,
Mar 13, 2017, 8:35:16 PM3/13/17
to
More rubbish. You fail to understand the very fundamentals of quantum
computing. The following simple question will make it clear:

How do you verify the output of a (hypothetical) quantum chess solver?

You didn't think of it, right?

Cheers,
Rainer

Offramp

unread,
Mar 13, 2017, 8:41:32 PM3/13/17
to
On Tuesday, 14 March 2017 00:35:16 UTC, Rainer wrote:

> How do you verify the output of a (hypothetical) quantum chess solver?
>
> You didn't think of it, right?

I think you are right.
***I HAVE RECOM
***MENCED ANALY
***SIS OF GAMES

raylopez99

unread,
Mar 16, 2017, 1:56:03 PM3/16/17
to
Wow you're denser than water Rainer. How you say? That's easy. If the output of a quantum computer assumes the state of the chess board, the value of the pieces at any given position would determine the potential of the position, and the results would flow naturally like water flows downhill. Example: the board position in a quantum computer shows all legal moves simultaneously. The best move to make is the one with the highest non-equilibrium in material for either Black or White. Example: equal position but mate in one means the 'mate in one' is the best move unless the other side can refute it, then, if not, it would be some other move. Possibly there's no best move that wins material so you pick a 'drawish' move (randomly, or using some trivial evaluation function, like centralize knights or rooks to open files). Unlike today's chess engine, there's NO NEED for an evaluation function, except a trivial one in the event there's no win of material move. No need because you're examining all moves in the chess tree, and depending on how much hardware you have, you can examine all the moves in the chess tree up to say 200 plies (if your quantum computer is powerful enough).

Nobel prize in physics follows for me.

RL

Offramp

unread,
Mar 31, 2017, 10:14:05 PM3/31/17
to
Please wait for an important announcement.

Offramp

unread,
Mar 31, 2017, 10:22:28 PM3/31/17
to
I have decided to substantially increase the size of this database by including games from chess.com, lichess.com, chess24.com and playchess.com (chessbase). I think I will be able to download every game since 1997.

Details and filenames will follow soon.

Offramp

unread,
Apr 1, 2017, 12:17:13 AM4/1/17
to
I have made an update.

Here is the torrent file for all recorded Internet chess games from January 1997 onwards.
The games have been converted into PGN format and each one has been played through and the score integrity has been checked. One game is incomplete: Huskisson1988354 v Shizekopf35477, FICS August 12th 2001: does anyone have the complete score?

Contents look like this:

Internet chess games
ICG-1997-01.pgn.bz2
ICG-1997-02.pgn.bz2
ICG-1997-03.pgn.bz2
... etc ...
ICG-2009-08.pgn.bz2
ICG-2009-09.pgn.bz2
ICG-2009-10.pgn.bz2
... etc ...
ICG-2016-12.pgn.bz2
ICG-2017-01.pgn.bz2
ICG-2017-02.pgn.bz2
ICG-2017-03.pgn.bz2

The total number of games is just over 19 trillion. If you have played internet chess over the last 20 years YOUR GAME WILL BE HERE!

The total download size is about 383Gb, zipped. Unzipped probably about 2.2 Yottabytes.

A valuable research tool for the aspiring chess master.

Offramp

unread,
Apr 1, 2017, 4:42:07 AM4/1/17
to
A quick search through the whole torrent file reveals some surprising statistics.

There are, in all, 57,988,266,801 complete games. How many do you think begin 1. e4 e5 2. Ke2?

Answer: 112,810,711! (That's just an exclam, not factorial.)

That sums up the huge educational value of the database. Where else would you find out how to defend against this attack?
0 new messages