A chess engine is a computer program that calculates chess position and chess moves.[1]
The chess engine decides what moves to make, but typically does not interact directly with the user. Most chess engines do not have their own graphical user interface (GUI) but are rather console applications that communicate with a GUI such as XBoard, WinBoard or glChess via a standard protocol. This allows the user to play against multiple engines without learning a new user interface for each, and allows different engines to play against each other.
Interface Protocol
The command line interface of GNU Chess became the initial de facto standard, called the Chess Engine Communication Protocol and was first supported by XBoard. When XBoard was ported to the Windows operating system as WinBoard this protocol was popularly renamed to 'WinBoard Protocol'. The WinBoard Protocol was itself upgraded and the two versions of the protocols are referred to as: 'WinBoard Protocol 1' (original version) and 'WinBoard Protocol 2' (newer version). There is another protocol, the Universal Chess Interface. Some engines support both major protocols, and each protocol has its supporters. The Winboard Protocol is more popular but many chess engine developers feel that the Universal Chess Interface is easier to implement. Some interface support both protocols whereas others, such as WinBoard, support only one and depend on subsidiary interpreters, such as Polyglot, to translate.
Some chess engines use endgame tablebases to increase their playing strength during the endgame. An endgame tablebase is a database of all possible endgame positions with small groups of material. Each position is conclusively determined as a win, loss, or draw for the player whose turn it is to move, and the number of moves to the end with best play by both sides. Endgame tablebases in all cases identify the absolute best move in all positions included (identifying the move that wins fastest against perfect defense, or the move that loses slowest against optimal opposition). Such tablebases are available for all positions containing three to six pieces (counting the kings) and for some seven-piece combinations. When the maneuvering in an ending to achieve an irreversible improvement takes more moves than the horizon of calculation of a chess engine, an engine is not guaranteed to find the best move without the use of an endgame tablebase, and in many cases can fall foul of the fifty-move rule as a result.
Many engines use permanent brain as a method to increase their strength.
Historically, commercial programs have been the strongest engines. To some extent, this is a self-fulfilling prophecy; if an amateur engine wins a tournament or otherwise performs well (for example, Zappa in 2005), then it is quickly commercialized. Titles gained in these tournaments garner much prestige for the winning programs, and are thus used for marketing purposes.
There are a number of factors that vary among the chess engine rating lists:
Missing from many rating lists are IPPOLIT and its derivatives (e.g. Fire [3]). Although very strong and open source, there are allegations from commercial software interests that they were derived from disassembled binary of Rybka.[11] Due to the controversy, all these engines have been blacklisted from many tournaments and rating lists. Although Rybka has been accused of being based on Fruit, it is not blacklisted from computer chess tournaments or rating lists.[12] In June 2011, the ICGA claimed Rybka was derived from Fruit and Crafty and Rybka has been banned from the International Computer Games Association World Computer Chess Championship, and its previous victories (2007, 2008, 2009, and 2010) has been revoked.[13] The ICGA was criticized for this decision by Dr. Søren Riis, a longstanding supporter of the Rybka program.[14] All rating lists continue to include Rybka.
Strategic Test Suite (STS) by Swaminathan and Dann Corbit, tests chess engine's strategical strength.[19]
The chess engine decides what moves to make, but typically does not interact directly with the user. Most chess engines do not have their own graphical user interface (GUI) but are rather console applications that communicate with a GUI such as XBoard, WinBoard or glChess via a standard protocol. This allows the user to play against multiple engines without learning a new user interface for each, and allows different engines to play against each other.
Interface Protocol
The command line interface of GNU Chess became the initial de facto standard, called the Chess Engine Communication Protocol and was first supported by XBoard. When XBoard was ported to the Windows operating system as WinBoard this protocol was popularly renamed to 'WinBoard Protocol'. The WinBoard Protocol was itself upgraded and the two versions of the protocols are referred to as: 'WinBoard Protocol 1' (original version) and 'WinBoard Protocol 2' (newer version). There is another protocol, the Universal Chess Interface. Some engines support both major protocols, and each protocol has its supporters. The Winboard Protocol is more popular but many chess engine developers feel that the Universal Chess Interface is easier to implement. Some interface support both protocols whereas others, such as WinBoard, support only one and depend on subsidiary interpreters, such as Polyglot, to translate.
[edit] Increasing strength
Main article: Computer chess
Chess engines increase in playing strength each year. This is partly due to the increase in processing power that enables calculations to be made to ever greater depths in a given time. In addition, programming techniques have improved, enabling the engines to be more selective in the lines that they analyse and to acquire a better positional understanding.Some chess engines use endgame tablebases to increase their playing strength during the endgame. An endgame tablebase is a database of all possible endgame positions with small groups of material. Each position is conclusively determined as a win, loss, or draw for the player whose turn it is to move, and the number of moves to the end with best play by both sides. Endgame tablebases in all cases identify the absolute best move in all positions included (identifying the move that wins fastest against perfect defense, or the move that loses slowest against optimal opposition). Such tablebases are available for all positions containing three to six pieces (counting the kings) and for some seven-piece combinations. When the maneuvering in an ending to achieve an irreversible improvement takes more moves than the horizon of calculation of a chess engine, an engine is not guaranteed to find the best move without the use of an endgame tablebase, and in many cases can fall foul of the fifty-move rule as a result.
Many engines use permanent brain as a method to increase their strength.
[edit] Comparisons
[edit] Tournaments
The results of computer tournaments give one view of the relative strengths of chess engines. However, tournaments do not play a statistically significant number of games for accurate strength determination. In fact, the number of games that need to be played between fairly evenly matched engines, in order to achieve significance, runs into the thousands and is, therefore, impractical within the framework of a tournament [2]. Most tournaments also allow any types of hardware, so only engine/hardware combinations are being compared.Historically, commercial programs have been the strongest engines. To some extent, this is a self-fulfilling prophecy; if an amateur engine wins a tournament or otherwise performs well (for example, Zappa in 2005), then it is quickly commercialized. Titles gained in these tournaments garner much prestige for the winning programs, and are thus used for marketing purposes.
- World Computer Chess Championship
- Dutch open computer chess championship
- Internet Computer Chess Tournament (CCT)
- International Paderborn Computer Chess Championship
- North American Computer Chess Championship
[edit] Ratings
Chess engine rating lists aim to provide statistically significant measures of relative engine strength. These lists play multiple games between engines on standard hardware platforms, so that processor differences are factored out. Some also standardize the opening books, in an attempt to measure the strength differences of the engines only. These lists not only provide a ranking, but also margins of error on the given ratings. Also rating lists typically play games continuously, publishing many updates per year, compared to tournaments which only take place annually.There are a number of factors that vary among the chess engine rating lists:
- Time control. Longer time controls, such as 40 moves in 120 minutes, are better suited for determining tournament play strength, but also make testing more time-consuming.
- Hardware used. Faster hardware with more memory leads to stronger play.
- 64-bit (vs. 32-bit) hardware and operating systems favor bitboard-based programs
- Multiprocessor vs. single processor hardware.
- Ponder settings (speculative analysis while the opponent is thinking) aka Permanent Brain.
- Transposition table sizes.
- Opening book settings.
- All listed engines are 64-bit
| Rating list | Time control (moves/minutes) | Year started | Last updated | Engine/platform entries | Games played | Top three engines | Rating |
|---|---|---|---|---|---|---|---|
| CCRL[2] | 40/40[3] Ponder OFF | 2005 | October 13, 2012 | 1241 | 417,873 | Houdini 2.0c x64 4CPU Critter 1.6a x64 4CPU Stockfish2.2.2 x64 4CPU | 3251 3243 3206 |
| CEGT[4] | 40/20[5] Ponder ON | 2006 | October 14, 2012 | 1067 | 620,333 | Houdini 1.5a x64 6CPU Critter 1.6 x64 4CPU Stockfish 2.3 x64 4CPU | 3101 3056 3056 |
| IPON[6] | 5m+3s ~16min/game Ponder ON | 2006 | October 17, 2012 | 110 | 213,250 | Houdini 3.0 STD x64 1CPU Komodo 5 x64 1CPU Critter 1.4a x64 1CPU | 3096 3011 2990 |
| SWCR[7] | 40/10 Ponder ON | 2009 | January 17, 2012 | 67 | 161,160 | Houdini 2.0c x64 1CPU Critter 1.4 x64 1CPU Rybka 4 Exp. 42 x64 1CPU | 3019 2980 2968 |
| SSDF[8] | 40/120 Ponder ON | 1984 | June 28, 2012 | 311 | 121,472 | Deep Rybka 4 x64 2GB Q6600 2.4GHz Naum 4.2 MP x64 2GB Q6600 2.4GHz Deep Shredder 12 x64 2GB Q6600 2.4GHz | 3221 3138 3115 |
| WBEC[9] | 40/40 Ponder ON | 2001 | May 15, 2011 | 226 (Historically: 850+[10]) | 100,749 | Rybka 4 x64 2CPU Stockfish 2.0.1 x64 2CPU Thinker 5.5.4A1 x64 2CPU | 3124 3121 3114 |
- Note that the listings in the above table only count the best entry for a given engine.
Missing from many rating lists are IPPOLIT and its derivatives (e.g. Fire [3]). Although very strong and open source, there are allegations from commercial software interests that they were derived from disassembled binary of Rybka.[11] Due to the controversy, all these engines have been blacklisted from many tournaments and rating lists. Although Rybka has been accused of being based on Fruit, it is not blacklisted from computer chess tournaments or rating lists.[12] In June 2011, the ICGA claimed Rybka was derived from Fruit and Crafty and Rybka has been banned from the International Computer Games Association World Computer Chess Championship, and its previous victories (2007, 2008, 2009, and 2010) has been revoked.[13] The ICGA was criticized for this decision by Dr. Søren Riis, a longstanding supporter of the Rybka program.[14] All rating lists continue to include Rybka.
[edit] Test suites
Engines can be tested by measuring their performance on specific positions. Typical is the use of test suites, where for each given position there is one best move to find. These positions can be geared towards positional, tactical or endgame play. The Nolot test suite, for instance, focuses on deep sacrifices.[15] Then there are the BT2450 and BT2630 test suites by Hubert Bednorz and Fred Toennissen. These suites measure the tactical capability of the engine[16] and have been used at least by REBEL.[17] There is also a general test suite called Brilliancy by Dana Turnmire. The suite has been compiled mostly from How to Reassess Your Chess Workbook.[18]Strategic Test Suite (STS) by Swaminathan and Dann Corbit, tests chess engine's strategical strength.[19]
[edit] Categorizations
[edit] Freely available
There are hundreds of freely available chess engines which conform to one of the above communication protocols. Many run on Windows or are open source. The top 50 strongest, freely available engines, according to the CCRL 40/40 rating list, are listed here.[20] Others may be found by examining the rating lists or external links.| Engine (strongest version) | Author (Country) | Elo | CPU | Source |
|---|---|---|---|---|
| Alaric v707 | Peter Fendrich (Sweden) | 2759 | Binary, non-commercial | |
| Bison v9.11 | Ivan Bonkin (Russia) | 2828 | GPLv3 | |
| Bobcat v2.75 | Alex Morozov (Ukraine) | 2807 | 64-bit | |
| Booot v5.1.0 | Alex Morozov (Ukraine) | 2947 | Source, non-commercial | |
| Bright v0.4a | Allard Siemelink (Netherlands) | 3002 | 4CPU | Binary, non-distributable |
| BugChess2 v1.9 | Francois and Jean-Philippe Karr (France) | 2867 | 64-bit | Binary, non-commercial |
| Chronos v1.9.9 | Guillermo Filia (Argentina) | 2843 | 64-bit | Binary |
| Colossus Chess 2008b | Martin Bryant (England) | 2746 | Binary | |
| Crafty v23.4 | Robert Hyatt (USA) | 2950 | 64-bit 4CPU | Source, non-commercial |
| Critter v1.2 | Richard Vida (Slovakia) | 3254 | 64-bit 4CPU | Binary |
| Cyclone v3.4 (derived from Fruit v2.1) | Fabien Letouzey (France), Thomas Gaksch (Germany), Norman Schmidt | 2971 | 2CPU | GPLv2+ |
| Cyrano v0.6b17 | Harald Johnsen (France) | 2745 | GPLv2 | |
| Daydreamer v1.75 | Aaron Becker (USA) | 2772 | 32-bit | Source, license? |
| Delfi v5.4 | Fabio Cavicchio (Italy) | 2824 | 2CPU | Source, non-distributable |
| Deuterium 10.01.27.213 | Ferdinand Mosca (Philippines) | 2770 | Binary | |
| Dirty 25Aug2011 | Pradu Kannan (USA), Fonzy Bluemers (Netherlands), Andres Valverde (Spain) | 2797 | ||
| Doch v1.2 | Don Dayley (USA) | 2990 | 64-bit | |
| E.T. Chess 13.01.08 | Eric Triki (France) | 2749 | Binary | |
| Frenzee Feb08 | Sune Fischer (Denmark) | 2877 | 64-bit | |
| Fruit v2.3.1 | Fabien Letouzey (France), Ryan Benitez (USA) | 2886 | GPLv2+ | |
| Gaviota v0.83 | FM Miguel A. Ballicora (Argentina) | 2758 | 64-bit 4CPU | |
| Glaurung v2.2 (now developed as Stockfish) | Tord Romstad (Norway) | 3001 | 64-bit 4CPU | GPLv3+ |
| Grapefruit v1.0 (derived from Toga II 1.4 beta5c) | Fabien Letouzey (France), Thomas Gaksch (Germany), Vadim Demichev (Russia) | 2977 | 32-bit 2CPU | GPLv2+ |
| Gull v1.2 | Vadim Demichev (Russia) | 3027 | 64-bit | Public Domain |
| Hannibal v1.1 | Edsel Apostol (Philippines), Sam Hamilton (USA) | 2988 | 64-bit | Binary |
| Houdini v1.5a | Robert Houdart (Belgium) | 3299 | 64-bit 4CPU | Binary, non-distributable |
| Jonny v4.00 | Johannes Zwanzger (Germany) | 2955 | 4CPU | Binary, distributable |
| Komodo v3.0 (last free version) | Don Dailey (USA) | 3216 | 64-bit SSE | Binary |
| Loop 13.6 (Loop 2007) (last free version) | Fritz Reul (Germany) | 2942 | 64-bit 4CPU | |
| Movei v00.8.438 (10 10 10) | Uri Blass (Israel) | 2769 | Binary, non-commercial | |
| Naraku v1.4 | Marco Meloni (Italy) | 2871 | Binary, distributable | |
| Naum v2.0 (last free version) | Aleksandar Naumov (Serbia/Canada) | 2800 | 64-bit | Binary |
| Nemo v1.0.1 | Michael Hoffmann (Germany) | 2947 | 64-bit | Binary |
| Pharaon v3.5.1 | Franck Zibi (France) | 2742 | 2CPU | Binary |
| Pro Deo v1.74 (freeware version of Rebel) | Ed Schröder (Netherlands) | 2736 | Binary, non-commercial | |
| Protector v1.4.0 | Raimund Heid (Germany) | 3090 | 64-bit 4CPU | GPLv3, except endgame |
| RedQueen 1.0.0 | Ben-Hur Carlos Langoni Junior (Brazil) | 2735 | 32-bit | GPLv3+ |
| Rybka v.2.3.2a (last free version) | Vasik Rajlich (Czech Republic) | 3124 | 64-bit 4CPU | Binary |
| Scorpio v2.7 | Daniel Shawul (Ethiopia) | 2851 | 32-bit | BSD |
| Slow Chess Blitz WV2.1 | Jonathan Kreuzer (USA) | 2737 | Source, non-commercial | |
| Spark v1.0 | Allard Siemelink (Netherlands) | 3081 | 64-bit 4CPU | Binary, non-distributable |
| Spike v1.4 Leiden | Volker Böhm and Ralf Schäfer (Germany) | 3129 | 4CPU | Binary, non-commercial |
| Stockfish v2.2.2 | Tord Romstad (Norway), Marco Costalba (Italy), Joona Kiiski (Finland) | 3264 | 64-bit 4CPU | GPLv3+ |
| Strelka v5.5 | Yuri Osipov (Russia) | 3240 | 64-bit | Binary |
| TheMadPrune v1.1.25 | Fabien Letouzey (France), WH Lowery Jnr (USA) | 2977 | 2CPU | |
| Thinker v5.4C Inert | Lance Perkins (Canada) | 3029 | 64-bit 4CPU | |
| Toga II v1.4.1SE (derived from Fruit v2.1) | Fabien Letouzey (France), Thomas Gaksch (Germany) | 3007 | 4CPU | GPLv2+ |
| Tornado v4.4 | Engin Ustun (Germany) | 2841 | 32-bit | Binary, non-commercial |
| Twisted Logic 20100131x | Edsel Apostol (Philippines) | 2864 | 64-bit | |
| Umko v1.0 | Borko Boskovic (Slovenia) | 2932 | 64-bit | GPLv3+ |
[edit] Pedagogical
These open source chess programs were expressly written to teach the craft of chess programming.- GNU Chess The goal of GNU Chess is to serve as a basis for research.
- ChessBin.com C# Chess Engine.
- Chess Program in C#.
- FirstChess (in C).
- Gerbil
- Gray Matter (in C++).
- MSCP (in C)
- Mizar (in C).
- Viper (in C++).
- Xadreco Chess Engine (in C).
- ChEngine (in C#).
- Stockfish (in C++).
- TSCP (in C).
- Micro-Max (chess engine), by H.G.Muller, a 133-line chess program in C (1433 characters)
- Toledo Nanochess, by Óscar Toledo G., an obfuscated chess program written in 1326 bytes of C.
[edit] Commercial
These chess programs are sold commercially. Most of these also include their own user interface.- Chess Genius, by Richard Lang of Mephisto fame
- Chessmaster
- Chess Tiger
- Fritz (single processor), Deep Fritz (multi processor)
- Gandalf
- Grandmaster Chess
- HIARCS
- Houdini 3 (Pro & Standard)
- Junior - (single- and multi-processor versions)
- Kasparov's Gambit, based on Socrates II
- The King - the engine of the commercial Chessmaster program
- Ktulu
- Loop (also the engine for Wii Chess)
- Naum versions 2.1 and later
- Onno
- Rebel - (see also ProDeo)
- Ruffian 2
- Rybka
- Shredder
- Deep Sjeng
- Smarthink
- SparkChess
- Zappa
[edit] Private
| Name | Author | Country |
|---|---|---|
| ApiChess | Max Himam | France |
| Azraël | Christopher Conkie | United Kingdom |
| Carnivor | Michael Sherwin | USA |
| Cheetah | Ralf Schäfer | Germany |
| Chepla | Mikael Bäckman | Sweden |
| Chimp | Andy Duplain | United Kingdom |
| Chiron | Ubaldo Andrea Farina | Italy |
| Cipollino | Giancarlo delli Colli | Italy |
| Cogito | Joerg Schaefer | Germany |
| Cowrie Chess | Chan Rasjid | Singapore |
| Czolgista | Tomasz Kazimierski | Poland |
| Diep | Vincent Diepeveen | The Netherlands |
| Dr. Theopolis | Corby Nichols | USA |
| EdlChess | Stephan Edlich | Germany |
| EGM | Pawel Kobylarz | Poland |
| Eichhörnchen | Wieland Belka | Germany |
| Ferret past winner of the World Computer Speed Chess Championship | Bruce Moreland | USA |
| Flywheel | Don Cross | USA |
| Freccia | Stefano Gemma | Italy |
| Grok | Peter Kappler | USA |
| HansDamf | Gerd Isenberg | Germany |
| Hector for Chess | Csaba Jergler | Hungary |
| Ikarus past winner of the World Computer Speed Chess Championship | ||
| Kallisto | Bart Weststrate | Netherlands |
| LearningLemming | Sam Hamilton | USA |
| MeneChess | Shaun Howe | United Kingdom |
| Moneypenny | Matt Shoemaker | USA |
| NaltaP312 | Yves Catineau | France |
| Nightmare | Joost Buijs | Netherlands |
| Now | Mark Lefler | USA |
| Nullmover | Michael Langeveld | Netherlands |
| Olympus | Joshua Shriver | USA |
| Pandix | Gyula Horváth | Hungary |
| Pebble | Adam Goodwin | USA |
| Philidor | Christian Barreteau and Bruno Lucas | France |
| Purple Haze | Vincent Ollivier | France |
| Sibyl | Milikas Anastasios | Greece |
| Sillycon | Lasse Hansen | Norway |
| Spandrel | Robert Purves | New Zealand |
| Symbolic | Steven Edwards | USA |
| Telepath | Charles Roberson | USA |
| Tinker | Brian Richardson | USA |
| TwilightChess | Tony Paletta | France |
| Tzunami | Ivo Tops | Netherlands |
| Vlad Tepes | Henk Fennema | Netherlands |
| WaDuuttie | Maarten Claessens | Netherlands |
| Waster | Geoff Westwood | United Kingdom |
| Weid | Jaap Weidemann | South Africa |
| XiniX | Tony van Roon-Werten | Netherlands |
| Z | Manuel Díaz | Spain |
| Zeta | Srdja Matovic | Montenegro |
| Ziggurat | David Norris | United States |
| Zilch | Mike Leany | USA |
[edit] Dedicated hardware
These chess playing systems include custom hardware or run on supercomputers. All are historical; chess supercomputers have not competed in computer tournaments since Hydra played in 2006.- Bebe, a strong bit-slice processor in the 1980s
- Belle
- Chess (Northwestern University)
- ChipTest
- Cray Blitz
- Deep Blue
- Deep Thought
- HiTech
- Hydra, predecessor was called Brutus
[edit] Commercial dedicated computers
In the 1980s and early 1990s, there was a competitive market for strong dedicated chess computers. Many form-factors were sold, from handheld peg-board computers to wooden auto-sensory boards with state-of-the-art processors. This market changed in the mid-90s when the economical embedded processors in dedicated chess computers could no longer compete with the fast processors in personal computers. Nowadays, most dedicated units sold are of beginner and intermediate strength.- Chess Challenger, a line of chess computers sold by Fidelity Electronics from 1977 [4] to 1992. These models won the first four World Microcomputer Chess Championships.
- ChessMachine, an ARM-based dedicated computer, which could run two engines:
- "The King", which later became the Chessmaster engine, was also used in the TASC R30 dedicated computer
- Gideon, a version of Rebel, in 1992 became the first microcomputer to win the World Computer Chess Championship
- Mephisto, a line of chess computers sold by Hegener & Glaser. The units programmed by Richard Lang won six consecutive World Microcomputer Chess Championships. They bought out Fidelity in 1989.
- Novag sells a line of tactically strong computers, including the Constellation, Sapphire, and Star Diamond brands.
- Saitek sells mid-range units of intermediate strength. They bought out Hegener & Glaser and its Mephisto brand in 1994.
- Excalibur Electronics sells a line of beginner strength units. Excalibur was started in 1992 by the son of the founder of Fidelity Electronics.
- Phoenix Chess Systems makes limited edition units based around StrongARM and XScale processors running modern engines and emulating classic engines
[edit] Historical
These chess programs run on obsolete hardware.- 1K ZX Chess
- Colossus Chess
- Kaissa
- Kotok-McCarthy
- Mac Hack
- Microchess was the first commercial game for a personal computer, developed first for the KIM-1 and later Commodore PET, Apple II, TRS-80 and others. Bobby Fischer played against MicroChess. [5]
- Sargon
- Socrates II
[edit] See also
- List of chess software
- Chess Engines Grand Tournament
- Computer chess
- Internet chess server
- Chess Engine Communication Protocol
- Universal Chess Interface
[edit] Notes and references
- ^ blog.chess.com Creating a chess engine from scratch (Part 1: Basics), Link date 28 June 2012
- ^ "CCRL 40/40 - Complete list". October 13, 2012. http://www.computerchess.org.uk/ccrl/4040/rating_list_all.html. Retrieved October 18, 2012.
- ^ Also available: 40 moves in 4 minutes
- ^ "CEGT 40/20". Chess Engines Grand Tournament. October 14, 2012. http://www.husvankempen.de/nunn/rating.htm. Retrieved October 18, 2012.
- ^ Also available: 40 moves in 4 minutes, 40 moves in 120 minutes
- ^ "IPON". IPON. August 29, 2012. http://www.inwoba.de/index.html. Retrieved August 29, 2012.
- ^ "SWCR". SWCR. January 17, 2012. http://www.amateurschach.de/. Retrieved May 21, 2012.
- ^ "The SSDF Rating List". Swedish Chess Computer Association. June 28, 2012. http://ssdf.bosjo.net/list.htm. Retrieved October 18, 2012.
- ^ "BayesianElo Ratinglist of WBEC Ridderkerk". May 15, 2011. http://wbec-ridderkerk.nl/html/BayesianElo_ed18.htm. Retrieved May 21, 2012.
- ^ http://home.scarlet.be/vincentlejeune/Ratings-WBEC-Ed-11-to-16.txt
- ^ Chess engine controversy at chessvibes.com, retrieved 28/May/2010
- ^ https://webspace.utexas.edu/zzw57/rtc/eval/eval.html
- ^ http://www.chessvibes.com/reports/rybka-disqualified-and-banned-from-world-computer-chess-championships/
- ^ Riis, Dr. Søren (02.01.2012). "A Gross Miscarriage of Justice in Computer Chess (part one)". Chessbase News. http://www.chessbase.com/newsdetail.asp?newsid=7791. Retrieved 19 February 2012.
- ^ Nolot test suite
- ^ BT2450 test suite
- ^ Rebel
- ^ Brilliancy suite TalkChess forum
- ^ [1] Strategic Test Suite
- ^ Elo ratings taken from "CCRL 40/40, full list". Computer Chess Rating List. January 29, 2012. http://www.computerchess.org.uk/ccrl/4040/rating_list_all.html. Retrieved February 9, 2012.
