How the expected score is calculated
A study's expected score is your probability of winning plus half your probability of drawing, $$\text{Expected score} = P(\text{win}) + \tfrac{1}{2}\, P(\text{draw}),$$ that is, the average number of points you score per game, with a win worth 1, a draw ½ and a loss 0. The probabilities are those of a game in which you always play your prepared moves and your opponents choose their moves like the players in the Opening Explorer.
It is calculated in two parts: a walk through your study, always the same, and a value for each line where your preparation ends, which you choose in the Stats page's calculation settings. This page describes both exactly, with a worked example. The numbers in the example are made up, so that the arithmetic is easy to follow.
The idea
Your study is a tree of moves. On your side, it has one prepared move per position; on your opponent's side, it can have several moves, the replies you've prepared for. The calculation walks this tree from the study's starting point, with three rules:
- You always play your prepared move.
- Your opponent replies as players do in the Explorer. In each position, each move the Explorer lists gets a weight: its number of games, divided by the total of the listed moves.
- Each line's end gets a value \(v\): the expected score of a game from that position on, between 0 and 1, from your side.
The expected score is then the weighted average of the values where the lines end: the probability \(P(\ell)\) of reaching the end \(\ell\) of each line, that is the product of your opponent's weights along it, times its value, summed over all lines:
$$\text{Expected score} = \sum_{\ell} P(\ell)\, v(\ell).$$
The walk and the weights \(P(\ell)\) only depend on your study and on the Explorer settings. The values \(v(\ell)\) are a choice: Lirep offers three, based on the Explorer's results, on Stockfish's evaluation, or on both.
Where a line ends
A line ends with its last answered move. Usually that's one of your moves, after which you haven't prepared anything. It can also be one of your opponent's replies that your study doesn't answer: it leaves your preparation, and the line ends with it. An opponent move in your study that you haven't prepared an answer to isn't an answered move, and is ignored, as if it weren't in the study.
Say a line ends with 5.b3 e4, and you're White: 5...e4 has no prepared answer, so the line ends with 5.b3. With Explorer values, the line is worth 5.b3's own results: the games in which 5.b3 was played in the position before it. The Explorer shows them on 5.b3's row when you look at the position after Black's 4th move. It isn't an average over Black's replies after 5.b3. With Stockfish, the line is worth the evaluation of the position after 5.b3.
The row of 5.b3 and the Explorer's totals in the position after it can differ: the Explorer stores games by position, and the totals also count the games that reached the same position by another move order. The row counts only the games that played 5.b3 in the position before it.
The value of a line's end
A line's value \(v\) is your expected score in a game that reaches the end of that prepared line: the points you can expect from there on, once your preparation stops, between 0 and 1. It can't be known exactly, so it is estimated, in one of three ways chosen in the calculation settings on a study's Stats page:
-
Explorer results, the default. A line is worth the results of its last move, as the Explorer lists them in the position before that move: its share of wins plus half its share of draws, from your side. The Stats page also shows the share of wins, draws and losses this gives.
-
Stockfish. A line is worth Stockfish's evaluation of the position where it ends, at the depth chosen in the calculation settings, turned into an expected score with the curve Lichess uses for its winning chances. With \(c\) the evaluation in centipawns, from your side:
$$v = \frac{1}{1 + e^{-0.00368208\, c}}.$$
Evaluation, in pawns 0.00 +0.50 +1.00 +2.00 +3.00 +5.00 Mate for you Value \(v\) 50% 54.6% 59.1% 67.6% 75.1% 86.3% 100% The curve gives an expected score, not wins, draws and losses apart, so the Stats page doesn't show them. Stockfish runs in your browser, like for the expected evaluation, and reuses the positions it has already evaluated on this device.
-
Hybrid. Explorer results, except where the last move's row has fewer than 100 games, or isn't listed at all: there the line is worth Stockfish's value instead. A score over 100 games still has a standard error of about 5 points, and the error grows quickly below that. The Stats page shows how many lines' ends Stockfish valued. As with Stockfish alone, there is no split into wins, draws and losses.
Where the Explorer has no games
If the Explorer has no games at all in a position where your opponent is to move, the walk can't go on: Explorer values count the position as 50%, the two others take its Stockfish value. If the study's starting point has nothing prepared, Explorer values take the Explorer's totals for that position. If your last move is too rare to be listed, Explorer values have no games to go by and count it as 50% too; the third value takes Stockfish's instead.
A worked example
We consider a study for White in the Caro-Kann Fantasy, starting after 3.f3, with lines' ends valued with Explorer results. In the Explorer, Black replies 3...dxe4 in 50% of the games, 3...e6 in 30% and 3...g6 in 20% (made-up numbers, as are all the numbers below).
From the ends of the lines back to the start:
- 3...dxe4 4.fxe4: your preparation stops with 4.fxe4, so this line is worth 4.fxe4's own results in the position after 3...dxe4: 58%.
- 3...e6 4.Nc3: Black's replies are weighted again. 4...Bb4 is answered by 5.a3, where your preparation stops: 55%, 5.a3's own results. 4...dxe4 isn't answered, so its own results count: 48%. This branch is worth 60% × 55% + 40% × 48% = 52.2%.
- 3...g6 isn't answered: its own results, 51%.
Expected score = 50% × 58% + 30% × 52.2% + 20% × 51% = 29 + 15.66 + 10.2 = 54.86%
The same sum, line by line: 4.fxe4 is reached with probability 50%, 5.a3 with 30% × 60% = 18%, 4...dxe4 with 30% × 40% = 12%, and 3...g6 with 20%. With Stockfish values, only the four values change: if Stockfish rated each of the four positions +0.50, each would be worth 54.6%, and so would the study.
Coverage
Coverage is the share of games still in your preparation after each of your opponent's moves, with the same weights. It starts at 100%. Your moves keep the games in; each opponent move splits them by how often each reply is played; games on a reply your study doesn't answer leave your preparation. So coverage can only go down, and only on your opponent's moves.
What the expected score does and doesn't say
Neither way of valuing a line's end gives your own score once you're out of book; each errs in its own way.
- Explorer results overstate your score where your preparation ends. They are the results of every player who reached the position: the average score of a player picked at random among them, including the ones who knew how to go on. What the expected score is after is the score of a player whose preparation stops there. In other words, the Explorer gives an average over all players where an average over unprepared players is wanted, and the prepared ones raise it.
- Stockfish's evaluations often differ widely from how people score. A position Stockfish calls equal can be much easier to play for one side, and an engine's advantage can be out of reach for a human. Over many lines, some of these errors cancel out, but not all: in a given opening they often lean the same way. The curve from centipawns to an expected score is also the same for every rating, while a pawn is worth more between strong players.
- Rare replies are left out. The Explorer lists the most played moves in a position (up to 12); the weights are shares of those listed moves.
- It depends on the pool. The rating band and time controls in the study's calculation settings choose the games; results calculated with different settings, or different values at the end of preparation, aren't strictly comparable. The leaderboards show each card's settings.
- Small samples are noisy. A move's results over a few dozen games are uncertain by several points; the game counts are shown in the Explorer. The third value, Hybrid, which uses Stockfish under 100 games, avoids the smallest samples.
Expected evaluation
The expected evaluation is the same calculation with Stockfish values, the same lines' ends and the same evaluations, but averages the evaluations themselves, in pawns, without turning them into scores:
$$\text{Expected evaluation} = \sum_{\ell} P(\ell)\, c(\ell),$$
with \(c(\ell)\) Stockfish's evaluation where the line \(\ell\) ends, from your side, capped at ±10 pawns; a mate counts as 10 pawns. Without the cap, a mate after a blunder in 0.1% of games would add a whole pawn to the average. Even with it, a few lines with a large advantage raise the expected evaluation more than the expected score: the curve flattens out, so each counts at most 100% in the score, but up to 10 pawns in the average. Once one of the two is calculated at a given depth, the other needs no new Stockfish evaluations. See the Stats documentation for both calculations' details.