New Chess Tool - Novelty Hunter!

Hello dear Lichess users! Today I'm very excited to share with you my new Chess Tool - Novelty Hunter. If you are a strong player (2200+) and find yourself spending a lot of time coming up with interesting opening ideas to throw your opponents off their ground, this tool can come in very handy. It is free and open-source :). You can find it on my website, josebarria.com/novelty-hunter.

What it is

Novelty Hunter is a very simple tool I created in about two weeks that takes in a pgn file you drop it and quickly scans through the games, trying to find rare moves. After finding each rare move, it will try to assess how interesting the move is and then presents you with information about the game and the final interest score it calculated. I'll get into what these things concretely mean later. For now, let me show you what it looks like:
This is what Novelty Hunter came back with after dropping it the games of the strong Bundesliga chess league, and selecting only 5 novelties. There's some useful information on the page like how many games there were before the move (Master Games Before) and how many there are after it (Master Games After).

How useful could this be?

If you play OTB chess tournaments with the least amount of regularity, you've probably experienced the massive theoretical improvement of the chess population lately. Everyone seems to know their stuff, especially in the mainlines. And so, the "meta" nowadays is to keep coming up with new opening lines. Fresh ideas to throw your opponent off. Maybe a little twist here, maybe a little nuance there. All that really matters is that they're finally thinking with their weak little brain and not the 3800 Silicon Monster's. And, obviously, this gets much worse the stronger you are. This phenomenon has caused players to devote the vast majority of their study time to the openings, and at the elite level, they even report to spend more than 90% of their time studying openings!

While Novelty Hunter is not the cure to this epidemic - for example, it cannot come up with moves that have never been played before (yet ( ͡º ͜ʖ ͡º) ) -, I firmly believe it should help out cut down plenty in the manual labour hours required to go and check every single game that is being played on every tournament, all the time.
Currently, it takes about 4/5 seconds to scan a game. It will take longer the more theory the players play.

Sweet! What do I need to do to start using it?

Nothing! It's free and takes minimal effort to set up. All you need is a pgn file (which you can easily download from any lichess broadcast) and a Lichess API key, which is equally easy to get.

How does it work?

As I said, Novelty Hunter is quite a simple tool. So simple, that after building it I came to the conclusion that many strong GM's surely use similar tools for themselves. There's more detailed info in the docs, but here are its main quirks:
What constitutes a rare move?
I defined a rare move to be a move where

• gamesBefore > 10 and
• gamesAfter < 500 and
• frequency < 5%

where frequency = gamesAfter / gamesAfterMostPopularMove, as opposed to the more normal gamesAfter / gamesBefore. I explained my reasoning for that, as well as other decisions, in the docs.

Okay, we have our rare move. But how rare is it? There should be some sort of difference between a move that has 499 games and 5% frequency vs a move that has 0 games. For this I calculate a rarity score.
Rarity score = 50% followUpScore + 50% frequencyScore. The first goes up the less follow up games there are, while the second goes up the closer the frequency is to 0%.

Cool, we have discriminated between kind of rare moves and super rare moves. But what if it is just rare because someone hung a piece? That is where the efficiency score comes in.
Efficiency score = 80% StockfishScore + 20% ResultScore. The first takes the difference in evaluation between the position 5 moves after the novelty, and right after it, to see in which side the advantage progressed. The second gives a bonus or a penalization based on the game result.

Let's also add a little ingredient that promotes novelties that are found early on in the game as opposed to something like move 14, and finally, we come to the final interest score.
*Final interest score *= 40% earlyNovScore + 40% efficiencyScore + 20% rarityScore.

These weights are, of course, very subjective, and I'll probably make them customizable in the future. The reason I decided to keep the rarityScore much lower than the other two is because for the interest score to be calculated, the move already had to have passed those initial rarity conditions.

Final notes

If you took the time to see how the tool works under the hood and you're still here, thank you! Novelty Hunter is an open-source tool, and as so, I'd love to read your feedback on its usage and the formulas. I'm sure there are tons of things that could be added and/or improved and this could become much more robust in the future!
Cheers and happy hunting!

Disclaimers

The idea for this tool was thought of by me, José Bárria. The scoring method was solely designed by me (for now). Almost all the code was generated by my good friend called Claude, with some revision by me. All of the text in this page is AI-free.