Monday, September 21, 2020

How good does your Tipping Bot actually need to be?

Here's an interesting question that I have been pondering.  How good does a Tipping Bot (model) need to be?




The first question is, what is the the target?
A few options:

1. Beat all the other models in the Squiggle Leaderboard.
    This would require tipping at the rate of between 66%-74% over the last few seasons.

2. Beat all the other models, and the punters on the Squiggle Leaderboard.
    In the last 4 seasons only one model has had more correct tips than the Punters, MasseyRatings in 2019 - so this is clearly a difficult task.

3. Beat all the tipsters on Footytips.com.au (or your preferred tipping site)

4. Tip every game correctly!   Lets ignore this option and assume it is impossible.


My goal:

For ZaphBot my original target was to do well in office tipping competitions, which I have (usually top-5 placing).  I've decided to step things up a notch and now target the Squiggle Leaderboard.

So, in order to do that what do I need to improve?


Right now, with one game left in the Home and Away season, the best bots on Squiggle are scoring 110-111, with the punters at 112.
Zaphbot, at this same moment, is at 105 - and lets focus on that one at the moment, since it is still good enough to place in the top 10 at Squiggle (just)

The difference between my bot, and the AFLalytics - the best bot on Squiggle - is just 6 tips (possibly 7 after Monday night).  Lets call it 7, out of 18 games.
If I could get 1 more tip right every 2 weeks, I'd be on top of the leaderboard.

But I'm going to diverge for a moment, and the numbers below are from before the start of Round 18 so forgive me if they don't match the above data - I started this post a week ago!

The gap between Bots and the best Humans:
Lets pretend we've got an algorithm equal to the current leader, AFLalytics and we want to beat all humans on footytips.com.au.  At the end of Round 17 there was an 8 game gap to the top place.  To be the best in the world it would need to change the tip for 8 out of the 34 tips that it got wrong.  Doesn't sound impossible.


My quest now is to see if we can find those 34 games.

Model Agreement:
Here's how the Models on Squiggle have agreed this year, game by game. I need a better way of visualising this, but it will do for now.  100% means all models agreed, 50% would be half - there is an odd number of bots most weeks so it doesn't hit 50% exactly:



So far this season there have been 65 occasions where every single model agreed on the outcome.  Of those games, they still got it wrong 16 times (~25% of the time).  So lets assume those 16 are genuine upsets and we were never going to get them right since nobody elses model did, and put them in the too-hard basket.


This leaves us with improving 8 out of (34-16) = 18 tips.  That's close enough to half, and in each case at least one of the models tipped the correct result, so it is not impossible.

The goal now becomes identifying those 18 games, while not flipping tips on other games we have correctly tipped.  If we can find just half of those 18 games, and have our model flip it's tip for them, then we'll be on top of the world.  At least for this season.

It may be that there are ~2 games per round (on average) that are true 50-50 games that are impossible to tip, but as a Hawthorn supporter who watched Geelong win almost every 50-50 game for many years, there may be something to look for.

My next analysis will be to look and see if this trend occurred in previous seasons, and to see if there is any pattern in the games that are tipped wrong.
Stay Tuned!

No comments:

Post a Comment

Round 5

Here we go again.  Absolute shocker of a week last week, I'm now languishing at the bottom of the Squiggle ladder, and I don't even ...