How much fairer is the new Super Rugby Competition?
First of all, this difficult article took a lot of thought to write, so I would be keen to answer any questions in the comments section to help people understand my thought process.
As most rugby fans are aware, Super Rugby has abandoned three teams and reformatted the competition to reduce flight distances and improve the fairness of the competition.
So how much fairer has it become? How is it compared to the format in 2015 before the expansion in 2016?
Observing past results
To test this, we will observe what the playoff seeds would have been under the 2015 and 2018 competition structures. This is based on the results of the 2017 tournament.
For each match in 2017, I’ve treated both teams as even test teams (referring to the IRB rankings formula for background). Then calculated the IRB ranking points gained/lost by each team for each match.
Then for each match that didn’t happen in 2017, (eg Jaguares against a New Zealand team), I computed the ranking points gained/lost by the average difference in ranking points gained/lost against oppositions they have both faced.
If Team A faced B, C, and D where Team E faced B, C and F during the season, the ranking points exchanged for the match between Team A and E (which didn’t actually happen) is calculated by:
[ (Ranking points exchanged for Team A vs B – Ranking points exchanged for Team E vs B) +
(Ranking points exchanged for Team A vs C – Ranking points exchanged for Team E vs C) ] / 2
Calculating using methodology
First, I’ve totalled all the ranking points gained/lost against every opponent for each team to create the “Perfectly fair 2017 standings”.
We call this table in this way because based on the margins, home/away grounds and the strength of oppositions each team faced in 2017, this is what the regular season standings would have been if the competition was a perfect round-robin with all matches played on neutral grounds.
We observe that the Brumbies are the only team to have seeded in the 2017 structure that wouldn’t have seeded if the competition was perfectly fair. And this is the exact opposite of the Blues. Therefore, the 2017 competition’s fairness was actually not as bad as most people might expect. Certainly based on “IRB rankings lens” that I have applied.
Now, we will transform this set of perfect round-robin neutral ground matches into the matches that took place in the 2015 season. The seedings based on the 2015 structure will be used.
When we compare this table to the “Perfectly fair 2017 standings”, the sum of the differences across all teams totals 38.82.
The Highlanders have the largest difference of -6.91. This means the Highlanders were the most disadvantaged team for the 2015 fixtures. Whereas the Waratahs with 4.73 were the most advantaged team.
Not surprisingly, all New Zealand teams have negative differences because the 2015 structure has more fixtures against your own country’s teams and recently the New Zealand teams overall have been the best out of the three countries.
On the other hand, the 2017 results based on the 2018 matches and structure looks like this:
Again, we compare this table to the “Perfectly fair 2017 standings”. The sum of the differences across all teams totals 41.09.
The Highlanders again have the largest difference of -8.21.
The Sunwolves with 4.73 is the most advantaged team for the 2018 fixtures. It is fair to say this is what we expect because the Sunwolves will join the Australian Conference that tends to be performing the worst out of the three countries.
The 2018 structure could be slightly less fair than the 2015 structure and obviously fairer than the 2017 structure. This is largely because each team will face a much wider variety of teams.
Of course, data can never reflect the entire story (for example, this table does not take into account the fairness of different travelling distances between the franchises). But based on the fixtures and the seeding system, the 2015 structure does a better job of seeding the higher performing teams.
Author: Kaito Goto