Hybrid Ranking System

Once upon a time before Semi Pro Ultimate or I wrote for Skyd, I (poorly) kept a personal blog, where I examined some different ways to incorporate the USAU Formula into determining bids for the Championships, in this case, at the DI and DIII college level. Realizing that changing the math or structure of the current formula might close current holes with the system but create other, additional problems, I thought incorporating human voting might be a good thing. Now, like the BCS (since it’s so popular) the human polls would only be part of the formula, with the computer retaining some influence. The reason for including the human element is that it can account for things the computer can’t, like if a team was playing at half strength at a tournament, if games had to be ended early as they were still in progress, if player injuries occurred that prevent them from playing in the series, last minute sanctioned games that occur under murky at best circumstances. I would argue that all of those are important factors when determining bids through a ranking system, things a computer is unable to account for.

From the Open Division let’s take my top 25, Ultiworld’s top 25, and the USAU Formula top 25. With those three as the sources of data, we’ll give 1/2 the weight to the USAU formula, 1/4 the weight to my top 25, and 1/4 the weight to Ultiworld. The BCS actually does 1/3 all around, but I think it’s important for the computer to have equal weight to the human rankings. That new, hybrid formula gives us this as the new top 25. The number next to the team is their new, average hybrid ranking. The numbers in parentheses are first the USAU Formula, then my ranking, then Ultiworld. Ties are awarded to whichever team is higher in the USAU system.

  1. Revolver              1             (1, 1, 1)
  2. Ironside               2             (2, 2, 2)
  3. Sockeye               4.5          (3, 4, 8)
  4. Chain Lightning 5            (4, 5, 7)
  5. Doublewide        5             (7, 3, 3)
  6. Johnny Bravo     5.5          (5, 7, 5)
  7. PoNY                     7             (6, 10, 6)
  8. Ring of Fire         7.5           (10, 6, 4)
  9. Truck Stop           8.75       (8, 9, 10)
  10. Machine              10            (9, 11, 11)
  11. GOAT                    10.25      (12, 8, 9)
  12. Boost Mobile     13.5        (15, 12, 12)
  13. Rhino                    13.75      (14, 14, 13)
  14. High Five              15.75      (11, 18, 23)
  15. Florida United   15.75       (13, 20, 17)
  16. Sub Zero              16            (17, 15, 15)
  17. Madison Club    16.75      (16, 17, 18)
  18. Prairie Fire          17            (18, 16, 16)
  19. Condors               20.25      (20, 19, 22)
  20. Madcow              21.5         (19, 24, 24)
  21. Voodoo               22            (22, 25, 19)
  22. Oakland               22            (23, 21, 21)
  23. Plex                       26            (28, 23, 25)
  24. Furious George                  (NR, 13, 14)
  25. HiP                                          (24, 22, NR)

Once I reached teams that weren’t ranked by all three, priority was given to teams ranked by two out of three. That left us with Furious (didn’t bring a valid roster to Terminus, thus unranked by USAU, also won’t end up in the final rankings) and HiP (not ranked by Ultiworld, probably because they haven’t played an elite tournament).

As you can see, since the computer is given half the weight the differences between the USAU formula and this hybrid approach are not too substantial. No bids change hands. Sub Zero climbs into the top 16, but replaces Madison Club, another North Central team. Florida United and High Five both fall, but not far enough to take a bid away from their region. It’s also somewhat surprising to see how close my rankings are to Ultiworlds. The only teams we seriously disagree (four spots or more difference) about are Sockeye, PoNY, High Five, HiP, Dark or Light, and Voodoo. I’m not sure why Ultiworld has Sockeye so low. They had a strong Terminus with a half roster. I suppose they’re putting more weight into the 8-13 loss to PoNY and thus feel compelled to put Sockeye behind New York. After PoNY and Sockeye the disagreements all come after #16, so they’re not too consequential.

The biggest problem with this hybrid system (if you’re ok with some subjectivity) is that it doesn’t take full advantage of the data given to us by the USAU formula. It treats the USAU rankings as discrete data when they’re really closer to continuous data. That is to say, the difference between Sockeye and Chain Lightning in the formula is miniscule, Sockeye has a 2023 rating and Chain has a 2022 rating. But the hybrid formula treats that difference the same as it does the difference between Chain Lightning and Johnny Bravo (1929) which is considerably more substantial. Another difference that some may say is a weakness that I say is a strength is the different methodologies in ranking teams. The criteria that Ultiworld uses for ranking teams is different than the criteria I use, which is different than the criteria the USAU formula uses. For example, my ranking is a bit more predictive than Ultiworld, which is more reflective. I think it’s best to incorporate both elements, while others might say that it’s inconsistent.

Ultimately this all probably boils down to navel gazing, since there isn’t much pressure on USAU to change their bid allocation system. Frankly the Triple Crown Tour does a fairly good job of encouraging elite teams to play enough games against elite (and select flight) opponents to help the formula get enough good data. But, the factors I mentioned earlier (teams sending small rosters, player injuries, weather issues, suspicious games) all still exist in ways the computer can’t account for. So even though this system or something similar to it is unlikely to ever be adopted, it’s still fun to talk about.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: