2010 USA Ultimate Club Women's Championships

[matthewbourland]

2010 USA Ultimate Club Women's Championships (http://scores.usaultimate.org/scores/#womens/tournament/7516)

This will serve as the official discussion thread for this event. Speculate about who will qualify, talk about the players to watch, hash out seeding, recap the results, and share links to photos/videos from the event.

[ambler]

Here is the field of 16 teams (http://www.usaultimate.org/competition/club_division/club_championships/womens_division_teams.aspx) that have qualified:

Central Region

1 - Nemesis (http://scores.usaultimate.org/scores/#womens/team/2343) (Chicago, IL)
2 - Pop (http://scores.usaultimate.org/scores/#womens/team/8158) (Minneapolis, MN)
3 - Revolution (http://scores.usaultimate.org/scores/#womens/team/11952) (St. Louis, MO)

Mid-Atlantic Region

1 - Scandal (http://scores.usaultimate.org/scores/#womens/team/6198) (Washington, DC)
2 - Phoenix (http://scores.usaultimate.org/scores/#womens/team/11483) (Triangle Area, NC)

Northeast Region

1 - Brute Squad (http://scores.usaultimate.org/scores/#womens/team/2345) (Boston, MA)
2 - Capitals (http://scores.usaultimate.org/scores/#womens/team/4197) (Toronto, OT)
3 - Bent (http://scores.usaultimate.org/scores/#womens/team/9609) (New York, NY)
4 - Storm (http://scores.usaultimate.org/scores/#womens/team/2354) (Montreal, QC)

Northwest Region

1 - Riot (http://scores.usaultimate.org/scores/#womens/team/2202) (Seattle, WA)
2 - Fury (http://scores.usaultimate.org/scores/#womens/team/2201) (San Francisco, CA)
3 - Traffic (http://scores.usaultimate.org/scores/#womens/team/6309) (Vancouver, BC)

South Region

1 - Showdown (http://scores.usaultimate.org/scores/#womens/team/5092) (Austin, TX)
2 - Ozone (http://scores.usaultimate.org/scores/#womens/team/2412) (Atlanta, GA)

Southwest Region

1 - Molly Brown (http://scores.usaultimate.org/scores/#womens/team/11891) (Boulder/Denver, CO)
2 - Safari (http://scores.usaultimate.org/scores/#womens/team/2359) (San Diego, CA)

[ambler]

Seeding input from captains is due this Wednesday.

The Score Reporter comparison tool (http://scores.usaultimate.org/scores/#womens/seeding/7516) has some suggestions, but here is what Riot is thinking about the seeds [* indicates the last time we saw a team play this year, if at all]:

1. Riot
2. Fury - [*NW Regionals, Oct]
3. Brute Squad (wins over Traffic, Phoenix, Nemesis, Showdown; won Philly Fusion, Sep) - [*ECC, Aug]
4. Capitals (wins over Scandal, Showdown) - [*Labor Day as Lotus, Sept]
5. Scandal (wins over Brute Squad, Phoenix, Bent, Showdown, Ozone; won Chesapeake Open, Aug)
6. Traffic (wins over Molly Brown, Bent, and Safari; NW historic strength) - [*NW Regionals, Oct]
7. Phoenix (1-1 with Bent, 2-1 over Showdown, wins over Capitals and Ozone; won Winston Cup Challenge, Aug) - [*Shootout, May]
8. Molly Brown (wins over Nemesis, Pop; won Colorado Cup, Aug) - [*Labor Day, Sep]
9. Nemesis (wins over Bent, Safari) - [*ECC, Aug]
10. Pop (wins over Traffic, Bent, Safari) - [*ECC, Aug]
11. Bent (1-1 with Phoenix and Showdown, 1-2 with Capitals, win over Safari) - [*ECC, Aug]
12. Showdown (1-1 with Scandal and Ozone, 1-2 with Bent and Phoenix; win over Storm) - [*WUCC, Jul]
13. Safari - [*Labor Day, Sep]
14. Ozone
15. Storm
16. RevoLOUtion

I'm sure there are lots of other opinions out there on the order, especially because so many teams have split games. What else do people think?

[michelle.ng]

Thanks for starting this discussion and sharing, Gwen. We're still discussing Showdown's official position, but here are some of my personal thoughts:

1. I am really confused as to what to think about Pop's DGP win over RevoLOUtion at Regionals.
2. I'm as pro-West Coast as they come, but if you use NW historic strength as a metric for seeding, can we justify moving Nemesis and Pop down based on CN historic performance at Nationals? (No disrespect to these teams- I mean this as more of a food for thought type of argument.)
3. I agree with your Top 5 (unsure about Capitals v. Scandal but that doesn't really matter) and your bottom 4. The rest is messy.

Our results are what they are, but I think I would be remiss not to mention that Showdown played part of Chesapeake with 11 players (half of them rookies) because of flight problems and the rest of the tournament with 16 players. We brought 19/24 to Fusion and we played every pre-Series tournament without Katey Forth (Great Britain Worlds team) and Holly Greunke ('07 Callahan Runner-Up). Again, we are responsible for our results, but a Worlds year and having to fly everywhere for competition have been brutal for us. :(

Michelle
Showdown

[angela.lin]

Hey. Thanks, Gwen for starting the discussion, and Michelle for continuing it. Wish more people were talking.

These are my own opinions, and not the official stance of Ozone.

Seems like there are two big things we think about when seeding for nationals. 1. how to rank teams consistently, which is difficult given the tournament-based nature of ultimate. 2. how to create a good draw for your own team (wrt thurs pools and potential fri power or lower pool crossovers) while keeping some semblance of objectivity and consistency so that it doesn't seem biased. I get that 1. allows for some flexibility for 2. And that each team is going to try to make the best situation for themselves.

But as far as "rules," we usually say current series results weigh heaviest w/ earlier results less impactful - giving us room to seed a team higher for nationals than a team they may have lost to earlier in the season - and historic performance is factored in somehow with "gut feeling" types of decisions as well as if a team hasn't attended many tournaments. Yes?

I agree w/ michelle that it's weird consistency-wise to consider NW historic strength while not considering other regions' historic strength or weakness, and it's worth discussing how to best use historic performance when ranking teams in the current series.

The DGP Pop RevoLOUtion thing def makes things a bit weird. good work central. =) But my gut feeling is that that region typically struggles at nats AND ranking pop 10 and rev 16 seems like a huge gap.

Ozone's official seeding vote will likely involve moving Traffic down, Phoenix up, Pop down, Showdown up (wrt what Gwen posted)...

The top is always reasonably clear (unless the reigning national champ [messes] up the seedings by placing 2nd in their region) as is the bottom. And the middle always fuzzier. And whatever, it all works itself out at the end of the month. =)

angela.

[a-yamaguchi-7]

Thread still kicking..

Angela> "1. how to rank teams consistently, which is difficult given the tournament-based nature of ultimate."

This can be a huge challenge, and not just the consistent application of some set of criteria across the field within a year. Weight criteria differently (order of application) and you can sometimes get different results, and those different results are more or less appealing to different teams in different years. So consistency of the criteria weightings from year to year can be part of the community expectations and challenges as well. With National Directors carrying over through multi-year terms, you can hope that this is managed well over the course of years.

Angela> "2. how to create a good draw for your own team (wrt thurs pools and potential fri power or lower pool crossovers) while keeping some semblance of objectivity and consistency so that it doesn't seem biased."

If I read this correctly, I think you just said 'game the system for your personal benefit while trying enough not to appear doing so'. Wow. You speak to not 'seeming' biased, but never speak to actually not BEING biased. Woah.

I think it's very dangerous to assume that what you do, or the way you and those you know approach the situation (in the South or anywhere), is indicative of how everyone else must also approach the situation. I understand that angle shooting is human nature, and that it is naive to believe that there won't always be someone, somewhere trying to work that angle, but your point of view excludes even the possibility that other people might not be quite so corruptible as you've experienced.

Michelle> I think I would be remiss not to mention
Angela> I agree w/ michelle that it's weird consistency-wise

I understand that Showdown had a hard year and has some circumstances that they consider 'special'. Would you agree that it's weird and inconsistent to consider any 'special circumstances' for Showdown without considering 'special circumstances' for all of the other teams in the field?

Does anyone, on any team, actually care about 'special' circumstances for any team but their own?

Did Showdown ask Scandal (best win) or Phoenix (second best win) if there were any 'special' circumstances in their wins, and publicly take them with an asterisk (*)?

I say this not to belittle the challenges Showdown may have faced this year. Qualifying for Worlds is a big deal and required a strong performance at the Club Championships last year. Results at Worlds were quite good (I believe.. haven't looked in a while). This is to be recognized and applauded. That said, it's foolish to believe that other teams have not faced obstacles as well. Every teams owns their challenges, shouldn't have to apologize for their advantages, and the totality of those experiences reflects in the records produced on the field.

-----

It seems like there may be a lack of understanding of how things _can_ work. This year is my first direct experience with the Women's Division, but here's what I've seen or learned from/about National Directors in Open and Masters. These folks talk to each other, so it wouldn't surprise me if this process is roughly the same across USA Ultimate competitions..

Michelle> "We're still discussing Showdown's official position"
Angela> "Ozone's official seeding vote "

You don't get a vote. It's not a democracy, or at least, it shouldn't be run that way. You get to provide your point of view. The overarching goal is fairness, both for what teams have done and for the competition to come. I believe that's how the NDs try to work the seeding process. At the end of the day, each ND is responsible for their division, and I believe they can go and ignore all the teams and create their own seeding if they wish. They own the consequences of such action of course.

The National Director has to set certain goals for the process and balance tradeoffs. For example, should seedings be reflective of what teams have 'earned' across some time period, or are seedings intended to be 'predictive' somehow of what will happen? In some of the other divisions at least, I believe 'predictive' is not part of the process.

If the ND is smart, she will go through the seeding exercise herself based on the weighted criteria she believes are most relevant to achieving her process goals. Hopefully those weighted criteria are the same as prior years and will be the same in future years. This provides consistency of expectation, limits the capability to game the system, and indicates for the future how teams should approach their seasons if seeding is a priority for them. There should be a way for everyone to agree seeding criteria and weighting for the future at least, given that the future is largely unknown and teams should have less stake to game things.

Having her own seeding concept, the ND can then compile the team seeding feedback. Hopefully there is a consensus that matches her concept. Otherwise, hopefully there is a consensus in the field at all.

Divergences from the consensus and divergences between the ND's concepts and the field's consensus can be illuminative, especially if decision logics are provided. Divergences indicate different criteria, different weightings of criteria, and/or attempts to game the system. Teams also sometimes get stuck only focusing on the teams directly above and below them in the seeding process, so matching feedback to sources can be helpful. The ND as a thinking human can process and filter to throw out the outlying criteria/weights/seedings.

It's just a feeling, but I think that in most years the process is actually pretty straightforward. The results across the draw and among teams line up and pretty much speak for themselves. The number of outlier wins and losses is small and gets overwhelmed by the total record and how teams performed against similar level teams. Consensus exists in the field and matches up well with the ND's seedings. Random arguments get auto-circular-filed.

Managing seeding via a process like this keeps things consistent and eliminates the need to try to compare 'fuzzy', qualitative attributes like 'special circumstances' and margins of victory vs. common opponents and such. Parcells' 'you are what your record says you are' speaks, essentially.


I went through this seeding exercise for the Women's Division myself the other day, and I think this is one of those straightforward years. My impression was that there were enough inter-regional and head-to-head results across the field that things mostly snapped into place. Maybe that's why this discussion thread is so quiet? Or maybe all the good talk is on r.s.d..

My approach when seeding a field, applied in the past as a TD at Club Sectionals, Club Regionals, CHCs, and so on, is to make rough groups of teams using overall record vs. the field (primarily) and/or RRI (secondary) to find the breakpoints. Thus, if you held a single outlier win over Riot this year, but had a mixed result up and down the field (where Riot has only lost to Fury otherwise), any argument to be seeded #1 overall gets filtered out. The 'quality win' over Riot can help you later against comparable teams in your group, but my first focus is on overall results and ignores cherrypicking of specific wins.

Then, within each group I consider the distribution of wins and losses within the group (primarily), results against common opponents (secondary), and then the balance of 'quality wins' against higher seeded teams and 'bad losses' to lower seeded teams (tertiary).

Series events have the added constraint of holding to prior Series event results.

The Championships seedings matrix is very helpful for this sort of task:
http://scores.usaultimate.org/scores/#womens/seeding/7516

Based on what I've described, you can probably replicate and discover my view. Does the DeAnna and/or the consensus of the field agree? We will find out..

[michelle.ng]

I think that EVERY team's circumstances should be taken into account. That is exactly why I put Showdown's circumstances on the table-- we cannot and do not expect other teams / Deanna to to know what we do not tell them. For example, if Riot loses a game to Team X and that is because Gwen, Surge, and Duffy weren't there, I absolutely want to know that when I'm doing my seedings. It's not a fact that them being there would have changed the outcome of the game, but I think it's important and pertinent information to consider when weighing the importance of a certain result. So yes, I DO care about special circumstances for teams other than my own. When I went through and made notes for myself about results for our seeding input, I wrote things like "Hoods and Walker were not there for Caps at Fusion" and "Riot was missing RV, Maddy, and Smalls at Sectionals." I think that every team is free to decide how much to incorporate this knowledge of outside circumstances... but in order for there to be that choice, we need to put the information out there. At the end of the day, our results are still our results-- we beat the teams we beat and we lost to the teams we lost to.

With regard to seeding, I don't know that it's ever "easy"-- sure, I think part of us saying it's not easy is because we're biased and have our own teams' interests in mind. But I've seeded college tournaments where my role has solely been organizer (e.g. Women's College Centex), and everyone always thinks their position is the "obvious" answer. It's pretty entertaining to see the wide range of ways that people think the seeding should go down. And I'd be lying if I said that I'd ever released the seeds and felt 100% happy with them... Even when it's "obvious," there are always results that muddle things. I think the purpose of discussions like this one is to give the people with input (team leaders) and the person doing the seeding (Deanna) as much information as possible. At the end of the day, Deanna will filter through what she deems important / relevant and probably tell all of us that we're severely biased and half-crazy. :)

I think both Angela and I were clarifying that what we posted is not necessarily representative of what our teams think. We don't want people going off and complaining about how crazy / dumb Showdown and Ozone are. You can just straight up blame us. :)

Michelle
Showdown

[kyle.weisbrod]

a-yamaguchi - Kudos on a smart, yet sassy post.

Michelle and Angela - you both raised questions about why Gwen would use historical information for the NW but not use it for the Central teams. Having gone through the exercise and come up with a very similar seeding myself my guess is that Gwen found herself having to make a decision between a handful of teams in a group with very limited (or conflicting) current season results. So, in that case she defaulted to historical performance as a last case tie-breaker.

For the Central teams, it appears that in the cases where historical performance might impact their seeding, the current season results are more clear cut.

That said I'd change a few things in Gwen's rankings:

7. Molly Brown - (8 in Gwen's ranking). While their RACO is similar, Phoenix' has some notable losses (specifically to Bent and Hot Metal). Molly Brown also has the HTH win over Nemesis.
8. Nemesis - (9 in Gwen's Ranking). Nemesis has a better RACO over Phoenix. Notably closer games against BS and Fury and 1-0 against Bent (while Phoenix is 1-1).
9. Phoenix - (7 in Gwen's Ranking). See 7 and 8 above.

Kyle
Affiliated w/ Ozone

[michelle.ng]

Kyle (and I guess tangentially Gwen)-- agreed about historical strength.

Here's a small example of a not-so-clear seeding situation (at least in my opinion):
We have Bent-Pop-Showdown. How do you seed those three teams? Bent DNQ last year, Pop finished 15th, Showdown finished 5th. Bent has beaten Showdown twice this season, Pop has beaten Bent, and there are no HTH results between Pop and Showdown. Did Bent bring a full roster to ECC? (Wish I had info about their circumstances). Bent came in 3rd but has the strongest region. Showdown took care of business in the South. Pop came in 2nd in the Central, but didn't have close games with Nemesis and almost lost to RevoLOUtion. Is looking at historical strength appropriate here? NE3 v. SO1 v. CN2 is pretty clear looking at past results, but perhaps that is not relevant. And hey, Pop has this win over Traffic too. (Was Traffic's roster set at ECC?) And then you have that confusing Showdown win over Scandal... (Was Scandal missing people?) Ahhh!

Akira-- I also want to clarify that I mean absolutely no disrespect to Nemesis or to the CN Region. I was just trying to put forth the argument that perhaps, as in the example above, it might be appropriate to look at historical strength. Last year, I wrote in the USAU preview that it was going to be the year the CN had one (or two) teams in the Quarters. Sadly I was wrong. I think Nemesis is a great team. No one can stop Candice's break throws, Erin is a huge pickup, and Court is one of my favorite players ever.

Ok, while we're on the topic of Bella Donna alums, Showdown is 17-1 with Holly Greunke in uniform... glad she'll be with us for Nationals. ;)

Michelle

[angela.lin]

a-yamaguchi:
I totally agree that there is math/logic that goes into seeding decisions. But if it was *all* math, we wouldn't have to discuss. An unbiased someone (the ND) would just use the algorithm and that would be it. I've also run tournaments and seeded fields of womens teams, some of whom I'd never seen play. For as much data as you can gather, input from the captains and players is still crucial. However, there is no person in any way affiliated w/ a women's team who can claim total non-bias. If you do, you should check your self awareness meters. I don't think it's "corruptible," I think it's "human nature" (as you said).

I appreciate your post, and appreciate the attention to the tools we have at our disposal with which to impart as much consistency/accuracy as possible. I'm a scientist, and I'm all for fairness and quantifiable methods to sort data and come to conclusions. There are many other factors, and perhaps they don't matter to everyone or even matter in the grand scheme of determining a champion.

a.

[a-yamaguchi-7]

Thanks Kyle!

Seeding notes are due to National Directors anytime now, right? So we can talk in a more academic sense about seeding and process..

Molly Brown, Nemesis, Phoenix, and Traffic.. the most interesting part of the deck. I have them:

6. Traffic
7. Phoenix
8. Molly Brown
9. Nemesis

I'll put my notes and reasoning at the bottom for anyone interested.

More importantly, I want to make a note that I think a good ND, RC, TD, etc. should publish all seedings (preliminary for feedback or otherwise) along with the more important reasoning for why seeding went down a certain way. Any last-minute arguments thereafter should apply to the reasoning and not the seeding, and any changes to the reasoning (should be ultra-rare) should apply to the whole field of teams.

Reasoning applied should be consistent from past years and into future years. In fact, one approach to ensure participation in the community interest would be to post seeding and reasoning but only allow changes to the reasoning for future years, leaving the current year's reasoning (assumed consistent with past years) and seeding intact.

Ok, on to the interesting seedings.. as I discuss above I use a grouping concept based on full season results against the Championships field, trying to account for logical groupings through inter-team results and sometimes (rarely) RRI. My top two groups were:

Group 1: Riot and Fury (records vs. Championship field head and shoulders above everyone else)
Group 2: Brute Squad, Capitals, and Scandal (records vs. Championship field a cut above the rest, plus clean round-robin winning records against each other)

For me, Group 3 was the most interesting.. here are my notes:

Group 3: Molly Brown, Nemesis, Phoenix, and Traffic

Why here:
Molly Brown is 4-3 against the Championship field with an in-group win over Nemesis.
Nemesis is 7-3 against the Championship field with no 'bad' out-of-group losses.
Phoenix is just 6-9 against the Championship field but holds wins over Group 2 Capitals and Scandal.
Traffic is just 3-8 against the Championship field but holds an in-group win over Molly Brown.

Why not higher:
Molly Brown has a loss to Group 1 (Fury), a loss in-group to Traffic, and a 'bad' out-of-group loss to Safari.
Nemesis is 0-2 against Group 1 (Fury) and Group 2 (Brute Squad), and lost in-group to Molly Brown.
Phoenix is 2-7 against Group 1 (Riot, Fury) and Group 2 (Brute Squad, Capitals, Scandal) with 'bad' out-of-group losses to Bent and Showdown.
Traffic is 0-7 against Group 1 (Riot, Fury) and Group 2 (Brute Squad), and has a 'bad' out-of-group loss to Pop.

Seeding:
Traffic is 1-0 against Molly Brown, who is 1-0 against Nemesis. The only question is where Phoenix fits.

Common opponent between Traffic and Phoenix tilts towards Traffic. Both have many losses to Riot, Fury, and Brute Squad, but Traffic is 1-0 against Bent where Phoenix is 1-1.

Overall record is of limited help. Phoenix has wins over Group 2 Capitals and Scandal, but that is balanced by more/worse 'bad' losses to Bent and Showdown compared to Traffic's 'bad' loss to Pop. Pop is 1-0 against Bent who is 2-1 against Showdown, and Phoenix has a loss to each of the latter two.

Traffic gets the nod. Maybe it's close-ish. (I think this is the closest and maybe only judgement call in the stack.)

After Traffic, Phoenix deserves consideration for their two 'higher' wins over Group 2 Capitals and Scandal, something Molly Brown and Nemesis cannot boast. (How much weight to give Phoenix's 'quality wins' here is maybe another call. Since DeAnna Ball de-emphasized RRI I tried to avoid employing such metrics.)

Molly Brown holds head-to-head over Nemesis.

#6 Traffic
#7 Phoenix
#8 Molly Brown
#9 Nemesis

-akira
(Coaching.. can you guess?)

Kyle- What is RACO?

[matthewbourland]

I'm guessing that stands for Record Against Common Opponents?

[kyle.weisbrod]

Akira, because Molly Brown went on to beat Safari at Regionals, I have discounted their pre-series loss to them. I'm guessing that's why our seedings are different in that middle area.

[a-yamaguchi-7]

Michelle> We have Bent-Pop-Showdown.

Understanding that we're taking completely different approaches, I didn't find this to be very complicated. In my world I imagine an ND who wants simplicity and consistency and says something like:

- Here's how it's going to be (criteria).
- How it is, that's how it's going to be going forward.
- Win your games.

I'm rushing a bit here, but my criteria looked something like (in order):

- Use only results vs. the Championship field (point worth debating perhaps.. but controlling complexity)
- Grouping / breakpoints by overall record and inter-team results (any disagreement? there is some magic here perhaps.)
- Inter-team results within the group (any disagreement?)
- Common opponents within the group (disagreement?)
- Balance of unrelated 'quality wins' over higher group teams and 'bad losses' against lower group teams

In that view, here are my notes..

Group 4: Bent, Pop, Showdown

Why not higher:
Bent is 2-11 against Groups 1, 2, and 3 and has an in-group loss to Pop.
Pop is 1-7 against Groups 1, 2, and 3.
Showdown is 2-9 against Groups 1, 2, and 3 and is 1-2 in-group against Bent.

Why here:
Bent is 2-0 against Group 5 teams (Safari, Storm) and 2-1 in-group against Showdown.
Pop is 3-0 against Group 5 teams (Revoloution, Safari) and 1-0 in-group against Bent.
Showdown is 3-1 against Group 5 teams (Ozone, Storm).

Seeding: Pop is 1-0 against Bent, who is 2-1 against Showdown.

#10 Pop
#11 Bent
#12 Showdown


To expand the thinking for those painfully interested...

I don't know if using non-Championship field results would affect things any. Most teams seem to play enough these days that their sample size gravitates around some theoretical expected performance level. Paring seeding results to just the Championships field provides incentives to go play good competition and also keeps things interesting as far as not knowing exactly which results will matter. You can't take your foot off the pedal against any potential high-Regionals finisher (a characteristic, I believe, of better teams). I suppose you could employ full season results for seeding as a backup plan in the event fuel costs exploded or something and inter-region play fell way (undefined) off.

There doesn't seem to be an argument against the grouping, at least not that I've seen. I think the grouping is sound because the records are similar against similar teams, both groups above (the ceiling) and the group below (the floor).

Within the group records interrelate in an ordered way (A over B over C).

I never got to common opponent for these teams. I can't see common opponent factoring over H-to-H. Can you?

You would have to make some sort of qualitative parsing of the balance of unrelated 'quality wins' and 'bad losses', ahead of the other criteria, to maybe end up with a different ordering? Qualitative judgements are more at-risk to subjectivity (opinion?) which is why I put them at the bottom. Of course, this is what RRI tries to do in a quantitative way, but I assumed that was out as a key criteria per the ND. It would be interesting to hear from the man behind the RRI to get his sense about sample sizes and so on before RRI converges enough with conventional views of quality and rank.

With teams playing so many games against so many opponents from all over, I didn't even consider past years or historical regional strength at all.

Another simple, clear, consistent system would be to take the prior year's finishes and hold the finishing slots by region.. so a Riot-Fury final would be NW1, NW2, and so on.. then plug in the next year's Regional finishers into the format in that order, with adjustments only for changing wildcards. There is a certain 'fairness' and clarity to that method as well.

[a-yamaguchi-7]

I'm guessing that stands for Record Against Common Opponents?

That sounds right. Good call, thanks. Learn something new every day..


Akira, because Molly Brown went on to beat Safari at Regionals, I have discounted their pre-series loss to them. I'm guessing that's why our seedings are different in that middle area.

We both have Traffic > Phoenix. We also both have Molly Brown > Nemesis by H-to-H.

My determinant was that both Phoenix and Molly Brown have 'bad losses' to team and groups below, and we can definitely play 'who's bad loss(es) were worse', but I didn't think it mattered. Phoenix had 'quality wins' to teams above as well, which neither Molly Brown nor Nemesis could claim.

[a-yamaguchi-7]

a-yamaguchi:
I totally agree that there is math/logic that goes into seeding decisions. But if it was *all* math, we wouldn't have to discuss.


As is obvious from this thread, I believe the discussion should revolve around what the right criteria should be, the ordering or weighting of said criteria, or how one simple, clear, consistent, reproducible, (and ideally) universally understood method for seeding compares to another, rather than trying to bump or lower certain results from certain games in a certain season.

In a sense, when you focus on building and improving systems, you are investing to make things easier in the future for the community at large, and those saved community energies can go to other things like starting teams, launching tournaments, outreach, coaching, or even sitting on the couch having a beer.



For as much data as you can gather, input from the captains and players is still crucial.


The value (to me) of polling the field of players and captains about seeding is to see if or how they apply their methods. The interesting stuff comes in the differences, and can serve to test your method and results. Edit: It's validation.



An unbiased someone (the ND) would just use the algorithm and that would be it.


I'm afraid you cannot depend on the ND not being a competitor, actually..

My argument with you is about this:


2. how to create a good draw for your own team (wrt thurs pools and potential fri power or lower pool crossovers) while keeping some semblance of objectivity and consistency so that it doesn't seem biased."


My paraphrase of this as 'game the system for your personal benefit while trying enough not to appear doing so' was not charitable, and I appreciate the complexities of diction, but the only element of fairness that this statement addresses is the APPEARANCE of objectivity, to be seen that way. There's no address to the actual state of being without bias, or even the attempt.

My claim is not that I am perfect, a robot, or without any bias. My claim is that I can produce a manageably simple, clear, reproducible, consistently applicable method for seeding that can be rigorously applied today and in the future. I'm sure I'm not the only one, and the larger community can doubtless improve. Such a system should be usable by completely different people to produce very consistently similar results, and the system should not require changing into a future where the results may be better or worse for your team, my team, or any team.

I believe that's the way to go, here and forward...

[ambler]

It looks like my proposed seeding is the same as Akira's which relied heavily on this season's results.

Every year the issue comes up of what carries the most weight in seeding. The Club Series Guidelines (http://www.usaultimate.org/competition/club_division/club_series/series_guidelines.aspx#seed) state that:

"Seeding for Sectionals and Regionals and the UPA Championships will be done by the coordinator using the following information: input solicited from all the captains of participating teams; results prior to the Series (e.g. head to head, common opponents, tournament finish); results of last year's Series; and other applicable information (e.g. team composition, conditions). The coordinator has the authority to adjust rankings according to the best information available."

There is no hierarchy listed for the different types of information that go into the seeding, so it makes sense that different teams are more likely to weight things differently based on the teams they saw in a given season, what they know about the teams' composition, how much they value historical results, etc. There's just nothing in the guidelines stating otherwise. Even the tradition of having seedings be "reflective" rather than "predictive" is not explicitly stated in the guidelines. Obviously the National Director (DeAnna) has the final word and has done a great job over the years of weighing all the information, but it often rarely seems like a cut and dry issue.

In my mind, the trickiest teams to seed are Phoenix, Traffic, and Showdown.

As a new team this year, Phoenix doesn't have direct results from last year to add to the discussion, although the team's core is from Backhoe which finished 7th last year. Ignoring the Texas Shootout results (that was in early May, after all), Phoenix has not played against any of the west coast teams which makes it harder to peg them against each other. Wins over the Capitals and Scandal are big and indicate a very strong team. But there is that loss to Bent (which the NC women since avenged).

At ECC and Labor Day, for the most part Traffic lost to teams seeded above it and beat teams below. The notable games are a DGP victory over Molly Brown and a close loss to Pop. Traffic's wins over Zeitgeist (at ECC and Regionals) are also an indication of the team's strength as zG has had close games with Riot and Brute Squad and has beaten other Nationals attendees like Molly Brown, Pop, Safari, and Nemesis. I could easily see Traffic being below Phoenix as an alternative ranking. But I think both of them should be above Molly Brown.

After finishing 5th at Nationals last year, Showdown had a very up and down season in 2010. It seems they never really played with their full roster, so perhaps the team's losses to Ozone and Bent can be attributed to being short-handed (especially as Showdown has beaten each team as well), but I don't think the losses get to disappear. Their DGP win over Scandal is a quality win over a top 8 team, but it does seem to be a bit of an anomaly even when looking at the scores Showdown posted from that tournament (Philly Fusion). I didn't like putting them all the way down at 12 because I know they have a ton of talent, but I just couldn't see putting them ahead of Bent when Bent has won their last two match-ups.

[hacklindrun923]

just food for thought....

When was the last time two Canadian teams made quarters at nationals in the women's division? It has always seemed to me that when Vancouver is strong, Ontario is weak. When Ontario is strong, Vancouver is relatively weak. I am not saying this will always hold true, but it has probably held true in recent history (since 2002?) for probably a reason. Maybe we could use that to seed Traffic appropriately given the lack of other results.

Sucks for whoever gets Showdown in their pool as the third, possibly fourth seed. Good teams/programs know how to win when it is time to win (and good teams know when it is time to win.....). Again, maybe taking into consideration previous Nationals results for this case may be appropriate.

[angela.lin]

As is obvious from this thread, I believe the discussion should revolve around what the right criteria should be, the ordering or weighting of said criteria, or how one simple, clear, consistent, reproducible, (and ideally) universally understood method for seeding compares to another, rather than trying to bump or lower certain results from certain games in a certain season.

In a sense, when you focus on building and improving systems, you are investing to make things easier in the future for the community at large, and those saved community energies can go to other things like starting teams, launching tournaments, outreach, coaching, or even sitting on the couch having a beer.

<etc>


I think you're smart, I like your ideas, and it's def hard to argue with your recent-results-based seeding determinations. and like I already said, I'm in favor of working towards a system that fairly and consistently produces seedings. I acknowedge that this means choosing certain metrics to be weighed more heavily than others and certain wins to be more valuable than others. I'd also like for you to be open to the possibility that someone may weigh the parameters differently than you do, and his/her method can be equally valid even if it involves a bit of "feeling" vs pure results. Which I think you do, though you come across a bit like, "clearly my way is the best."

It's clear that you've given it a lot of thought and articulate your reasons well. I find a lot of value in discussing methodology with you, so I appreciate that you've taken the time to give lengthy responses.

I like this in a lot of ways (copying/pasting from one of your other posts):
- Use only results vs. the Championship field (point worth debating perhaps.. but controlling complexity)
- Grouping / breakpoints by overall record and inter-team results (any disagreement? there is some magic here perhaps.)
- Inter-team results within the group (any disagreement?)
- Common opponents within the group (disagreement?)
- Balance of unrelated 'quality wins' over higher group teams and 'bad losses' against lower group teams

With the first I agree there's worth in debating. With the second, I think the "magic" can be related to something like the last bullet and historic performance, so I think there's some grey. But simplifying everything to relevant results is def one way to go w/ seeding, and I respect its merits.

The method I use is def less strict, but I also create groupings and look at relative results (incl W-L records H-H, against same opponents, other significant games, and also scores of signif games). And I use some general thoughts around regional and historic nationals exp to fill in where there's possible confusion. yeah, it's mostly this last part that isn't as quantitatively reproducible and leaves room for some different interpretations. Some would argue that that's what makes this process interesting.

I agree w/ Gwen that the interesting, more challenging teams to seed are Phoenix, Traffic, Showdown, and I'd add to that the whole Pop - RevoLOUtion issue. We run into the problems of "it's hard to rank this team so low, but given the results there arent really any options," as I paraphrase Gwen's thoughts on Showdown at the 12 seed. Or, in my opinion, Pop w/ a DGP win over the clear 16 seed but quality wins over Traffic and best record w/in-group makes for a logically sound but somewhat confusing placement at 10. And I think having played Phoenix this year and Traffic in years past (tho less relevant I know), I was leaning towards Phoenix ahead of Traffic.

My argument with your argument with me is that you assumed I didn't have overall fairness in mind when I said teams try to create the best draw for themselves... If you knew me, you'd know that I'm one of the most fair players in ultimate. Of course I wouldn't condone a seeding suggestion that was grossly unfair or created one seemingly easier pool and three harder ones in favor of my own team. Come on. And of course everyone looks at what their pool matchups will be and considers them within the confines of the relatively loose rules governing the process. The seeding guidelines accommodate a bit of this. There's some leeway in each team's opinion on what is equitable and accurate. Teams typically (or at least ours does) provide notes and justifications for their seeding suggestions, and I assume no one clearly disregards results in favor of a purely opinion based seeding scheme that makes their route to the top easiest.

Maybe I came across as stepping along some slippery slope. But I feel like I was just calling everyone out on existing (and perhaps subconscious) bias. I think pure objectivity by a captain or coach is impossible. And fair seeding and even pools can be an eye of the beholder kind of thing anyway - one team's easy money can be another team's nightmare. Maybe I'm wrong about teams checking out their thurs/fri pools, looking to see if their initial seeding ideas create desirable pool matchups, but I don't think I am. Mostly I'm sort of offended that you immediately assumed the worst path I could mean regarding fairness and legitimacy.

Yes. I realize that I forgot DeAnna is playing. My bad, DeAnna. But my point still stands. If a purely mathematical method was implemented, or even if it was like the BCS, anyone (biased or not) or even a chimp could input everything and come up with the "correct" seedings. Maybe that's what we want to move towards. I wouldn't be opposed, as it does work towards less effort in the future. But I currently like that the human opinion gut factor is allowable, and yes I do like feeling like we get to exert another small bit of control in a world largely filled with uncontrollables.

It would be interesting I think to retro-apply your methods to years past and see how that matches up with actual seedings by the ND as well as final standings, which could maybe elucidate the reflective vs predictive question. Have you tried this? Or, if we do the same with a method that quantitatively factors in strength of schedule and historic performance, see how well that matches. I've strangely never looked at my seedings or the official seedings and compared them to final results from any year. weird I guess.

a.

[kyle.weisbrod]

That sounds right. Good call, thanks. Learn something new every day..



We both have Traffic > Phoenix. We also both have Molly Brown > Nemesis by H-to-H.

My determinant was that both Phoenix and Molly Brown have 'bad losses' to team and groups below, and we can definitely play 'who's bad loss(es) were worse', but I didn't think it mattered. Phoenix had 'quality wins' to teams above as well, which neither Molly Brown nor Nemesis could claim.

But according to you Molly Brown's only "bad loss" was to Safari. Considering that they went on to beat them at Regionals (where it actually matters) they've (effectively) eliminated that loss. So (at least how I look at it) Nemesis and Molly Brown have no bad "out of group" losses, while Phoenix still has two.

Granted, Phoenix has two out of group wins, but both were against teams that neither Molly Brown nor Nemesis played during the season. For Nationals teams that the three teams did play their scores were:

Phoenix - Losses: Fury 13-5, Bent 13-10, Brute Squad 15-3. Wins: Bent 6-15
Molly Brown - Fury 15-11
Nemesis: Losses: Fury 15-11, Brute Squad 10-8. Wins: Bent 7-15.

Molly Brown and Nemesis have both performed better against common nationals opponents than Phoenix this season. Phoenix were blown out by Fury and BS, while Molly Brown and Nemesis have played them significantly closer, and Nemesis beat Bent in their only meeting while Phoenix split games with Bent.

Moreover, both Molly Brown (2954) and Nemesis (2976) have better RRI's than Phoenix (2933).

Frankly, it looks to me that Phoenix's wins over your group 2 teams (which Molly Brown and Nemesis didn't play) are the only thing keeping them in Group 3 at all. Given that all other results comparing those teams is against Phoenix (common opponents and RRI), I believe that the Traffic>Molly Brown>Nemesis>Phoenix order makes the most sense.

But, I can understand how you would prefer Molly Brown in your pool over Phoenix...

[berklaur]

Just to add my 2 cents after skimming this thread...

I just do not see why Traffic is seeded as high as they are. Even the historical argument doesn`t seem to make sense since they ended up 9th last year. They only beat Molly Brown by 1 point, and the same can be said for their game against Safari (both of which are relatively recent games). The only quality win they have over a championship team is Bent.

I guess my suggestion would be:

6. Phoenix
7. Molly Brown
8. Nemesis
9. Pop
10. Traffic

[a-yamaguchi-7]

I think you're smart, I like your ideas, and it's def hard to argue with your recent-results-based seeding determinations.

Hi Angela- Thank you. I likewise appreciate the discussion and give-and-take as well, especially in a thread that was a ghost town earlier this week.

I'll speak to a couple comments from yourself and Kyle here, and I think there also room for me to try to prioritize, clarify, and tighten up my points.

One point I want to make first- I don't think you're a bad person or anything like that. Far from it, actually. This discussion has been helpful and illuminative to me, and would not have been so without your participation and the participation of others. By reputation you're a great player and even better person, and I don't feel differently now in this thread.

I think I earlier granted that my reading of your first post may have been less than charitable. I don't think I took any liberties with your content though:


2. how to create a good draw for your own team (wrt thurs pools and potential fri power or lower pool crossovers) while keeping some semblance of objectivity and consistency so that it doesn't seem biased."

As I read this, I take semblance to mean some appearance or substance. And I read seem to be an appearance or impression, the state of looking a certain way, independent from actually being or even attempting to be a certain way.

So yes, I object that the abstract team necessarily approaches seeding as an opportunity to position themselves and that a/the foremost concern is figuring out how to do that while looking clean enough somehow to anyone observing.

My earlier point about charity was trying to grant that your verbiage as written may not have been the intended meaning of the point or post, which if so, is fine. We all do it, and grimace afterwards. I just didn't have anything else to go off of, this thread containing three posts or such at that time.


My A-#1 desire in all of this is to contribute to the debate in a way that may someday lead to getting rid of seeding discussions entirely. I think they are a waste of community energy (people disagree of course :)) Thus my desire for some mechanistic process with limited-to-no ambiguity or questions along the way. Somewhere further down I'll post what I think may be an even better alternative than the process described so far.

There's a certain justice in sport and life to 'getting what you deserve'. It doesn't always happen. There are many huge factors introducing variance on a scale way beyond our lifetimes, and scores often deviate from the run of play and quality. You can do everything right and still get beat (though I would argue that you can then hold your head high, having done all possible). Etc.

I don't believe seeding is one of those areas in Ultimate where a skill premium should apply. I don't believe the community best interests are furthered by the practice of everyone's lawyering, lobbying, or oratory chops, to move the mass to support the particular flavors of criteria most beneficial to them for the current year. I don't directly blame anyone who engages in these kinds of activities. I've done it. But I generally find them unseemly, unpleasant, and a waste of the participant's life forces.

Mostly I blame the system. The system as constructed opens the possibility that such efforts can make a difference and invites those kinds of activities.

I recognize that other folks can, and likely do weigh criteria differently. Also, by nature, some people are more driven by feelings and perspectives than data (a Myers-Briggs F vs. T). I have difficulty coming up with systems that include many feelings, because I do not think feelings can be fairly and consistently compared (comparison being something I believe seeding as we do requires).

Who's pain is worse? Someone who endured a lifetime of slavery? Or someone who's entire family and bloodlines were murdered in the 90s in Rwanda?

What about a person who's immediate family was wiped out vs. someone who's immediate family survived, but who personnally suffered the traumas of violence and rape?

This is a very extreme case with no comparisons to ranking teams for a recreational sports tournament, but I think it can help illustrates how impossible and absurd comparing feelings can be. No one's pain is 'worse' or 'better'. The feelings are all valid. The person holding the pain suffers the most for their own experience, and it's nutty to even ask them to weigh their own vs. another's.

I think this is part of the basis of 'human nature' as we've discussed and certain very difficult to avoid and filter 'biases'. Every team and player takes an event and experiences it in a very personal and often unique way.

So I think my view of seeding is 'best'. Per the certain key criteria for the system, my way, your way, Kyle's way, or another, I care far less.


One comment I should have made about specific criteria, it's possible to take criteria and split them and order them apart.. for example, rather than taking the balance of quality wins and losses as some catch-all last resort, USA Ultimate or an ND could weight 'quality wins', then 'Head-to-Head within the group, THEN 'quality losses'. Criteria can be stacked and ordered to incent or disincent certain community goals.

I neglected to articulate that I pretty much looked only at wins and losses and not the timeline or scores. That should be obvious, and it was me simplifying to 'win your games'. There may be methods and value to decaying results somehow or accounting for margin of victory or loss, although the method matters and the value needs to be weighed against the effort or additional complexity. That's a factor in your Pop point.. Revoloution was not part of Pop's grouping so had no bearing on my in-group results, and didn't factor into 'common opponent'. I have to review my notes to expound further, but when I listed my criteria, they were intended as an ordered list of application.

Also, I took the 'reflective' (Gwen's excellent word choice) vs. 'predictive' elements from past conversations I had with other NDs. I have not backtested any system.. but given (at least certain) divisions lean towards 'reflective', the intention of seeding isn't to predict the results anyways. I usually look at all the result boards after Day 1 to see what (by seed) upsets there were... and I know NDs look at that too, but that can't be much judge of the seedings given the 'reflective' tradition, and I've heard NDs say exactly that while studying the board.

-akira

(Who completely unrelatedly has a prop. bet on regarding Women's CN3 and finish below (or not) all other teams in the final standings..)

[a-yamaguchi-7]

But according to you Molly Brown's only "bad loss" was to Safari. Considering that they went on to beat them at Regionals (where it actually matters) they've (effectively) eliminated that loss. So (at least how I look at it) Nemesis and Molly Brown have no bad "out of group" losses, while Phoenix still has two.

My grouping was Molly Brown, Nemesis, Phoenix, and Traffic. You can see why above under 'Why here' and 'Why not higher'. The grouping does not seem to be an area of contention between us.

Next for me is in-group results. Traffic > Molly Brown > Nemesis. Phoenix has no results in group so we don't know where to put them yet. We have to compare vs. the part we know.

Starting at the top, I decided to see how Phoenix and Traffic compared. We both gave the nod to Traffic, so I'll leave that to prior work.

So then Molly Brown vs. Phoenix. Remember that we grouped them together and it made sense. We tried to fit Phoenix in with Molly Brown and the other group teams by in-group results, but there aren't any. For me, common opponent (Bent, Traffic 1-0, Phoenix 1-1) decided place #6 to Traffic.

Place #7 is between Molly Brown and Phoenix. Again, grouping was fine. There are no in-group results.

Common opponent lends Fury (Group 1, 'quality loss', used for grouping but not applied by me otherwise), who both teams lost to. No help at all (I am not considering margins of victory or any such).

So that leaves the balance of 'quality wins' and 'bad losses' Before breaking the teams down, I'll explain the whole set of my terms, some prior unsaid:

'Quality win': Win over a higher group team which can be considered among the balance of 'quality wins' and 'bad losses'.
'Quality loss': Loss to a higher group team, used for grouping but not used as a strike against you.
'Bad win': Win over a lower group team, used for grouping but not used as a point for you.
'Bad loss': Loss to a lower group team which can be considered among the balance of 'quality wins' and 'bad losses'.

'Quality wins' and 'Bad losses' are differentiators. They provide credentials for being high and strikes towards being low. For example, Phoenix should get some credit at some stage for their win over Scandal even though the H-to-H record is 1-4. I believe this is a good thing as an incentive to play tough competition. 'Quality losses' and 'bad wins' are the basis for grouping teams and are accounted for there (and thus do not get re-used later). In the Phoenix case, Phoenix has no argument to be seeded above Scandal despite the one win. The four 'quality losses' to Scandal put a ceiling on Phoenix during the grouping stage.

Also (and the key difference perhaps), I don't erase anything. With me, you can't avenge an earlier loss to a bad team later. Winning later helps of course, but why would the first loss go away? What were you doing that day? Why the face? (Remember.. philosophically 'win your games'). Correspondingly you don't get hurt very much losing to better teams. Playing up should be incented. You can't be seeded above the teams you lost to earlier in the season, but it shouldn't hurt you in the comparables against other teams who've done no different. My differentiators are overachieving wins and underachieving losses.


So Molly Brown and Phoenix then, considering the balance of 'quality wins' and 'bad losses'":

Molly Brown
Fury (Group 1, 'quality loss', considered as part of 'common opponent', irrelevant here)
Traffic (same group, considered as part of in-group result, irrelevant here)
Nemesis (same group, considered as part of in-group result, irrelevant here)
Pop (lower group, 'bad win', irrelevant here)
Safari (lower group, 'bad loss', highly relevant here)


Phoenix
Fury (Group 1, considered as part of 'common opponent', irrelevant here)
Riot (Group 1, 'quality loss', irrelevant here)
Brute Squad (Group 2, 'quality loss', irrelevant here)
Capitals (Group 2, 'quality win', highly relevant here)
Scandal (Group 2, 'quality win', highly relevant here)
Bent (lower group, 'bad loss', highly relevant here)
Showdown (lower group, 'bad loss', highly relevant here)
Ozone (lower group, 'bad win', irrelevant here)

We can compare 'quality wins', 'bad losses', or both, and I put Phoenix ahead of Molly Brown.

Wins: Molly Brown has none to be considered here (W over Nemesis has been counted earlier in group results and Molly Brown cannot be below Nemesis). Phoenix has beaten Capitals and Scandal.

Losses: Molly Brown has lost to Safari. Phoenix has lost to Bent and Showdown. I believe losing to Safari is worse because Bent has beaten Safari and has 6 wins against the Championship field, Showdown has beaten Bent (1-2) and has 6 wins against the Championship field, and Safari has 1 win against the Championship field.

Thus Traffic > Phoenix > Molly Brown > Nemesis (the last pair being from in-group H-to-H as mentioned).

Per prior, I didn't look at margin of victory and that sort of data.



For Nationals teams that the three teams did play their scores were:
Phoenix - Losses: Fury 13-5, Bent 13-10, Brute Squad 15-3. Wins: Bent 6-15
Molly Brown - Fury 15-11
Nemesis: Losses: Fury 15-11, Brute Squad 10-8. Wins: Bent 7-15.


You bring up an interesting point about how results can differ if you consider two teams one grouping at a time or three teams together. It's the same thorny problems as can occur in electoral systems. I didn't consider three (or four) teams together partly for complexity but also because I tried to account for such comparison at the in-group results stage and the common opponent stage, BEFORE getting to considering the balance of 'quality wins' and 'bad losses'.

I assume you chose to put Traffic #6 by comparing Traffic and Phoenix. How or why did you choose to compare Traffic and Phoenix against each other, then elected to compare Molly Brown, Nemesis, and Phoenix as a group? I'm not certain that I am, but I thought I was being consistent comparing Phoenix and Traffic, then if not Phoenix comparing Phoenix and Molly Brown, and so on.

I've already mentioned that I don't like to deal with margins of victory (complexity), but it's also that Phoenix and Nemesis have played Fury, Brute Squad, and Bent, but Molly Brown has only played Fury. This is highly illustrative of the myriad outcomes that can be produced by changing the groupings and re-stacking the criteria.

Here in Kyle's example, if we allow for his grouping and prioritize common opponent then head-to-head, we'd get Molly Brown -tied- Nemesis > Phoenix. Applying head-to-head thereafter results in Molly Brown > Nemesis > Phoenix, a somewhat different result than mine.

Imagine too if we added back Traffic and their results against Fury and Brute (all losses), their win over Molly Brown, win over Bent, loss to Pop and win over Safari. PLUS add in Molly Brown and Nemesis' results against Bent and Safari. Could Traffic end up not #6 b/c of their loss to Pop who has been dominated by Nemesis and Molly Brown? And so on.

All of these methods can produce good seeding. Kyle and I have indicated independent arguments for Phoenix above Molly Brown or below Nemesis. Given the way this can be nudged in different ways, it makes sense to study and agree one method and use that forever.



Moreover, both Molly Brown (2954) and Nemesis (2976) have better RRI's than Phoenix (2933).

I understood RRI to be not part of the preferred mix of criteria, per the division ND. So I avoided using if everywhere, even for grouping teams.


Frankly, it looks to me that Phoenix's wins over your group 2 teams (which Molly Brown and Nemesis didn't play) are the only thing keeping them in Group 3 at all.

Looking at it now, I would agree. I think I would have grouped Phoenix with Bent-Pop-Showdown in the next level down. Mixed records with Bent and Showdown would definitely have supported that. However, they did post wins over Capitals and Scandal. Their record is too mixed to be part of that group, but also too mixed to be part of the Bent-Pop-Showdown grouping. So they are in between. Once group, that Phoenix ends up at the top or bottom of the group is, to me, a different set of rules.



Given that all other results comparing those teams is against Phoenix (common opponents and RRI), I believe that the Traffic>Molly Brown>Nemesis>Phoenix order makes the most sense.

I think I've established how I found that common opponent is no help here. Now, I don't allow a team to 'take back' a prior loss with a future win..



But, I can understand how you would prefer Molly Brown in your pool over Phoenix...

:P

The trick will be to see if in future years I am arguing the exact same system no matter the possible outcomes or personal stake. Alternatively, if DeAnna publishes a system applied this year, then it would also be consistent for me to argue that we should continue to utilize the same system exactly into the future no matter the projected outcomes or personal stake.

I have no idea what's going to come out. In fact, I feel reasonably sure that DeAnna and I think very differently. So I have no idea on any of this.

To get to where we all want to go, you have to beat a lot of good teams. Sometimes you play different teams than someone else. And other times you play the same teams, but in a different order. All you can do is be ready and win your games.

-akira

Explaining a system takes real work!

[a-yamaguchi-7]

Everybody knows now that I think talking about seeding is a waste of the community's energies. At the risk of boring everyone to tears, I want to describe an area of Ultimate where I think we now do a very good job (not) focusing, in contrast to the past: formats.

A long time ago, when #1 Chicago Open team 'Z' was in the middle of two runs to semis, the CP section had a few other decent teams. Chicago Open #2 and #3 as well as Northern Indiana #1 were all reasonably close in performance. There was a clear hierarchy to seed by (in that order), but everyone was gearing up for some really tough games at Sectionals.

Someone(s) actually came up with, and worked/lobbied/forced through (my best recollection of) this format.. 4 pools, 16 teams, snake seeding. If 'Z' won all of their games they go straight to the finals. If Chicago #2, Chicago #3, and Indiana #1 win all of their games, then Indiana #1 would play Chicago #3, the winner of which would play Chicago #2, the winner of which would meet 'Z' in the finals. Crazy. Oh the controversy.

This may actually have been a reasonably fair way to do things. 'Z' was way, way ahead of the crappy ultimate everyone else was playing. But holy crap! This was at Sectionals!

As it turned out, things didn't quite work out as some hoped, with Chicago #3 beating Indiana #1, Chicago #3 beating Chicago #2 for the first time in a close game, and then 'Z' smoking Chicago #3 in the finals. That format didn't really turn out to be so fair. After an entire season of '<your team Chicago #3> will never beat <our team Chicago #2>' type talk and so on, Chicago #3 clearly had the hardest road in this gerrymandered format. Winning did not change that the format was not equitable.


This never happens anymore, even at unsanctioned, random tournaments. A bunch of smart people went into a room in a vacuum somewhere and worked out a bunch of fair formats balancing a variety of competing constraints. For just about any number of teams, to determine some number of advancing places, there are usually at most two formats. For many event types there's only one format. When there is a choice between pool and bracket formats, the event director usually has a preference of some sort, or there is an area tradition, or whatever, usually driven by factors such as field space, time charges for fields, distance teams traveled and perceived expense, etc. Teams still need to be seeded (argh!) but the placement of teams in the draw down to the order of games has been worked out to try to be a fair as possible and to do right for all the variations of competitive field and number of important places. The teams and their preferences vis-a-vis the competition just aren't a factor, and thank god for it.

[a-yamaguchi-7]

No surprise, I have some crazy ideas now to deal with this (seeding). I haven't thought through what all of the tradeoffs would be, but I think there are some definite advantages to the administration of the Club Championships and to the community.

1. Total Random Draw
- 16 teams.
- Current format.
- All 16 team names go in a pot.
- Randomly draw places #1 - #16 from a pot.
- Results to be certified by the accountants at Numbers & Loblaw.
- Primetime Webcast Championship Draw show from USAU HQ in Boulder, CO.
- USAU members sign-in and answer a very short demographic survey (3-5 randomized questions).
- Visitors sign-in and answer a short demographic survey (5-10 randomized questions).
- Schedule start time, start 2 minutes late (allow sign-ins/sell advertising).
- Sell (hopefully non-endemic) advertising for the start, between every row of draws, and so on.
- Sell placement (backdrop, microphones, drawing pot) or bundle as part of Club Championships merchandising/sponsorship.
- Opportunity for a USAU-led post-draw discussion show.

Pros
- No seeding, no waste.

Cons
- Fury and Riot could end up in the same pool (the format can handle this).
- The top 8 teams could end up together in two pools, condemning 2 from quarters (format does not handle this well).
- There will be easy pools and 'pools of death (may be a 'pro' for some).


2. Bucketed Draw (FIFA World Cup Style)
- 16 teams.
- Current format.
- 16 teams are seeded (argh!) into 4 pots.
- Alternatively, 4 pots could be broken by geography, etc.
- Random draw from pot 1 places into #1 - #4.
- Random draw from pot 2 places into #5 - #8.
- Random draw from pot 3 places into #9 - #12.
- Random draw from pot 4 places into #13 - #16.
- All else same.

Pros
- Much simpler seeding.
- Seeding is by row -- no one can know placement and matchups.
- Seeding discussions are simpler as there is less room to maneuver.
- Roughly balanced pool strengths.
- Fits well with current format allowing for play-in to quarters.

Cons
- Seeding still required into 4 groups.


3. Limited Random Draw
- 16 teams.
- Current format.
- Top 4 teams are seeded.
- Bottom 4 teams are seeded.
- Middle 8 team names go in a pot.
- Random draw places #5 - #12.

Pros
- Much simpler seeding.
- Seeding is by row -- no one can know placement and matchups.
- Seeding discussions are simpler as there is less room to maneuver.
- Fits well with current format allowing for play-in to quarters.

Cons
- Seeding still required for top 4 and bottom 4.
- 3 of the top 5 teams could end up in one pool (format allows play-in from below).
- 6 of the top 8 teams could end up in two crossing pools (format allows play-in from below, but one team would be left out).
- There will be easy pools and 'pools of death (may be a 'pro' for some).


4. Total Random Draw-Totally Modified Format
- Total Random Draw, plus..
- Format change:
- 16 teams, 4 pools, into
- 16 team double(-like) elimination tournament a la the AVP Winners/Contenders brackets
- 3 pool games Thursday, 5 (Winners) or 6 (Contenders) Friday and Saturday games to reach the finals
- http://www.bvbinfo.com/Tournament.asp?ID=1545&Process=Matches

Pros
- No seeding, no waste.
- This format can handle Fury and Riot in the same pool, or other crazy permutations.
- Finalists will likely have beaten both other semifinalists.

Cons
- Fury and Riot could end up in the same pool (the format can handle this).
- Are pool games devalued in this format?
- Top 8 teams could end up together in two pools destined to cross immediately after pool play.


Overall: Think about the interest! Turn administative drudgery into marketable entertainment!

One, more, or a combination of these concepts has to be better than what we do today... maybe?

[kyle.weisbrod]

Personally, I enjoy these types of discussions. I suppose though, if there is a standardized system for seeding, these types of intellectual efforts can be spent on other activities (like developing the best fantasy Ultimate scoring structure for October Madness!)

What Akira has illustrated is the importance of the criteria in the outcome for seeding. Our conflict is rooted in our different perspective of what criteria should be implemented and how.

Perhaps I should have started here when I jumped in to this discussion, since this is where seeding starts:

I agree with Akira's concept of breaking the field of team in to smaller competitive groups. From there I believe that the criteria to be applied to seed within each group (in order of application) should be:

1. Head to Head (H2H) - Head to head provides a direct comparison of the teams' strength within the group
2. Record Against Common Opponents (RACO) - Using common opponents provides the ability to measure teams against one or more "constants." This should include point differential in games for the purpose of increasing the importance of points and games in pre-series tournaments.
3. RRI or some other Rankings Algorithm - If H2H and RACO do not provide a clear indicator of which team has performed better over the course of the season. A ranking that utilizes an algorithm to measure strength by including all the teams' results over the season is preferable to us picking and choosing which tertiary or quaternary results should or should not matter. A rankings algorithm is itself a standard application of criteria to measure team strength (which is what we're trying to do here). Obviously, there is a lot of discussion on how that algorithm should be set-up, if there is a time decay, if there is a point where a win (or loss) by more than some number of points shouldn't be counted (I believe in the old USA College rankings a 15-7 win was valued the same as a 15-1 win to discourage blowouts and/or reduce the importance of games against unevenly matched opponents. Of course, this punished teams that competed far below their level and benefited teams that played up).
4. Historical Results - in the case where two or more teams' have identical or nearly identical current season performance (as measured by the above criteria) then (and only then) should historical results be used to seed teams. I can imagine some small range in RRI that would trigger this final criteria to be applied (like if the teams were within 10 or 20 points).

Where I disagree with Akira:
- I don't believe we should apply "Quality Wins" or "Bad Losses" that are not comparing common opponents between the teams. I believe this is inherently unfair and myopic. If Team A is being compared to Team B and Team A has a quality win against Team C, which Team B did not play but all other evidence about their seasons indicates that Team B is better, there is no reason to think the Team B couldn't have had the same result against Team C had the two teams played. If team A's win was significant enough it would impact their RRI (or other ranking) to be higher than Team B.
- I do believe that a team's losses to a regional opponent that the team finished higher than at Regionals should be negated when breaking up original groups or when discussing H2H and RACO within a group. I'm okay with keeping those pre-series games in the ranking algorithm only because it would lead to rankings inconsistencies where Regional finishes impact past RRI. I'll touch on this a little more in the next section.

Values:
I should admit that there are a couple of values that I hold that lead me to the criteria that I have listed above. There may be more but these are the ones that I can identify. I can understand how others may hold different values that lead them to different rankings criteria. Here are mine:
1. I want every point of the regular season to matter, not just Ws and Ls - This leads me to include point differential in the RACO measurement as well as using RRI as a criteria.
2. I want Regionals to have a significantly higher value than the regular season - This leads me to discard pre-series Regional results between regional competitors and utilize just their Regional results for Nationals ranking discussions
3. I value results from inter-regional pre-series competition higher than intra-regional pre-series competition - Again, this leads me to discard pre-series regional results.
4. I believe that the Nationals seedings should mirror the USA-Ultimate tiebreaking guidelines as closely as possible to create consistency in systems. Obviously, breaking a tie within a pool is significantly easier since all USAUltimate formats have teams playing a standardized schedule. Still, I think you can apply the same concepts for the first tie-breaking criteria: H2H (W-L), H2H (point differential), Point differential between common opponents.

OK, having backed up a bit, I'm hoping this can lead to even more discussion here and help better explain my seeding of those 7-9 seeded teams.

Kyle

[parinell]

Rather than a direct H2H, I think performance at common tournaments is a better indicator of strength and performance. Many or most tournaments these days are elite only and every game matters. The winning team often loses one or more games, and using just direct H2H would give a leg up to that team they lost to.
Alternatively, if you must use H2H, weight elimination games 2x or more.
Kyle is right that H2h is always the first tiebreaker, but I think it's way too early to call it a tie.

[matthewbourland]

Seedings have been finalized:
http://www.usaultimate.org/news/2010-club-championships--schedules-and-seedings/

Pool A:

(1) Riot
(8) Molly Brown
(9) Showdown
(16) Revoloution

Pool B:

(2) Fury
(7) Phoenix
(10) Nemesis
(15) Storm

Pool C:

(3) Brute Squad
(6) Traffic
(11) Pop
(14) Ozone

Pool D:

(4) Capitals
(5) Scandal
(12) Bent
(13) Safari

[ambler]

A preview article about some of the top teams going to Sarasota was just published in the USA Ultimate magazine. Check out the article, written by Phoenix's Leila Tunnel, on page 26 and 27 of the magazine's online version (http://www.usaultimate.org/multimedia/usa_ultimate_magazine/default.aspx).

[tamada]

Rather than a direct H2H, I think performance at common tournaments is a better indicator of strength and performance. Many or most tournaments these days are elite only and every game matters. The winning team often loses one or more games, and using just direct H2H would give a leg up to that team they lost to.
Alternatively, if you must use H2H, weight elimination games 2x or more.


Yeah, although it's practically universal in American (and, AFAICT international) sports to put a high value on H2H results, I suspect that mathematically that is the wrong thing to do. Especially when we're talking about high quality teams who are close in rank to each other. When they play H2H, if they are indeed close in rank then the outcome of that game is going to be determined less by quality and more by chance (because the teams are close in quality). Or it might be determined by quirks of matchups, which might be even worse than using random chance, in terms of coming up with results which are useful for coming up with overall ranks.

Better to put more weight on more robust, wide-ranging set of results instead of the narrow, small sample-sized, somewhat random H2H results.

This goes against most people's intuition, but when placed in a nice list amongst other criteria, as with Kyle's list, I think most people will see the validity and usefulness of those other measures, and won't mind that H2H is not #1 on the list.

[ambler]

A preview of all the women's teams by pool (http://www.usaultimate.org/news/2010-club-championships-preview--womens-division/) has been posted to the USAU site. Carolyn Matthews breaks it all down with some predictions for the tournament's results.

Also, for those getting excited for the upcoming tournament, Five Ultimate is hosting a bracket contest, Florida Fever (http://fiveultimate.com/the-sport/usa-ultimate-club-championships/). Fill out a bracket and submit it before games start on Friday.

[matthewbourland]

All the 1st round women's games go according to seed. Scandal comes back from 8-11 to win 15-12 over Safari.

[matthewbourland]

Standings after Thursday:

Pool A:
Riot (3-0)
Molly Brown (2-1)
Showdown (1-2)
Revolution (0-3)

Pool B:
Fury (3-0)
Nemesis (2-1)
Phoenix (1-2)
Storm (0-3)

Pool C:
Brute Squad (3-0)
Traffic (2-1)
Ozone (1-2)
Pop (0-3)

Pool D:
Capitals (3-0)
Scandal (2-1)
Bent (1-2)
Safari (0-3)

Sets up the following pools for tomorrow:

Pool E:
Riot (1-0)
Brute Squad (1-0)
Traffic (0-1)
Molly Brown (0-1)

Pool F:
Fury (1-0)
Capitals (1-0)
Scandal (0-1)
Nemesis (0-1)

Pool G:
Showdown (1-0)
Ozone (1-0)
Pop (0-1)
Revoloution (0-1)

Pool H:
Phoenix (1-0)
Bent (1-0)
Safari (0-1)
Storm (0-1)

[matthewbourland]

The day 1 women's division recap (http://www.usaultimate.org/news/club-championships--day-1-open-division-wrap/) is now available.