Last summer, the United States Soccer Federation released what, at the time, was a remarkable document -- and has become even more extraordinary with another year gone by. It was titled, "A BREAKDOWN OF THE RIGOROUS PROCESS IN THE SEARCH FOR THE USMNT HEAD COACH."
"When the U.S. Soccer Federation hired Matt Crocker after years of service with English Football Association and Southampton in the English Premier League," the federation wrote, "he entered the soccer landscape in the United States with a fresh pair of eyes and a rigorous set of criteria for his primary responsibility at the outset of his tenure: identifying the best coach to lead the U.S. men's national team."
After what was described as "a global search for candidates" and a process that utilized "advanced data analytics, sophisticated metrics, and cutting-edge hiring methods," Crocker hired ... no one. Instead, he just renewed the contract of the previous USMNT manager, Gregg Berhalter.
No matter how you look at it, this supposedly rigorous and data-driven dog and pony show was an utter disaster. The reasoning for the hire, at the time, was an absurd McKinsey-ification of something quite simple: just tell us that he did a decent job over the previous World Cup cycle and the players liked him.
But let's say that you (A) believe in this jargoning of work as a marketing tool, (B) think hiring processes should be data-driven and incredibly complex, and (C) accept that such a process could, coincidentally, lead Crocker to hire the same manager who was already there. Even if you buy into all of that, the result was still a complete failure.
After the USMNT's early ouster in the group stage of the 2024 Copa América, Berhalter was fired last week. In a conference call, Crocker suggested his next attempt at finding the right manager won't be much different than the first one -- just that he's better positioned to do it right: "I think [I'm] now in a better place to have much more of a targeted search where I'll be more inclined to go hard and go early with specific candidates that I feel meet the criteria that we're looking for," he said.
So, what should the criteria be? And how should it differ from whatever it was that led the federation to land on Berhalter? Let's take a look back at how U.S. Soccer got the USMNT hire so wrong to assess how it ought to move forward.
What is the responsibility of the USMNT manager?
That answer might seem simple, and the job of a manager is pretty simple: ensure that his or her group of players performs at least to its level of talent, if not exceeds it.
Legendary Italian manager Giovanni Trapattoni had this to say about the work of the soccer coach: "A good manager makes a team 10% better, and a bad manager makes it 30% worse." There are more bad coaches than good coaches, and per Trapattoni's framework, bad coaches are more impactful than good coaches. There's a very real chance, then, that you hire a manager who actually makes your team worse.
It's no fun, but U.S. Soccer landing a new coach who doesn't improve the team but who also doesn't actively harm the team would be a success. Don't believe me? Go look at the rosters for Portugal, England and Brazil -- and then watch them play.
But however you insist on defining the manager's task, defining it simply makes it simple to judge progress. You find a way to benchmark the team's level of talent -- aggregate wage data, use estimated transfer fees, create a player-performance model, whatever -- and then use a stable metric like expected goals (not results), adjust for the location of the game and the quality of the opponent and work from there. At Brentford, one of the sharpest sports teams in the world, they operate from what they call the "Table of Justice," which ignores results and instead only shows underlying performance data, in order to assess the progress of their squad.
No such process seems evident in U.S. Soccer's document from 2023. There appear to be 22 different responsibilities outlined for the USMNT manager, and those responsibilities are divvied up into two subcategories, titled: "LEAD: Owning the overall strategy" and "GUIDE: Empowering the overall team."
Strangely, the words "performance" and "results" do not appear anywhere in the document. And if U.S. Soccer doesn't want to judge the USMNT manager on performance or results, then that's fine! Avant garde, even! But they, quite clearly, are judging managerial performance based on results. Otherwise, they would not have cut ties with Berhalter after a three-game stretch in Copa América where the driving factors in the one unsatisfactory result were an early red card and some shoddy goalkeeping.
Most of the responsibilities listed by U.S. Soccer make the U.S. manager out to be some kind of high-level functionary or figurehead who also occasionally coaches the national team. I'm talking about things like:
• "Act as an ambassador for sponsorship activities"
• "Create and maintain a positive media profile on/off camp"
• "Drive player off camp engagement/connections"
The federation needs to simplify the next manager's responsibilities. Not only are many of these listed responsibilities things that most coaches consider part of the process of developing a team -- i.e. making sure your players don't hate each other -- but a concise capsule of coaching tasks would also make it easier to figure out what you want from the next guy.
Who should the USMNT manager be?
There are basically two ways to assess a manager: Does he have a track record of making teams better? And how do his teams play?
There are secondary concerns, too. Like, does he trust young players? What's his man-management style? Does he play his best players? And is he -- I don't know -- the kind of guy who might get caught on tape talking about how to circumvent financial regulations?
But, again, the main task of a manager is to make his players play at a higher level than the average manager would. And within that, there are legitimate concerns to how he does it. If you're a club or a national team, you want some consistency of approach. If you're cycling between defensive coaches who only want to score on set pieces and play on the counter, and attacking managers who want to defend with the ball and focus on nothing else, your rosters are going to need to constantly change to fit your new manager. That makes any real long-term progress unlikely, if not impossible
It's quite easy to measure how a manager likes to play. You can go back and look at all of his coaching stops and quantify how much attacking-third possession his teams had, how aggressively they pressed, how quickly they moved the ball up the field, how often they crossed the ball into the box, etc. You can also compare this to how the team played before and after he left and how the other teams in those leagues played. You can't perfectly model tactics, but most coaches maintain some kind of stylistic data footprint wherever they go. And if a coach has no consistency from team to team, that also tells you something.
You can also easily look at other things, like how much he was willing to trust young players, how well his teams performed on set pieces and whether he played his best players. Or was he like Nigel Pearson, who apparently had better players than Harry Kane and Jamie Vardy while he was managing Leicester City?
Of course, things like "constantly improves his teams" and "successfully implements [insert tactical approach]" are also completely absent from the "HEAD COACH COMPETENCIES" slide of the U.S. Soccer document. Instead, here's what it wanted:
• "Drives a vision-led identity"
• "Builds relationships"
• "Effective planner"
• "Self manages"
• "Values communication"
• "Decisive decision maker"
• "Creative innovator"
• "Passionate developer"
It's just more corporate gobbledygook -- a painfully vague collection of words that could be summed up in four words: "Is a soccer coach." None of it even mentions winning soccer games.
Stop overcomplicating this! Figure out how you want the USMNT to play. Find the coaches with a track record of improving their teams within that style. And then see who's interested in the gig.
How to hire the USMNT manager
I've talked to some of the smartest people in the sport, those who U.S. Soccer would kill to hire, and all of them say that judging managerial performance and quantifying coaching is next to impossible. Outside of Jurgen Klopp, Pep Guardiola and Carlo Ancelotti, there are basically no other managers out there who you could be confident would make your team better if you hired them.
And there are even legitimate concerns with all of them. Can Klopp's pressing style work with a group of specific players he doesn't get to select? Can Ancelotti's man-management-centric approach work without the best players in the world? And what happens if Guardiola doesn't have all of the best passers in the world in one place?
At The Athletic, John Muller looked at the "expected goal difference" for every team in Europe based on their wage bill (a proxy for talent level) this past season, and then he awarded the under-performance and over-performance to the coach. The process highlighted pretty much all of Europe's hottest managerial prospects: Xabi Alonso, Sebastian Hoeness, Michel, Andoni Iraola, Thiago Motta and others.
And so, there you have it. Control for wages, look at team performance and bam: you know who the best managers are. Right? Not so fast, my friend.
As Muller writes:
One important trait for a good sports stat is stability, or how much it varies from season to season. If last year's performance can't predict next year's because the number is too sensitive to context, you probably don't want to make it the sole basis for any expensive hiring decisions.
By that standard, our manager metric is a bust. For head coaches who change jobs, there's no correlation whatsoever between the previous year's performance above or below expectations at their old club and their first season at their new club. Even though goal difference added seemed pretty good at identifying this season's hottest managers, it has zero predictive value for new hires.
This is because it's impossible to really know what effect a manager is having on a team's performance.
Maybe he's improved the players so much that they're now getting paid more and it no longer seems like they're overachieving. Maybe the team just happened to have a couple underpaid stars who just so happened to bloom into great players right when he took over. Maybe some assistant coach was actually providing all the value. Maybe an average player had a one-off career year. Maybe the coach's best player was having problems at home and brought those struggles onto the field.
Herculez Gomez reacts to Jesse Marsch saying a "big shift" is needed at U.S. Soccer if he is to ever manage the USMNT.
These complex interactions are so hard to identify and even harder to measure. Anyone who tells you they know how to do it is lying. And that brings us to the worst part of U.S. Soccer's dossier on the process that brought them to bringing back Berhalter -- a section that explains how Crocker landed on his final choice:
"Within these categories, he utilized advanced data analytics, sophisticated metrics, and cutting-edge hiring methods to profile and rank each candidate. During the course of several weeks, candidates were evaluated through all of these filters and went through a battery of practical and psychological testing. In the end, after a multitude of domestic and international coaches were considered, the choice, in the end, was clear and convincing."
Despite what Crocker & Co. say, there are no great metrics to predict future managerial performance. Because of that, you actually can't land on a "clear and convincing" option -- unless somehow Klopp or Pep become an option. It's all, of course, even more absurd when you realize that this clear and convincing option was the guy who was already the coach, the guy they would then fire a year later.
Hiring is hard, across all industries -- and it's especially true when it comes to soccer coaches. The psychological tests that U.S. Soccer mentions all seem silly -- what are you measuring beyond how good a person is at these tests? Most research into hiring processes lands on two different insights: (1) ignore interviews, and (2) find a way to fight back against your biases.
Just because someone says something in an interview, it doesn't mean they'll actually do it once you hire them. On top of that, studies show that we all form our opinions of interviewees pretty early into the interviews. To actually get a sense of how someone might perform on the job, you need to talk to lots of people whom they've worked with -- above them and below them.
More pertinent, though, is the second point. Both after the Berhalter hiring and firing, Crocker has centered himself as the sole decision maker. He talks about himself and how he figures out who he should hire. This is a problem because we all have our own biases we're unaware of. Even as U.S. Soccer talks about "evidence-based" evaluations, that evidence itself is biased because someone is deciding what evidence is being considered, and those decisions are shaped by their own biases. For example: So many hiring algorithms lead to an outsized number of white men getting identified as ideal candidates.
Crocker's biases -- seemingly toward a McKinsey-type coach who can speak the technocrat language, nail the test scores and function as some kind of visionary figurehead -- led him to rehire Berhalter. If he follows the same path for his second hire, he'll likely land on someone who fits a very similar mold.
To avoid that, they should totally scrap the convoluted process, simplify what they're looking for, put a much bigger focus on a manager's history of improving his teams, talk to as many people as they can who've worked with their candidates and find a way to get some impartial feedback on whatever shortlist of options they end up with.
After that? Have some humility. No one is good at this. All you can really do is be aware of your biases and try to limit your mistakes. Crocker can't afford another one.