How FIFA plan to advance technology in football: From synthetic feeds to skeletal tracking

Polish referee Szymon Marciniak (2nd-R) and match officials pose for pictures before the start of the Qatar 2022 World Cup final football match between Argentina and France at Lusail Stadium in Lusail, north of Doha on December 18, 2022. (Photo by Jewel SAMAD / AFP) (Photo by JEWEL SAMAD/AFP via Getty Images)
By Adam Leventhal
Mar 26, 2024

Some people are so riled by VAR and the delays that it brings they would get rid of it altogether. FIFA are not on that page. In fact, they are looking at innovations that will take the involvement of technology even further. Penalties, fouls, handballs, interference with goalkeepers and even AI that aids referee announcements are all being considered.

Advertisement

“I was watching a German Cup game (in which VAR wasn’t being used) and Dortmund scored, the player was offside, and the text commentary read: ‘This goal would not have counted had VAR been in use’,” Nicholas Evans – FIFA’s leading technology problem solver — tells The Athletic. “You have to start thinking, what are we doing here? Why would you allow that to happen?”

There is no question in his mind that VAR and the continuing integration of technology into football makes sense.

“The vision that I and FIFA have is it’s better than what was there before,” he says. “It’s a safety net and you will never be perfect, but you’re taking the stress off the referee at a moment in time when they’re making a decision.”

Evans is head of football research & standards within FIFA’s technology innovation team that is responsible for implementing solutions to solve the game’s problems. He’s speaking to The Athletic overlooking the pitch at the Tottenham Hotspur Stadium, which witnessed the Premier League’s most embarrassing VAR error of the season when Liverpool’s Luis Diaz was denied a legitimate goal. Miscommunication engulfed the VAR room when the incorrect on-field call of offside wasn’t overturned.

(Marcus Jamieson-Pond — JamPondPhotography)

“If something like that were to happen in a World Cup or build-up event, we would look to learn from that and see what we can do to improve,” says Evans. “We’re seeing more of a disconnect between what is happening on the pitch and the fans.”

The Professional Game Match Officials Limited (PGMOL) — the body in charge of referees in English football — has enlisted the help of air traffic controllers and pilots to improve the clarity of communication, but enhanced offside technology could have also aided that process. Semi-automated offside technology (SAOT) is used in FIFA and UEFA competitions and Serie A, which means there is no need to draw or activate lines, with a virtual line generated instead. It accurately identifies skeletal data points on multiple players and ball tracking shows exactly when the ball was kicked.

Advertisement

“I know the Premier League are looking at it. They’ve seen our tournaments, they know the facts, it’s really now how they interpret that for their own purposes,” says Evans.

Publicly, the Premier League are non-committal on bringing SAOT in any time soon.

“We don’t believe it will improve the accuracy of decision-making,” the league’s chief football officer Tony Scholes said last month. “What it will do is speed up the time of decision-making. It’s extremely important in that regard.” Tests are taking place, but England’s top tier only has a tentative plan to “go to clubs for a decision later in the year” but “that’s not a definite at the moment”, said Scholes. The earliest any vote from Premier League clubs could take place is this summer’s AGM, which would involve a tight turnaround for next season.

So why the wait? “From a fan’s perspective, I fully understand that argument,” says Evans, who has confirmed both La Liga and Bundesliga are looking at SAOT integration, too. “You need to understand how you justify that to your audience. You need to explain to fans what to do about these five-minute reviews. I think we can try and avoid them by giving the referee tools that avoid that.”

Evans acknowledges that factors behind the scenes play a key part like logistics, training and input of broadcasters. He also believes there is a “legitimate conversation to have” around ensuring investment in technology doesn’t dwarf the amount paid to the referees: “It shouldn’t be underestimated and I understand refereeing bodies saying, ‘Hang on, if you’re going to spend another £5million on technology while our guys are still making a couple of thousand per game, then we need to first talk about the human development’.”

(Marcus Jamieson-Pond — JamPondPhotography)

FIFA are pushing on with research and development in other areas. Evans says the “sheer amount of data from any given game is mindblowing” and has seen accelerated growth and opportunity for accuracy. Traditional performance data capture via devices commonly worn by players allows analysis of approximately 600,000 data points per team. Skeletal tracking, now used in some officiating decisions, takes that figure to 172million. “We have more and more information, which becomes data, which becomes insight,” says Evans, who spoke about the topic at the International Sport Convention in March.

Advertisement

So where is this heading? And how will it help officials’ and fans’ experience of the game?

“One of the biggest error rates that we’re seeing is with corner versus goal kick, especially at the end of the game that can have a massive impact,” says Evans. “With connected ball technology, you know exactly who made the final touch.”

Part of this conversation is whether the ball has gone out of bounds in the build-up to goals. Arsenal manager Mikel Arteta called the officiating “a disgrace” after November’s 1-0 defeat at Newcastle when there was no clear angle to see if the ball had gone out of play or if there was a foul in the build-up to Anthony Gordon’s winner.

At the 2022 World Cup, Japan profited from a VAR call going their way against Spain in the final group games, which was made on the strength of enhanced camera pictures and a greater number of angles. FIFA are now carrying out further trials using a combination of ball tracking and a variety of cameras to get an even clearer picture. “You put the cameras on the line or close enough and use the data as a model to say we are beyond doubt that that ball is out,” says Evans. “This then informs the referee directly and in the interests of transparency you can easily give this information to a fan via a second-screen experience. That part of the journey could come sooner rather than later.”

FIFA are also looking at “synthetic feeds”, which allow first-person views and data integration to help with in-game decision-making, including whether offside players are interfering with the view of a goalkeeper. PGMOL head Howard Webb admitted Nathan Ake’s goal for Manchester City against Fulham in September should have been disallowed after Manuel Akanji, who was offside, obstructed Bernd Leno.

“The technology is good enough that you could go into that first-person view and actually use a synthetic, animated feed from a goalkeeper view to make an officiating decision,” says Evans. “You could play that through and then make an objective assessment.”

Part of the research is adding a visual guide in the form of a corridor to help decision-making and information supplied to fans. “Those are the sort of officiating use cases that are very practical to help overcome some of these subjective decisions,” says Evans.

Advertisement

“Handball is another example and it’s relatively easy to detect if the ball touched the hand,” he says regarding another area that is seeing extensive research.

During the World Cup, ball sensors outlined whether or not Cristiano Ronaldo touched the ball with his head before the ball went in against Uruguay. Evans thinks it can be used for all associated elements of handballs, including whether the ball hit another body part before hitting the hand.

Helping referees understand whether tackles are careless or reckless and thus warrant a yellow or red are also being researched via data modelling. One example is looking at tackles from behind. “You could measure the angle very easily. How high was the foot? Was the player accelerating or decelerating? Is the player making an active movement? Skeletal data will let you do all of that,” says Evans, who sees no barrier to further analysis.

“How do you quantify a foul? There is no such thing as a foul. It’s a handball, holding, pushing, tripping, kicking and all of those you can quantify. Am I tripping you or not? Well, we just look at the contact between the two limbs.”

Officials explaining their decisions to stadium and TV audiences was seen at the most recent Women’s World Cup and that will continue in all FIFA competitions. “I personally believe it is an intermediate step to a wider opening,” says Evans. While FIFA understand there are hurdles to overcome to make all VAR conversations public all of the time, the aim is to move to greater detail to on-field explanations including AI to provide real-time translations for a global audience.

(Glyn Kirk/AFP via Getty Images)

Less technologically advanced was the suggestion that a blue card could be used to indicate that a player would be sent to a sin bin. Evans was surprised by the backlash. “The whole thing was completely blown out of proportion in the sense of saying, ‘Oh, FIFA have gone crazy and want a blue card’,” he says. “It was probably just someone saying there could be a visual mechanism to differentiate between a yellow and red.”

Sin-bin trials continue, in line with International Football Association Board (IFAB) planning, at grassroots levels. The current signal is a yellow card and the referee signalling to a temporary dismissal area with two hands.

Advertisement

The “negative media backlash” has had an impact according to Evans: “It’s a shame because it means that the problem which was being addressed has now sort of been pushed down the road. It took away its legitimacy by being ridiculed. Forget whether it’s blue or not, that’s not what it’s about.”

While any plan would need to be fully approved by IFAB — the guardians of the game’s rules — Evans feels a step forward in trials may be during junior tournaments such as the Under-17 World Cup. He thinks it is unlikely the decision-making timetable will see it in place for the next senior World Cup in 2026.

(Top photo: Jewel Samad/AFP via Getty Images)

Get all-access to exclusive stories.

Subscribe to The Athletic for in-depth coverage of your favorite players, teams, leagues and clubs. Try a week on us.