How Artificial Intelligence May be Tennis's Ace for Serving Grand Slam Fans

IBM Watson's Cognitive Highlights dashboard shows off some of the top men’s tennis highlights. IBM

When you think of artificial intelligence, you may picture a network of computers making impossibly complex calculations, or even a robot uprising threatening the world. What you likely aren't thinking about is professional tennis, but that is exactly what IBM is using the company's Watson AI for right now.

IBM Program Director Elizabeth O'Brien spoke to Newsweek about the different ways Watson can improve the digital experience for U.S. Open attendees and viewers, as well as the business applications of the technology.

"I think Watson has really arrived in a much bigger way at the U.S. Open this year," O'Brien told Newsweek , while explaining the two major applications for the AI. The first is a natural language search engine called Cognitive Concierge, and the second is a way to instantly find major highlights across the hours of footage from different tennis matches.

Cognitive Concierge works like a conversation with another person. "For example, you can say 'I'm hungry,' and Watson will respond by saying 'What kind of food do you want to eat?'" O'Brien said. "It understands contexts. It's something that's growing."

Knowing what the attendees at the U.S. Open want to see allows the USTA to fix issues for future events. An example given by O'Brien was a trend in questions this year about tickets and what each ticket would grant access to at the event. For future events, the USTA may be able to answer questions even before they are asked.

Along with the Cognitive Concierge, Watson can be used to find match highlights over the course of the tournament. With more than 800 matches being played, there's too much tennis for even the most die-hard fans to possibly take in. However, an AI can watch any match that's on video, and can even find the most exciting parts from each match.

"Watson is watching video from seven courts," O'Brien explained. "It's watching and listening for three things: it's listening to the crowd noise, it's watching player gestures and it's following the match specifics so it knows when there's a pressure point. When there's the first point of a match, it's not as dramatic."

All this gets combined into a number, ranking each clip. Higher numbers indicate the more exciting clips to watch. Called Cognitive Highlights, USTA's social media team uses this technology to serve up shareable content. It also may give attention to players who aren't normally in the tennis spotlight.

"You can see the up-and-coming stars more easily," said O'Brien. "Of course, we're going to look at the Federer highlights, but what about the lesser-known players? We can see the truly best moments of the U.S. Open much more easily."

The technology isn't perfect yet, but any miscalculations by Watson become opportunities for the AI to learn and grow. O'Brien mentioned that at this year's Wimbledon, Watson misread player gestures.

"Venus Williams was calling for her towel to wipe herself in between points. That arm movement and hand to the forehead looked like a celebration to Watson," O'Brien said. "The team recognized that and trained Watson that towelling off is not a player celebration. They also trained Watson to look at a player's facial expression too. So from Wimbledon to now, Watson no longer thinks towelling off is a player celebration gesture."

O'Brien says there are applications for this expression recognition technology in many different areas of business. "If you're a retailer and you have video in your store, you might want to see how customers are reacting to a certain display, what kind of noises are being generated around the display. What's working, what's not. This helps you better understand your customer."

The tech is so new, the main use for it probably hasn't even been thought up yet. "There are applications beyond tennis," said O'Brien. "This is something that didn't exist six months ago."