Never Miss a Game: Pixellot’s AI-Powered Cameras Are Rapidly Transforming How Sports Are Covered and Consumed
By Adam Noyes
Automated sports production company Pixellot has been broadcasting high school, college and professional games since 2013 by using AI-powered cameras. Some of these games have appeared on the NFHS Network and FloSports, and the technology is being used by coaches in international soccer circuits such as the English Premier League, La Liga, Liga MX, and Bundesliga.
The latest major partnership for the Israeli-headquartered startup is a deal with ESPN in which Pixellot will augment the number of America East conference sports streamed on ESPN+. To discuss that deal and the company’s increasing focus on youth sports—where it has already partnered with SportsEngine—SportTechie caught up with Dave Shapiro, Pixellot’s president of its U.S. youth division and the head of all U.S. sales.
SPORTTECHIE: Certainly Pixellot has been involved in automating live broadcasts for a while, but when you start talking about ESPN and Division I college sports, it becomes a much bigger deal. What are the details of this arrangement and how long had you been testing?
SHAPIRO (pictured right): The testing has been going on for quite a while with ESPN. They had two years ago what they call an Auto Production Summit, where they brought in the different players in the auto-production industry and had a show-and-tell in Connecticut for a day. Coming out of that, we were selected along with one other company as the leaders in the space in ESPN’s eyes and then started to do some testing. The first testing that we did was at UT-Austin for the Longhorn Network where we did several women’s volleyball matches for them. That was successful, and then after that, last season we did a test for the Big West at UC Irvine where we installed our technology in the Bren Center, which is their basketball-volleyball center.
Automated sports production company Pixellot has been broadcasting high school, college and professional games since 2013 by using AI-powered cameras. Some of these games have appeared on the NFHS Network and FloSports, and the technology is being used by coaches in international soccer circuits such as the English Premier League, La Liga, Liga MX, and Bundesliga.
The latest major partnership for the Israeli-headquartered startup is a deal with ESPN in which Pixellot will augment the number of America East conference sports streamed on ESPN+. To discuss that deal and the company’s increasing focus on youth sports—where it has already partnered with SportsEngine—SportTechie caught up with Dave Shapiro, Pixellot’s president of its U.S. youth division and the head of all U.S. sales.
SPORTTECHIE: Certainly Pixellot has been involved in automating live broadcasts for a while, but when you start talking about ESPN and Division I college sports, it becomes a much bigger deal. What are the details of this arrangement and how long had you been testing?
SHAPIRO (pictured right): The testing has been going on for quite a while with ESPN. They had two years ago what they call an Auto Production Summit, where they brought in the different players in the auto-production industry and had a show-and-tell in Connecticut for a day. Coming out of that, we were selected along with one other company as the leaders in the space in ESPN’s eyes and then started to do some testing. The first testing that we did was at UT-Austin for the Longhorn Network where we did several women’s volleyball matches for them. That was successful, and then after that, last season we did a test for the Big West at UC Irvine where we installed our technology in the Bren Center, which is their basketball-volleyball center.
We produced men’s and women’s basketball and also men’s and women’s volleyball games. It went directly to ESPN3. That again was successful, but there were a couple of things coming out of that that they wanted to see—mostly the enhancement to 1080 P and 60 frames per second. We had a 1080 Pm but we were at 30 frames per second. So we improved our products over the past six months and launched a new product called Pixellot Prime, which is 1080 P and 60 frames per second. We did our first test with ESPN with that product at NBA Summer League, where we produced about 35 games in Las Vegas. That was successful again, and that led to this America East deal, which is the most significant partnership that we’ve had so far with ESPN.
SPORTTECHIE: What is required from Pixellot to produce these and how much does it streamline the process for ESPN? How many different cameras will be involved and what human manpower will they still need?
SHAPIRO: To start, we are installing units at UMass-Lowell, the University of Vermont, and then we’re talking about a few other universities. The fields that we’re installing on are the soccer field, which will capture men’s and women’s soccer, and then we’re on the field hockey and lacrosse field.
Where this comes from is ESPN has the rights to the American East, but they were not producing every sport because of the cost of production. For some of the smaller sports that might not have as big of an audience, it just doesn’t add up. But with Pixellot, the cost of production goes way down because it’s automated and therefore gives them the opportunity to produce some of the sports that they weren’t producing before. So right now we’re not replacing production of what they were doing in the 2018 school year. We’re bringing new events in the 2019-2020 school year.
SPORTTECHIE: Is it just the one primary camera angle that pivots back and forth? What else goes into it?
SHAPIRO: It is just one camera angle, but it’s four cameras. The cameras are all housed in one unit. That’s about, call it, 18 inches by 6 inches. That’s mounted at midfield. Each camera is pointed to capture a quadrant of the soccer field. It’s being fed into our computer and processed locally where we’re automatically stitching those four cameras together and then using our AI to determine where the action is in the game. The output we have is a 180-degree panorama the field, but the broadcast view is a zoom-pan-tilt of where the action of the game is. So when you watch it, you would think that this is a production that has multiple people involved—different cameraman, production crew that’s deciding which camera to go to—but the fact is [that] is that it’s 100% automated. There’s zero people involved in the video production.
Now we do tie in typically with ESPN announcers, so they have announcers on site, and then that feed is being plugged into our broadcast. The scoreboard is also automated. We have two options. One is there’s a separate technology called Sportscast, which basically pulls the data from the scoreboard and then we create the graphics within our computing system. So you have a bottom line ticker like you do on any ESPN broadcast that has the score, the time in the game, the quarter, etc. That’s being done automatically either through Sportscast or we have an OCR camera solution also where we point the camera at the scoreboard in venue, turn that into data and then create that bottom line ticker.
SPORTTECHIE: When will the program begin?
SHAPIRO: We’ve already done our first two games. We’ve done two field hockey games. That unit is up and running, so any game that’s on the UMass-Lowell field hockey field is being produced right now. Two have successfully gone to ESPN+. We’re installing the soccer field [unit] next Wednesday. And then we’re installing, I believe, next Friday at the University of Vermont.
SPORTTECHIE: Earlier this year, Pixellot announced an extension of a partnership with PlayOn! Sports and the NFHS that will lead to the production of more than one million live-streamed events per year. What’s the update on how that’s going?
SHAPIRO: It’s ramping up faster than we even expected when we announced that extension with NFHS Network. We’re installing about 250 units every month right now in high schools across the country and having a ton of success with that partnership. So we’re ahead of pace, actually, on getting to the 20,000 units over the next seven years.
Just to give you some numbers in terms of hours of production: Two years ago, Pixellot was doing about 720 hours a month of sports production, which we thought, ‘Wow, this is groundbreaking. We’re leading the industry.’ Last month in September we did 55,000 hours of production [across multiple sports]. Any given Friday night right now, now that the football season is in full swing, we’re doing close to 2,000 football games on high school fields across the country.
SPORTTECHIE: The algorithms must only continue to improve with each game that’s ingested. How sophisticated has the production and editing gotten and how much farther can it go?
SHAPIRO: That’s the benefit of having so many games. All this is based on machine learning, so with each game that we produce, our algorithms are learning and improving. The production is getting better and better. That’s really the significance behind that 55,000 hours in a month. A lot of times, our production, I think, can be better than the human camera operator. A human camera operator can be tired or falling asleep or get a text message, and they’re not following the action. That’s not going to happen with our technology.
If we miss a play, for example—this has happened, this happened previously, two years ago or so in basketball. We’re focused on where the players are and the ball. Well, at the end of the game, that could create an issue, right? Because you’ve got one person that’s on a fast break. Your focus where the inbound play is and the nine other players, and they throw a full court pass and the camera’s late getting to that layup, right? We learned from that and were able to teach the system. ‘Okay. If you’ve got not much time left on the clock and this is the set up of the [court], then we need to zoom out and go to a wider angle because this type of thing might be happening in play.
SPORTTECHIE: A lot of major international soccer leagues are using the system. Is that primarily on practice fields or game fields and what type of software helps the tactical coaches work with the players?
SHAPIRO: We have a tactical view where you can just see every player on the field. That’s typically the view that coaches want to see, right? They want to know what their defense is doing while they’re on attack, for example. So they can get a tactical view. They can also get a 180-degree panorama where you have the entire field of play that you see. They can use that to upload into whatever coaching software they might be using to break down the film and use from a coaching perspective.
Our solution does hit both of those needs for our partners. You meet the broadcast and streaming needs and then also what you need from a coaching perspective. You can really save costs and/or put the time of your staff in the right place. Instead of having somebody who’s on a lift capturing that video, you’re going to have all that video captured through us. You can have that person that’s focused on breakdown and analyzing that video.
SPORTTECHIE: Does it track how far a player runs or those kind of metrics as well? Or does that raw footage get uploaded to another system to compute that?
SHAPIRO: It would get uploaded to another system. We do not do that yet. You could use our video to do that, but we don’t have that technology where we are measuring how much they run or the speeds they’re running at. That’s just not what Pixellot is focused on.
What we are focused on building and we’re getting very close on this is automating highlights. Through optical recognition, we can track the players by the numbers. So you can say, ‘I just want all the highlights from number 12,’ and we can automate those highlights.
SPORTTECHIE: That would potentially be not only for broadcast, but also a parent who can’t make it back home for his son’s or daughter’s game, correct?
SHAPIRO: Exactly. ‘I don’t want to watch a two-hour game after it’s happened, and I know what’s happened, but I want to see all of my kid’s plays.’ You couldn’t get that without technology. The other thing we do is automatic condensed games. Our technology has been taught to identify the action plays. So in basketball, for example, our condensed game is, for every single basket scored, it pulls four seconds before, four seconds after (except for free throws because that’s not really an action moment unless it’s at the end of the game). If you just want to see each basket, instead of watching a two-hour game, you would get a condensed game in roughly 8-to-10 minutes.
Automated sports production company Pixellot has been broadcasting high school, college and professional games since 2013 by using AI-powered cameras. Some of these games have appeared on the NFHS Network and FloSports, and the technology is being used by coaches in international soccer circuits such as the English Premier League, La Liga, Liga MX, and Bundesliga.
The latest major partnership for the Israeli-headquartered startup is a deal with ESPN in which Pixellot will augment the number of America East conference sports streamed on ESPN+. To discuss that deal and the company’s increasing focus on youth sports—where it has already partnered with SportsEngine—SportTechie caught up with Dave Shapiro, Pixellot’s president of its U.S. youth division and the head of all U.S. sales.
SPORTTECHIE: Certainly Pixellot has been involved in automating live broadcasts for a while, but when you start talking about ESPN and Division I college sports, it becomes a much bigger deal. What are the details of this arrangement and how long had you been testing?
SHAPIRO (pictured right): The testing has been going on for quite a while with ESPN. They had two years ago what they call an Auto Production Summit, where they brought in the different players in the auto-production industry and had a show-and-tell in Connecticut for a day. Coming out of that, we were selected along with one other company as the leaders in the space in ESPN’s eyes and then started to do some testing. The first testing that we did was at UT-Austin for the Longhorn Network where we did several women’s volleyball matches for them. That was successful, and then after that, last season we did a test for the Big West at UC Irvine where we installed our technology in the Bren Center, which is their basketball-volleyball center.
We produced men’s and women’s basketball and also men’s and women’s volleyball games. It went directly to ESPN3. That again was successful, but there were a couple of things coming out of that that they wanted to see—mostly the enhancement to 1080 P and 60 frames per second. We had a 1080 Pm but we were at 30 frames per second. So we improved our products over the past six months and launched a new product called Pixellot Prime, which is 1080 P and 60 frames per second. We did our first test with ESPN with that product at NBA Summer League, where we produced about 35 games in Las Vegas. That was successful again, and that led to this America East deal, which is the most significant partnership that we’ve had so far with ESPN.
SPORTTECHIE: What is required from Pixellot to produce these and how much does it streamline the process for ESPN? How many different cameras will be involved and what human manpower will they still need?
SHAPIRO: To start, we are installing units at UMass-Lowell, the University of Vermont, and then we’re talking about a few other universities. The fields that we’re installing on are the soccer field, which will capture men’s and women’s soccer, and then we’re on the field hockey and lacrosse field.
Where this comes from is ESPN has the rights to the American East, but they were not producing every sport because of the cost of production. For some of the smaller sports that might not have as big of an audience, it just doesn’t add up. But with Pixellot, the cost of production goes way down because it’s automated and therefore gives them the opportunity to produce some of the sports that they weren’t producing before. So right now we’re not replacing production of what they were doing in the 2018 school year. We’re bringing new events in the 2019-2020 school year.
SPORTTECHIE: Is it just the one primary camera angle that pivots back and forth? What else goes into it?
SHAPIRO: It is just one camera angle, but it’s four cameras. The cameras are all housed in one unit. That’s about, call it, 18 inches by 6 inches. That’s mounted at midfield. Each camera is pointed to capture a quadrant of the soccer field. It’s being fed into our computer and processed locally where we’re automatically stitching those four cameras together and then using our AI to determine where the action is in the game. The output we have is a 180-degree panorama the field, but the broadcast view is a zoom-pan-tilt of where the action of the game is. So when you watch it, you would think that this is a production that has multiple people involved—different cameraman, production crew that’s deciding which camera to go to—but the fact is [that] is that it’s 100% automated. There’s zero people involved in the video production.
SPORTTECHIE PRO DAY: Meet and Learn From Dave Shapiro on Dec. 12 in New York City
Now we do tie in typically with ESPN announcers, so they have announcers on site, and then that feed is being plugged into our broadcast. The scoreboard is also automated. We have two options. One is there’s a separate technology called Sportscast, which basically pulls the data from the scoreboard and then we create the graphics within our computing system. So you have a bottom line ticker like you do on any ESPN broadcast that has the score, the time in the game, the quarter, etc. That’s being done automatically either through Sportscast or we have an OCR camera solution also where we point the camera at the scoreboard in venue, turn that into data and then create that bottom line ticker.
SPORTTECHIE: When will the program begin?
SHAPIRO: We’ve already done our first two games. We’ve done two field hockey games. That unit is up and running, so any game that’s on the UMass-Lowell field hockey field is being produced right now. Two have successfully gone to ESPN+. We’re installing the soccer field [unit] next Wednesday. And then we’re installing, I believe, next Friday at the University of Vermont.
SPORTTECHIE: Earlier this year, Pixellot announced an extension of a partnership with PlayOn! Sports and the NFHS that will lead to the production of more than one million live-streamed events per year. What’s the update on how that’s going?
SHAPIRO: It’s ramping up faster than we even expected when we announced that extension with NFHS Network. We’re installing about 250 units every month right now in high schools across the country and having a ton of success with that partnership. So we’re ahead of pace, actually, on getting to the 20,000 units over the next seven years.
Just to give you some numbers in terms of hours of production: Two years ago, Pixellot was doing about 720 hours a month of sports production, which we thought, ‘Wow, this is groundbreaking. We’re leading the industry.’ Last month in September we did 55,000 hours of production [across multiple sports]. Any given Friday night right now, now that the football season is in full swing, we’re doing close to 2,000 football games on high school fields across the country.https://www.youtube.com/embed/Hutns7smpqI
SPORTTECHIE: The algorithms must only continue to improve with each game that’s ingested. How sophisticated has the production and editing gotten and how much farther can it go?
SHAPIRO: That’s the benefit of having so many games. All this is based on machine learning, so with each game that we produce, our algorithms are learning and improving. The production is getting better and better. That’s really the significance behind that 55,000 hours in a month. A lot of times, our production, I think, can be better than the human camera operator. A human camera operator can be tired or falling asleep or get a text message, and they’re not following the action. That’s not going to happen with our technology.
If we miss a play, for example—this has happened, this happened previously, two years ago or so in basketball. We’re focused on where the players are and the ball. Well, at the end of the game, that could create an issue, right? Because you’ve got one person that’s on a fast break. Your focus where the inbound play is and the nine other players, and they throw a full court pass and the camera’s late getting to that layup, right? We learned from that and were able to teach the system. ‘Okay. If you’ve got not much time left on the clock and this is the set up of the [court], then we need to zoom out and go to a wider angle because this type of thing might be happening in play.
SPORTTECHIE: A lot of major international soccer leagues are using the system. Is that primarily on practice fields or game fields and what type of software helps the tactical coaches work with the players?
SHAPIRO: We have a tactical view where you can just see every player on the field. That’s typically the view that coaches want to see, right? They want to know what their defense is doing while they’re on attack, for example. So they can get a tactical view. They can also get a 180-degree panorama where you have the entire field of play that you see. They can use that to upload into whatever coaching software they might be using to break down the film and use from a coaching perspective.
Our solution does hit both of those needs for our partners. You meet the broadcast and streaming needs and then also what you need from a coaching perspective. You can really save costs and/or put the time of your staff in the right place. Instead of having somebody who’s on a lift capturing that video, you’re going to have all that video captured through us. You can have that person that’s focused on breakdown and analyzing that video.
SPORTTECHIE: Does it track how far a player runs or those kind of metrics as well? Or does that raw footage get uploaded to another system to compute that?
SHAPIRO: It would get uploaded to another system. We do not do that yet. You could use our video to do that, but we don’t have that technology where we are measuring how much they run or the speeds they’re running at. That’s just not what Pixellot is focused on.
What we are focused on building and we’re getting very close on this is automating highlights. Through optical recognition, we can track the players by the numbers. So you can say, ‘I just want all the highlights from number 12,’ and we can automate those highlights.
SPORTTECHIE: That would potentially be not only for broadcast, but also a parent who can’t make it back home for his son’s or daughter’s game, correct?
SHAPIRO: Exactly. ‘I don’t want to watch a two-hour game after it’s happened, and I know what’s happened, but I want to see all of my kid’s plays.’ You couldn’t get that without technology. The other thing we do is automatic condensed games. Our technology has been taught to identify the action plays. So in basketball, for example, our condensed game is, for every single basket scored, it pulls four seconds before, four seconds after (except for free throws because that’s not really an action moment unless it’s at the end of the game). If you just want to see each basket, instead of watching a two-hour game, you would get a condensed game in roughly 8-to-10 minutes. https://www.youtube.com/embed/w8Vhhy-JaS0
SPORTTECHIE: In what other new markets should we be looking for Pixellot’s name to emerge?
SHAPIRO: The youth sports marketplace is a big focus of ours. We’re doing a lot of product development and R&D right now in that space. Then, in 2020, you’ll see us coming out with products to reach that marketplace. Today, our technology certainly can reach the youth sports market—it’s just that you have some hurdles. Really, the technology has been built mostly for a fixed-installation environment, where you can install it at the UMass-Lowell soccer field and you’re going to capture everything there.
That works very well on the high school, collegiate and professional market—not so much in the youth market because the youth teams are usually rotating from different city fields, and you don’t have the infrastructure or security to install these things. And they’re just playing at different locations a lot. So we’re developing products that will be more mobile and able to reach that big marketplace.
In the U.S., there are 40 million kids that are playing youth sports. It’s a huge marketplace and a market that wants video very badly, both from a recruiting perspective to help kids get to college but also just from a personal memory standpoint—to have that video of the play that Johnny had when he was 10 because he might not be playing when he is 12. I want to be able to capture that. Right now, most of youth sports is being captured on cell phones, and our product is going to be a lot better than that. You’re going to be able to clip highlights a lot easier and really have production that looks like ESPN-type production for Johnny’s 10-year-old games.
SPORTTECHIE: What is the biggest highlight of your extended partnership with HockeyTech?
SHAPIRO: We’ve been partnered with HockeyTech for about two years now. They have our technology in every AHL stadium, which is being used mostly for a coaching perspective, and we’ve been doing a lot of work for HockeyTV, which is a subset of HockeyTech in the junior hockey space. That’s the real competitive part of the market. The new partnership we just formed with them is to create what’s called the Community Network, to bring our technology to more of the youth marketplace. They have about 40 units going into local hockey rinks to produce the younger age group hockey.
SPORTTECHIE: How active is Pixellot in looking for partners to fill in the gaps of what could be a one-stop offering or does Pixellot prefer to develop solutions in house?
SHAPIRO: We are actively talking with partners in the wearable space where we could integrate, and you could have the two of those working together so you have a fully-packaged solution. We are not going to develop our own wearable technology. That would be through partnership with another company.
SPORTTECHIE: Are there other trends in the sports tech sector that have caught your attention recently?
SHAPIRO: This isn’t necessarily related to sports tech, but it’s an area that I think sports tech is going to be used, which is betting. As it gets legalized in more and more states in the U.S.—it’s already huge in Europe—but as it gets legalized here across the board, I think that there are going to be other applications for sports technology through the industry that’s going to build around betting.
Specific to Pixellot, if you’re betting on games, you want to be able to watch those games. As betting becomes more widespread, I think there’s going to be a need for our technology from those companies that are in the betting space so that their customers can both see the game and see highlights, the past games, all that stuff.
Read the original article here!