In one of his first public appearances since he joined Facebook to lead his artificial intelligence initiatives, Vice President Jérôme Pesenti expressed concern over the increasing use of computing power needed to create powerful artificial intelligence systems.
"I can tell you that this keeps me awake at night," Pesenti said. "The maximum calculation for companies like Facebook and Google can afford an experiment, we're already doing it."
More software innovation will be required for artificial intelligence to grow without disruption and hardware and software optimization.
Examples of systems that rely less on computing for innovative breakthroughs include Pluribus, and the AI system developed by Facebook AI Research and Carnegie Mellon University, introduced today, that can take on world-class poker players. In an article in science, the researchers said that Pluribus only required $ 150 in cloud computing to train.
The end of Moore's Law means that the calculation needed to create the most advanced AI is going up.
Pesenti quoted and the OpenAI analysis that found the calculation necessary to create cutting-edge systems has increased 10 times each year since 2012.
"We still see gains with the calculation increase, but the pressure of the problem is simply going to increase," Pesenti said. "I think we will continue using more computation, it will not stop, but it will slow down, because it can not keep up with the rhythm of 10 times a year, which is simply not possible."
Last month, an analysis found that the costs of training systems such as OpenAI's GPT-2 can exceed the carbon emissions of the life of five cars.
Pesenti, who brings artificial intelligence to Facebook, on the stage of the VentureBeat Transform conference, spoke about the unique challenges faced by Facebook in implementing artificial intelligence systems for 2,800 million unique users around the world, such as analyzing nuances as if a publication qualifies as hate speech or if it is a video. It simply modifies or deekfake.
The road block companies can find on their journey to deploy the AI can be cultural or logistical, or simply not recognize that the AI stack is the same as the typical engineering stack.
AI plays a role in virtually every aspect of Facebook services, from what is shown to recommendations on Facebook or Instagram to content moderation, as well as new customer experiences, such as the Portal's smart camera.
Many Facebook services work with Intel CPUs, said Facebook engineering manager Kim Hazelwood last year.
Pesenti, as executives of Google, Microsoft and Airbnb in his talks on Transformation, also spoke about the importance of diversity in contracting and making sure that artificial intelligence works in the same way for everyone.
He believes that the bias usually comes from data sets rather than the creators of artificial intelligence systems.
"We are making progress. It is still far from where we should be, "he said." We need to do everything possible to increase diversity in the field. "
Facebook shared new statistics related to the company's diversity earlier this week, but did not disclose statistics about race or gender diversity within divisions such as Facebook AI research devoted entirely to artificial intelligence.
The analysis carried out by Data & Society Fellow and the co-author of the Mutale Nkonde Algorithmic Responsibility Law found that Facebook AI Research currently employs 146 people, none of whom is of African descent.
An inclusive artificial intelligence program created by Facebook's AR / VR business leader, Lade Obamehinti, is currently being used internally to learn products by bias. Obamehinti created the program a year ago after she discovered that Portal's Intelligent AI camera did not work on people like her with darker skin pigments.
However, measuring AI diversity by team may not be up to date soon. Pesenti wants Facebook developers to be part of every team and division of the company.
"My goal is to make every engineer in the organization an ML engineer, and that number has multiplied by three in the last year, so we are talking about thousands and thousands of engineers that are not on my team and that Actually, they are not ML Engineers, "he said.