Putting Continual Learning on AI’s Map Posted on : Feb 12 - 2019

The capability of a machine to “learn” on its own is the subject of some debate. With traditional supervised machine learning, decisions can be optimized, but the machine isn’t really learning by itself. Now a startup called Cogitai is hoping to push the limits of a machine’s capability to learn continuously using reinforcement learning techniques.

Cogitai was founded in 2015 by some of the earliest innovators in the reinforcement learning (RL) field, including Mark Ring, Peter Stone, and Pete Wurman. The Orange County, California is hoping to leverage the collective RL knowledge work of its founders and the 15 or so PhD computer scientists in the firm to change the course of AI applications.

Ring, Cogitai’s CEO, paints a stark contrast between the supervised machine learning (ML) that he says powers 99% of AI applications today, and the newer RL approaches that Cogitai is taking with its cloud-based platform, called Continua, which becomes generally available today.

“Supervised learning data has to be labeled by people, by humans who provided the data labels, then algorithms are really just finding patterns in the data that match the labels supplied by the people,” Ring tells Datanami.

“It’s extremely powerful,” he continues, “but also very limited. In particular, the ability to make decisions and take actions in the world and then to learn from that decision-making process is completely missing from that picture.”

Reinforcing Learning

Cogitai is utilizing reinforcement learning techniques to power an array of AI applications via its Continua cloud service.

RL is a branch of machine learning that has been getting lots of attention recently, especially in certain industrial use cases, such as robotic process automation. Compared to supervised ML, where a data scientists is typically employed to periodically retrain the algorithm to keep its accuracy high, the feedback loop is built directly into the RL process.

While a supervised ML algorithm will look for external cues to gauge its accuracy or performance – from a data scientist or the line of business – an RL algorithm has a built-in feedback mechanism that allows it to continually adjust its settings and try new approaches to reach a desired outcome. This is an important distinction that Cogitai hopes to exploit with its new cloud service.

The key component in Cogitai’s RL system is a learning agent. According to Ring, the Continua learning agent determines what actions to take and then passes those decisions back to the client. After the client executes the action, the result from that action is sent back to the learning agent, which assesses whether it met certain performance metrics determined b the client.

“The learning agent tries to maximize that performance metric,” Ring says. “It tries to increase that metric as much as possible by taking different actions in different situations that lead to the maximal possible reward signal.”

This RL approach allowed Google to train a computer model called AlphaGo to beat the world champion at that game. After iterating across millions of games, AlphaGo was able to optimize its decision making, without any  direct input from human programmers.

In other words, the system learned not only what works, but what doesn’t. This RL capability to recall success and built skills worked spectacularly well for Google’s video game killer, Ring says, and the same process can be applied to address specific industrial challenges, such as automobile control systems, manufacturing optimization, and logistics operations, among others. View More