{"id":13944,"date":"2022-06-17T09:09:46","date_gmt":"2022-06-17T09:09:46","guid":{"rendered":"https:\/\/www.searchenginejournal.com\/google-limoe\/454673\/"},"modified":"2022-06-17T09:09:46","modified_gmt":"2022-06-17T09:09:46","slug":"google-limoe-a-step-towards-goal-of-a-single-ai-via-sejournal-martinibuster","status":"publish","type":"post","link":"https:\/\/marketingnewsbox.com\/?p=13944","title":{"rendered":"Google LIMoE \u2013 A Step Towards Goal Of A Single AI via @sejournal, @martinibuster"},"content":{"rendered":"
<\/div>\n

Google announced a new technology called LIMoE that it says represents a step toward reaching Google\u2019s goal of an AI architecture called Pathways.<\/p>\n

Pathways is an AI architecture that is a single model that can learn to do multiple tasks that are currently accomplished by employing multiple algorithms.<\/p>\n

LIMoE is an acronym that stands for Learning Multiple Modalities with One Sparse Mixture-of-Experts Model<\/em>. It\u2019s a model that processes vision and text together.<\/p>\n

While there are other architectures that to do similar things, the breakthrough is in the way the new model accomplishes these tasks, using a neural network technique called a Sparse Model.<\/p>\n

The sparse model is described in a research paper from 2017 that introduced the Mixture-of-Experts layer (MoE) approach, in a research paper titled, Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer<\/a>.<\/p>\n

The sparse model is different from the the \u201cdense\u201d models in that instead of devoting every part of the model to accomplishing a task, the sparse model assigns the task to various \u201cexperts\u201d that specialize in a part of the task.<\/p>\n

What this does is to lower the computational cost, making the model more efficient.<\/p>\n

So, similar to how a brain sees a dog and know it\u2019s a dog, that it\u2019s a pug and that the pug displays a silver fawn color coat, this model can also view an image and accomplish the task in a similar way, by assigning computational tasks to different experts that specialize in the task of recognizing a dog, its breed, its color, etc.<\/p>\n

The LIMoE model routes the problems to the \u201cexperts\u201d specializing in a particular task, achieving similar or better results than current approaches to solving problems.<\/p>\n

An interesting feature of the model is how some of the experts specialize mostly in processing images, others specialize mostly in processing text and some experts specialize in doing both.<\/p>\n

Google\u2019s description of how LIMoE works shows how there\u2019s an expert on eyes, another for wheels, an expert for striped textures, solid textures, words, door handles, food & fruits, sea & sky, and an expert for plant images.<\/p>\n

The announcement about the new algorithm describes these experts:<\/p>\n

\n

\u201cThere are also some clear qualitative patterns among the image experts \u2014 e.g., in most LIMoE models, there is an expert that processes all image patches that contain text. \u2026one expert processes fauna and greenery, and another processes human hands.\u201d<\/p>\n<\/blockquote>\n

Experts that specialize in different parts of the problems provide the ability to scale and to accurately accomplish many different tasks but at a lower computational cost.<\/p>\n

The research paper summarizes their findings:<\/p>\n

\n