Groq is a new artificial intelligence (AI) model that is causing quite a stir on social media thanks to its lightning-fast response time and innovative technologies that could eliminate the need for GPUs.After public benchmark testing on the social media site X went viral, demonstrating that Groq outperformed ChatGPT, a well-known AI chatbot, in terms of computation and reaction speed, the product became an overnight sensation.
The reason for this is that the Groq team created a unique application-specific integrated circuit (ASIC) chip specifically designed for large language models (LLMs), which enables it to produce about 500 tokens every second.By contrast, the model’s publicly accessible version, ChatGPT-3.5, has a token generation rate of about 40 per second.Rather than using the expensive and rare graphics processing units (GPUs) that are usually used to run AI models, the creator of this model, Groq Inc., claims to have invented the first language processing unit (LPU) that powers its model.
The business that created Groq is not new, though.It was established in 2016 and registered the term Groq as a trademark.When Elon Musk’s AI model, also named Grok (spelled with a “k”), began to gain popularity in November of last year, the creators of the original Groq wrote a blog post criticizing Musk for the name selection.
“We can see why you might want to adopt our name. You like fast things (rockets, hyperloops, one-letter company names) and our product, the Groq LPU Inference Engine, is the fastest way to run large language models (LLMs) and other generative AI applications. However, we must ask you to please choose another name, and fast.”
Since Groq went viral on social media, neither Musk nor the Grok page on X has made any comment on the similarity between the name of the two models.
However, a lot of users on the platform have started contrasting the LPU model with other well-known GPU-based models.For goods that need minimal latency—the amount of time it takes to execute a request and get a response—one user who works in AI development referred to Groq as a “game changer.”
According to a comment made by another user, Groq’s LPUs may provide a “massive improvement” over GPUs in the future for meeting the demands of AI applications. It may also prove to be a viable substitute for the “high performing hardware” of Nvidia’s highly sought-after A100 and H100 chips.This occurs at a time when big AI developers are trying to create their own chips internally in order to not be dependent on Nvidia’s models alone.According to reports, OpenAI is looking for money from governments and investors throughout the globe totaling billions of dollars in order to construct its own processor and solve scalability issues with its products.