Artificial Analysis’ Post

Thanks for the support Andrew Ng! Completely agree, faster token generation will become increasingly important as a greater proportion of output tokens are consumed by models, such as in multi-step agentic workflows, rather than being read by people.

Cerebras has set a new record for AI inference speed, serving Llama 3.1 8B at 1,850 output tokens/s and 70B at 446 output tokens/s. Cerebras Systems has just launched their API inference offering, powered by their custom wafer-scale AI accelerator chips. Cerebras Inference is achieving the fastest speeds we have ever benchmarked on Artificial Analysis for Llama 3.1 8B and 70B. Pricing is also competitive at $0.1 per 1M tokens for Llama 3.1 8B and $0.6 per 1M tokens for Llama 3.1 70B. Cerebras is currently serving both models with an 8K context window (compared to the the Llama 3.1 series’ native 128k context). Cerebras’ Llama 3.1 8B offering is nearly 10x faster than the speeds offered by OpenAI, Google and Anthropic for their current small models - GPT-4o mini, Gemini 1.5 Flash and Claude 3 Haiku. Cerebras Inference is powered by the Cerebras WSE-3, a custom 5nm AI chip built on a unique wafer-scale design. A single WSE-3 chip is over 50x larger in total area than a Nvidia H100 and hosts 900,000 cores with 44GB of on-chip memory (SRAM). Faster AI inference enables a new generation of AI applications, from sophisticated agentic workflows to instant search experiences. See below for further charts and links to our full benchmark results.

AI21 Labs launches Jamba 1.5 Mini and Large! These models utilize a hybrid state space/transformer architecture to maintain output speed with long context inputs Both Mini and Large launch with 256k context windows, the largest of all major open weights models. Their hybrid state space/transformer architecture supports the models in maintaining high performance over long prompt lengths. Jamba 1.5 is leading in speed compared to models of a similar intelligence class (comparing to the median performance across providers for each model). Artificial Analysis has independently benchmarked Jamba 1.5 across quality, API performance and price. See below for further charts and a link to our full analysis of AI21’s models.

Ideogram’s v2 Text to Image model released today is launched on our Image Arena! We have crowdsourced >200k user preferences to independently evaluate the quality of Text to Image models. Stay tuned for how Ideogram's new model compares to Midjourney v6.1, Black Forest Labs's FLUX.1 [pro], Stability AI's Stable Diffusion 3 Medium & other models as preferences are submitted in the coming days. Early comparisons suggest a strength of Ideogram v2 is in adding text clearly to images and in-line with style of the image (as per image generated below), something Text to Image models have historically struggled with. We have also commenced API performance benchmarking of Ideogram’s API which is currently in beta. Generations are priced at $80 per 1k images, in-line with Midjourney. See below for a link to join in Artificial Analysis’ Text to Image Arena 👇

Groq has just launched their record breaking Distil-Whisper endpoint! With a Speed Factor of 240x, it is the fastest Speech to Text endpoint we have benchmarked. It is also the lowest-cost Speech to Text endpoint we benchmark at $0.33 per 1000 minutes of audio. This means you can transcribe all Star Wars movies (~27 hours) in under 7 minutes for less than $1 ($0.53). However, important to note is that Distil-Whisper is English language only and has a higher Word Error Rate than Whisper v3. Per our independent quality evaluation, Distil-Whisper has a Word Error Rate of 12.7%, higher than Whisper v3’s 10.3%. As such, this endpoint is suited for those that have English-language use-cases and want to prioritize speed and cost over a marginal decrease in accuracy. Link to our analysis below 👇