AI model repo promises lower costs, broader compatibility for NIMs competitor
Hugging Face this week announced HUGS, its answer to Nvidia's Inference Microservices , which the AI repo claims will let customers deploy and run LLMs and models on a much wider variety of hardware.NIMs, Hugging Face Generative AI Services are essentially just containerized model images that contain everything a user might need to deploy the model.
For comparison, Nvidia charges $1 per hour per GPU for NIMs deployed in the cloud or $4,500 a year per GPU on-prem. If you're deploying a larger model, say Meta's Llama 3.1 405B, that spans eight GPUs, Hugging Face's offering will be significantly less expensive to deploy. What's more, support for alternative hardware types means customers won't be limited to Nvidia's hardware ecosystem.
In terms of models, Hugging Face is fairly conservative and focuses on some of the most popular open models, including:Alibaba's Qwen 2.5 7B
United Kingdom Latest News, United Kingdom Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Woman's Death After 'Misdiagnosis' Highlights Urgent Need for Face-to-Face GP ConsultationsA grieving husband has spoken out after his wife tragically died from endometriosis, claiming that her condition was misdiagnosed over the phone by her GP. Laura, who was just 31 years old, suffered excruciating pain but received inadequate medical attention before succumbing to the illness. Her husband believes face-to-face consultations are crucial for accurate diagnoses and preventing similar tragedies.
Read more »
Japanese man who spent more than 50 years on death row receives face-to-face apologyLocal police chief Takayoshi Tsuda visited Mr Hakamada at his home to apologise in person, telling the former boxer, 'we are sorry to have caused you unspeakable mental distress and burden. We are terribly sorry'.
Read more »
Burglar left hanging off a drainpipe after coming face-to-face with homeowner'That's what you get for breaking into someone's house'
Read more »
New DWP update for people on PIP due to attend a face-to-face assessmentThe updated guidance can help new and existing PIP claimants before a concussion.
Read more »
When Trump is on the Joe Rogan podcast, and their past quarrels explainedThe podcaster and former president will come face-to-face this week after previous spats online
Read more »
Elena Sadiku digs out Rangers again ahead of derby clashThe rivals face face at Broadwood on Thursday night
Read more »