TensorFuse (YC W24) reposted this
Lately, I’ve been writing a lot about running serverless GPUs. However, very few people actually understand how hard it is to build it. Here’s something we are doing under the hood: 1. Rewriting our own Docker from scratch 2. Rewriting our own file system 3. Making it compatible with K8S and auto-scalers like Karpenter, Knative, etc. 4. All this while operating within the deep, dark forest called AWS All this to solve for cold start and bring it down to less than 5 seconds for any type of model image, from Llama3 to Stable Diffusion. If you are a systems engineer with experience in Go or Rust, come join us at Tensorfuse. We’re a lean team building the next gen of serverless computing!
Harshil Shah
1mo
Why you need to rewrite the kernel?
Akshay Jain
1mo
Rewriting Docker and a filesystem from scratch is no small feat, especially with the cold start challenges in serverless computing - the under-the-hood glimpse is super fascinating!
Harshil Shah
1mo
With serverless gpu how is it different than serverless functions?
Sounds some super interesting stuff that!
Founder @ Tensorfuse (YC W24)
1moYou forgot about the changes to the kernel!