TensorFuse (YC W24) reposted this
Lately, I’ve been writing a lot about running serverless GPUs. However, very few people actually understand how hard it is to build it. Here’s something we are doing under the hood: 1. Rewriting our own Docker from scratch 2. Rewriting our own file system 3. Making it compatible with K8S and auto-scalers like Karpenter, Knative, etc. 4. All this while operating within the deep, dark forest called AWS All this to solve for cold start and bring it down to less than 5 seconds for any type of model image, from Llama3 to Stable Diffusion. If you are a systems engineer with experience in Go or Rust, come join us at Tensorfuse. We’re a lean team building the next gen of serverless computing!