#deployment
Every summary, chronological. Filter by category, tag, or source from the rail.
Tag · #deployment
Choosing Between Llama.cpp and vLLM for Local LLM Inference
Llama.cpp is optimized for running LLMs on consumer hardware via quantization, while vLLM is designed for high-throughput production environments using techniques like continuous batching and PagedAttention.
IBM TechnologyDeploying Qualcomm AI Hub Models: From PyTorch to On-Device
A practical guide to using the Qualcomm AI Hub SDK to load, test, and deploy models like MobileNet-V2 and YOLOv7, including cloud-based hardware profiling and TFLite compilation.
MarkTechPost
Rainbow Deploys: Git SHA Kubernetes for Stateful Drains
For stateful services like websocket backends needing hours to drain connections, deploy Kubernetes with git SHA-named Deployments, switch Service selectors to new ones, and manually delete old after traffic burns down—avoids mass reconnects unlike rolling updates.
__oneoff__
Showing 4 of 4