Frontend Feeds

Category

The Giants

6 Sources

  • DEV CommunityThe Giantsread at source

    NVIDIA GPU Operator on RKE2: Passthrough to Kubeflow (The Sports Car in the Garage)

    Episode 7 of the homelab build series. The foundation is poured, the walls are up, the alarms are wired, the storage floor is down and the safe is locked. Now for the fun room: three NVIDIA GPUs, one in each machine, so GPU capacity stays highly available when any one of them is down, managed by the NVIDIA GPU Operator and wired into Kubeflow. nvidia-smi side by side on the three GPU nodes, one per datacenter. Introduction Three GPUs. Three datacenters. Zero schedulable. I had three NVIDIA cards in three machines, and Kubernetes swore I had none. I asked for one GPU in a test pod and got this: 0/21 nodes are available: 21 Insufficient nvidia.com/gpu. The Proxmox hosts could see them. lspci could see them. nvidia-smi on the host lit up. But to the cluster, three cards' worth of Ampere silicon were parked, idle and invisible. That's the most expensive kind of idle there is. And the whole MLOps platform from the earlier episodes (Kubeflow, Katib, KServe, the training pipelines) was built to train and serve real models. You can't schedule what the scheduler doesn't know exists. Fitting the cards took a Saturday. Getting the kernel module, the container runtime, the device plugin and the scheduler to agree took a lot longer, and taught me thirteen different ways a GPU can hide from Kubernetes. Here's where it ended up. Three RTX A1000s, one in each "datacenter", which here means one in each of three mini PCs on a shelf. Lose a whole machine and two cards are still there. The NVIDIA GPU Operator builds the driver, wires up the runtime and advertises the cards on its own. And the part I'm proud of: when the whole lab loses power, all three cards come back and pass their CUDA self-test without me touching anything. This is how I got there, every trap along the way, and the parts that still aren't done. And at the end there's a real job on a real card: a 7-billion-parameter language model answering at about 30 tokens a second, against 4 on the CPU beside it. Where this sits in the build Foundation: Terraform on Proxmox. Framing: Ansible building the five-master HA RKE2 cluster. Security: Kyverno, KubeArmor and Trivy, with Falco parked for now to save CPU. Workshop: Tekton, Harbor, SonarQube and Chains. Storage floor: Rook-Ceph and CloudNativePG. The safe: HA HashiCorp Vault. The sports car: this one. This is where the cluster stops being general-purpose Kubernetes and gains an accelerator pool. Before the operator went in, the three GPU workers were scheduled like any other node, by CPU and memory, with no idea a GPU was attached. The repo side is small: one namespace, one chart, one values file, and one ClusterPolicy the operator builds everything from. kubernetes-addons/0-boostrap/gpu-operator/ namespace.yaml # gpu-operator namespace, privileged Pod Security gpu-operator-values.yaml # annotated Helm values; every trap fix lives here dcgm-custom-metrics.yaml # extra DCGM fields for the dashboard nvidia-dcgm-dashboard.yaml # Grafana dashboard examples/ # multi-GPU best-practice manifests The hardware in the garage The cards: three NVIDIA RTX A1000s. Ampere GA107GL , 8 GB of GDDR6, 2,304 CUDA cores, about 50 W. It draws all its power from the PCIe slot, with no extra connector, which is exactly why it fits where a bigger card wouldn't. The node label says it plainly: $ kubectl get node rke2-worker-dc1-gpu-a \ -o jsonpath = '{.metadata.labels.nvidia\.com/gpu\.product} {.metadata.labels.nvidia\.com/gpu\.memory}MiB' NVIDIA-RTX-A1000 8188MiB The hosts: three Minisforum MS-01s. Intel i9-13900H, 96 GB of DDR5, and, rarely for a mini PC, one real half-height PCIe slot wired to the CPU: x16 in length, x8 electrically, which is plenty for this card. That slot is the whole reason this build exists. Datacenter Proxmox host GPU worker GPU VRAM dc1 proxmox-dc-1 rke2-worker-dc1-gpu-a RTX A1000 8 GB dc2 proxmox-dc-2 rke2-worker-dc2-gpu-a RTX A1000 8 GB dc3 proxmox-dc-3 rke2-worker-dc3-gpu-a RTX A1000 8 GB Three cards, three machines: PCIe passthrough from Proxmox host to RKE2 worker. That's 24 GB of GPU memory across the fleet. One 8 GB card runs a 4-bit 7B model with room to spare; I measure that further down. It's great for inference, fine for small fine-tunes, and marginal for training from scratch, which is a rented-A100 job anyway. Why one card per datacenter instead of three in one box? The same answer as storage: failure domains. If a machine goes down, GPU work still has two healthy places to land. In a homelab, rack density is worth nothing and HA is worth a lot. Why bother with Kubernetes when the GPU is right there on the host? Three reasons. Each project gets its CUDA version from an image tag instead of eight CUDA installs fighting each other. The runtime becomes a pinned image, so results are reproducible. And the scheduler is the only sane way to say "this job gets one card, that one gets another". PCIe passthrough: from Proxmox to the VM Each card is bound to vfio-pci on its Proxmox host and passed into exactly one VM. An Ansible playbook handles the nouveau blacklist, the vfio binding, IOMMU and the reboot. All three cards sit at the same address, 0000:01:00 , which keeps the Terraform pleasingly uniform. The passthrough itself is Terraform with the bpg provider, through a gpu_pci_hosts map per datacenter. One gotcha deserves its own sentence: raw hostpci needs the root@pam Proxmox user. A normal API token can't assign raw PCI devices, and the error doesn't tell you that. cd terraform/environment/prod-rke2/bpg terraform plan -target = module.rke2_worker_dc2_gpu -target = module.rke2_worker_dc3_gpu # 0 to add, 2 to change, 0 to destroy (in place: +machine=q35, +hostpci0) terraform apply -target = module.rke2_worker_dc2_gpu -target = module.rke2_worker_dc3_gpu machine=q35 and hostpci0 only take effect after a full power cycle of the VM, not a soft reboot. The best moment of the whole build came here. Once dc1 worked, I passed the cards through on dc2 and dc3 and rebooted them. The card appeared, the node got labelled, and the operator rolled the entire stack onto both nodes by itself. The first node took me twelve hours. The other two took about ninety seconds each, and I didn't touch them. Three hardware notes that earned their place: Driver version is pinned to 570.133.20 . The version stays put. What has to follow the kernel is the module, which the driver container rebuilds against whatever kernel the VM is running. No vGPU, no MIG. The A1000 supports neither. Secure Boot never gets a say. The GPU VMs boot with SeaBIOS, not UEFI, so the unsigned NVIDIA module loads without any key enrolment. The Proxmox hosts themselves do have Secure Boot on; it doesn't matter, because the module loads inside the VM. If your VMs use UEFI with Secure Boot, you'll have to sign the module or switch it off there. What the GPU Operator actually does What the GPU Operator runs: a tower where every floor waits for the one below. One Helm release of nvidia/gpu-operator (chart v26.3.2) runs a set of DaemonSets from a single ClusterPolicy . Sticking with the garage: Component In plain terms What it really does Operator the garage manager watches the ClusterPolicy and keeps everything below in shape NFD (Node Feature Discovery) the inspector scans hardware and labels nodes that have an NVIDIA PCI device GFD (GPU Feature Discovery) the spec sheet adds product, memory, count, driver and CUDA labels Driver the mechanic compiles and loads nvidia.ko against the running kernel, inside a container Container Toolkit the licence plate registers the nvidia runtime with containerd Device Plugin the parking attendant advertises nvidia.com/gpu to the kubelet DCGM Exporter the dashboard per-GPU metrics for Prometheus Validator the MOT runs a CUDA test on every card and reports the result The dependency chain is the thing to remember. NFD labels the nodes that have a GPU. Everything else only runs where that label is. So the driver, toolkit and device plugin land on the three GPU workers and nowhere else. No NFD label, no driver. No driver, no GPU. And every other piece waits for the driver. When the bottom of that tower is broken, everything above it sits in Init . That gave me the rule that saved the most time: debug from the driver up. There's no point staring at the device plugin while the driver is still crash-looping. What I deliberately don't run: the MIG manager (the A1000 has no MIG), the vGPU manager (that's for licensed shared vGPU), the sandbox and Kata pieces (the card is passed through once, at Proxmox), GPUDirect Storage (no hardware for it), and a standalone DCGM (the exporter brings its own, and two would fight over one socket). The stack lives in the gpu-operator namespace with privileged Pod Security. The driver, toolkit and device plugin load kernel modules and touch host paths, and a restricted namespace blocks every one of them. kubectl get pods -n gpu-operator mid-cascade: a wall of Init:0/1 and one very sad nvidia-driver-daemonset . The "before" shot. From PCIe slot to scheduled pod From PCIe slot to scheduled pod: once per node, then once per pod. Once per node: The worker boots with the GPU present as a PCI device. NFD labels the node. The driver pod compiles nvidia.ko for the running kernel and loads it. The toolkit registers the nvidia runtime with containerd. The device plugin tells the kubelet the node has nvidia.com/gpu: 1 . GFD adds nvidia.com/gpu.product=NVIDIA-RTX-A1000 and friends. The DCGM exporter starts and metrics flow. Then once per pod: A workload asks for nvidia.com/gpu: 1 . Kyverno adds the taint toleration and the GPU node rule (more on that in the next section). The scheduler finds a GPU node with a free card and binds the pod. The NVIDIA runtime injects the device and driver libraries. nvidia-smi inside the container sees its card. All of it is automatic once the operator is in. That's the whole argument against a hand-rolled Ansible role that builds kernel modules per node and breaks on every kernel upgrade. Keeping GPU pods on GPU nodes Two bits of metadata mark these nodes, and you want both: A label so workloads can find them: nodepool=gpu , plus everything NFD and GFD add. A taint so ordinary workloads can't drift onto them and starve a GPU job: nvidia.com/gpu=true:NoSchedule . $ kubectl get node rke2-worker-dc1-gpu-a -o jsonpath = '{.spec.taints}' [{ "effect" : "NoSchedule" , "key" : "nvidia.com/gpu" , "value" : "true" }] Normally that means every GPU pod needs a toleration, a node selector and a resource limit, and forgetting any one gives you a confusing failure. On this cluster a Kyverno policy, gpu-pods-to-gpu-nodes , does the first two for you. Any pod that asks for nvidia.com/gpu is mutated to tolerate the taint and require a nodepool=gpu node. So a GPU pod here needs exactly one line: resources : limits : nvidia.com/gpu : 1 The operator's own DaemonSets carry the toleration in the values file, and NFD's master also needs a control-plane toleration: daemonsets : priorityClassName : system-node-critical tolerations : - key : nvidia.com/gpu operator : Exists effect : NoSchedule nfd : master : tolerations : - key : node-role.kubernetes.io/control-plane operator : Equal value : " " effect : NoSchedule Prove it works The quickest proof is NVIDIA's own CUDA sample. It adds two vectors on the card and prints one line if the whole chain held: driver, runtime, device plugin, scheduler. apiVersion : v1 kind : Pod metadata : name : cuda-vectoradd namespace : default spec : restartPolicy : OnFailure securityContext : runAsNonRoot : true runAsUser : 1000 seccompProfile : type : RuntimeDefault containers : - name : cuda-vectoradd image : nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda12.5.0-ubuntu22.04 securityContext : allowPrivilegeEscalation : false capabilities : drop : - ALL resources : limits : nvidia.com/gpu : 1 Look at what isn't there: no toleration, no node selector, no runtimeClassName . The only GPU line is the limit. The securityContext blocks have nothing to do with the GPU; my default namespace enforces the restricted Pod Security Standard, so every pod there needs them. $ kubectl apply -f cuda-vectoradd.yaml $ kubectl get pod cuda-vectoradd -o wide NAME READY STATUS RESTARTS AGE NODE cuda-vectoradd 0/1 Completed 0 16s rke2-worker-dc1-gpu-a $ kubectl logs cuda-vectoradd [ Vector addition of 50000 elements] Copy input data from the host memory to the CUDA device CUDA kernel launch with 196 blocks of 256 threads Copy output data from the CUDA device to the host memory Test PASSED Done Sixteen seconds from apply to Completed , image pull included. The pod landed on a GPU worker without being told to, and you can see what Kyverno wrote into it on the way in: $ kubectl get pod cuda-vectoradd -o jsonpath = '{.spec.tolerations}' [{ "effect" : "NoSchedule" , "key" : "nvidia.com/gpu" , "operator" : "Equal" , "value" : "true" }] $ kubectl get pod cuda-vectoradd -o jsonpath = '{.spec.affinity.nodeAffinity}' { "requiredDuringSchedulingIgnoredDuringExecution" : { "nodeSelectorTerms" :[ { "matchExpressions" :[ { "key" : "nodepool" , "operator" : "In" , "values" :[ "gpu" ]}]}]}} Delete the pod when you're done. It has finished, so it isn't holding the card, but it's clutter. kubectl logs cuda-vectoradd ending in Test PASSED . The hero shot. The runtime: getting containerd to know about GPUs containerd doesn't know what a GPU is. Something has to register an nvidia runtime that injects the device and driver libraries into containers. The classic way is to patch containerd's config.toml , and RKE2 regenerates that file from a template , throwing your patch away on every restart. Two things make it work here. CDI is on, so the toolkit writes a device spec at /etc/cdi/nvidia.yaml that containerd reads at container start: cdi : enabled : true nriPluginEnabled : false And the toolkit is pointed at RKE2's own containerd rather than the default one: toolkit : env : - name : CONTAINERD_CONFIG value : /var/lib/rancher/rke2/agent/etc/containerd/config.toml - name : CONTAINERD_SOCKET value : /run/k3s/containerd/containerd.sock - name : CONTAINERD_RUNTIME_CLASS value : nvidia - name : CONTAINERD_SET_AS_DEFAULT value : " true" The subtle bit is that CONTAINERD_CONFIG points at the generated config.toml , not the .tmpl . The toolkit re-applies its setup every time it starts, so when a reboot regenerates the config without nvidia, the toolkit simply puts it back. That's what makes it heal itself across reboots. Point it at the template and the live config sits there without a GPU runtime until someone restarts rke2-agent by hand. Because nvidia is the default runtime, workloads here don't need runtimeClassName: nvidia . On a cluster without that setting they do, and that one line is what confuses people copying manifests between clusters. Sharing the cards Each A1000 advertises exactly one nvidia.com/gpu , so by default one pod owns the whole card. Mode What it does Isolation Here Exclusive one pod owns the card full ✅ what runs today Time-slicing several pods take turns on the card compute only, memory is shared available, not switched on MIG hardware partitions with their own memory full ❌ the A1000 has no MIG MPS processes share one GPU context partial available, not used The node labels confirm it: nvidia.com/gpu.replicas=1 and sharing-strategy=none on all three. Time-slicing is a ConfigMap plus a patch to the ClusterPolicy : apiVersion : v1 kind : ConfigMap metadata : name : time-slicing-config namespace : gpu-operator data : a1000 : |- version: v1 sharing: timeSlicing: resources: - name: nvidia.com/gpu replicas: 4 # one A1000 looks like four schedulable GPUs kubectl patch clusterpolicies.nvidia.com cluster-policy --type merge \ -p '{"spec":{"devicePlugin":{"config":{"name":"time-slicing-config","default":"a1000"}}}}' Create the ConfigMap first, then patch. I ran the patch on its own once, and the device plugin's replacement pod sat in Init with configmap "time-slicing-config" not found , which took that node's card from one schedulable GPU to zero until I cleared the reference again. The DaemonSet replaces one pod at a time, so the rollout stalled there and the other two cards kept working. Before you turn it on, remember it's oversubscription, not partitioning: No memory isolation. Four pods share 8 GB. Four 2 GB models fit; a fourth 3 GB model dies with CUDA out-of-memory. No fault isolation. One pod that wedges the card affects its neighbours. Sharing, not guarantees. Right for notebooks and bursty inference, wrong for anything with a latency target. My plan is to time-slice dc1 and dc2 and leave dc3 whole for jobs that need all 8 GB. How reliable is it? Here's what the GPU tier survives, and the evidence for each: What goes wrong What happens How I know A whole machine goes down Its card is gone, and so is any job that was running on it; new GPU pods land on the other two One card per machine (dc1, dc2, dc3), each advertising nvidia.com/gpu: 1 The whole lab loses power Every node rebuilds its GPU stack and re-tests its card, hands-off After the last power cut, the toolkit, device plugin, DCGM exporter and validators came back on all three nodes by themselves, and the CUDA validator Succeeded on every card RKE2 regenerates containerd's config on reboot The toolkit re-applies the nvidia runtime CONTAINERD_CONFIG points at the generated file, not the template Someone forgets the GPU toleration Kyverno adds it, plus the GPU node affinity gpu-pods-to-gpu-nodes mutation policy A notebook is abandoned with a card It should be stopped after 24 hours idle and the card freed Config only: ENABLE_CULLING=true , CULL_IDLE_TIME=1440 . I have not seen it fire. The lab is powered off overnight, and two notebooks sat on cards until I stopped them by hand A driver or runtime problem The operator stops and reports it ClusterPolicy state ready ; validators gate everything above the driver And the honest limits. "Highly available" here means GPU capacity : a job running on a card dies with its machine and has to start again elsewhere. The operator itself is a single replica; if it's down, running GPU pods carry on and nothing new gets configured. A kernel upgrade on a GPU node still needs a driver rebuild (see the traps), there are no GPU alerts wired yet, and nothing caps how many cards one namespace can take. Those last three are on the list at the end. kubectl get pods -n gpu-operator -l app=nvidia-cuda-validator : three validators, one per datacenter, all Completed . The traps, from the driver up Thirteen ways a GPU hides from Kubernetes, by layer, from the driver up. Thirteen of them, grouped by layer, in the order you should debug them. Most of them bit me. A few I only know from other people's write-ups, and those say so. Every piece waits on the driver, so a broken foundation makes everything above it look broken too. The driver The driver gets OOM-killed, exit 137. The log dies right after an nvidia-drm warning, then nvidia-smi: not found , and it crash-loops. It looks like a driver bug. It isn't: the driver container compiles the kernel module , running gcc against the kernel headers, and a 512Mi limit kills it mid-compile. driver : resources : limits : cpu : " 2" memory : 4Gi # 512Mi OOMs the kernel-module compile failed to install nvidia kernel module: distro 'ubuntu24.04' not supported . The driver image ships pre-built modules for a fixed list of distro and kernel pairs, and mine wasn't on it. Build from source instead: driver : kernelModuleType : auto # open modules first, proprietary as fallback usePrecompiled : false # build from source, about 3 minutes on first boot nvidia: Unknown symbol __register_chrdev after a host kernel upgrade. The cached module was built for the old kernel. Force a rebuild: kubectl -n gpu-operator rollout restart daemonset/nvidia-driver-daemonset The real lesson is a runbook entry, not a config change: a GPU node's kernel upgrade is a special operation , with a driver restart on the checklist. A plain power cut doesn't need it, because the kernel hasn't changed. Missing kernel headers. The driver builds against the running kernel, so a kernel upgraded without linux-headers-$(uname -r) fails the build. The Ansible role for the GPU pool installs them, and the error is plain in the driver pod log. The validator nvidia-validator: command not found , exit 127. Version skew, and the error tells you nothing. Older charts shipped the validator as a separate image; the 26.x chart puts the binary inside the main operator image. Values written for one layout against a chart of the other produce a missing binary. Pin the chart ( v26.3.2 here), deploy it from the NVIDIA repo rather than a stale vendored copy, and check what's actually running with helm list -n gpu-operator , not what your values file assumes. The toolkit and runtime no runtime for "nvidia" is configured . The RKE2 trap the generic docs miss. The toolkit looks for /run/containerd/containerd.sock ; RKE2 runs its own containerd at /run/k3s/containerd/containerd.sock . The fix is the toolkit.env block above. Failed to initialize NVML: Driver/library version mismatch inside a pod, while the host shows the same version. The toolkit had patched config.toml directly and RKE2 regenerated it. CDI or the generated-config setting fixes it. KubeArmor's AppArmor profile blocks the toolkit. The same family of trap that once bit Cilium here: the toolkit can't write what it needs, containerd is silently never configured, and pods report the nvidia RuntimeClass as missing. If you hit it, run the toolkit container unconfined. Today it runs fine under its own KubeArmor-generated profile. The toolkit writes the CDI spec and containerd never reads it. Pods schedule, but nvidia-smi hangs. CDI needs containerd 1.7 or newer, so check what your RKE2 release ships before you switch it on. I didn't hit this one: my containerd is 2.x. Missing runtimeClassName: nvidia on a cluster where nvidia isn't the default runtime. The pod schedules, runs with no GPU injected, and looks exactly like broken hardware. Not an issue here because of CONTAINERD_SET_AS_DEFAULT , which is why it's worth knowing: the requirement depends on the cluster. The device plugin and scheduling Insufficient nvidia.com/gpu while three healthy GPUs sit idle. The driver and toolkit were running, but the device plugin was Pending : 0/21 nodes are available: 21 node ( s ) had untolerated taint { nvidia.com/gpu: true } , that the pod didn 't tolerate. No device-plugin pod means nothing tells the kubelet about the card. The upstream chart ships the right toleration, but a trimmed values file copied from a blog about an untainted cluster quietly drops it. Same trap one layer down: NFD's master without its control-plane toleration never labels anything, so nothing else schedules either. NFD takes a minute on first install. The operator comes up before NFD's first scan, so for about a minute it looks like the operator is running and nothing is happening. It isn't broken. Don't start changing things. Telemetry The DCGM exporter gets OOM-killed right after logging DCGM successfully initialized! . It loads its whole field set into memory and 256Mi isn't enough: dcgmExporter : resources : limits : memory : 1Gi All of these fixes live in gpu-operator-values.yaml , . Who holds a card, and how to get it back This is the part I'd most have wanted to read, because it's invisible until a GPU job won't schedule. nvidia.com/gpu can't be oversubscribed. Without time-slicing, a pod with nvidia.com/gpu: 1 owns that physical card for its whole life. Three cards means three GPU pods at once, full stop. That makes long-lived notebooks the quiet problem. At one point two Kubeflow notebooks were each holding a whole A1000, one of them for three and a half months, while asking for 0.6 and 1.2 CPUs. A notebook that asks for less than one CPU and holds an entire GPU is the shape of the problem. It also quietly capped a Katib sweep at one GPU trial at a time. And they cost more than cards: together they used over half of their namespace's CPU quota, enough to make a daily training pipeline time out because its last job could never be scheduled. The claim is on the pod , not the Notebook, the StatefulSet or the volume. So stopping a notebook frees the card. Kubeflow's "stop" sets an annotation, the controller scales to zero, and the pod goes away: kubectl -n annotate notebook \ kubeflow-resource-stopped = " $( date -u +%Y-%m-%dT%H:%M:%SZ ) " --overwrite I sampled the free-GPU count every ten seconds while doing it: [ 01] sts replicas = 0 pod = Terminating FREE GPUs = 1 [ 02] sts replicas = 0 pod = gone FREE GPUs = 2 Look at the first line. The notebook was "stopped" by every API you'd think to check, and the card was still claimed , because the pod hadn't finished terminating. replicas=0 isn't proof a card came back. On a cluster where KubeArmor can wedge pods in Terminating , that matters. The reliable check counts from the pods and cross-checks the scheduler: # what pods claim kubectl get pods -A -o json | jq '[.items[] | select(.status.phase=="Running" or .status.phase=="Pending") | .spec.containers[].resources.limits["nvidia.com/gpu"] // "0" | tonumber] | add' # what the scheduler has allocated on a node kubectl describe node | awk '/Allocated resources/,/Events/' | grep nvidia.com/gpu As I write this, both GPU notebooks are stopped and all three cards are free. Culling is on as a safety net. The notebook controller stops notebooks that have been idle for 24 hours. It stops, it never deletes: the volume and every file survive, and restarting is one click. What's lost is in-memory kernel state. A running cell counts as busy, so a long training run won't be culled. And the setting is global and not GPU-aware, so it catches abandoned notebooks; it doesn't manage contention. The better lever is not giving notebooks a GPU by default. ⚠️ One more surprise: nvidia.com/gpu isn't in the profile namespaces' ResourceQuota, and a quota only limits what it names. So GPU use per namespace is unbounded even though CPU and memory are capped. Putting the GPUs to work Making nvidia.com/gpu schedulable was the prerequisite, not the goal. Who on the platform can ask for a card Nothing on this platform needs a GPU to run. Every Kubeflow component is a CPU service; what changes is the work users launch through them. Here's each door to a card, and whether anything has walked through it yet: Platform piece How it asks for a card Has it used one here? Kubeflow Notebooks a GPU dropdown in the spawner form Yes, and too well: two notebooks held cards while idle Katib nvidia.com/gpu in the trial template No. Both sweeps ran on CPU Kubeflow Trainer resourcesPerNode on a TrainJob No. The runtimes are installed ( torch-distributed , three torchtune LLM fine-tunes); no job has run KServe a GPU limit on the predictor No. The one live model is XGBoost on CPU Kubeflow Pipelines a GPU limit on a single step No. The daily pipeline is all CPU Spark executor GPU settings plus the RAPIDS plugin No, and not planned: these jobs are small A plain pod one line, nvidia.com/gpu: 1 Yes: the LLM benchmark below Everything else in the data platform (Kafka, Airflow, MLflow, the databases) never touches a card and shouldn't. So the honest summary is two real users so far: notebooks, which took cards and did nothing with them, and one benchmark pod, which took a card and drove it at 98%. Hyperparameter search with Katib Katib runs an Experiment that launches trials as pods, each reporting a metric back through an injected sidecar. Getting it to run at all taught me more than the GPUs did. It couldn't run in any Kubeflow profile namespace. An Experiment sat at Created with zero trials, and the UI only said "Couldn't find any successful Trial." The real error was two levels down, on a ReplicaSet: pods " -tpe-..." is forbidden: minimum memory usage per Container is 16Mi, but request is 10Mi Katib asks for 10Mi of memory for its suggestion pods, and the Kyverno-generated LimitRange in every profile namespace has a 16Mi floor. Eleven of twelve algorithms had no explicit resources, so eleven of twelve couldn't schedule. Fixing those exposed the same problem on the trial pods, because the metrics sidecar is injected by a webhook and its default never shows up in the template. The fix is to raise Katib to meet the floor, not lower the floor: runtime : suggestions : - algorithmName : tpe image : ghcr.io/kubeflow/katib/suggestion-hyperopt:v0.19.0 resources : requests : cpu : 50m memory : 128Mi limits : cpu : 500m memory : 512Mi metricsCollectors : - kind : StdOut image : ghcr.io/kubeflow/katib/file-metrics-collector:v0.19.0 resources : requests : cpu : 50m memory : 32Mi limits : cpu : 200m memory : 128Mi Three more gotchas from getting a sweep to finish: katib-config is mounted with subPath , which never receives updates. Restart the Katib controller after every config change; waiting does nothing. Never put a multi-line python3 -c script in a trial template. The stdout collector rewrites the command into a shell string, and the inline script doesn't survive. Pass a file path and flags instead. A sub-second trial can finish before the collector attaches , which shows up as random failures with an empty observation. Pace the objective and set backoffLimit: 1 . With all of that fixed, a 12-trial TPE sweep finished 12 out of 12 and found the known optimum, on CPU, because for a demonstration sweep a GPU changes nothing (more on that below). And with no time-slicing, the real ceiling for GPU trials is three at a time, one per card. A parallelTrialCount higher than that just leaves trials Pending . The Katib trial leaderboard with the best trial highlighted. Training and serving A GPU training job in Kubeflow Trainer V2 looks like this. I haven't run this one yet, so treat it as the shape, not a tested manifest. With one card per machine, two workers have to land on two different machines: apiVersion : trainer.kubeflow.org/v1alpha1 kind : TrainJob metadata : name : resnet-finetune namespace : terranetes spec : runtimeRef : name : torch-distributed trainer : numNodes : 2 resourcesPerNode : limits : nvidia.com/gpu : 1 For serving, a KServe InferenceService with minReplicas: 0 gives the card back when the model is idle and cold-starts it on demand. That works because the GPU claim is per pod: scale to zero, the pod goes, the card is free. With three cards, scale-to-zero is the difference between one always-on model and a fleet of models sharing three GPUs. My live models today don't need a GPU at all. The football match predictor is an XGBoost model on tabular data, and it trains and serves on CPU in seconds. The first real GPU TrainJob and a properly served LLM are both on the list at the end. The first real job: a 7B model on one card For three months these cards passed their self-test and did nothing else. Prometheus has the receipts: 0% utilisation and 0 MiB of GPU memory on all three, the whole time. So before writing another word about what an A1000 can do, I made one do it. The job is deliberately plain. One pod, one card, Ollama running Mistral 7B in 4-bit ( Q4_K_M , a 4.1 GB download), the same prompt every time with a fixed seed. The whole request for hardware is one line, nvidia.com/gpu: 1 . Kyverno added the toleration and the node rule, the scheduler put it on dc2, and nvidia-smi inside the container saw the card. NAME SIZE PROCESSOR CONTEXT mistral:7b 5.8 GB 100% GPU 4096 Then the same model on the same node with the GPU switched off, four threads against the pod's four-core limit: Tokens per second Notes RTX A1000 about 30 (four runs: 29.9 to 31.6) 5,031 MiB of the 8,188 MiB used, 98-99% utilisation CPU, 4 threads (i9-13900H) about 4.2 (two runs: 4.1 and 4.3) same model, same prompt, same seed Mistral 7B on one RTX A1000 against the CPU beside it: about 30 tokens a second against about 4.2. Seven times faster, on a 50-watt card that fits in a mini PC. A 400-token answer took 13 seconds on the GPU. At the CPU's rate it would take over a minute and a half. That is the difference between a tool you'd use and one you wouldn't. Three things the run taught me that the spec sheet didn't: It runs hot. The card idles at about 60 °C in the MS-01 and sat at 88 to 90 °C within a minute of load, a few degrees over its own 86 °C target, with the driver reporting a software thermal slowdown. Speed drifted from 31.6 to 29.9 tokens a second across the runs. It never reached the 96 °C hard slowdown, but a small case and a single-slot cooler have a ceiling, and sustained training will live at it. It can't tell you its power draw. nvidia-smi reports power.draw as [N/A] on this card under passthrough. So the missing power metric isn't an exporter setting I forgot. Threads matter more than cores. My first CPU run was left to pick its own thread count. It saw 20 cores, started 20 threads inside a four-core limit, and was still going ten minutes later. Pinned to four threads it produced 60 tokens in about fifteen seconds. If you benchmark CPU inside a container, set the thread count to the limit. The manifest and the exact commands are in examples/llm-benchmark-pod.yaml . It's a benchmark, not a service: the pod holds a whole card while it exists, so I deleted it when the numbers were in, which is this article's own advice. 📸 SCREENSHOT: the Grafana DCGM dashboard for dc2 across the run: utilisation jumping from 0 to 98%, memory to 5 GiB, temperature from 60 to the high 80s. The first non-zero lines those panels have ever drawn. Multi-GPU, and does a GPU change the result? With more than one card, the internet will tell you to reach for tensor parallelism and model sharding. On three 8 GB cards over PCIe with no NVLink, most of that advice is wrong. Strategy Splits Use when On these cards Data parallel (DDP) the batch, full model per GPU the model fits in 8 GB ✅ the default Sharding (FSDP, ZeRO) parameters and optimiser state the model won't fit one GPU ⚠️ prefer offloading to CPU RAM Tensor parallel individual layers needs NVLink ❌ slower over PCIe than not doing it Pipeline parallel layer stages very deep models, many GPUs ⚠️ rarely worth it here Use single-GPU tricks first (bf16, gradient checkpointing, smaller batches), then DDP for throughput, and only then sharding. For inference, don't shard: quantise. A 7B model in int4 fits on one card, and splitting it across two PCIe cards is strictly worse. Does a GPU change the result, or just the speed? It changes how fast you get a result, not what the result is. With a fixed number of trials, a GPU gets you the same answer sooner. With a fixed amount of time, faster trials mean more trials, and more trials is how a search finds a better optimum. That gain comes from trial count, not from the GPU. And it can make results slightly worse: GPU reductions aren't always deterministic, so training often isn't bit-for-bit reproducible. So putting a CPU-bound job on a GPU is theatre, and it costs a card something else needed. Monitoring The DCGM exporter publishes per-GPU metrics, and a ServiceMonitor puts them in Prometheus, with NVIDIA's dashboard in Grafana. The ones worth watching: DCGM_FI_DEV_GPU_UTIL : utilisation DCGM_FI_DEV_FB_USED and DCGM_FI_DEV_FB_FREE : memory used and free DCGM_FI_DEV_GPU_TEMP : temperature DCGM_FI_DEV_PCIE_REPLAY_COUNTER : link errors on the PCIe slot Two I want and don't have. DCGM_FI_DEV_XID_ERRORS isn't in my exporter's field list yet. DCGM_FI_DEV_POWER_USAGE isn't either, and adding it won't help: this card doesn't report power draw under passthrough at all. The most useful panel is utilisation by pod: sum(DCGM_FI_DEV_GPU_UTIL{Hostname="rke2-worker-dc2-gpu-a"}) by (exported_namespace, exported_pod) Mind the label. On my setup pod is the exporter's own pod, and the workload is in exported_pod . Group by pod and every card looks like it's being used by the DCGM exporter. What's missing is any alerting. The metrics are there, but nothing fires on a hot card, full memory or an XID error yet. And the metric I want most isn't on any dashboard: allocated versus used . A card that's 100% allocated and 0% used is the notebook problem, and utilisation alone makes it look exactly like a free card. NVIDIA has since open-sourced something aimed at exactly that gap: the GPU Usage Monitor , an Apache 2.0 Helm chart that pairs the DCGM exporter with kube-state-metrics, Prometheus and Grafana, and ships dashboards for allocated against used, by namespace and by pod, plus GPU pods stuck Pending . I haven't installed it. I already run every one of those four pieces, so what I'd want from it is the dashboards, not a second Prometheus. And its platform filter is named for datacenter cards (Hopper, Blackwell), so I don't yet know how it treats an A1000. It's on the list. The Grafana DCGM dashboard showing all three A1000s. What it costs Three used RTX A1000s for about £600, in slots the MS-01s already had. No extra power. The card is rated at 50 W; I can't give you a measured figure, because it doesn't report one. Cooling is the honest cost: nothing extra was fitted, and under load the card sits at its thermal limit. The operational cost is mostly kernel upgrades on GPU nodes, which need the driver restart. Adding a GPU node is now Terraform the passthrough, label and taint it, and let the operator do the rest. What's Next? The sports car runs, survives a power cut and comes back on its own. What's left, roughly in order: Try NVIDIA's GPU Usage Monitor dashboards against the Prometheus I already have, to get allocated versus used on a screen. Export XID errors , then wire up GPU alerts: temperature (now that I know it reaches 90 °C), memory nearly full, XID errors, and a card allocated but unused for hours. Turn on time-slicing on dc1 and dc2, and keep dc3 whole. Put nvidia.com/gpu in the profile ResourceQuota , so one namespace can't take every card. Run a real GPU TrainJob across two machines, and serve that LLM properly behind KServe with scale-to-zero, not from a benchmark pod. Automate the driver restart after a GPU node's kernel upgrade, so it stops being a checklist item. If you've got a GPU sitting in a box that Kubernetes can't see, I hope this saves you the thirteen traps on this list. Follow me on LinkedIn Author: George Ezejiofor

  • DEV CommunityThe Giantsread at source

    Quick Tips: 20 One-Liners to Boost Your Front-End Skills

    Introduction: Supercharge Your UI Development Hey there, fellow UI developers! Are you ready to take your front-end skills to the next level? Whether you're just starting out or you've been in the game for a while, we all know that staying up-to-date with the latest tricks and techniques is crucial in our fast-paced field. That's why I've put together this list of 20 one-liners that will give your front-end skills a serious boost. These quick tips are designed to be easy to remember and implement, but they pack a punch when it comes to improving your code and workflow. So, let's dive in and explore these game-changing one-liners! The Power of One-Liners in Front-End Development Before we jump into our list of 20 one-liners, let's talk about why these little snippets of code are so valuable. One-liners are concise pieces of code that perform a specific task or solve a particular problem in a single line. They're like the Swiss Army knives of programming – compact, versatile, and incredibly useful. For UI developers, one-liners can be particularly helpful because they allow us to: Streamline our code and make it more efficient Quickly solve common problems without writing lengthy functions Improve our understanding of language features and best practices Impress our colleagues with our coding prowess (okay, maybe that's just a bonus!) Now that we understand the importance of one-liners, let's get to the good stuff – our list of 20 one-liners that will boost your front-end skills! HTML One-Liners: Markup Magic 1. Viewport Meta Tag for Responsive Design name= "viewport" content= "width=device-width, initial-scale=1" > This one-liner is essential for responsive web design. It ensures that your website scales correctly on different devices by setting the viewport width to the device width and initial zoom level to 1. 2. Preload Critical Resources rel= "preload" href= "critical-style.css" as= "style" > Use this one-liner to preload critical resources like CSS files. This tells the browser to start downloading the resource as soon as possible, improving page load times. 3. Async JavaScript Loading Adding the async attribute to your script tags allows the browser to continue parsing the HTML while the script is being downloaded, which can significantly speed up page load times. 4. Defer JavaScript Loading Similar to async , the defer attribute delays the execution of the script until after the HTML document has been parsed, which can be useful for non-critical scripts. 5. Favicon for Multiple Devices rel= "icon" type= "image/png" sizes= "32x32" href= "/favicon-32x32.png" > This one-liner sets up a favicon for your website, ensuring it looks good on different devices and browsers. You can include multiple sizes for better coverage. CSS One-Liners: Styling Shortcuts 6. CSS Reset in One Line * { margin : 0 ; padding : 0 ; box-sizing : border-box ;} This powerful one-liner resets default margins and paddings, and sets the box-sizing property to border-box for all elements, giving you a clean slate to work with. 7. Center Anything Vertically and Horizontally .center { display : flex ; justify-content : center ; align-items : center ;} Use this one-liner to easily center any element both vertically and horizontally within its parent container using flexbox. 8. Custom Scrollbar Styling ::-webkit-scrollbar { width : 10px ;} ::-webkit-scrollbar-thumb { background : #888 ;} This two-part one-liner allows you to customize the appearance of scrollbars in webkit-based browsers (like Chrome and Safari), making them more visually appealing. 9. Text Truncation with Ellipsis .truncate { white-space : nowrap ; overflow : hidden ; text-overflow : ellipsis ;} Apply this class to any text element to truncate it with an ellipsis (...) if it exceeds its container's width. 10. Smooth Scrolling for the Entire Page html { scroll-behavior : smooth ;} This simple one-liner enables smooth scrolling for the entire page, creating a more pleasant user experience when navigating between sections. JavaScript One-Liners: Coding Wizardry 11. Get a Random Item from an Array const randomItem = arr => arr [ Math . floor ( Math . random () * arr . length )]; This handy one-liner function allows you to quickly grab a random item from any array, which can be useful for creating dynamic content or randomizing elements on your page. 12. Check if the Current Year is a Leap Year const isLeapYear = year => new Date ( year , 1 , 29 ). getMonth () === 1 ; Use this clever one-liner to determine if a given year is a leap year. It works by checking if February 29th exists for that year. 13. Generate a Random Hex Color const randomColor = () => ' # ' + Math . floor ( Math . random () * 16777215 ). toString ( 16 ); This one-liner generates a random hex color code, which can be useful for creating dynamic color schemes or testing different color combinations. 14. Debounce Function const debounce = ( fn , delay ) => { let timeout ; return (... args ) => { clearTimeout ( timeout ); timeout = setTimeout (() => fn (... args ), delay );};}; While this one-liner is a bit more complex, it's incredibly useful. It creates a debounce function that can be used to limit the rate at which a function is called, which is great for performance optimization in things like scroll or resize event handlers. 15. Copy Text to Clipboard const copyToClipboard = text => navigator . clipboard . writeText ( text ); This simple one-liner uses the Clipboard API to copy any text to the user's clipboard, making it easy to implement "Copy" functionality in your UI. Performance One-Liners: Speed Up Your Site 16. Lazy Loading Images src= "placeholder.jpg" data-src= "actual-image.jpg" loading= "lazy" > This HTML one-liner implements lazy loading for images, which can significantly improve page load times by only loading images when they're about to enter the viewport. 17. Minify Your CSS .class { property : value } While not exactly a one-liner, this represents the concept of minifying your CSS by removing all unnecessary spaces and line breaks, which can reduce file size and improve load times. 18. Use Short CSS Color Codes .element { color : #f00 ;} /* Instead of #ff0000 */ When possible, use the shorthand version of hex color codes. This small optimization can add up when you have a large stylesheet. 19. Preconnect to External Domains rel= "preconnect" href= "https://fonts.googleapis.com" > Use this one-liner to preconnect to external domains that you'll be fetching resources from. This can speed up subsequent requests by establishing the connection early. 20. Cache-Control Header Cache-Control: public, max-age=31536000 While this is technically a server-side configuration, it's important for front-end developers to understand. This header tells browsers to cache the resource for one year, which can greatly improve load times for returning visitors. Putting It All Together: Implementing One-Liners in Your Workflow Now that we've gone through our list of 20 one-liners, you might be wondering how to best incorporate these into your daily workflow as a UI developer. Here are some tips to help you make the most of these powerful snippets: Create a Cheat Sheet : Keep a document or note with these one-liners for quick reference. You can categorize them by language (HTML, CSS, JavaScript) or by function (performance, styling, functionality). Practice Regularly : Try to use at least one new one-liner in each project you work on. The more you use them, the more naturally they'll come to you when you need them. Understand the Underlying Concepts : While one-liners are great for quick solutions, it's important to understand how they work. Take some time to break down each one-liner and learn about the concepts behind them. Customize and Expand : These one-liners are great starting points, but don't be afraid to modify them to fit your specific needs or combine them to create more complex solutions. Share with Your Team : If you're working in a team, share these one-liners with your colleagues. You can even create a shared repository of useful snippets that everyone can contribute to and benefit from. Stay Updated : The world of front-end development is constantly evolving. Keep an eye out for new one-liners and techniques that can further enhance your skills. Test and Measure : When implementing performance-related one-liners, always test and measure the impact. Use browser developer tools and performance testing sites to ensure you're actually seeing improvements. Consider Browser Compatibility : While these one-liners are generally widely supported, always check browser compatibility, especially for newer features. Conclusion: Elevate Your Front-End Game And there you have it – 20 powerful one-liners to boost your front-end skills! From HTML tricks to CSS shortcuts and JavaScript wizardry, these snippets cover a wide range of front-end development scenarios. By incorporating these one-liners into your toolkit, you'll be able to write more efficient code, solve common problems quickly, and ultimately become a more proficient UI developer. Remember, the key to mastering these one-liners is practice and understanding. Don't just copy and paste them blindly – take the time to understand how they work and why they're effective. As you become more comfortable with these techniques, you'll find yourself naturally reaching for the most efficient solutions in your day-to-day work. So, fellow UI developers, I challenge you to start using these one-liners in your projects today. Pick one or two that seem most relevant to your current work and give them a try. You might be surprised at how much of a difference these small changes can make in your overall development process. And hey, don't stop here! The world of front-end development is vast and ever-changing. Keep exploring, keep learning, and keep adding new tricks to your arsenal. Who knows? Maybe you'll come up with the next game-changing one-liner that revolutionizes the way we approach UI development. Happy coding, and may your front-end skills continue to grow and flourish!

  • DEV CommunityThe Giantsread at source

    Beta Blast: A Bouldering Bubble Shooter Built with Claude

    I asked Claude for a Puzzle Bobble clone. It built a climbing game with a global leaderboard. Beta Blast : shoot coloured climbing holds at a wall, pop three or more, send the route. Playable in your browser right now, free, no install. It started with one question: "What's a good tech stack to make a replica of Puzzle Bobble?" A few hours of chatting later I had a finished, deployed game. I wrote none of the code and that's OK. What's in it 🧗 Endless grades. V0, V1, V2 and on, getting harder with no ceiling 📅 Daily Boulder. One route a day, the same for everyone ⏱️ Survival. The wall keeps dropping, the music speeds up, and the stages ramp up 🛠️ Route Setter. Build your own wall and share it as a link 👥 2-player over the internet. Pop holds to send rows to your opponent 🏆 A global leaderboard , with a climber icon next to every score 🧢 Customisable climber. Beanie, shirt and chalk bag colours unlock as you play 🔌 A public API and a one-line embed , so any website can show the leaderboard The stack Game TypeScript, Phaser 3 , Vite Art and sound Drawn and synthesised in code. There are no image or audio files. 2-player PeerJS (WebRTC), so no game server Leaderboard Cloudflare Workers + D1 (SQLite), on the free plan Hosting GitHub Pages and a custom domain, deployed by GitHub Actions Quality 236 tests, a bot that plays every grade to check the difficulty, and CI on every pull request Why it was fun I made the creative calls, Claude did the typing. I picked the theme (climbers shooting holds), the name, and which feature came next. Claude proposed options, built them, and opened the pull requests. I mostly said "yes, do that." It checked its own work. It ran the game in a headless browser, looked at its own screenshots and fixed what looked wrong. When I asked for a full review it found real bugs, such as a network blip that ended live matches, and fixed them. It handled the boring, essential parts. Tests, CI, deploys, the Cloudflare setup, the README and the screenshots in it. Anti-cheat was a good problem. Scores come from the browser, so anyone can fake one. The game now records every shot and can replay a run on the server through the same engine to check the score. It's off on the free plan, because a replay costs more CPU than the free tier allows. Claude measured that, told me, and let me pick. That kind of trade-off is what I'd want from a human colleague too. The honest bits It wasn't one prompt. It was a long conversation, with me playing the game and steering. Things broke along the way: a deploy that needed Pages switched on, a leaderboard that looked empty because it opened on the wrong board. We fixed each one. Claude can't tap a real phone screen or feel whether a shot is satisfying. That part is still me. Try it 🎮 Play: https://thesecretstage.vip/Beta-Blast/ 💻 Code: https://github.com/OwainWilliams/Beta-Blast Add the leaderboard to your own site with one line: If you've been sitting on a "wouldn't it be cool if" project, this is your sign. Open a conversation, describe it, and see how far an afternoon goes. I've a load of other features I want to add to this, one will be an Umbraco themed version, a 4 player version and maybe some other features that I liked from old online games such as TetriNet (anyone else remember that?) If you have a feature you'd like to see, create an issue on the repo and we can chat about it.

  • DEV CommunityThe Giantsread at source

    Hotwire Marten: Updating Just the Parts That Changed

    In the first part , we wired up Hotwire in a Marten application and built a small task list. Clicking "Create task" loaded a form into a Turbo Frame. Pretty nice for something that barely involved JavaScript! But a task app where you can't complete a task feels a little broken. Nothing is done or is everything already done? Or maybe I just forgot to add some mechanisms for that 🤷🏻. Let's fix that with Turbo Streams. New tasks should appear in the list without a full page reload, completing a task should update just that item, and deleting one should remove it. Turbo, Stimulus, and importmap are already in place from the first part, so we can focus on the behavior this time. We'll start with task creation. Render a task once Before adding the streams, let's extract the markup for a task into src/templates/tasks/_task.html : id= "{% dom_id task %}" > {{ task.title }} The dom_id tag comes from marten-turbo. For a task with ID 42, it produces task_42 . We'll use that ID later to update or remove exactly this element. In tasks/list.html , replace the existing list with: id= "tasks" > class= "empty" > No tasks yet. {% for task in records %} {% include "tasks/_task.html" %} {% endfor %} The surrounding "Create task" link and empty new_task frame stay as they were. The list now has an ID too, so we can append things to it. One small detail: the empty message is always in the HTML. A little CSS handles its visibility: #tasks .empty :not ( :only-child ) { display : none ; } When a task appears, the message hides. When we delete the last task, it comes back. Conveniently, our stream responses don't have to do anything about it. Modern CSS is awesome! Also add ordering :id to Tasks::ListHandler . We're going to append new tasks at the bottom, so the initial page should use the same order. Otherwise a reload could rearrange our list. Small but important! Creating a task with Turbo Streams Our create handler already knows how to validate and save a task. We can keep that and switch its parent class to the Turbo-aware version: class Tasks :: CreateHandler MartenTurbo :: Handlers :: RecordCreate model :: Task schema TaskSchema template_name "tasks/create.html" turbo_stream_name "tasks/create.turbo_stream.html" record_context_name "task" success_route_name "home" end turbo_stream_name tells the handler which template to render after a successful Turbo submission. record_context_name makes the created record available as task , matching the variable our partial already uses. Now create src/templates/tasks/create.turbo_stream.html : {% turbo_stream 'append' 'tasks' template: 'tasks/_task.html' %} {% turbo_stream 'replace' 'new_task' do %} id= "new_task" > {% end_turbo_stream %} That's the entire response template. After saving, the task appears in the list and the form closes. Why replace the frame instead of removing it? Our "Create task" link still needs that frame as its target. Replacing it also clears the src attribute Turbo added when it loaded the form, so clicking the link again can fetch a fresh form. What about validation? A successful submission is the easy part. Let's submit an empty title. We want the form to stay open and show the error from Marten. For that, move the frame and form into src/templates/tasks/_form.html : id= "new_task" > action= "{% url 'tasks_create' %}" method= "post" > {% csrf_input %} for= "{{ schema.title.id }}" > Title: type= "text" id= "{{ schema.title.id }}" name= "{{ schema.title.id }}" > {% for error in schema.title.errors %} class= "input-error" > {{ error.message }} {% endfor %} type= "submit" > Create Then replace the frame in tasks/create.html with {% include "tasks/_form.html" %} . Keep the page's template inheritance and content block around it. I've removed the input's required attribute here so we can actually send an empty title to Marten and see its validation response. The schema still validates the title on the server: class TaskSchema Marten :: Schema field :title , :string , max_size: 255 end The old completed field is gone from this schema. Creating a task only needs a title; completion will get its own schema in a moment. Add one more setting to Tasks::CreateHandler : turbo_invalid_stream_name "tasks/create_invalid.turbo_stream.html" And create that template: {% turbo_stream 'replace' 'new_task' template: 'tasks/_form.html' %} The handler returns a 422 response for invalid submissions and renders the form with Marten's validation errors. The form stays open so we can try again. Finally, completing some work Let's add a checkbox. First, a small schema in src/schemas/task_completion_schema.cr : class TaskCompletionSchema Marten :: Schema field :completed , :bool , required: false end And an update handler in src/handlers/tasks/complete_handler.cr : class Tasks :: CompleteHandler MartenTurbo :: Handlers :: RecordUpdate http_method_names :post model :: Task schema TaskCompletionSchema turbo_stream_name "tasks/complete.turbo_stream.html" record_context_name "task" success_route_name "home" end Add its route inside Marten.routes.draw : path "/tasks/ /complete" , Tasks :: CompleteHandler , name: "tasks_complete" The handler only accepts POST requests. We don't need a separate page for editing one checkbox. Its stream template, tasks/complete.turbo_stream.html , is another one-liner: {% turbo_stream 'replace' task template: 'tasks/_task.html' %} This time we pass the record itself as the target. The tag derives its DOM ID, so task becomes something like task_42 , matching the from our partial. Now update _task.html : id= "{% dom_id task %}" {% if task.completed %} class= "completed" {% endif %} > action= "{% url 'tasks_complete' pk: task.pk %}" method= "post" > {% csrf_input %} type= "hidden" name= "completed" value= "0" > type= "checkbox" name= "completed" value= "1" aria-label= "Completed" {% if task.completed %} checked {% endif %} > type= "submit" > Save {{ task.title }} And give completed tasks a little styling: #tasks .completed span { color : #888 ; text-decoration : line-through ; } Check the box and click Save. The task is saved and shown as completed. There's a slightly sneaky detail in the form: the hidden input before the checkbox. An unchecked checkbox sends no value at all. We need an explicit 0 to clear an existing completion value. When checked, both values are sent, and Marten takes the last one: 1 . If you've used Rails form helpers, this probably looks familiar. A little Stimulus, finally The Save button works, but clicking it after every checkbox change feels unnecessary. This is a good job for Stimulus. Generate a controller: marten stimulus generate controller autosubmit Replace its generated contents with: import { Controller } from " @hotwired/stimulus " export default class extends Controller { submit () { this . element . requestSubmit () } } Then add the controller and action to the completion form: action= "{% url 'tasks_complete' pk: task.pk %}" method= "post" data-controller= "autosubmit" data-action= "change->autosubmit#submit" > The checkbox's change event bubbles up to the form, where Stimulus calls submit() . We can put the Save button inside so it's only displayed when JavaScript is disabled: type= "submit" > Save Our importmap and controller loading were already configured in the first part. Rebuild and restart the server after adding the controller so its pin appears in the generated importmap. Now clicking the checkbox submits the form, saves the task, and replaces the list item. Stimulus also connects the controller on the replacement form, so the next click works too. And that's all the custom JavaScript for this feature. The controller doesn't even know what a task is. It just submits its form. Deleting a task We can create and complete tasks now. Let's also add a way to delete them. Create src/handlers/tasks/delete_handler.cr : class Tasks :: DeleteHandler MartenTurbo :: Handlers :: RecordDelete http_method_names :post model :: Task success_route_name "home" def turbo_stream MartenTurbo :: TurboStream . remove ( record ) end end The handler also lets us return a MartenTurbo::TurboStream directly from Crystal. For deletion, that saves us a template file. Add the route: path "/tasks/ /delete" , Tasks :: DeleteHandler , name: "tasks_delete" And add a second form after the title's inside _task.html : action= "{% url 'tasks_delete' pk: task.pk %}" method= "post" > {% csrf_input %} type= "submit" > Delete Click Delete, the record is deleted, and the stream removes its list item. If that was the last task, our empty message becomes visible again through CSS. Where we ended up We now have a task list where creating, completing, and deleting tasks updates just the relevant elements on the page. What I like about the Marten integration is how little we had to add to our existing handlers. We switched to the Turbo-aware base classes, configured the stream templates, and reused the same task partial throughout. The only custom JavaScript is our small autosubmit controller, but it feels SPA like nonetheless!

  • DEV CommunityThe Giantsread at source

    # TypeScript Made Simple: 7 Concepts Every Developer Should Know

    If you have worked with JavaScript, you have probably faced situations where your code looked perfectly fine but failed because of a small mistake. Maybe you passed a string instead of a number. Maybe an API returned something you were not expecting. Or maybe you spent 20 minutes debugging a property that did not exist. This is one of the reasons I enjoy working with TypeScript. TypeScript does not magically eliminate bugs, but it helps us catch many common mistakes before our code even runs. Let's understand seven useful TypeScript concepts with simple examples. 1. Type: Tell TypeScript What You Expect Imagine you are building an application where every user has a name and an age. In JavaScript, you can accidentally assign the wrong value without getting an immediate warning. TypeScript helps you define what values are expected. let userName : string = " Alex " ; let age : number = 25 ; age = " twenty-five " ; // TypeScript error Think of types as basic rules for your data. They help you catch mistakes while writing code instead of discovering them later. 2. Interface: Define the Shape of an Object Suppose your application receives user information from an API. Instead of guessing which properties the user object should contain, you can define its structure. interface User { id : number ; name : string ; email : string ; } const user : User = { id : 1 , name : " Alex " , email : " alex@example.com " , }; Now TypeScript can warn you if you forget a required property or provide an incorrect type. In simple words: An interface describes what an object should look like. 3. Type vs Interface: Are They the Same? This is one of the most common questions developers ask. Both can describe an object's structure. interface Product { name : string ; price : number ; } type Customer = { name : string ; email : string ; }; So, why have both? The main difference becomes clearer when you need more advanced features. For example, type can represent a union of different values: type Status = " pending " | " success " | " failed " ; An interface is often a natural choice for object contracts, while a type alias is particularly useful for unions, intersections, and other type combinations. There is some overlap, so you do not need to overthink the choice for every object. 4. Union Types: Allow More Than One Possibility Imagine an API that returns a user ID as either a number or a string. You can represent both possibilities with a union type. let userId : string | number ; userId = 101 ; userId = " USER-101 " ; The | symbol means the value can be one type or another. You can also use this for application states: type OrderStatus = | " pending " | " shipped " | " delivered " | " cancelled " ; This is much safer than allowing any random string. For example, "shiped" would be rejected because it is not one of the allowed values. 5. Generics: Write Code That Works With Different Types Suppose you need a function that returns whatever value you pass to it. You could write separate functions for strings, numbers, and objects. But that would create unnecessary repetition. Generics solve this problem. function getValue T > ( value : T ): T { return value ; } const name = getValue ( " Alex " ); const age = getValue ( 25 ); Here, T is a placeholder for a type. TypeScript can infer that name is a string and age is a number. You get reusable code without losing type information. Think of generics as a way to make one function work with different data types while still respecting their types. 6. Utility Types: Reuse Existing Types Sometimes you already have a type but need a slightly different version of it. For example, you have a user model but want to create a user without requiring every property. interface User { id : number ; name : string ; email : string ; } type CreateUser = Omit User , " id " > ; Now CreateUser contains only name and email . Here are a few useful utility types: Partial — makes all properties optional. Pick — selects specific properties. Omit — removes specific properties. Required — makes all properties required. Readonly — makes properties read-only at the type level. These become especially useful when working with API requests, database models, and update operations. 7. Type Narrowing: Check Before You Use a Value Imagine a function accepts either a string or a number. You cannot safely call string-specific methods until you know that the value is actually a string. function printValue ( value : string | number ) { if ( typeof value === " string " ) { console . log ( value . toUpperCase ()); } else { console . log ( value . toFixed ( 2 )); } } TypeScript understands the condition and narrows the type inside each branch. This makes your code safer and easier to understand. You can also narrow types using checks such as in , equality comparisons, and custom type guards. One Important Thing to Remember TypeScript checks types during development and compilation, but its types do not automatically validate incoming data at runtime. For example, an API response can still contain unexpected data even if you have defined an interface for it. For external data, use runtime validation when necessary. TypeScript helps you write safer code, but it is not a replacement for proper validation and testing. Final Thoughts You do not need to master every advanced TypeScript feature to start using it effectively. Start with basic types, interfaces, union types, and type narrowing. Once you are comfortable with those, explore generics and utility types. The goal is not to write complicated types everywhere. The goal is to make your code easier to understand, maintain, and debug. What TypeScript concept took you the longest to understand? Share your experience in the comments. It might help another developer who is learning the same thing.

  • SitePointThe Giantsread at source

    Reverse Engineering with AI: Trace Features Across Electron and Native Apps

    Trace features through Electron IPC boundaries and native binaries using REA, an open-source MCP server and CLI for AI coding agents. Continue reading Reverse Engineering with AI: Trace Features Across Electron and Native Apps on SitePoint .

  • Hacker News Front PageThe Giantsread at source

    French supermarkets suffer refrigerator shutdown after unprecedented cyberattack

  • Hacker News Front PageThe Giantsread at source

    GLM-Legal: Advancing Agentic Legal Intelligence

  • Hacker News Front PageThe Giantsread at source

    French supermarkets suffer fridge shutdown after 'unprecedented' cyberattack

  • Hacker News Front PageThe Giantsread at source

    Fridges in French supermarkets hit by cyber attack

  • Hacker News Front PageThe Giantsread at source

    Deep dive into Azure Maia 300 architecture

  • SitePointThe Giantsread at source

    Designing a Safer Account Recovery Backend for Consumer Apps

    null Continue reading Designing a Safer Account Recovery Backend for Consumer Apps on SitePoint .

  • SitePointThe Giantsread at source

    Beyond the Password Reset: Building Recovery Experiences That Protect Online Shoppers

    null Continue reading Beyond the Password Reset: Building Recovery Experiences That Protect Online Shoppers on SitePoint .

  • SitePointThe Giantsread at source

    Designing Reliable Offline Experiences for Modern Web Applications

    null Continue reading Designing Reliable Offline Experiences for Modern Web Applications on SitePoint .

  • SitePointThe Giantsread at source

    Beyond Lazy Loading: How to Build Faster Websites with Responsive Images

    null Continue reading Beyond Lazy Loading: How to Build Faster Websites with Responsive Images on SitePoint .

  • SitePointThe Giantsread at source

    Building Accessible Web Forms with HTML, CSS, and JavaScript

    null Continue reading Building Accessible Web Forms with HTML, CSS, and JavaScript on SitePoint .

  • SitePointThe Giantsread at source

    Test README with Users and Docker: Two-Layer Verification

    Verify your project setup documentation using a two-layer testing workflow: automate commands in a clean Docker container to catch mechanical setup errors, then observe a real user think aloud to identify missing prerequisites and unclear steps. Continue reading Test README with Users and Docker: Two-Layer Verification on SitePoint .

  • SitePointThe Giantsread at source

    AI Agent Tool Calling in TypeScript: Guardrails Against Runaway Loops

    Add bounded retries, execution timeouts, and a best-effort budget guard to your TypeScript AI agent tool-calling loop to stop runaway LLM API loops and token overruns. Continue reading AI Agent Tool Calling in TypeScript: Guardrails Against Runaway Loops on SitePoint .

  • SitePointThe Giantsread at source

    Self Host Next.js in Production with Docker Standalone

    Learn how to self-host Next.js in production using Docker standalone output, Nginx reverse proxying, health checks, and shared cache handling across containers. Continue reading Self Host Next.js in Production with Docker Standalone on SitePoint .

  • SitePointThe Giantsread at source

    Tailwind CSS v4 Dynamic Themes Using Semantic CSS Tokens

    Combine multiple color themes and dark modes in Tailwind CSS v4 using @theme inline and CSS custom properties, enabling utilities like bg-primary to adapt at runtime without dark: variants. Continue reading Tailwind CSS v4 Dynamic Themes Using Semantic CSS Tokens on SitePoint .

  • SitePointThe Giantsread at source

    Tilde in PATH Security: How Quoted Tildes Cause Relative Execution

    Quoting a tilde in PATH suppresses expansion, creating a literal '~' entry that resolves relative to the current working directory. If an attacker can write to that relative path and PATH search order reaches it, the shell executes the foreign binary. Here is how to audit your shell configuration and replace literal tildes with $HOME. Continue reading Tilde in PATH Security: How Quoted Tildes Cause Relative Execution on SitePoint .

  • SitePointThe Giantsread at source

    Flexbox space-between Not Working on Wide Screens? How to Fix It

    Understand why justify-content: space-between fails on wide screens, how flex-grow consumes distributable free space, and how to restore alignment using proper flex item sizing. Continue reading Flexbox space-between Not Working on Wide Screens? How to Fix It on SitePoint .

  • SitePointThe Giantsread at source

    Building Reliable Marketplace Listing Workflows: A Developer’s Guide

    null Continue reading Building Reliable Marketplace Listing Workflows: A Developer’s Guide on SitePoint .

  • Smashing MagazineThe Giants6 min read

    A Practical Guide To Naming Things

    A practical guide to naming UI components, with useful resources, naming conventions, and structures. You’ll find more [useful guides](https://www.smashingmagazine.com/category/guides/) and [online workshops](https://smashingconf.com/online-workshops) here on Smashing Magazine, so make sure to browse and explore.

  • Smashing MagazineThe Giants12 min read

    Under Autumn’s Spell (October 2026 Wallpapers Edition)

    October is just around the corner, which means it’s time for some new desktop wallpapers! Designed by the community for the community, they’re here to brighten your screens, and maybe spark some creativity along the way.

  • Smashing MagazineThe Giants5 min read

    Stop Treating CSS Container Queries Like Traditional Media Queries

    Despite broad browser support, container queries remain surprisingly underused and frequently misunderstood. Let’s look at how they differ from media queries, when to reach for each, and how container queries help reusable components respond naturally to the contexts in which they appear. - CSS - Tools - Techniques

  • CSS-TricksThe Giantsread at source

    Let’s Use the Emergent CSS random() Function in all the Browsers

    The journey to create a polyfill for the upcoming CSS random() function that works in all browsers. Let’s Use the Emergent CSS random() Function in all the Browsers originally handwritten and published with love on CSS-Tricks . You should really get the newsletter as well.