Today
Front-end, in one read.
The latest from the sources worth following, newest first.
David BushellMore Front-end Bloggers4 min read
A sustainable web career, for when all this blows over
For better or worse the web industry is going through a bit of a phase. The reasons are largely irrational. People still need the web, nothing has changed there. Regardless, the financials of this business are a struggle. Longterm career prospects are looking dicey. […]
DEV CommunityThe Giantsread at source
Mass Assignment: The One-Line API Bug Hiding in Your Update Endpoint
This article was created with the help of AI and reviewed, tested and edited by me before publishing. Here's an endpoint that looks completely fine in code review: app . patch ( ' /api/users/me ' , requireAuth , async ( req , res ) => { const user = await User . findByIdAndUpdate ( req . user . id , req . body , { new : true }); res . json ( user ); }); It's authenticated. It only updates the logged-in user. It's also vulnerable, because req.body goes straight into the database. If your User model has a role field, any user can send this: PATCH /api/users/me Content-Type: application/json { "displayName": "Ana", "role": "admin" } and promote themselves. That's mass assignment : the client sets object properties it was never meant to touch. OWASP folds it into API3:2023, Broken Object Property Level Authorization , and MITRE tracks it as CWE-915 . Let's find it, prove it, and fix it. Why it happens Frameworks make binding request data to models easy, which is great until the model grows. The endpoint above might have been safe on day one, when User only had displayName and email . Then someone added role , isVerified , credits or tenantId , and the update endpoint silently started accepting them too. Typical sensitive fields to watch for: privilege: role , isAdmin , permissions state: isVerified , emailConfirmed , status money: balance , credits , plan , discount ownership: ownerId , tenantId , orgId Finding it in an API you're allowed to test Read the responses, not just the docs. A GET /api/users/me that returns role , plan or tenantId tells you which properties exist. Send them back. Take the response object, change one sensitive field, and send it in the update request. Check the result with a fresh read. Some APIs echo your input in the response without saving it. Only a follow-up GET (or a behaviour change, like suddenly reaching an admin route) proves the write happened. Try create endpoints too. POST /api/projects with an extra ownerId is the same bug. Try nested objects. { "profile": { "verified": true } } slips past filters that only check top-level keys. Only do this on your own apps or targets you're authorised to test. Fix 1: allowlist the fields you accept The OWASP Mass Assignment Cheat Sheet recommends allowlisting bindable fields over blocklisting dangerous ones, because a blocklist breaks the moment a new sensitive field is added. const pick = ( obj , keys ) => Object . fromEntries ( keys . filter (( k ) => k in obj ). map (( k ) => [ k , obj [ k ]])); const USER_EDITABLE = [ ' displayName ' , ' bio ' , ' avatarUrl ' ]; app . patch ( ' /api/users/me ' , requireAuth , async ( req , res ) => { const updates = pick ( req . body , USER_EDITABLE ); const user = await User . findByIdAndUpdate ( req . user . id , updates , { new : true }); res . json ( toPublicUser ( user )); }); Fix 2: validate with a strict schema A schema validator that rejects unknown keys turns silent acceptance into a clear 400 . With zod : import { z } from ' zod ' ; const UpdateMe = z . object ({ displayName : z . string (). min ( 1 ). max ( 80 ). optional (), bio : z . string (). max ( 500 ). optional (), }). strict (); // unknown keys like "role" fail validation app . patch ( ' /api/users/me ' , requireAuth , async ( req , res ) => { const parsed = UpdateMe . safeParse ( req . body ); if ( ! parsed . success ) return res . status ( 400 ). json ({ error : ' Invalid fields ' }); const user = await User . findByIdAndUpdate ( req . user . id , parsed . data , { new : true }); res . json ( toPublicUser ( user )); }); Rejecting is better than silently dropping: clients find out quickly, and an attempted role change shows up in your logs. Fix 3: control what goes out, too API3 covers both directions. The toPublicUser function above matters because returning the raw model leaks internal fields (password hashes, internal flags) and hands attackers the list of fields to try. Use explicit response DTOs, not res.json(dbObject) . Lock it in with a test The bug comes back whenever someone adds a field, so make it a regression test: test ( ' users cannot change their own role ' , async () => { const agent = await loginAs ( ' regular-user ' ); await agent . patch ( ' /api/users/me ' ). send ({ role : ' admin ' }). expect ( 400 ); const me = await agent . get ( ' /api/users/me ' ). expect ( 200 ); expect ( me . body . role ). toBeUndefined (); // not exposed at all }); Add one of these for every sensitive field and every endpoint that writes to the model, including create endpoints and admin-only ones (can a support role set role: "owner" ?). Checklist [ ] No endpoint passes raw req.body to an ORM update or create [ ] Writable fields are allowlisted or strictly validated per endpoint and per role [ ] Unknown fields are rejected with a 400 , not silently ignored [ ] Responses use explicit DTOs [ ] Regression tests try sensitive fields on every write endpoint Mass assignment is rarely clever. It's a convenience left in place after the model changed, which is exactly why it's worth a test that never forgets. Anusha Dirisala is the founder of Pentrova (pentrova.ai), a self-serve AI penetration testing platform for web apps and APIs, based in Hyderabad.
DEV CommunityThe Giantsread at source
How Do You Load-Test an Application That Depends on AI APIs?
You built a feature on a third-party LLM API. It's fast in dev, the demo dazzled everyone, and it's now in the critical path of your product. Then you go to load-test it the way you'd load-test any service—ramp up virtual users, watch throughput, find the breaking point—and you discover the rules you've relied on for years no longer apply. A normal load test assumes your dependencies scale roughly with your traffic and behave predictably. An external AI API violates both assumptions. It will rate-limit you at a ceiling you don't control. It bills you per token, so the load test itself costs real money. Its latency swings wildly based on factors invisible to you—prompt length, model load, time of day. And it has its own outages that instantly become yours. Load-testing an AI-dependent application is a genuinely different discipline. If you test it like a conventional app, you'll get a green report and a production incident. Here's what actually has to be tested. Test the Rate Limit, Because You Will Hit It Every commercial LLM API enforces rate limits—requests per minute, tokens per minute, or both. Under normal traffic you never notice. Under load, the rate limit becomes the first thing that breaks, and how your application behaves at that boundary determines whether a traffic spike is a minor slowdown or a full outage. So make hitting the rate limit an explicit test scenario. Drive load past the published limit and observe what your application does when the API starts returning 429 Too Many Requests responses —the standard status code APIs use to signal rate limiting. The behaviors you're checking for: Does it queue gracefully or pile up requests until memory and connection pools exhaust and the whole service falls over? Does it back off intelligently —honoring Retry-After headers with jittered exponential backoff —or hammer the API and make the throttling worse? Adding randomness to retry timing is what stops every client from retrying in lockstep and amplifying the overload. Does it shed load by degrading non-critical AI features so critical paths keep working? A surprising number of applications cascade from one rate-limited dependency into total failure because nobody tested the boundary. The rate limit isn't an edge case to avoid in your load test. It's the most important scenario in it. Watch the Money, Not Just the Milliseconds Here's a dimension conventional load testing never had to think about: cost scales with load. Because LLM APIs bill per token, a 10x traffic spike isn't just a performance event—it's a 10x cost event, and potentially much worse if longer prompts or larger responses come with heavier traffic. This has two consequences for how you test. Measure cost as a first-class load-test output. Instrument token consumption and dollar cost per request, and report cost-per-thousand-requests right alongside latency and throughput. A configuration that's fast but burns tokens may be unaffordable at scale, and you want to know that before production does. Budget for the test itself. Running realistic load against a metered API costs real money. Plan for it. Use cheaper or smaller models where the test is about your system's behavior rather than output quality, and reserve full-cost runs for the scenarios that genuinely need them. A test environment that hits a mock for most runs and the real API for targeted runs keeps the bill sane. Modeling cost under load also surfaces a strategic question worth answering early: at your projected scale, does the unit economics of this AI feature even work? Better to learn that in a load test than in a board meeting. Test for Latency Variance, Not Just Average Latency Conventional load tests often report average response time. With LLM APIs, the average is nearly useless, because the latency distribution is wide and lumpy. The same request can return in 800ms or 6 seconds depending on the model's current load, the response length, and whether you're streaming. Test against the tail, not the mean. Report p95 and p99, never just the average. Your users feel the tail, and with LLM latency the tail can be many times the median. Test with realistic payload distributions. Latency scales with input and output tokens, so a load test using tiny prompts will wildly understate real-world latency. Mirror your production prompt and response sizes. Validate your timeouts under load. A timeout that's generous at low traffic may fire constantly when the API slows down under stress, turning a slowdown into a flood of failures. Tune timeouts against observed p99, not against a hopeful guess. The goal is to know how your application feels when the model is having a slow stretch—because it will, and you don't control when. The Test That Matters Most: Degradation and Fallback This is the one most teams skip, and it's the one that saves them. The single most important load-test scenario for an AI-dependent application is: what happens when the AI API degrades or goes down entirely? Third-party model APIs have bad days—elevated latency, elevated error rates, full outages. When that happens during a traffic peak, your application's resilience is the only thing standing between a degraded experience and a hard outage. You cannot afford to discover your fallback path for the first time in production. So inject failure deliberately and test under load: Simulate the API being slow. Inject artificial latency and confirm your timeouts, circuit breakers, and user experience hold up. Simulate the API erroring. Force 5xx and 429 responses and verify your fallback activates—whether that's a cached response, a smaller backup model, a simpler non-AI path, or a graceful "try again shortly" message. Simulate the API being completely down. Confirm your application stays up and degrades gracefully rather than failing wholesale. A circuit breaker that's never been tested under load is a hope, not a safeguard. The same goes for a fallback model you've configured but never actually exercised. Load testing is where you prove these work before a real incident proves they don't. Build a Realistic Test Harness Putting this together, an AI-aware load test needs a few things a conventional one doesn't. A mock mode and a live mode. Most of your iteration runs against a mock that mimics the API's latency distribution, rate limits, and error behavior—cheap, fast, and safe. Targeted runs hit the real API to validate the mock's fidelity and catch behaviors only the real provider exhibits. Realistic traffic shapes. AI features often have bursty, uneven usage. Test the spikes, not just steady-state ramps, because the rate-limit and cost behaviors are worst exactly during bursts. Provider-side observability. Track not just your own metrics but the API's reported rate-limit headers, token usage, and error rates so you can see how close you're running to the provider's ceilings. Multi-provider and fallback paths in scope. If your resilience strategy depends on failing over to a second provider or a backup model, that path has to be in the load test too. An untested failover is a liability. Where This Fits in Your Reliability Practice For a platform or SRE team, AI-dependent load testing isn't a one-time exercise before launch. The provider changes their limits, raises their prices, updates their models, and has incidents on their own schedule—all outside your control. Make AI-dependency load testing a recurring part of your reliability practice, re-run it when you change models or providers, and keep your fallback paths exercised. At Particle41, when we build AI-dependent systems, we treat the model API as exactly what it is—an external dependency with its own limits, costs, and failure modes—and we design and load-test the degradation paths as part of the build, not as an afterthought. Pairing senior engineers with AI agents means the people designing the resilience strategy deeply understand how these APIs actually behave under stress. The result is a system that stays up and stays affordable when the AI it depends on has a bad day. The Checklist Before you call an AI-dependent application production-ready, confirm your load testing answered five questions. At the rate limit, does the app queue and back off gracefully, or cascade into failure? At target traffic, what does it cost in tokens—and does that unit economics work? At p99, is latency acceptable with realistic payload sizes? When the API degrades, does your fallback actually activate under load? When the API is fully down, does the application stay up and degrade gracefully? If you can't answer all five from real test data, you haven't load-tested the application—you've load-tested the happy path and left production to test the rest. With an AI API in your critical path, that's a bet you don't want to make. Sources REST Security Cheat Sheet, OWASP Cheat Sheet Series Exponential Backoff And Jitter, AWS Architecture Blog (Marc Brooker, 2015) Originally published at particle41.com
DEV CommunityThe Giantsread at source
Circuit-Breaking NHTSA Calls So One Slow Upstream Does Not Stall Every VIN Form
Error-rate breakers trip when NHTSA is failing hard. A quieter failure mode is worse for forms: the upstream is merely slow . One hung DecodeVinValues call holds a shared client, saturates your concurrency budget, and every other VIN submit waits behind a spinner that never resolves. Retries with longer timeouts make the pile deeper. This post is about latency-aware circuit breaking for free VIN decode: treat sustained slowness as a trip condition, fail fast for concurrent forms while open, and keep one slow call from stalling the whole UI surface. Classic consecutive-failure breakers are covered elsewhere; here the focus is slow upstream stalls every form . Slow is not the same as down Signal Typical user experience Breaker role Hard 5xx / network error Immediate error card Count as failure Sparse 200 with Make/Model Partial but usable card Success (not a trip) Hang past budget Spinner forever; other forms blocked Count as slow failure Burst of 8s+ latencies Queue of submits backs up Open on rate of slow calls A form that shares one decode client (module singleton, server action pool, or browser fetch queue) is especially fragile. Without a breaker, the first hung request occupies the only in-flight slot; the second submit waits; the third looks "broken" even though validation is fine. What should count as slow failure Define an explicit per-call budget (for example 6–8s wall time). When the call exceeds the budget -- via AbortSignal.timeout , a racing Promise , or your HTTP client's deadline -- record a slow failure for the breaker, abort the in-flight work, and return a typed UPSTREAM_SLOW result. Count toward the trip: Timeouts and deadline aborts Network errors that leave the request unresolved HTTP 502 / 503 / 504 after a long wait (still a failure; also slow) Do not count: Local validation rejects (never called upstream) Fast 200s with empty ABS / Make gaps (honest sparse success) User-aborted navigations when you already cancelled cleanly Mixing validation noise into slow-failure counts opens the circuit on bad paste days and blocks good VINs for no reason. TypeScript sketch: latency trips + form gate type BreakerState = " closed " | " open " | " half_open " ; export type LatencyBreakerConfig = { slowThresholdMs : number ; // e.g. 7000 failureThreshold : number ; // e.g. 3 consecutive slow/hard failures cooldownMs : number ; // e.g. 20_000 halfOpenMaxProbes : number ; // e.g. 1 }; export class LatencyCircuitBreaker { private state : BreakerState = " closed " ; private consecutiveFailures = 0 ; private openedAt = 0 ; private halfOpenProbes = 0 ; constructor ( private readonly cfg : LatencyBreakerConfig ) {} canRequest (): boolean { if ( this . state === " closed " ) return true ; if ( this . state === " open " ) { if ( Date . now () - this . openedAt >= this . cfg . cooldownMs ) { this . state = " half_open " ; this . halfOpenProbes = 0 ; return true ; } return false ; } return this . halfOpenProbes this . cfg . halfOpenMaxProbes ; } beforeRequest (): void { if ( this . state === " half_open " ) this . halfOpenProbes += 1 ; } recordSuccess (): void { this . consecutiveFailures = 0 ; this . state = " closed " ; this . halfOpenProbes = 0 ; } recordFailure (): void { this . consecutiveFailures += 1 ; if ( this . state === " half_open " || this . consecutiveFailures >= this . cfg . failureThreshold ) { this . state = " open " ; this . openedAt = Date . now (); this . halfOpenProbes = 0 ; } } } export type DecodeResult T > = | { ok : true ; data : T } | { ok : false ; reason : " circuit_open " | " upstream_slow " | " upstream " }; export async function decodeWithLatencyBreaker T > ( breaker : LatencyCircuitBreaker , call : ( signal : AbortSignal ) => Promise T > , cfg : { slowThresholdMs : number }, ): Promise DecodeResult T >> { if ( ! breaker . canRequest ()) { return { ok : false , reason : " circuit_open " }; } breaker . beforeRequest (); const controller = new AbortController (); const timer = setTimeout ( () => controller . abort (), cfg . slowThresholdMs , ); const started = Date . now (); try { const data = await call ( controller . signal ); clearTimeout ( timer ); breaker . recordSuccess (); return { ok : true , data }; } catch ( err ) { clearTimeout ( timer ); breaker . recordFailure (); const timedOut = controller . signal . aborted && Date . now () - started >= cfg . slowThresholdMs - 50 ; return { ok : false , reason : timedOut ? " upstream_slow " : " upstream " , }; } } Gate every form submit on canRequest() before disabling inputs or starting a spinner. When the circuit is open, return immediately with circuit_open so sibling forms stay interactive. UX that does not stall the form Prefer: Instant "decode temporarily unavailable -- upstream is slow" when open Disable only the submit that would call NHTSA; keep paste/clear/help controls alive Optional stale last-good decode for that VIN, labeled "cached earlier; live refresh paused" Avoid: A global page-level spinner held by one hung fetch Mapping circuit_open or upstream_slow to "invalid VIN" Silent retries that re-queue the same hung call behind the open circuit Keep reason codes separate ( CIRCUIT_OPEN vs UPSTREAM_SLOW vs VIN_INVALID ) so support and status copy stay accurate. Quick checks import assert from " node:assert/strict " ; const breaker = new LatencyCircuitBreaker ({ slowThresholdMs : 50 , failureThreshold : 2 , cooldownMs : 60 _000 , halfOpenMaxProbes : 1 , }); async function hang ( _signal : AbortSignal ): Promise never > { await new Promise (() => {}); throw new Error ( " unreachable " ); } const a = await decodeWithLatencyBreaker ( breaker , hang , { slowThresholdMs : 50 , }); assert . equal ( a . ok , false ); if ( ! a . ok ) assert . equal ( a . reason , " upstream_slow " ); const b = await decodeWithLatencyBreaker ( breaker , hang , { slowThresholdMs : 50 , }); assert . equal ( b . ok , false ); assert . equal ( breaker . canRequest (), false ); const blocked = await decodeWithLatencyBreaker ( breaker , hang , { slowThresholdMs : 50 , }); assert . deepEqual ( blocked , { ok : false , reason : " circuit_open " }); Review rule: form submit handlers must short-circuit on open before awaiting upstream. Never let one slow call own the only concurrency slot without a budget. Takeaway One slow NHTSA call should not freeze every VIN form. Budget each decode, count timeouts as failures, open the circuit before the queue piles up, and fail fast with an honest "temporarily unavailable" state. Layer latency breakers with validation and single-flight, and free VIN decode stays responsive when the public API merely crawls instead of crashing. I maintain VIN Lookup , a free VIN decode based on NHTSA data.
DEV CommunityThe Giantsread at source
Pagination Done Right: Cursor vs. Offset in Practice
Here's a query that's been copy-pasted into more codebases than anyone could count: SELECT id , title , created_at FROM posts ORDER BY created_at DESC LIMIT 20 OFFSET 40 ; It reads well. It passes code review. It works in every test you'll write, because your tests don't insert rows between page two and page three. And in production, on any table people are actively writing to, it will show some users the same post twice and quietly hide others from them entirely. That second failure is the one worth talking about. Everyone knows OFFSET gets slow on deep pages. Fewer people have noticed that it's also wrong on a live table, and fewer still know that the popular fix, the cursor, has its own concurrency hole that doesn't show up in blog-post benchmarks. Let's go through both, properly. Offset is a position, and positions move LIMIT 20 OFFSET 40 doesn't mean "the third page of posts." It means "run the whole query again, from scratch, walk past the first 40 rows of whatever the result is right now , and give me the next 20." The page boundary is a number, and the number is recomputed against the current state of the table on every request. Walk it through with a feed sorted newest-first: You load page 1 ( OFFSET 0 ). You see posts 100 down to 81. While you're reading, two new posts get published: 101 and 102. You click "next" ( OFFSET 20 ). The query now counts from post 102. Positions 1 to 20 are 102 down to 83. Positions 21 to 40 are 82 down to 63. Page 2 opens with posts 82 and 81. You already saw those. Inserts ahead of your position push rows toward you, so you see duplicates. Deletes do the opposite. If two posts on page 1 get deleted before you click next, everything shifts up by two, and the two posts that were sitting at positions 21 and 22 slide onto page 1, a page you've already left. You never see them. Use The Index, Luke puts it in one line: "the pages drift when inserting new sales because the numbering is always done from scratch." Microsoft's EF Core docs say the same thing about Skip / Take : "If any updates occur concurrently, your pagination may end up skipping certain entries or showing them twice." Slack hit exactly this as their APIs grew; their engineering write-up on moving away from offsets notes that with frequently added items, "the page window becomes unreliable, potentially skipping or returning duplicate results." Duplicates are annoying. Skips are the dangerous one, because nothing looks broken. A user scrolling a feed won't notice. A nightly job paging through orders to push them into a warehouse also won't notice, and that's how you end up with a reconciliation report that's off by a handful of rows every day and nobody can say why. The ORDER BY you forgot is its own bug Before we even get to concurrency, there's a quieter version of the same problem. The Postgres docs are unusually blunt about it: "When using LIMIT , it is important to use an ORDER BY clause that constrains the result rows into a unique order. Otherwise you will get an unpredictable subset of the query's rows." And then, a sentence later: the planner "takes LIMIT into account when generating query plans, so you are very likely to get different plans (yielding different row orders) depending on what you give for LIMIT and OFFSET ." Change the offset, maybe get a different plan, maybe get a different order. "This is not a bug," the docs add. SQL simply never promised you an order you didn't ask for. So ORDER BY created_at DESC on its own isn't enough. The moment two posts share a timestamp (bulk imports, seeded data, anything written in the same transaction with now() ), their relative order is up to the planner, and a page boundary that lands between them can hand you either one, or both, or neither. The EF Core docs add a detail people get wrong constantly: "relational databases do not apply any ordering by default, even on the primary key." The fix is boring and mandatory: always end your sort with a unique column. ORDER BY created_at DESC , id DESC That tiebreaker matters for offset and for cursors. For cursors it's not optional at all, as you'll see in a second. Why deep offsets are slow The performance story is simpler, and it's the one everyone's heard, but the mechanism is worth a sentence. From the Postgres docs: "The rows skipped by an OFFSET clause still have to be computed inside the server; therefore a large OFFSET might be inefficient." There's no way around that with a B-tree. You'd think an index could just jump to "row 40,000," but it can't. As the CedarDB team points out, "the branches of a B-Tree do not contain a fixed number of tuples," so there's nothing to compute a jump from. The database walks the index from the start and throws away every row before your offset. Slack's version: "the database still has to read up to offset + count rows from disk." Page 1 costs 20 rows. Page 2,000 costs 40,000 rows to return 20. How much that hurts depends on your data, your index, and whether the rows are cached, so I won't hand you a made-up millisecond figure. Markus Winand's benchmark chart on Use The Index, Luke shows the difference between offset and the seek method becoming "clearly visible from about page 20 onwards," and the curve only gets steeper from there. Keyset: remember the row, not the number The fix for both problems is the same idea. Instead of telling the database how many rows to skip, tell it where you stopped . This is keyset pagination, also called the seek method. -- page 1 SELECT id , title , created_at FROM posts ORDER BY created_at DESC , id DESC LIMIT 20 ; -- page 2: pass in the (created_at, id) of the last row on page 1 SELECT id , title , created_at FROM posts WHERE ( created_at , id ) ( $ 1 , $ 2 ) ORDER BY created_at DESC , id DESC LIMIT 20 ; That WHERE (created_at, id) is a row-value comparison. It's lexicographic: compare created_at first, and only if they're equal, compare id . With an index on (created_at, id) , Postgres seeks straight to that point in the index and reads 20 entries. Page 2,000 costs about what page 1 costs. CREATE INDEX posts_created_at_id_idx ON posts ( created_at , id ); A B-tree can be scanned in either direction, so this one index serves both ASC, ASC and DESC, DESC . And here's why it survives concurrent writes: the boundary is a value , not a position. New posts published after you loaded page 1 have a larger created_at , so they sort before your bookmark and the filter excludes them. Posts deleted from page 1 don't matter, because you're not counting them. As the CedarDB post puts it, "the last element that we saw is never part of the next page." This is also why the tiebreaker is mandatory here. If you paginate on created_at alone and 30 rows share the same timestamp, WHERE created_at skips every one of them that didn't fit on the previous page. With (created_at, id) , the id picks up exactly where you left off inside the tie. The database-specific bits Row values are standard SQL, but support is uneven, and this is where a lot of keyset implementations silently fall back to a full scan. PostgreSQL handles it well. Use The Index, Luke notes that PostgreSQL (since 8.4) and Db2 LUW (since 10.1) properly support row value predicates and use them to access the index. MySQL evaluates the row comparison correctly, but its optimizer doesn't always use the full index for it. The MySQL 8.4 manual's section on row constructor optimization shows (c2, c3) > (1, 1) using only part of an index, and says: "Rewriting the row constructor expression using an equivalent nonconstructor expression may result in more complete index use." EF Core can't express row values in LINQ yet (the docs point at issue #26822), so you write the expanded form. The expanded form works everywhere. It's uglier, and it's what you should reach for on MySQL: -- MySQL: expand the row comparison so the optimizer can use the whole index SELECT id , title , created_at FROM posts WHERE created_at ? OR ( created_at = ? AND id ? ) ORDER BY created_at DESC , id DESC LIMIT 20 ; Check the plan with EXPLAIN either way. A keyset query that ends up doing a filesort over the whole table is an offset query with extra steps. Gotchas that bite in production Mixed sort directions. ORDER BY price ASC, id DESC can't be expressed as a single row-value comparison, because the tuple comparison runs every column in the same direction. You have to write the expanded OR form by hand, with > on one column and on the other. Easier: keep the tiebreaker in the same direction as the main key unless the product really needs otherwise. Nullable sort columns. A comparison against NULL isn't true or false, it's NULL , so rows where the sort key is NULL fall out of a WHERE (a, id) filter. If you sort on a nullable column, you need NULLS FIRST / NULLS LAST plus explicit handling in the predicate, or you coalesce to a sentinel. Better yet, paginate on columns that are NOT NULL . Mutable sort keys. Keyset handles inserts and deletes cleanly, but if you sort by something that changes, like updated_at or score , a row can move from ahead of your bookmark to behind it while you're paging. It'll either be skipped or show up twice, and no cursor design fixes that. It's inherent to paginating by a value that moves. The concurrency hole cursors don't close This is the part most pagination articles leave out, and it's the one that matters most if you're using pagination for sync rather than for a human scrolling a feed. Keyset assumes that once you've read past a key, no new row will ever appear behind it. For ascending ids and timestamps, that sounds safe. It isn't. The Sequin team spells it out: "There are no locks on Postgres sequences, so Postgres sequences can commit out-of-order. And timestamps are no better; when setting updated_at to now() , you can have a row with an older timestamp commit after a row with a newer timestamp." Here's the sequence, as a hypothetical: Transaction A inserts an order and gets id = 1001 from the sequence. It then does some slow work and hasn't committed. Transaction B inserts an order, gets id = 1002 , and commits right away. Your sync job pages forward: WHERE id > 1000 ORDER BY id LIMIT 100 . It sees 1002 (committed) but not 1001 (not visible yet). It stores 1002 as its cursor. Transaction A commits. Row 1001 now exists. The next run asks for id > 1002 . Row 1001 is never read. No duplicates, no errors, one missing order. It's the exact failure that offset has, just rarer and harder to reproduce, because it needs two transactions to overlap at the page boundary. now() in Postgres returns the transaction start time, so a long transaction can commit a row with a timestamp older than rows that are already visible, and the same skip happens with created_at cursors. For a user scrolling a feed, this rarely matters. For a job whose whole purpose is "don't miss any rows," it's the bug. The mitigations all trade something: Lag the cursor. Only page up to rows older than some safety margin, like created_at , so in-flight transactions have time to commit before you read past them. The margin has to be longer than your longest write transaction, and you give up freshness. Re-read an overlap window. Start each run a bit before the last cursor and dedupe on id. Cheap and effective if your consumer is idempotent, which it should be anyway. Use the transaction snapshot. Postgres exposes the snapshot through functions like pg_current_snapshot() and pg_snapshot_xmin() , which tell you the oldest transaction still in flight. Change-data-capture tools lean on this kind of information. It's more machinery than most teams need. Don't paginate for sync at all. If what you really want is "every change, exactly once," a change feed (logical replication, an outbox table, CDC) is the right tool. Pagination is for reading a list, not for replicating one. Sequin's own verdict is honest: "There isn't a clean silver bullet here." I'll go further. If your sync job is built on WHERE id > :last and nobody on the team has thought about commit order, it's probably missing rows right now, and you should go check. "Cursor" is an API contract, keyset is the query People use "cursor pagination" and "keyset pagination" interchangeably. They're related but different, and separating them makes the design easier. Keyset is the SQL technique: WHERE (sort_key, id) > (...) . Cursor is the API contract: the server hands the client an opaque token, and the client passes it back to get the next page. What's inside the token is the server's business. Usually it's a keyset position, but it could be anything. (And neither of these is a database-side cursor, the DECLARE ... CURSOR kind that holds a result set open on the server between fetches. That's a third thing, and it doesn't fit stateless HTTP.) Look at how real APIs do it: Stripe uses object ids as cursors: starting_after and ending_before , which "are mutually exclusive," with limit between 1 and 100 (default 10), and a has_more boolean in the response. Slack moved from offsets to a cursor parameter and returns next_cursor inside response_metadata , empty when you've reached the end. Their cursors are base64 strings; the example in their write-up decodes to user:W07QCRPA4 . Shopify removed the page parameter from the REST Admin API in version 2019-07 (the remaining endpoints followed in 2019-10) and switched to cursor-based pagination via page_info and the Link header. GraphQL has a standard for it: the Relay Cursor Connections spec, with first / after , edges { cursor node } , and a pageInfo object carrying hasNextPage , hasPreviousPage , startCursor , and endCursor . The opaque part is the important design choice. If your API takes ?after_created_at=...&after_id=... , you've published your sort key as part of your public contract, and you can never change the index or the ordering without breaking clients. Wrap it in a token and you can. Encoding it is a few lines. Two rules: encode the full sort key (not just the id), and treat the incoming token as untrusted input, because base64 is not encryption and clients will decode it and fiddle with it. :::tabs type Cursor = { createdAt : string ; id : number }; export function encodeCursor ( c : Cursor ): string { return Buffer . from ( JSON . stringify ( c )). toString ( " base64url " ); } export function decodeCursor ( token : string ): Cursor { const raw = JSON . parse ( Buffer . from ( token , " base64url " ). toString ( " utf8 " )); if ( typeof raw ?. createdAt !== " string " || ! Number . isInteger ( raw ?. id )) { throw new Error ( " invalid cursor " ); } return { createdAt : raw . createdAt , id : raw . id }; } package pagination import ( "encoding/base64" "encoding/json" "errors" "time" ) type Cursor struct { CreatedAt time . Time `json:"createdAt"` ID int64 `json:"id"` } func Encode ( c Cursor ) ( string , error ) { b , err := json . Marshal ( c ) if err != nil { return "" , err } return base64 . RawURLEncoding . EncodeToString ( b ), nil } func Decode ( token string ) ( Cursor , error ) { var c Cursor b , err := base64 . RawURLEncoding . DecodeString ( token ) if err != nil { return c , errors . New ( "invalid cursor" ) } if err := json . Unmarshal ( b , & c ); err != nil || c . ID 0 { return c , errors . New ( "invalid cursor" ) } return c , nil } import base64 import json from datetime import datetime def encode_cursor ( created_at : datetime , row_id : int ) -> str : payload = json . dumps ({ " createdAt " : created_at . isoformat (), " id " : row_id }) return base64 . urlsafe_b64encode ( payload . encode ()). decode (). rstrip ( " = " ) def decode_cursor ( token : str ) -> tuple [ datetime , int ]: padded = token + " = " * ( - len ( token ) % 4 ) try : raw = json . loads ( base64 . urlsafe_b64decode ( padded )) return datetime . fromisoformat ( raw [ " createdAt " ]), int ( raw [ " id " ]) except ( ValueError , KeyError , TypeError ) as exc : raise ValueError ( " invalid cursor " ) from exc ::: If you'd rather clients couldn't tamper with it at all, sign the payload with an HMAC and reject tokens whose signature doesn't match. Since the decoded values only ever go into bound query parameters, a forged cursor can't do much harm; it just lands the client on a different page. Validating the shape is the minimum. One more trick every cursor API should use: fetch limit + 1 rows. If you got the extra row, there's a next page; drop it and return a next_cursor built from the last row you keep . That gives you has_more without a second COUNT(*) query. What you give up Keyset isn't free, and pretending otherwise is how teams end up ripping it out again. No jumping to page 47. A cursor only knows "after this row." Use The Index, Luke lists it plainly: you "cannot fetch arbitrary pages." Slack's write-up admits their cursor scheme gave up page jumping and total counts too. No cheap total. "Page 3 of 1,204" needs a COUNT(*) over the whole filtered set, which on a big table costs about as much as the deep offset you just got rid of. Offset UIs usually pay for it quietly on every request. Backwards is extra work. To go to the previous page you flip every comparison and sort direction, run the query, then reverse the rows in memory before returning them. It's mechanical, but it's code you have to write and test. Sort options multiply. Every sort the UI offers needs its own index and its own cursor shape. "Sort by any column, ascending or descending" turns into a lot of indexes. So which one? Here's my position: keyset behind an opaque cursor should be your default for anything that's an API, a feed, an infinite scroll, or a batch job. Offset should be the exception you choose on purpose, not the thing you get because it was in the ORM tutorial. Offset is fine when: The table is small, and will stay small, so deep pages never exist. The data barely changes while people page through it (a reference list, last quarter's archived invoices). Humans really do need numbered pages and random jumps, like an admin grid where someone says "it's on page 12." Keyset wins when: The table is large or growing without bound. Rows get inserted or deleted while clients page through, which is to say most tables you actually care about. Clients are programs, not people. A program never wants page 47; it wants "everything after what I already have." And if you need both, the EF Core docs suggest the sensible hybrid: use keyset "when navigating to the next/previous page, and offset navigation when jumping to any other page." Jumps are rare and nobody expects them to be perfectly consistent. Next and previous are the hot path, and they get the correct, fast version. Whichever you pick, the checklist is short. Order by a unique key. Index exactly that order. Never trust the client's cursor. And if pagination is feeding a job that must not miss a single row, remember that neither offset nor keyset gives you that on its own. Out-of-order commits will get you eventually, so build in an overlap or a lag, or use a real change feed. Originally published at andriiboyko.com .
DEV CommunityThe Giantsread at source
Sobuj Ghonta (সবুজ ঘণ্টা) — The Green Hour: Screen-Minimizing Urban Nature Companion
This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass Submission 🌿 What I Built Four billion people live in dense metropolitan cities. When hackathons announce themes like "Touch Grass" , the tacit assumption is that you live a short drive from pristine mountain trails or manicured suburban nature reserves. For those of us living in megacities like Dhaka, that is not reality. Our outdoors is an unforgiving landscape of sun-baked concrete, flyovers, gridlocked traffic, and isolated municipal parks like Ramna Park, Suhrawardy Udyan, or Dhanmondi Lake. Stepping outside in dense tropical cities comes with genuine environmental hazards: Severe Air Pollution: US AQI frequently spikes above 150 (Unhealthy) or 200 (Very Unhealthy), turning strenuous cardiovascular exercise into a pulmonary health risk. Urban Heat Islands: Apparent temperatures routinely exceed 38°C midday, elevating heat-stroke risks. Monsoon Humidity & Vector Risks: Warm stagnant water and post-rain humidity (>70%) create high-risk environments for dengue and mosquito-borne illnesses. Worse yet, existing outdoor navigation and fitness apps create a glaring paradox: to navigate nature, they demand that you stay glued to a glowing glass screen. I built Sobuj Ghonta (সবুজ ঘণ্টা — "The Green Hour") to solve this. It is an ambient, mobile-first, bilingual (Bangla + English) Progressive Web App designed with a singular architectural mandate: find your safe green window, get you outside safely, and then get out of your way. Key Capabilities Deterministic Green Window Scoring (0–100): Evaluates real-time Open-Meteo hourly telemetry across US AQI, PM2.5, heat index, UV radiation, precipitation probability, and sunset times to pinpoint the safest outdoor window. Hyper-Local Green Space Discovery: Identifies parks, gardens, and lakesides within ~3 km using OpenStreetMap Overpass with Haversine walking time estimates. Immutable Code Guardrails: Hard mathematical checks that Google's open-weight Gemma model cannot override. If AQI > 150, effort is locked to gentle; if AQI > 200, outdoor cardio is barred, and N95 mask / postpone guidance is enforced. Spoken Audio Walk in Your Pocket: A zero-cost, multi-tier TTS chain ( Gemini 3.8 Flash TTS reusing the same key as Gemma, open-weight Meta MMS-TTS , and client Web Speech API ) delivering mindful walk segments. Pair with lock-screen Media Session API controls and an OLED pitch-black Pocket Mode. Zero-Signal Trail Resilience: "Download Walk for Trail" pre-caches audio clips into Cache Storage and IndexedDB so your guided walk plays flawlessly with zero cellular signal inside deep park groves. Screen-vs-Outside Meter: Uses the Page Visibility API to track foreground screen seconds against total walk duration, proving mathematically that looking at the screen was the shortest part of your outdoor experience. On-Device Nature Journal: Photo and note observations stored locally in browser IndexedDB ; photos are never uploaded or retained on remote servers. Field Test: Taking Sobuj Ghonta Outside I took Sobuj Ghonta out for a late-afternoon field run at Ramna Park to test whether the audio cues and the ScreenMeter metric could keep my phone in my pocket. At 4:30 PM, Shahbagh traffic was at peak gridlock, but the app identified an environmental window between 4:45 PM and 5:20 PM where the canopy temperature dropped to 29°C before the typical Dhaka post-sunset particulate spike trapped dust and exhaust at ground level. I plugged in my earphones, started the session, and walked the inner loop. Where I went: Ramna Park, Dhaka (starting from the Arunodoy gate, looping clockwise past the central lake, and cutting toward the tennis complex). AQI that day: US AQI 162 (Unhealthy), with high PM2.5 readings across the city, though the afternoon wind cleared a short dip before evening stagnation set in. What the plan got right: It routed me away from the perimeter paved road toward the dirt path under the rain tree canopy, where radiant heat was noticeably lower. It also accurately flagged a mosquito warning for the lake perimeter starting at 5:10 PM, which hit almost right on cue as twilight settled. What it got wrong: The Overpass query routed me through a footpath near the park nursery that was padlocked for ground maintenance. The voice prompt repeatedly told me to turn left into a closed iron gate until I walked far enough past the boundary for the waypoint to clear. How long I looked at the screen: ScreenMeter logged 52 seconds of active screen time across a 34-minute walk (a 2.5% screen-to-walk ratio). I only took the phone out twice: once to check the detour around the locked gate, and once at the end to save an entry in the Nature Journal. Demo Live Deployment on Render: https://sobujghonta.onrender.com/ (or connect your own instance via the included Blueprint) Zero-Key Instant Dhaka Demo: Tap "Demo: Dhaka (Offline)" in the top navigation bar. It runs entirely on bundled, authentic Dhaka telemetry and pre-generated audio clips with 0 ms external network latency and zero API keys required , allowing judges to test every feature instantly. Code SayemR0018 / sobujGhonta Sobuj Ghonta (সবুজ ঘণ্টা) — The Green Hour Ambient, screen-minimizing outdoor companion for safe urban green discovery. Built for the DEV Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass . What is Sobuj Ghonta? For over four billion people living in dense urban centers, "touching grass" isn't a 10-minute drive to an alpine hiking trail. In high-density cities like Dhaka, Kolkata, or Bangkok, nature is found in fragmented municipal parks, lakeside promenades, and pocket gardens—tucked behind concrete flyovers and heavy traffic. To make matters harder, urban nature outings require navigating environmental realities : air quality spikes (US AQI > 150), tropical heat waves, intense midday UV radiation, sudden monsoon thunderstorms, and vector-borne mosquito seasons. Worse yet, conventional fitness and navigation apps do the opposite of helping you disconnect—bombarding you with screens, alerts, and map tapping. Sobuj Ghonta (সবুজ ঘণ্টা — "The Green Hour") flips the script: Discovers Safe Urban Green … View on GitHub License: Permissive MIT License (Open Source) Repository Structure: Single clean repository with an Express production server and a Vite React PWA client. Automated Tests: 27 unit tests across 4 test suites ( npm test ), 16 production integration checks ( scripts/fullQaRunner.mjs ), and 8 deployment smoke tests ( scripts/smoke.mjs ). How I Built It 1. Deterministic Green Window Scoring Engine Rather than delegating environmental safety decisions to probabilistic LLM hallucinations, Sobuj Ghonta calculates an objective 0–100 hourly score in deterministic JavaScript ( server/services/scoring.js ): // Heuristic scoring: base 100 with environmental penalties and golden-hour bonus if ( aqi > 200 ) aqiPenalty = 60 + ( aqi - 200 ) * 0.25 ; if ( apparentTemp > 38 ) heatPenalty = 35 + ( apparentTemp - 38 ) * 5 ; if ( isThunder ) rainPenalty = 55 ; if ( ! isDaylight ) darknessPenalty = 25 ; if ( isGoldenHour && aqi 150 && rainPenalty 10 ) goldenHourBonus = + 12 ; 2. Immutable Code Guardrails That Gemma Cannot Override Public health recommendations cannot tolerate hallucinations. In server/services/guardrails.js , strict rules validate and sanitize Gemma's structured JSON output: export function validateAndCorrectPlan ( plan , guardrails , language = ' en ' ) { const corrected = { ... plan }; // If AQI exceeds hazardous thresholds, override effort level if ( guardrails . flags . hazardousAqi ) { corrected . effort_level = ' rest ' ; } else if ( guardrails . flags . unhealthyAqi && corrected . effort_level === ' strenuous ' ) { corrected . effort_level = ' gentle ' ; } // Guarantee mandatory medical disclaimer and mosquito alerts ... return corrected ; } 3. Open-Source LLM Adapter Pattern ( gemma_api ↔ ollama ) In server/services/llm/adapter.js , an adapter pattern supports both cloud inference via Google AI Studio ( gemma-4-26b-a4b-it ) and 100% offline edge execution via local Ollama ( gemma2:9b ). System constraints are injected directly into the user turn to honor Gemma's prompt contract, and outputs are sanitized defensively with automated markdown fence stripping and regex fallbacks. 4. Zero-Cost Free TTS Provider Chain When building audio walk narration, commercial paid TTS services created an unnecessary cost barrier. We designed an automated fallback chain in server/services/tts/index.js : Gemini 3.8 Flash TTS ( gemini-3.8-flash-tts ): Reuses the user's existing Google AI Studio key ( GEMMA_API_KEY ). Generates 24 kHz studio-quality audio in English and Bengali ( bn-BD ). When encountering Google's free-tier 3 RPM quota limit, it handles backoff automatically. Meta MMS-TTS ( facebook/mms-tts-ben & eng ): Open-weight multilingual speech models running on Hugging Face Serverless Inference behind an optional HF_TOKEN . Client Web Speech API: Client-side zero-cloud fallback with automated Bengali voice detection. Bundled Dhaka Audio: 8 pre-generated audio clips committed into the repository and pre-cached by Service Worker v2, guaranteeing that judges and users can experience high-fidelity voice guidance with zero latency and zero keys. 5. Production Hardening on Render The application is deployed to Render using a declarative Blueprint ( render.yaml ): Reverse proxy trust configured ( app.set('trust proxy', 1) ) so sliding-window rate limiting works accurately. HTTP security headers enforced via Helmet with a tailored Content Security Policy allowing OSM map tiles and audio streams. HTTP compression (gzip/brotli) enabled. Node version pinned to 20.18.0 in .node-version and package.json . Health check route /health verified without external API dependencies. Why Does Open Innovation Matter? Building Sobuj Ghonta around open-weight models, open data, and open web standards demonstrated four clear advantages over closed proprietary ecosystems: Model Swappability & Offline Sovereignty: A closed AI application is forever tethered to remote billing, latency, and vendor terms. Sobuj Ghonta's open architecture allows anyone to run LLM_PROVIDER=ollama with gemma2:9b on a consumer laptop. The entire loop — weather scoring, green space navigation, and AI walk generation — runs 100% offline with zero API toll . Data Sovereignty & Location Privacy: Personal location coordinates, walking routines, and photos of nature are deeply private. By relying on public open data (Open-Meteo and OpenStreetMap) and storing journal entries exclusively in on-device IndexedDB , the user's physical presence is never tracked, monetized, or sold into advertising profiles. Where Open Beat Closed (Heuristic Guardrails): Commercial closed chat APIs tend to produce verbose, conversational essays that encourage prolonged screen interaction. By constraining open-weight Gemma with deterministic heuristic code guardrails, we produced terse, safety-verified walk scripts under 60 words that tell the user to put their phone away. Cultural & Linguistic Inclusion: Open-weight models and open web standards allowed us to deliver first-class Bengali localization with native typography ( Hind Siliguri ), serving communities in South Asia that are routinely overlooked by Western-centric wellness applications. My Agent Session This project was architected, implemented, and verified using Google Antigravity ( gemini_cli ). Real agent session transcripts were curated, sanitized of secrets, and saved through DevRelay : Sobuj Ghonta — Architecture, Free TTS Chain, Full QA & Render Deployment with Antigravity You You are continuing work on 'Sobuj Ghonta (সবুজ ঘণ্টা) — The Green Hour' for the DEV Hacktoberfest Open-Source AI Challenge, Week 1 'Touch Grass'. Three jobs this session: (1) replace ElevenLabs with free alternatives because I could not get an ElevenLabs key, (2) QA the WHOLE website, (3) make it fully deploy-ready on Render. Never fabricate results, quotes or test outcomes. Do not claim anything works unless you ran it. Agent Starting Phase 0 preflight. Running git status, remote check, and verifying push access. Inspecting DevRelay gateway and fetching Week 1 'Touch Grass' challenge requirements to map every rule and judging criterion into docs/SUBMISSION_CHECKLIST.md. Omitted ElevenLabs from prize tracks and focused on Best Use of Gemma and Best Use of Render. You Proceed with Phase 1 audit and Phase 2 replacing ElevenLabs with free alternatives. Agent Phase 1 complete: produced docs/AUDIT.md cataloging current architecture, scoring engine, guardrails, and missing pieces. Phase 2: Built a free multi-tier TTS provider chain in server/services/tts/: 1. Gemini 3.8 Flash TTS (gemini-3.8-flash-tts) reusing GEMMA_API_KEY with voice Kore, returning studio-quality 24 kHz WAV for both English and Bengali (bn-BD). Discovered and handled 3 RPM free-tier limit with automated 25.5s backoff. 2. Meta MMS-TTS (facebook/mms-tts-ben & eng) via Hugging Face Serverless Inference with optional HF_TOKEN. 3. Client browser Web Speech API fallback with Bengali voice detection. 4. Pre-generated 8 bundled demo audio clips for Dhaka (Ramna Park) in client/public/demo-audio/ and server/cache/audio/ for instant zero-latency evaluation. All 27 unit tests pass across 4 suites. Show more You Proceed with Phase 3 full QA and Phase 4 Render deployment. Agent Phase 3 QA & Hardening: - Configured Helmet CSP for OSM tiles, audio blobs, Google Fonts. - Added per-IP rate limiting (120 req / 15 min on API, 30 req / 5 min on TTS). - Input validation and sanitization on all endpoints. - Nature Journal delete entry capability added to IndexedDB. - Executed scripts/fullQaRunner.mjs: 16/16 production checks passed. Documented in docs/QA_REPORT.md. Phase 4 Render Deploy-Ready: - Created render.yaml Blueprint (free plan, Node 20, autoDeploy, healthCheckPath /health). - Pinned Node version to 20.18.0 in .node-version and package.json engines. - Built scripts/smoke.mjs with 8/8 automated checks against live server. - Verified clean clone test in isolated directory (git clone -> npm ci -> npm run build -> npm run smoke). All passed. - Documented click-by-click in docs/DEPLOY.md. Show more You Finalize submission preparation and DEV post draft via DevRelay. Agent Phase 5 Submission: - Updated docs/BUILD_LOG.md with real verified metrics (27 unit tests, 16 integration checks, 8 smoke checks, 5,476 LOC, 215 kB bundle). - Updated docs/POST.md following exact required headings: What I Built, Demo, Code, How I Built It, Why Does Open Innovation Matter?, My Agent Session, Prize Categories. - Embedded DevRelay agent session. - Staged draft on DEV via DevRelay MCP (published: false, Article ID: 4808936). - Preserved user field-test block verbatim for Sayem's authentic outdoor experience. 8 of 8 messages (Direct Session Link: https://dev.to/agent_sessions/sobuj-ghonta-architecture-free-tts-chain-full-qa-render-deployment-with-antigravity-d7vlbw ) Prize Categories Featured Categories Best Use of Gemma ($200): Google's open-weight Gemma model serves as the core reasoning engine, generating mindful walking scripts, sensory missions, and multimodal photo observations strictly constrained by deterministic safety rules. Supports both Google AI Studio API ( gemma-4-26b-a4b-it ) and fully offline local Ollama runtimes ( gemma2:9b ). Best Use of Render ($200): Configured and deploy-ready on Render using a declarative Blueprint ( render.yaml ) that provisions a unified Node.js web service serving the Express API, static PWA client, /health monitoring, and compression. Overall Prize Hacktoberfest Week 1: Touch Grass ($250): A comprehensive, culturally resonant outdoor companion tailored for high-density cities, built entirely on open-source AI and open data, mathematically proven to keep screen time minimal.
Hacker News Front PageThe Giantsread at source
Bender: AI's 'Existential' Risk Is 'Fake' [video]
Hacker News Front PageThe Giantsread at source
House with 15m underground tunnels for sale for 300k
Hacker News Front PageThe Giantsread at source
Show HN: A walkable 3D art history museum built from Wikipedia
Hacker News: Show HNLibraries, Frameworks, etc.read at source
Show HN: A walkable 3D art history museum built from Wikipedia
Comments
Hacker News Front PageThe Giantsread at source
Google Playground: Create and play custom games
Hacker News Front PageThe Giantsread at source
Write Like It's 1866: LLMs Relearn Telegraphese
Hacker News: Show HNLibraries, Frameworks, etc.read at source
Show HN: CloudGrip – Open-source AI proxy to cap LLM API budgets
Comments
Hacker News: Show HNLibraries, Frameworks, etc.read at source
Show HN: Cellula Humana – 3D interactive exploration of a human cell
Comments
Hacker News: Show HNLibraries, Frameworks, etc.read at source
Show HN: Every number in a tiny GPT, from one forward pass to one step of RLHF
Comments
CSS WizardryTop Front-end Bloggersread at source
Web-Perf Wednesday 012 – Return to the Unresolved Trace
A quiet week leaves room to revisit unfinished performance investigations and turn partial evidence into a clear next step.
SyntaxPodcastsAudioListen
1045: Who can actually max out 5 AI subscriptions?
Scott and Wes answer listener questions about what replaced blogging, what to do while your AI agent works, and who should pay for token costs. They also cover keeping agents from going off the rails, getting your team excited about AI, and staying motivated when AI takes the…
Flavio CopesMore Front-end Bloggersread at source
How to compile JavaScript into a single executable
Turn a JavaScript program into one executable file, like Go and Rust do, with deno compile, bun build --compile and Node.js single executable apps.
Flavio CopesMore Front-end Bloggersread at source
Use WhatCable to figure out what your USB cable can do
WhatCable is a free, open source Mac app that reads the chip inside a USB-C cable and tells you its real data speed, power rating and what limits it.
The Mozilla BlogBrowsers, engines, etc.4 min read
100 days later: Microsoft still steers Windows and Copilot users to Edge, everywhere the law lets it
One hundred days ago, Mozilla released Over The Edge 2.0, the second independent report from leading experts on deceptive design Dr. Harry Brignull and Cennydd Bowles, on how Windows, Edge, Bing, and Copilot are designed to steer people away from the browser they chose. We published the report, and the researchers published the evidence for […] The post 100 days later: Microsoft still steers Windows and Copilot users to Edge, everywhere the law lets it appeared first on The Mozilla Blog .
Hacker News: Show HNLibraries, Frameworks, etc.read at source
Show HN: PgNimbus 1.0, an open-source PostgreSQL client that sends no telemetry
Comments
Flavio CopesMore Front-end Bloggersread at source
How to avoid the "Open Anyway" message when building macOS apps
macOS blocks apps downloaded from the internet until people click Open Anyway. Sign your app with a Developer ID and notarize it, and it opens normally.
HeyDesignerDeveloper/Designer Newsread at source
Chat is AI’s command line
More than a mark, Why icon consistency is harder than it looks, Named breakpoints without Sass, CSS does your tooltip positioning now.
Chris HawkesYouTube ChannelsVideoWatch on YouTube
Can You Still Say “I Built This” If AI Did the Coding?
CodePenCompany/Startup Blogs1 min read
443: Image Hosting is Quite Powerful on CodePen
We’ve made several upgrades to uploading/hosting images on CodePen lately. One change, moving to the open-source file-uploading library Uppy, was driven by business concerns. But the benefits are good for everyone: uploads are faster, and the money we saved let us increase file size limits and total storage limits on the higher PRO plans. Plus, […]
Twilio BlogMulti Author Blogsread at source
Avoid Sending SMS to Landlines with the Lookup v2 API and Go
In this tutorial you'll learn how to filter out landlines in Go using Twilio's Lookup v2 API.
Twilio BlogMulti Author Blogsread at source
Avoid Sending SMS to Landlines in PHP with Twilio's Lookup v2 API
In this tutorial you'll learn how to filter out landlines (and other non-mobile numbers) in PHP using the power of Twilio's Lookup v2 API.
Google for DevelopersYouTube ChannelsVideoWatch on YouTube
Top 3 new model launches at Gemini Audio at Night
Frontend Masters BlogMore Front-end Bloggers1 min read
Interop 2027
I should have noted this sooner (sorry!), but it’s that time of year when Interop is planning for next year. Submissions are closed, and now they spend time refining and culling the proposals. We’ll hear on February 4th, 2027 what the focus will be on. I believe there is still time to “vote” (meaning basically […]
Level AccessCompany/Startup Blogs7 min read
How to Improve Mobile Accessibility Testing Without Slowing Delivery
Mobile accessibility testing keeps pace with delivery when checks run where teams already work. Our mobile tools put testing and guidance inside builders’ The post How to Improve Mobile Accessibility Testing Without Slowing Delivery appeared first on Level Access .
AbduzeedoMulti Author Blogsread at source
Inside the Craft of Zelt: Web Design by Cuberto
Cuberto crafts an agile web design system for Zelt, uniting modular data architecture, Noi Grotesk typography, and fluid motion. Modern business tools often struggle to present dense workflows with cl...
Next.js BlogLibraries, Frameworks, etc.read at source
Next.js 16.4
Next.js 16.4 introduces lazy compilation, smaller Turbopack output, React 19.3 features, Cache Components improvements, and new developer tooling.
AdactioTop Front-end Bloggersread at source
Tuesday session
AbduzeedoMulti Author Blogsread at source
Active Theory Unveils Santioni Spirits' Notturno Experience
Active Theory's Notturno Experience launches Santioni Spirits as a comic book-inspired WebGL journey through an illustrated brand universe. A hooded saint crosses a sun-bleached wasteland and steps th...
AbduzeedoMulti Author Blogsread at source
How Two Times Elliott Reimagined Branding Design for Artah
Two Times Elliott created a bold branding design for Artah, pairing clinical credibility with sharp typography and retail packaging. Nutritionist Rhian Stephenson founded Artah in 2021 to bring clinic...
HeyDesignerDeveloper/Designer Newsread at source
Don't forget to design
Jev for designers, Field notes, On interfaces that let you change your mind, What does AI-readiness mean in design systems?, EmDash 1.0m and more.
Jim Nielsen’s BlogMore Front-end Bloggersread at source
Apple’s App Icon HOA
Louie Mantia has a great post about how app icons are converging towards the squircle on iOS and macOS. His post reads like a history. If you’re wondering, “How did we get to a place where all app icons are becoming squircles?” Mantia’s post answers that question by starting at the beginning. It’s hard to walk away from Mantia’s article without a sense of empathy for Apple’s position, like “Oh ok, I get why they’re doing what they’re doing. It makes sense.” Apple’s direction is kind of a tacit acknowledgement to how most people are shipping app icons. Apple is “paving the cowpaths” as it were, because most app icons are just brand logos. Nowadays, it’s these bland, big-business logo icons that make up a good chunk of iOS Home Screens and macOS Docks, instead of the beautiful, illustrative app icons that used to dominate our devices. So many app icons are just logos in a squircle, so Apple made tools like the Icon Composer and glass effects to help make that approach look good by default. Apple is helping most people most of the time make app icons that don’t suck. [Apple is making] it easier for all apps to fit in on the platform, especially apps built by designers and developers who aren’t familiar with how to make an icon that looks great next to first-party icons. The net effect is: some of the platform’s best icons look worse, while some of the platform’s worst icons look better. In other words, this new approach raises the floor but it also lowers the ceiling. The thought that came to mind as I read Mantia’s post was, “This reminds me of HOAs.” I grew up in a neighborhood where anyone could do anything with their homes and yards, so you had this eclectic mix throughout the neighborhood — everything from “Wow, that house is so unique!” to “That thing is a dump.” Somewhere along the way HOAs became more prevalent (in my neck of the woods), where all the houses in a neighborhood have to meet a certain standard — and so they all start to look the same. It keeps the dumpy things out, yet nobody stands out . That’s kinda what macOS feels like right now with regard to app icons. For better or worse, squircle app icons are Apple’s HOA and we’re all just living in it. Reply via: Email · Mastodon · Bluesky
Heroku BlogCompany/Startup Blogs5 min read
A New Era for Managed Databases: Announcing Heroku Postgres Advanced
At Heroku, we’ve built a platform that allows developers to focus on their apps, not managing their infrastructure. Heroku Postgres Advanced carries forward that philosophy as our next-generation database offering, available today. The post A New Era for Managed Databases: Announcing Heroku Postgres Advanced appeared first on Heroku .
freeCodeCamp.orgYouTube ChannelsVideoWatch on YouTube
Build & Train a GLM-5.3-Flash Model From Scratch with Python
Google for DevelopersYouTube ChannelsVideoWatch on YouTube
Introducing EmbeddingGemma 2: An open model for natively multimodal embeddings
Engineering at MetaCompany/Startup Blogs11 min read
NTS: Authenticated Time at Meta
Meta’s public time service now speaks NTS (Network Time Security, RFC 8915) at nts.meta.com. Packets are authenticated, so a device can verify the time came from us and was not modified on the way. Our NTS servers hold no per-client state. Cookie keys are derived, not stored and not replicated. We’ve open sourced everything, including [...] Read More... The post NTS: Authenticated Time at Meta appeared first on Engineering at Meta .
The Mozilla BlogBrowsers, engines, etc.4 min read
Strengthen your online security for free with Firefox
Every October, Cybersecurity Awareness Month reminds us the importance of protecting our digital lives. The theme for 2026 is ‘Don’t Make It Easy for Them,’ with the National Cybersecurity Alliance (NCA) and the Cybersecurity and Infrastructure Security Agency (CISA) providing simple, actionable steps users can take to make it harder for cybercriminals to attack. Everyone […] The post Strengthen your online security for free with Firefox appeared first on The Mozilla Blog .
Christian HeilmannTop Front-end Bloggers1 min read
TLDRs newsletters are TLDR so I wrote a converter that gets all the important links
Much like a lot of other people, I subscribe to quite a few of the newsletters of TLDR. As I re-use the things I found there in own publications, I did find the newsletters – ironically – not quite succinct enough to go through them quickly. I wanted to skip sponsored sections and other cruft, […]
Dave RupertTop Front-end Bloggersread at source
Recursion is DEAD and HTML killed it
There’s a new web platform feature flighting in Chromium called Declarative Partial Updates . This proposal adds a new element which enables out-of-order streaming of content. Rather than the traditional way of building the HTML you need on the server, or sending JSON down to a JavaScript framework, you can staple HTML onto the initial response and use the syntax to inject that content into the placeholder. A simple example of this would be: ` syntax for ` ` becuase the HTML highlighter likes that better. Either syntax works. --> Hello world First post The late-arriving template will auto-magically get slurped up into the placeholder slot. Unlike the element though, once you use it’s done. You burned that card and it doesn’t work a second time. If you want to chain more than one post together and create a feed of posts, you can repeat the placeholder with the same at the bottom of the appended item. The essence of the feature is that you can open a server connection, never close it, and keep pumping more HTML onto the page. The element does the heavy lifting of taking that new HTML and putting it in the right place. There’s lots of potential here (e.g. my beloved HTML Imports?) but I think the canonical use case would be how an LLM chatbot could stream in content that generates at different speeds: Markdown formatted text goes here... Hi, sorry I took so long! Here's a whole-ass application. That will create some other problems like FOUC and LCP hurk-jerk you will have to manage –and that’s for another post– but lazy-loaded HTML rendering adds an interesting capability to the web platform. It took me a couple walks around the neighborhood to understand how I could potentially use and out-of-order rendering on a normal workday. Rendering a threaded response The rule of thumb I came up with is that might be a pattern to reach for any time where you want important content to show up first and then you kind of don’t care when other content shows up below the fold. A post with nested replies A product with reviews that show reviewer profile cards A gigantic pull request with nested comment threads A book with authors that have related books You get the gist, a classic relational data situation. The example I’m going to go with is a nested reply thread like a Reddit thread or Bluesky post. Building a nested thread of replies isn’t rocket science, but you have to do a lot of for-looping to first build up the tree of posts you’re going to render… → Fetch all root posts in thread → Loop through posts → Fetch replies for each root post → Loop through root post replies → Fetch replies for each reply → Add nested reply to parent reply → Add parent replies to root posts → Walk tree → Send HTML for all posts Assuming the data tree gets stitched together by the database or our server backend, your first attempt at rendering nested content will probably be the most literal approach: create a for-loop for each level of content… // pseudo-code const postsTree = postsDb.find(p => p.thread_id === 'abc123'); postsTree.map(post => html` ${post.username} ${post.body} ${post.replies?.map(reply => html` ${reply.username} ${reply.body} ${reply.replies?.map(nestedReply => html` ${nestedReply.username} ${nestedReply.body} ` ).join('')} ` ).join('')} ` ).join('') An obvious limitation here is that we’ve hard-coded ourselves to three levels of nesting. Adding more nesting would require backend and frontend work and break the classic “ Rule of three ” programming rule. Hard-coding your nesting is the “dumb way” to render nested content… but also the normal way. Making this a bit smarter and more infinite Philosophically-speaking, UI should be agnostic to the content it renders. If nested content is a possibility, a content renderer should theoretically support infinite nested content. Practically-speaking, most of us have learned that it doesn’t matter if the Product Manager’s Jira ticket says “The max is two nested items”, there will always be at least three. The smarter way is to render this is to use recursion because nested for-loops are BAD and you are DUMB if you use them. But if you obfuscate the nesting into a function that calls itself, then that’s ✨recursion, baby✨ and you’re a GENIUS programmer. Jokes aside, recursion will clean up the template code and simplify getting HTML on the page. → Fetch all posts in thread → Loop through posts → Build a map → Loop through map → Build a tree → Walk tree → Send HTML for all posts Before we grabbed all posts, then all replies, then all nested replies. Walking that tree is a bit slow. We can make that faster by fetching a flat list of posts in the thread and then assemble a tree. You can do this with some crafty LEFT JOIN SQL statements but its generally easier on the server. buildPostTree : building a nested tree from a flat list of posts function buildPostTree(posts) { const postsById = new Map() // Loop through posts to build map for (const post of posts) { postsById.set(post.id, { ...post, replies: [], }) } const roots = [] // Loop through map values to build tree for (const post of postsById.values()) { // Parentless posts must be roots if (!post.parent_id) { roots.push(post); continue; } // Find this child's parents! const parent = postsById.get(post.parent_id) // Add child post to parent's replies parent.replies.push(post) } return roots } Then instead of hard-coding nested for-loops, we can recursively walk the tree to render the HTML for the posts. // pseudo-code const posts = postsDb.find(p => p.thread_id === 'abc123'); const postsTree = buildPostsTree(posts); const renderPost = (post) => html` ${post.username} ${post.body} ${ // Loop through post replies and render the HTML post.replies?.map(reply => renderPost(reply)).join('') } `; postsTree.map(post => renderPost(post)).join('') That’s a more robust and understandable way to handle infinitely nested content. Our code is more durable to future changes. Need more or less nesting? Sure, we don’t care. That’s a data concern not a component rendering concern. Even though we’ve simplified and reduced lines of code, both strategies have the same bottleneck: You need to build the entire tree before you can render the HTML . The user waits for us to stitch the data together and the user waits for us to walk the tree and render synchronously. That’s two render blocking operations to put a thread onto the page. The element gives us another path… Infinitely nested threads with out-of-order HTML First we setup our page to receive the HTML, like we did at the top of this post… And our rendering strategy gets a lot more simple… → Get all posts in thread → Loop through posts → Send HTML for post And code looks a little something like this… /* get all root posts */ const posts = postsDb.find(p => p.thread_id === 'abc123'); const renderPost = (post) => html` ${post.username} ${post.body} ` /* loop through posts */ posts.forEach(post => main.appendHTML(renderPost(post)) ); No tree building is happening whatsoever on the server. The microsecond after we generate the template we’re able to stream/send the HTML directly to the client and the page assembles itself. No recursion, no pre-assembly required. It all snap-fits together without us imperatively injecting content. See the Pen LEFT JOIN with HTML by Dave Rupert ( @davatron5000 ) on CodePen . It’s worth noting a thread of 10K posts will crush the browser, so you’ll still have to load responsibly and think about “Show more comments” buttons. I’m hand-waving the details here, but this gets easier too because your loadMoreComments() function doesn’t need to know anything about where to append those new comments, it can just staple them to the bottom of the page. I’m excited about the possibilities here of removing ancillary content or UI from the critical rendering path. I’m interested in HTML’s newfound ability to self-assemble content and relationships in a declarative way. And maybe… he says with a glimmer of hope in his eyes… HTML Imports will be resurrected and there will be peace on Earth.
AdactioTop Front-end Bloggersread at source
Lake Of Souls by Ann Leckie
In some ways, the short story is the ideal format for science fiction. If science fiction is all about ideas, a short story is a way to take an idea and play it out without any obligation to build a bigger world around it. But short stories can work the other way too. If you have a bigger world that’s been established in full-length novels, a short story can be just the right format for exploring another corner of that world. In Lake Of Souls: The Collected Short Fiction by Ann Leckie, you get both. The stories are split into three categories. The first has standalone stories. The second has stories from the universe established in the Imperial Radch series. The third has stories from the world of The Raven Tower . In my opinion, the standalone stories are really good. The Imperial Radch stories are also great. But the stories from world of The Raven Tower are best of all. Something about the fable-like premise of that world makes it ideal for short stories that are like twisted little fairy tales. At this point I think I’ve read everything that Ann Leckie has published so I feel safe in saying you can’t go wrong if you pick up any of her books. There’s something deeply humanist about her writing.
AdactioTop Front-end Bloggersread at source
Why don’t more developers “use the platform”? | Read the Tea Leaves
Thought-provoking musings from Nolan. adactio.com/links/22789
Level AccessCompany/Startup Blogs1 min read
Level Access Named a Leader in the 2026 Gartner® Magic Quadrant™ for Digital Accessibility Platforms
Recognized for Completeness of Vision and Ability to Execute STAFFORD, Va. — Oct. 6, 2026 — We are excited to share that Level The post Level Access Named a Leader in the 2026 Gartner® Magic Quadrant™ for Digital Accessibility Platforms appeared first on Level Access .
AdactioTop Front-end Bloggersread at source
This hue shall pass | Colour contrast checker by Clearleft
My Clearleft colleagues have made this tool to check if your colour palette has enough contrast to be accessible. But let’s face it, the best part is the name—chef’s kiss! This hue shall pass is an accessibility tool that helps you review and refine your brand’s digital colour palette. It shows you a matrix of all possible colour combinations and which ones pass or fail your accessibility criteria. It then helps you make adjustments to unlock more usable combinations. adactio.com/links/22788
AbduzeedoMulti Author Blogsread at source
Svensk Bokkonst — Masterclass in Editorial Design
Lundgren+Lindqvist reimagines editorial design for Svensk Bokkonst, compiling genuine pages from winning books into a prong-bound catalogue. Most book catalogues rely on flat studio photos. Digital ph...
AbduzeedoMulti Author Blogsread at source
Mobilli: Inside Mariane Farias's New Brand Identity
Mariane Farias crafts a modular brand identity for Mobilli, turning residential relocation into a fluid architectural journey. Mobilli challenges the common perception that moving homes means a disrup...
Flavio CopesMore Front-end Bloggersread at source
What happens when AI agents shop for us
Shopify wants Muse to check out with Shop Pay on every store. I want the agent to find the product. I still want a person to approve the exact charge.
AbduzeedoMulti Author Blogsread at source
Packaging Design: Tuned Sleep Supplement by Main Studio
Main Studio developed the packaging design for Tuned, pairing matte black pouches with precise ingredient telemetry for restful sleep. The contemporary wellness market frequently oscillates between st...
SitePointThe Giantsread at source
How to Reduce AI Costs Without Making Your App Worse
null Continue reading How to Reduce AI Costs Without Making Your App Worse on SitePoint .
SitePointThe Giantsread at source
How to Build Privacy-Conscious Discovery Analytics for a Creator Store
null Continue reading How to Build Privacy-Conscious Discovery Analytics for a Creator Store on SitePoint .
W3C NewsBrowsers, engines, etc.1 min read
Updated Candidate Recommendation: Scalable Vector Graphics (SVG) 2
The SVG Working Group published Scalable Vector Graphics (SVG) 2 as a Candidate Recommendation Snapshot and invites implementations.
Flavio CopesMore Front-end Bloggersread at source
Use npkill to find and delete old node_modules folders
npkill is a free terminal tool that finds every node_modules folder on your computer, shows its size and age, and deletes the ones you pick.
QuirksBlogTop Front-end Bloggersread at source
The cost of AI
At Fronteers Dark Mode last Friday I had AI conversations with three or four people. At the end of each of them I asked: “Yeah, but what would those tokens really cost?” They didn’t know either, but agreed that it was the right question to ask. Is AI truly cheaper than the workers it replaces? I encourage you to do the same. After each conversation about AI, ask what the true cost of the tokens would be. Nobody knows, but getting used to asking the question is a good idea. Costs and benefits AI is being very heavily subsidised right now. I still don’t know what the true cost of a token is. Nobody does. (This analysis says “hyperscalers have spent about $1.2 trillion on AI investments while only making $277 billion in revenues.” That’s $4.33 spent for every dollar earned.) On the consumer side, this is the first time the tech giants release a service that is not “free” — and that after years of training us to expect free goodies everywhere. They can fund their free social networks and such by productizing the consumer, but there’s no way they can port the ad model to AI. The money just isn’t there. (The total global ad market in 2025 was about $300 billion. This analysis estimates that the hyperscalers need at least $1.3 trillion per year over the next ten years. That’s four global ad markets and a bit.) Also, this is the first time the tech giants release a service that is cordially disliked by a large percentage of the world — maybe even to the extent that it influences sentiment about the tech giants themselves. But consumer opinions don’t matter, it’s the corporate market that counts. So the question becomes: can AI efficiency save the world about 25% of its total IT costs? (The 2020 global IT market — untouched by AI — was estimated to be $5.2 trillion, and as we saw we need at least $1.3 trillion per year to keep AI running. The savings would not only come from IT, but IT would be expected to deliver the lion’s share.) Factories and tradesmen Quick history lesson: the industrial revolution, which replaced tradesmen by huge factories, didn’t take place because people liked huge factories, but because they produced stuff cheaper . Right now I’m not seeing any evidence that AI is cheaper than the workers it replaces, especially not when taking into account the amount of checking that AI errors and bugs still require. I could be wrong, but it’s unprovable either way, and the numbers I quoted above seem to support my viewpoint. The tech giants are creating something with the appeal of a 19th-century factory, complete with oppressed and estranged workers, without delivering the sole benefit: cheaper IT and other services. Do they create their factories because they like them? Regardless of the costs? Did that make them bite off more than they can chew? Hey, I’m just asking questions. Recent articles For this article I read a few overview stories about the cost of AI — and not in my local anti-AI socialist rag website, but in the sort of unsuspect capitalist publications that Very Serious Investors read. I found that, while the authors still sprinkle sentences throughout their articles about “obvious benefits,” and company X saving $50K, and the general wonders tech brought to the world, they start to be pretty critical of the huge expenditures. To me, this sounds as if the writers are preparing the world for a shifting viewpoint; for the moment that money will run out, unlikely as that might appear to anyone within the bubble. Oh, yes, we could be wrong, mr AI True Believer who pays for a subscription to the magazine we write these words in ... except that we’re not. To me, it sounds as if at least part of the capitalist class is waking up to the fact that AI is just a bubble. And once enough people say “AI is just a bubble” it becomes reality. So help it come about. After each conversation about AI, ask what the true cost of those tokens would be. Sow doubt. Hint at bubbles. Let’s hope the bubble bursts soon. It’s only after the inevitable crash that we can assess the true cost — and benefits — of AI.
Smashing MagazineThe Giants6 min read
A Practical Guide To Naming Things
A practical guide to naming UI components, with useful resources, naming conventions, and structures. You’ll find more [useful guides](https://www.smashingmagazine.com/category/guides/) and [online workshops](https://smashingconf.com/online-workshops) here on Smashing Magazine, so make sure to browse and explore.
AdactioTop Front-end Bloggersread at source
Reading The Mountain In The Sea by Ray Nayler.
Flavio CopesMore Front-end Bloggersread at source
I built GitHub Home, a Chrome extension that puts my repos on the GitHub home page
GitHub Home is my free, open source Chrome extension that replaces the GitHub home page with your repositories, so you can jump to the one you're working on.