Celebrate 50 years of Microsoft with the company's original source code

An image of an archeologist adventurer who wears a hat and uses a bullwhip

Build an 8-bit computer from scratch

Curl-impersonate: Special build of curl that can impersonate the major browsers

The Slow Collapse of Critical Thinking in OSINT Due to AI

Overengineered Anchor Links

Senior Developer Skills in the AI Age

Supervisors often prefer rule breakers, up to a point

New nanoparticle therapies target two major killers

Was Houdini a Spy? (2021)

AI cheats: Why you didn't notice your teammate was cheating

Hey HN - this is Alexander from Hatchet. We’re building an open-source platform for managing background tasks, using Postgres as the underlying database.Just over a year ago, we launched Hatchet as a distributed task queue built on top of Postgres with a 100% MIT license (<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=39643136">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=39643136</a>). The feedback and response we got from the HN community was overwhelming. In the first month after launching, we processed about 20k tasks on the platform — today, we’re processing over 20k tasks per minute (&gt;1 billion per month).Scaling up this quickly was difficult — every task in Hatchet corresponds to at minimum 5 Postgres transactions and we would see bursts on Hatchet Cloud instances to over 5k tasks&#x2F;second, which corresponds to roughly 25k transactions&#x2F;second. As it turns out, a simple Postgres queue utilizing FOR UPDATE SKIP LOCKED doesn’t cut it at this scale. After provisioning the largest instance type that CloudSQL offers, we even discussed potentially moving some load off of Postgres in favor of something trendy like Clickhouse + Kafka.But we doubled down on Postgres, and spent about 6 months learning how to operate Postgres databases at scale and reading the Postgres manual and several other resources [0] during commutes and at night. We stuck with Postgres for two reasons:1. We wanted to make Hatchet as portable and easy to administer as possible, and felt that implementing our own storage engine specifically on Hatchet Cloud would be disingenuous at best, and in the worst case, would take our focus away from the open source community.2. More importantly, Postgres is general-purpose, which is what makes it both great but hard to scale for some types of workloads. This is also what allows us to offer a general-purpose orchestration platform — we heavily utilize Postgres features like transactions, SKIP LOCKED, recursive queries, triggers, COPY FROM, and much more.Which brings us to today. We’re announcing a full rewrite of the Hatchet engine — still built on Postgres — together with our task orchestration layer which is built on top of our underlying queue. To be more specific, we’re launching:1. DAG-based workflows that support a much wider array of conditions, including sleep conditions, event-based triggering, and conditional execution based on parent output data [1].2. Durable execution — durable execution refers to a function’s ability to recover from failure by caching intermediate results and automatically replaying them on a retry. We call a function with this ability a durable task. We also support durable sleep and durable events, which you can read more about here [2]3. Queue features such as key-based concurrency queues (for implementing fair queueing), rate limiting, sticky assignment, and worker affinity.4. Improved performance across every dimension we’ve tested, which we attribute to six improvements to the Hatchet architecture: range-based partitioning of time series tables, hash-based partitioning of task events (for updating task statuses), separating our monitoring tables from our queue, buffered reads and writes, switching all high-volume tables to use identity columns, and aggressive use of Postgres triggers.We&#x27;ve also removed RabbitMQ as a required dependency for self-hosting.We&#x27;d greatly appreciate any feedback you have and hope you get the chance to try out Hatchet.[0] <a href="https:&#x2F;&#x2F;www.postgresql.org&#x2F;docs&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.postgresql.org&#x2F;docs&#x2F;</a>[1] <a href="https:&#x2F;&#x2F;docs.hatchet.run&#x2F;home&#x2F;conditional-workflows">https:&#x2F;&#x2F;docs.hatchet.run&#x2F;home&#x2F;conditional-workflows</a>[2] <a href="https:&#x2F;&#x2F;docs.hatchet.run&#x2F;home&#x2F;durable-execution">https:&#x2F;&#x2F;docs.hatchet.run&#x2F;home&#x2F;durable-execution</a>

Show HN: Hatchet v1 – A task orchestration platform built on Postgres

The cultural evolution of distortion in music

Microsoft reports several bootloader vulnerabilities

Aviator (YC S21) is hiring engineers to build AI DevEx platform

A university president makes a case against cowardice

Reasoning models don't always say what they think

How Big Is VMS? – VMS Software, Inc

Binary Distribution Rebuilds

AI 2027

Show HN: GitMCP is an automatic MCP server for every GitHub repo

John Cage recital set to last 639 years recently witnessed a chord change

Hey HN! I posted this on April 1st when it launched, and though it didn&#x27;t get traction here, it was a minor hit on reddit! Now that we&#x27;ve got a few thousand monkeys under our belt, wanted to give it another shot here!Happy to talk about the technical details of running the site - using supabase&#x2F;postgres and constantly putting out fires from the traffic.

Hacker News Application