Replication, failover, graceful shutdown, autoscaling and alerting usually arrive as managed services. Inside a bank, you build them from Kubernetes, an operator or two and a lot of small decisions. Here is how each one works, with the code.
Field n...
An S3 bucket full of Parquet files is not a table. Iceberg turns it into one. Here is what happens under the hood, the traps we fell into and the settings that made our queries faster and our bill smaller.
Field notes from running Iceberg tables for...