Forget CDK and AWS's insane costs. Pulumi and DigitalOcean to the rescue
github.com93 points by mavdi 9 hours ago
93 points by mavdi 9 hours ago
Pulumi is really a royal piece of shit. Why the f*ck am I writing code to do "deployment". In C# --> new Dictionary<string, object> when dealing with a values.yaml for instance. The whole need to figure out when and when not to use Apply.
Give me Terraform (as much as I hate it) any day.
As much as I like it, I find C# to be too inflexible of a language for infrastructure code. I tried with Pulumi for a while but moved to TypeScript as it works so much better. Structural typing makes your life a lot easier.
As SRE dealing with former Pulumi, "Hey Devs can use code to deploy infrastructure" is not great idea you think it is. I've seen some real ugly conditional behavior where I'm like "Is this or is this not going to run? I honestly can't tell."
Apply is really straightforward. The dictionary stuff is very annoying overhead but it’s nice keeping everything in one language.
> Give me Terraform (as much as I hate it) any day
Terraform sure is a quirky little DSL ain’t it? It’s so weirdly verbose.
But at the same time I can create some azure function app, setup my GitHub build pipeline, get auth0 happy and in theory hook up parts of stripe all in one system. All those random diverse API’s plumbed together and somehow it manages to work.
But boy howdy is that language weird.
I wish CDK was fully baked enough to actually use. It's still missing coverage for some AWS services (sometimes you have to do things in cloudformation, which sucks) and integrating existing infra doesn't work consistently. Oh and it creates cloudformation stacks behind the scenes and makes for troubleshooting hell.
> sometimes you have to do things in cloudformation, which sucks
All of CDK does things in cloudformation, which made the whole thing stillborn as far as I’m concerned.
The CDK team goes to some lengths to make it better, but it’s all lambda based kludges.
The biggest hurdle I've encountered is cross-stack resource sharing, especially in case of bidirectional dependencies like KMS keys and IAM roles.
The biggest hurdle is when you want to refactor your stacks, and you pretty well just can't, without risk of deleting everything
I also had a really rough go with cdk. I personally found the lack of upsert functionality -- you can't use a resource if it exists or create if it doesn't -- to make it way more effort than I felt was useful. Plus a lack of useful error messages... maybe I'm dumb, but I can't recommend it to small companies.
Cant use bun to deploy CDK, CDK fails as it looks for package-lock yarn-lock or pnpm’s exclusively
So dumb. Trying to move to SST for only that reason
but if you add cdk to the path, you can still deploy, its just that your cicd and deployment scripts are not all using bun anymore
Kubernetes no thanks. Terraform + Kamal [1] on Digital Ocean is the way I deploy/run apps now.
Plain Podman systemd integration is way more powerful and secure, as it does not mess with firewall and allows to run rootless containers using services. It's even possible to run healthchecks and enforce building images just before starting service making on-demand containers using systemd-proxyd possible. Check example: https://github.com/Mati365/hetzner-podman-bunjs-deploy
> way more powerful and secure
I don't care about powerful. That's the opposite of what I want. I could just use k8s if I cared about that.
It looks like you don't even care about opening documentation before pressing reply. Podman is a simple hammer without any moving parts, that used properly can be used to build fancy stuff without much knowledge.
I'm aware of what Podman and Systemd are. Apparently you are not aware of what Kamal is. Open documentation, then press reply.
Does it support zero downtime deploys?
Why not? Install trafeik or any other load balancer, setup two services, and restart one after one.
https://kamal-deploy.org/docs/configuration/proxy/
I think GP's point was that Kamal has all of these things already, so you don't have to set them up.
Ah yes my favourite thing to have to do, rolling my own deploys and rollbacks.
It’s stuff like this that’s just a thousand papercuts that dissuades me from using these “simpler” tools. By the time you’ve rebuilt by hand what you need, you’ve just created a worse version of the “more complex” solution.
I get it if your workload is so simple ir low requirement that zero-downtime deploys, rollbacks, health/liveness, automatic volumes, monitoring etc are features you don’t want or need, but “it’s just as good, just DIY all the things” doesn’t make it a viable alternative in my mind.
Sure but Kumal getting all those features means it strays close to Kubernetes in complexity and it quickly because "Why not Kubernetes? At least that is massive popular with a ton of support."
I've looked into Kamal but it feels so "It's as complex as Kubernetes but isn't so support is going to be nightmarish."
Why is this better then Ansible + Docker Compose?
You could certainly implement Kamal just with Ansible and Docker Compose. It's just an abstraction that does it for you and handles all the edge-cases. (Kamal doesn't use Ansible, it has its own SSH lib).
Kamal is simply NIH K8s made by an unreliable company with poor leadership. No thanks, not for my prod infra!
Pulumi genAI-based documentation is trashed. I've moved to terraform and i was able to achieve much better results in shorter time thanks to higher documentation level for terraform.
Worth noting that most of the terraform documentation for classic pulumi providers (providers build on top of TF providers) is still relevant to Pulumi.
I don’t think Digital Ocean is all that much better for pricing, but using Pulumi over CDK is a pure win as far as I’m concerned.
Agreed. On the bright side, I was able to migrate managed k8s on DO to managed k8s in GCP with very minimal work since it was managed via pulumi.
Yeah, I've been really disappointed with Digital Ocean so far. Not just from a pricing perspective but from a customer service perspective.
Anyone using CDK should switch to Pulumi though.
Hi everyone,
We've gone through a lot of pain to get this blueprint working since our AWS costs were getting out of hand but we didn't want to part ways with CDK.
We've now got the same stack structure going with Pulumi and Digital ocean, having the same ease of development with at least 60% cost reduction.
Keep an eye on reachability and performance. I’ve seen DO consistently perform terribly and/or drop connections for months (that is, didn’t look like some brief routing glitch somewhere) for some US and Canadian routes (not, like, Sri Lanka or something) on excellent Internet connections. The fix was moving to AWS, problem gone. It felt like a shitty-peering-agreements issue.
People will pretend that this quality difference doesn’t exist in networking, uptime, server quality.
It’s not a drop in replacement. It might be worth it depending on what you’re doing.
Frustratingly, it’s also something that doesn’t meaningfully appear on any features list or comparison sheet.
Please change the title text unless you add some discussion of the cost differences to the page you linked. However useful your tool is, nothing on this page mentions AWS or costs.
One thing about managing EKS with Pulumi, Terraform, etc. if you deploy things like Istio that makes changes to infrastructure. Do a Terraform destroy - no luck, you are hunting down maybe some security groups or other assets Istio generated that TF doesn't know about. Good times.
It’s only “insane costs” if you don’t know what you’re doing.
and even if you do, it’s usually a system design problem that you’re maintaining
on one hand, I can see how this is an unfalsifiable standard, on the other hand I can see the utility of solving a friction for people that messed up
Pulumi is very neat with straight AWS, too. I suspect this is the primary use case.
EKS has become a clusterf*ck to manage and provision. This looks very useful. Bare metal k8s, even running on EC2, might be another option.
What's your issue with EKS? I operate several very simple and small single-tenant clusters, and I have to touch the infrastructure only once a year for updates
You don't choose EKS because it's easy to manage. You choose it because you intend to use the bevy of other AWS hosted services. The clusterfuck of management is directly related to that.
The alternative, which I feel is far too common (and I say this as someone who directly benefits from it): You choose AWS because it's a "Safe" choice and your incubator gets you a bunch of free credits for a year or two. You pay nothing for compute for the first year, but instead pay a devops guy a bunch to do all the setup - In the end it's about a wash because you have to pay a devops guy to handle your CI and deploy anyway, you're just paying a little more in the latter.
This title text is nowhere on the linked page. Please get rid of the editorialization. DO is not that much cheaper for a baseline instance.
Is this an Ad?
[dead]
Digital Ocean isn't really a "real" cloud. Maybe use Digital Ocean if you're hosting video game servers, but no serious business should be on it.
I wouldn't even use DO for that, unless it's like a private server for just your friends.
I won't touch DO after they took my droplet offline for 3 hours because I got DDoS'd by someone that was upset that I banned them from an IRC channel for spamming N-bombs and other racial slurs.
What's your definition of real cloud?
And can you name a real cloud that charges a half-reasonable price for bandwidth? I consider $10/TB to be half-reasonable.