Go Cryptography State of the Union
words.filippo.io115 points by ingve 7 hours ago
115 points by ingve 7 hours ago
I agree with the author’s sentiment about FIPS 140. I find NIST to be incredibly slow. I understand there must be some stability, but they are too slow. For example, I think it's horrible that they are still recommending PBKDF2 in 2025.
A big part of the problem I have with it is that it's a "ceiling" on security. Things like electrical code or building code are a "floor" on quality, you have to be at least as good as the code requirements, but can freely be better. FIPS-140 bounds you both ways. If you could more easily do better it'd be much less of a problem that NIST are slow.
In fairness, it's one thing for an implementation like a building to be as over-enginereed as possible in its own right, but it's another when a standard has to ensure that multiple implementations can interoperate. I'm not saying FIPS-140 has only that kind of limitation (far from it), just that this isn't the best analogy.
Is any of FIPS about ensuring interoperability?
Yeah, there's a ton of correctness testing involved. That's mostly at the algorithm, rather than the module level, so it'll fall under CAVP/ACVP rather than CMVP.
That's not for interop, that's for "are you actually doing the crypto you said you'd do". It's designed to prevent broken crypto, not to ensure coordination between parties.
I don't love FIPS either, but cryptosystems don't work the same way as buildings and electrical codes. It's very easy to have "secure cryptosystem A" and "secure cryptosystem B", and then have massive security holes in "cryptosystem A + B". This happens all the time, and is one of the main reasons for the classic "don't roll your own crypto" admonition. The FIPS "whole system" mandate is meant to forestall this failure mode.
I don't know why the standard library crypto packages insist on passing around `[]byte` for things like a seed value, or why we can't just pass in a seed value to a single unambiguous constructor when generating asymmetric keys. Or how the constructor for a key pair could possibly return an error, when the algorithm is supposed to be deterministic.
It all just seems a bit sloppy. Asking for a seed value like `[32]byte` could at least communicate to me that the level of security is at most 256 bits. And removing all dependencies on rand would make it obvious where the entropy must be coming from (the seed parameter). Cloudflare's CIRCL[0] library does a bit better, but shares some of the same problems.
> I don't know why the standard library crypto packages insist on passing around `[]byte` for things like a seed value
These are actually very deliberate choices, based on maybe unintuitive experience.
We use []byte instead of e.g. [32]byte because generally you start with a []byte that's coming from somewhere: the network, a file format, a KDF.
Then you have two options to get a [32]byte: cast or copy. They both have bad failure modes. If you do a ([32]byte)(foo) cast, you risk a panic if the file/packet/whatever is not the size you expected (e.g. because it's actually attacker controlled). If you do a copy(seed, foo) it's WAY WORSE, because you risk copying only 5 bytes and leaving the rest to zero and not noticing.
Instead, we decided to move the length check into the library everywhere we take bytes, so at worst you get an error, which presumably you know how to handle.
> why we can't just pass in a seed value to a single unambiguous constructor when generating asymmetric keys
I am not sure what you are referring to here. For e.g. ML-KEM, you pass the seed to NewDecapsulationKey768 and you get an opaque *DecapsulationKey768 to pass around. We've been moving everything we can to that.
> Or how the constructor for a key pair could possibly return an error, when the algorithm is supposed to be deterministic.
Depends. If it takes a []byte, we want to return an error to force handling of incorrect lengths. If the key is not a seed (which is only an option for private keys), it can also be invalid, deterministic or not. (This is why I like seeds. https://words.filippo.io/ml-kem-seeds/)
> removing all dependencies on rand would make it obvious where the entropy must be coming from (the seed parameter)
Another place where experience taught us otherwise. Algorithms that take a well-specified seed should indeed just take that (like NewDecapsulationKey768 does!), but where the spec annoyingly takes "randomness from the sky" (https://words.filippo.io/avoid-the-randomness-from-the-sky/) in an unspecified way, taking a io.Reader gave folks the wrong impression that they could use that for deterministic key generation, which then breaks as soon as we change the internals.
There is only one place to get entropy from in a Go program, anyway: crypto/rand. Anything else is a testing need, and it can be handled with test affordances like the upcoming crypto/mlkem/mlkemtest or testing/cryptotest.SetGlobalRandom.
> If you do a ([32]byte)(foo) cast, you risk a panic if the file/packet/whatever is not the size you expected (e.g. because it's actually attacker controlled)
Can you give an example of a situation where that is actually a concern? It doesn't really seem like a realistic threat model to me. Knowledge of the key is pretty much how these algorithms define attackers vs. defenders. If the attacker has the key that's gg.
There are lots of things in Go that can panic. Even in syntax, the conversion is very similar to an interface conversion, and those haven't been a problem for me in practice, partly because of good lint rules to force checking the "okay" boolean.
A cloud service that lets users upload their certificates and private keys, to be served by the service's CDN. Here the attacker is attacking the system's availability, not the key.
(But also, it's easy to see how this is a problem for public keys and ciphertexts, and it would be weird to have an inconsistent API for private keys.)
It's been many years since I wrote any Go for a living, but does Go seriously lack a way to say "foo is probably 32 bytes, give me the 32 byte array, and if I'm wrong about how big foo is, let me handle that" ?
If the caller was expected to provide a duration and your language has a duration type, you presumably wouldn't take a string, parse that and if it isn't a duration return some not-a-duration error, you'd just make the parameter a duration. It seems like this ought to be a similar situation.
> It's been many years since I wrote any Go for a living, but does Go seriously lack a way to say "foo is probably 32 bytes, give me the 32 byte array, and if I'm wrong about how big foo is, let me handle that" ?
Not in the static type signature, but you can do that as a runtime check either by casting and handling the potential panic (as described above) or by checking the size and returning an error if it's not as expected, which is what the library does.
Those aren’t arguments for having []byte instead of [32]byte like you think they are. They’re arguments for having an unambiguous IV type that can be constructed from a []byte or [32]byte, or responsibly generated on your behalf. The error-handling logic can be expressed once in the conversion process, and the rest of your crypto APIs can assume the happy path.
Of course, this isn’t really reasonable given golang’s brain-dead approach to zero values (making it functionally impossible to structurally prevent using zero IVs). But it just serves as yet another reminder that golang’s long history of questionable design choices actively impede the ability to design safe, hard-to-misuse APIs.
I'm not sure what you are referring to, but we were talking about keys, not IVs.
Also, "an unambiguous key type that can be constructed from a []byte or responsibly generated on your behalf" is exactly what crypto/mlkem exposes.
What’s wrong with zero values? They free the developer from guessing hidden allocations. IMO this benefit outweighs cast riddles by orders of magnitude.
Zero values prioritize implementation convenience (we always have a zero value so we don't need to handle any cases where we don't have a value, just says those are zero) over application convenience (maybe my type should not have a default and the situation where it has no value is an error)
Take either of Rust's library types named Ordering - core::cmp::Ordering (is five less than eight or greater?) or core::sync::atomic::Ordering (even if it's on another core this decrement definitely happens before that check) neither of these implements Default because even though internally they both use the zero bit pattern to mean something, that's a specific value, not a default.
They're like PHP: silent failures that ever push forwards through the system.
I'm not nearly as angsty as the parent on this subject, but they don't really free the developer from guessing about hidden allocations--Go's allocations are still very much hidden even if developers can reasonably guess about the behavior of the escape analyzer. I think it would have been better if Go _required_ explicit assignment of all variables. That said, despite not being a big fan of this particular decision, Go is still by far the most productive language I've used--it strikes an excellent balance between type safety and productivity even if I think some things could be improved.
boggle
I’m not a golang programmer, but I find this quite bizarre. Sure, C++ arrays are awful and even std::array may not be able to legally alias a vector. But Rust (no surprise) gets this right — there is a properly fallible conversion from slice reference to array reference.
But I guess Go doesn’t. This seems silly to me.
Right now, type casts (called conversions in the spec) always produce a single value. The idiomatic way to have checked casts would IMO be a two-value form, as this would be consistent with type assertions, channel receives, and map indexing, off the top of my head. End result would be something like:
var x []byte
y, ok := [32]byte(x)
// ...
But it would be a subvariant of a much more common pattern in code - destructuring which is quite noisy now but could just be:
a, b, rest := strings.Split(somestr, "/")
I usually wind up using something like the samber/lo library to reduce the noise here. You wind up doing this all the time.
I must disagree. Destructuring is nifty, but it is almost completely unrelated. The second value in any of the builtin two-value assignment forms is invariant on the type or size of RHS; it's always a boolean, and its meaning relates to the "kind" of RHS, not the exact type:
msg, ok := <-ch // ok means "channel ch is not closed"
val, ok := m[key] // ok means "map m contains key"
dyn, ok := i.(T) // ok means "interface i's dynamic type is T or implements T"
arr, ok := [N]T(s) // ok means "slice s has len(s) == N"
Try creating new AES cypher, you will see that you have to provide 16, 24 or 36 bytes in order to get AES-128, AES-192, or AES-256. There is no single [32]byte, for example, as the length cannot be inherently fixed due to this.
I'm more of a C person than a Go person, but I am unbelievably happy that someone in that community is using the word "cryptography" to mean cryptography and not Bitcoin.
Wasn't it just the shorthand "crypto" that got co-opted by the Shitcoin Industrial Complex? I think "cryptography" still means what it always meant regardless of who you ask.
That's mostly the case, but I've seen job postings for "cryptography experts" that are, as best I can tell, looking for block chain hucksters. But I'm unlikely to work for Microsoft, so I just ignore them.
Well yeah, someone hired to work with the low-level nuts and bolts of blockchains would ideally need to know their way around actual bona-fide cryptography.
Respectfully disagree. My experience is that they've read a couple chapters from Applied Cryptography and think "IACR" is a router manufacturer.
But we all see different parts of the industry. Happy to hear you're encountering more capable people in that industry.
There are definitely better cryptographers than me working at Zcash, for example.
I was at Microsoft for meetings once and one of the guys asked if I was a cryptographer. I replied that it depends on who else is in the room. If it's just me and my boss, I'd probably qualify as being a cryptographer. But if I'm in the room with Niels Ferguson or Adi Shamir, I should be considered a software engineer who is somewhat decent at reading math papers. At about the same time, Niels Ferguson walked in the room and I quickly commented "you should not consider me a cryptographer for the rest of this meeting."
Oh no I'm not speaking from experience, just saying that in principle you would want actual cryptography experts in that position, so there's no contradiction in terms. How that tends to play out in practice is another question.
Downvoted for mentioning that people confuse cryptography with Bitcoin? Good thing I didn't mention I think we're in an AI bubble. Or that I prefer emacs to vi.
Let's revert the self-scrutiny trend and actually enjoy a fellow mentioning his downvote. At least I am enjoying, your perception is valid.
I'm curious about how GC languages handle crypto. Is it a risk that decrypted stuff or keys and things may be left in memory (heap?) before the next GC cycle?
You might find this proposal and the upcoming runtime/secret package interesting.
What we did with Java (J/SAFE) was to add explicit methods to zero out sensitive info. It was a bit of a PITA because Java's never had consistent semantics about when final(ize,ly) methods were called. Later we added code to track which objects were allocated, but no longer needed, which also wasn't much fun.
Back in the Oak days Sun asked us (I was at RSADSI at the time) to review the language spec for security implications. Our big request was to add the "secure" storage specifier for data. The idea being a variable, const, whatever that was marked "secure" would be guaranteed not to be swapped out to disk (or one of a number of other system specific behaviors). But it was hard to find a concrete behavior that would work for all platforms they were targeting (mostly smaller systems at the time.)
My coworker Bob Baldwin had an existing relationship with Bill Joy and James Gosling (I'm assuming as part of the MIT mafia) so he led the meetings. Joy's response (or maybe Goslings, can't remember anymore) was "Language extension requests should be made on a kidney. Preferably a human kidney. Preferably yours. That way you'll think long and hard about it and you sure as hell won't submit 2."
I evaluated but didn't adopt https://github.com/awnumar/memguard in Go. No matter how well-implemented and reliable it is, I can't pass its secrets to https://github.com/FiloSottile/age.
I assume all process memory may contain residual secrets. As a mitigation in a password manager and an encrypted file editor, I prevent process memory from being swapped to disk with https://pkg.go.dev/syscall#Mlockall.
If you have access to the local machine no language will save you.
Sure. But there are several graduations of threat between "zero access" and "complete access." On the intarwebs, every request is from a potential attacker. Attackers are known for violating RFC3514, so it is frequently useful to build a trust model and use existing access control mechanism to deny "sensitive" data (or control functions) to protocol participants who cannot verify their identity and/or access permission.
These models can get complex quickly, but are nevertheless important to evaluate a system's specified behaviour.
No system is perfect and your mileage may vary.
To oversimplifiy, it's like the same-ish risk level as JS or PHP or Ruby? (assuming the underlying algorithm is good)
It can be, another risk it that a secret value is left on the stack, and is never overwritten because the stack doesn't get to that memory address again, so it's never overwritten or zerod.
Go really just needs a few `crypto.Secret` values of various sizes, or maybe a generic type that could wrap arrays. Then the runtime can handle all the best practices, like a single place in memory, and aggressive zeroing of any copies, etc.
It's not that simple! What about intermediate values during arithmetic computations? What about registers spilling during signal handling?
I honestly thought it could not be done safely, but the runtime/secret proposal discussion proved me wrong.
What’s up with all these bots posting 3-4 sentence summaries in the comment section?
This is a super helpful overview. Love how Go’s crypto ecosystem keeps getting more opinionated and safer.
[flagged]
[flagged]
[flagged]