Softmax forever, or why I like softmax
kyunghyuncho.me184 points by jxmorris12 5 months ago
184 points by jxmorris12 5 months ago
An aside: please use proper capitalization. With this article I found myself backtracking thinking I’d missed a word, which was very annoying. Not sure what the authors intention was with that decision but please reconsider.
I agree.
I'm all for Graham's pyramid of disagreement: we should focus on the core argument, rather than superfluous things like tone, or character, or capitalisation.
But this is too much for me personally. I just realised I consider the complete lack of capitalisation on a piece of public intellectual work to be obnoxious. Sorry, it's impractical, distracting and generates unnecessary cognitive load for everyone else.
You're the top comment right now, and it's not about the content of the article at all, which is a real shame. All the wasted thought cycles across so many people :(
Graham's Hierarchy, in "How to Disagree": https://paulgraham.com/disagree.html
It's a fad associated with AI, popularised by Sam Altman especially.
It's the new black turtleneck that everyone is wearing, but will swear upon their mother's life isn't because they're copying Steve Jobs.
Twitter and all forms of instant messaging (SMS, WhatsApp, Discord, and the older ones like AIM/MSN/ICQ) have normalized it for many years. Sam is just one of the few large company CEOs to tweet in the style other Twitter users usually use. He's adopting the native culture rather than setting a trend.
Sam still uses capitalization in all of his essays, as do most people (including young people). In essays, like this one, it's distracting without it. I predict in 10 years the vast majority of people will all-lowercase on places like Twitter but almost no one will do it for essays.
>Sam is just one of the few large company CEOs to tweet in the style other Twitter users usually use.
Just looked at the algorithmic feed on Twitter to makes sure trends haven't shifted overnight, and zero people in that sample of hundreds of tweets used all lower case in the tweets. Not in science. Not in AI. Not in maths or politics or entertainment or media.
Sam is trying to bE dIFFERENT. He isn't adopting a norm but instead he's trying to make one. It looks ridiculous.
Most people on my feed do. It's true that professionals, executives, and public intellectuals generally don't use lowercase and that Sam is trying to normalize that, but my main point was that the medium matters. Lowercase looks ridiculous for longform essays and normal for short messages and microblog posts.
> Not in science. Not in AI. Not in maths or politics or entertainment or media.
That is the professional Twitter class and is not at all representative of the norm. Click through to the replies. Sentence case is probably in the vast minority.
I promise you that it is neither a fad nor started with the AI bro.
Black turtlenecks existed before Steve Jobs wore them.
Nobody is arguing that Sam Altman doesn't type in lowercase or that Steve Jobs didn't wear black turtlenecks.
Oh you are the one who made the original claim lmao. What would it take for you to ever admit that you are wrong?
> Twitter and all forms of instant messaging (SMS, WhatsApp, Discord, and the older ones like AIM/MSN/ICQ) have normalized it for many years.
Half true. In SMS it was just easier, but in IM it mostly was a thing because the IM client's message boundaries acted as markers for beginning/end of sentence, making the formatting unnecessary. That's why using correct capitalization and periods for single sentences came to be associated with a more formal/serious tone, it was unnecessary so including it meant you wanted to emphasize it.
Even back then we'd use regular formatting outside of IM or when sending multiple sentences in a single message.
> He's adopting the native culture rather than setting a trend.
If this was the intent, it's really coming off as that "Hello, fellow kids" meme, rather than genuine.
Look at AOL Instant Messenger in 1999 and this is how everyone in school wrote.
Guess what… the people who used AIM in 1999 are now middle aged…
the article has lots of caps it’s a branding and stylistic choice very e e cummings and so on not everyone likes it but clearly the author sees some utility in using caps since they name papers appropriately they also say “UPDATE” in all caps
> He's adopting the native culture rather than setting a trend.
Maybe he should consider there are different registers of language, and code-switching is a thing. This now increasingly applies to written language, not just spoken language.
Writing this way in structured, formal discourse would be equivalent to a CEO using internet memes and trendy slang in board meetings.
It's 2025 and my phone has been automatically capitalizing the first letter of each sentence for about a decade. Doesn't it take more effort to do it the wrong way?
I and most others I talk to disable autocapitalization on our phones. First thing I do when I get a new phone.
That is so incredibly dumb. I can get it in a tweet, but please, please, please properly capitalize in anything longer than a few words!
i don't want to press the shift-key everytime i need a capitalized letter on my phone and i disable auto-correct because it constantly messes with native languages etc.
wasn't aware that this makes me a steve jobs copier :(
EDIT: people are seriously so emotionally invested in capitalization that i get downvoted into minus, jeez.
When you consciously choose to save yourself effort in writing, at the expense of the readers who are trying to make sense of what you are saying, the people onto whom you've transferred the cognitive load are not likely to appreciate your laziness.
your comment contains one, single, capitalized letter. if the first W in your comment would have been small, would that have made your comment so much harder to read?
does it make my comment so hard to read just because i don't start my sentences with big letters and don't capitalize myself(i)? really don't get the fuzz.
of course i capitalize letters in "official" texts, but we're in a comment section.
i find it doubly funny because english doesn't capitalize lots of things, anyways.
Hey, sorry! I don't want you to feel bad, and I don't think you should.
I think there are legitimate reasons to struggle with things like capital letters, and you've named a few: non-native language and interface device limitations. There's other accessibility reasons too, like I have some dyslexic family members who use less capitalisation than most. Also, direct or casual communication with individuals, the impact of the extra cognitive load is minimal - 1 or 2 people - so again, no real issue.
The problem I have with this piece is that it's clearly meant to be an intellectual or academic-adjacent piece, and it's clearly meant to be public/read by many people - that's why we're reading it on Hackernews. The author is not putting in the extra few seconds required to fix the problem when writing, and as a result, many thousands of people lose a few seconds each when reading. I feel there must be a point where the cost of the extra reading time to humanity outweighs the benefits of the intellectual contribution - I can't really tell because even if I overlook the capitalisation, I'm not smart enough to understand it anyway.
> Hey, sorry! I don't want you to feel bad, and I don't think you should.
no worries, but apologies accepted, and sorry that i make you read my non-capitalized comments :)
> The problem I have with this piece is that it's clearly meant to be an intellectual or academic-adjacent piece
that's a good point i hadn't considered. i am of course for correct capitalization and grammar in "serious" documents etc., HN for me is more like a blog. but HN links to third-party, "serious" sites, where such things should matter.
> many thousands of people lose a few seconds each when reading.
i guess i can't really comprehend why some struggle with reading non-capitalized texts in english, because it doesn't matter to my brain. but valuable to know, that other people prefer it.
> I'm not smart enough to understand it anyway.
+1. at least we can discuss about language, so it's not that we're too dumb in general ;)
> EDIT: people are seriously so emotionally invested in capitalization that i get downvoted into minus, jeez.
I find it weird that you would be surprised that people care about the quality of textual communication
maybe because i use downvotes differently than others. downvote for me means, that someone either outright lies, is very disrespectful or adds nothing to the discussion.
i don't see it as a "i don't agree with this comment"-button. opinions differ, i guess :)
For completeness: I also disable all autocorrect/autocomplete on mobile because it's more trouble than it's worth, but I leave auto-capitalisation on. This is a thing, they're independent settings.
i absolutely HATE every automated keyboard-"helper", ever.
type in multiple languages constantly and all of these helpers constantly default to english usage. plus it would be weird to me if every sentence starts with a capital letter but the rest is left as it is. seems like such an arbitrary solution.
> It's a fad associated with AI, popularised by Sam Altman especially.
I know this is true but does anyone understand why they do it? It is actually cognitively disruptive when reading content because many of us are trained to simultaneously proof read while reading.
So I also consider it a type of cognitive attack vector and it annoys me extremely as well.
It's not true at all. It was very common, nearly norm in all online communication until phones started auto correcting with capitalization. You could always tell who was a mac/phone user by their use of capitalization. Sam is older than when that happened. He almost certainly spent the majority of his online life typing in lowercase, as I did. Go look at any old IRC chat log, forum, etc from his era.
The sibling comment to yours mentions that this is pretty common on Twitter, and I'd guess that it started as a way to making firing off tweets from a phone easier (since the extra effort to hit shift when typing on a phone keyboard is a bit higher, and the additional effort to go back and fix any typos that happen due to trying to capitalize things is also higher compared to using a traditional keyboard). Once enough people were doing it there, the style probably became recognizable and evoked a certain "vibe" that people wanted to replicate elsewhere, including in places where the original context of "hitting the shift key is more work than it's worth" doesn't hold as well.
> since the extra effort to hit shift when typing on a phone keyboard is a bit higher, and the additional effort to go back and fix any typos that happen due to trying to capitalize things is also higher compared to using a traditional keyboard
I'm a bit confused about this. Do people turn off auto capitalisation on their phones? I very rarely have to press shift on my phone
I and everyone I know turns it off. On many platforms and in many cultures, capitalization often implies Solemness or even rudeness in 1-on-1 conversations, and otherwise comes across as being out of touch in other kinds of communication.
Wow then I guess everyone finds me very rude. I capitalize, use correct grammar and spelling to the best of my ability in text messages just like any written communication. I find it rude when people don’t as I interpret it as they don’t even care enough about our communication to take the small effort to make their writing easy to comprehend and understand!
I’ve never encountered anyone turning it off. I avoid socialising with people who think it’s rude to use capitalisation.
It's not directly rude, it's more like a serious tone of voice. But it only works like that when used unnecessarily, like in chat or IM where the message boundary styling doubles as a sentence boundary.
Using the chat/IM style outside of that context just doesn't work and looks really odd, like it's obviously someone who didn't learn those norms and is now mimicking them without understanding them.