Python Polars Cheatsheet (based on our O'Reilly book)

Is there an .MD version? For, uh, reasons :)

Agreed, as an R and polars user. The fundamental advantage R holds over other languages/libraries is expressions. The ability to reference columns directly AND interoperate with vectorized base ops in R is unfair. Of course, this super power is equally confusing to learners, fraught for production code, etc.

Less friction, considerably faster. I have a statistician friend who's recently made the jump away from R. I think he would agree with you.

I've seen people with exactly that frustration use "import polars.col as c" and use c("colname") instead!

The bare R experience is not that great, to put it mildly, but it's a whole other story if you add tidyverse on top of it. The data work becomes really easy then, but I still prefer Python because of familiarity and a better experience & ecosystem when you want to do anything beyond data wrangling & analysis.

I found `polars` to be a better experience than `pandas` even though I'd say it leaks some "Rustisms" in its Python APIs. But LLMs alleviate those pains and it's easy enough to review. I'd say it's even easier when there's less of a chance of implicit behavior.

If your work is more focused on statistics or pure modeling, then I agree R wins hands down. The issue is that most projects have "unclean" parts where you have to gather data from multiple sources, use connectors for services, S3 buckets and whatnot; dealing with that mess is where Python really shines.

AI probably changes the equation to some extent, but I still believe I'd rather maintain a complicated data pipeline like that in Python rather than R.

I first learned about R at a python user group meeting. It was when Pandas was new, and we were having a bunch of talks about it. Wes McKinney even came to give one before going to Pycon.

Anyway, the general consensus at the time was that R was much nicer once you had your data, and if all you had to do was transform it. But that everything else was better in Python.

One of our group members did an experimental project, where you could open R inside of python and share memory. So you could theoretically do your API calls and screen scraping and whatnot in Python, then transform your data in R, then take the output and use it to do something else in Python. It was pretty cool, but I think it was just a POC and never really went anywhere.

I tried learning R after that, but didn't get very far with it.

Data science is a broad church, but my own branch of it (molecular biology, genomics and epigenetics, bioinformatics) and my partner's (ecology) are still very much R-based and entrenched. If you primarily care about statistical modeling, then R still wins (easily).

That doesn’t justify lots of stuff that R lacks vs Python. Anytime someone new joins my corp, they reluctantly move off of R from their academic days on Python and once in the ecosystem, they never look back

[dead]

Same. Almost every time I would use its streaming interfaces in Python, it would STILL materialize everything into memory. That was like 6 months ago. Maybe streaming interfaces actually work, but I found them to be leaky abstractions that required a ton of hand holding to make sure they didn't build a bunch of memory pressure, if you're lucky enough to even have a way to do it.

For example, last time I used it, you couldn't do NDJSON streaming scans from S3 (looks like fixed with PR #26563).

Yep, and chdb as well. Said this before, chdb's DatStore is a pretty neat pandas replacement too.

even for just in-memory quick data analysis?