Opaque Types in Python
blog.glyph.im59 points by lumpa 3 days ago
59 points by lumpa 3 days ago
Java made opaque types possible by private and package-private constructors.
It's sad to see that many features regarding object-oriented programming and static typing are implemented worse in Python than Java. Various examples: __str__() vs. toString(), underscore vs. private, @staticmethod/@classmethod vs. static, generic types are so clunky in Python, types are not given in the official Python standand library documentation, __init__() doesn't force you to call super() whereas it's mandatory in Java.
Funny, I ran into the same pattern just a few months ago!
In practice, I found it difficult for coworkers to read and understand so I dropped the idea.
Another limitation I found is that it breaks down when you start using inheritance. For example:
```
class _A: pass
A = NewType("A", _A)
class _B(_A): pass
B = NewType("B", _B)
def foo(a: A) -> None: pass
b = B(_B())
foo(b) # Mypy is not happy: Argument 1 to "foo" has incompatible type "B"; expected "A"
foo(A(b)) # Mypy is OK
```
The main problem with such approach is that `class _RealShipOpts:` is very ugly to write unit tests for. You need to import a private entity in tests. I would slightly change the presented approach, and move the "public" `ShippingOptions`, `shipFast`, etc., into a new module that is a public API, for my users to use something like `from my_lib.shipping.api import ShippingOptions`.
That way, I can use "normal" naming in `class RealShipOpts:...`, and be explicit that it's not really public for the end users (they should use the `.api` module instead).
You're holding it (Python) wrong. Python OO was a counter reaction to the bondage and discipline that languages like C++ had with private members and protected inheritance.
If you have members that users probably shouldn't touch, you prepend them with an underscore. This is just a hint; It doesn't actually change anything. We're all adults here and we know the consequences of reaching into implementation details.
I agreed with this 100% for a long time. Then I started working on a library at $WORK with dozens of downstream users abusing the hell out of my idiomatic underscore usage, especially in the context of lazy tests with folks writing endless mocks. When I’d “break” their test suite (blocking some time sensitive release) I’d get all kinds of shit. But _they_ were breaking the contract. Unfortunately I had little (if any) control on the path of application code making it to production (yeah yeah not great engineering org, but it’s the world I lived in). Strategies like this post would be helpful for said situations.
There’s always the extra idiomatic __SECRET_INTERNALS_DO_NOT_USE_OR_YOU_WILL_BE_FIRED for coworkers that can’t take a hint.
Something similar happened to me. I told those groups to pound sand because they knew they were relying on something which they should not. Manager had my back, they whined a lot but they had to change and improve their processes.
> We're all adults here and we know the consequences of reaching into implementation details.
I wish you were right but, IMHE, it requires a lot of communication once teams grow and many team member do not fully understand the consequences of what they do. It is nice to have something that helps when reviewing code.
> If you have members that users probably shouldn't touch, you prepend them with an underscore
Well, this is precisely what TFA does. It prepends the constructor with an underscore.
I think you missed the issue at hand:
> even if you keep all your fields private, the constructor is still, inherently, public.
ShippingOptions and the literals / enums are part of the public API, so the user would just be writing
ShippingOptions(Carrier.USPS, Conveyance.Air)
Dataclasses do have a `kw_only` option, but I'm not sure how well underscore prefixes would be understood as private parameters / a private ctor, whereas wrapping a clearly "private" type should be clear to everybody.
Glyph is not entirely correct on the "any class" bit as you can always break the default init path:
class ShippingOptions:
_ship: Literal["fast", "normal", "slow"]
__init__ = None
def shipFast() -> ShippingOptions:
opts = object.__new__(ShippingOptions)
opts._ship = "fast"
return opts
"Glyph" knows. He has been in the Python inner circle for decades back to when the circle promoted "spam and eggs" and "consenting adults".
Like the rest of that circle, he moves with the times, supports public shaming of Tim Peters and others and now promotes poorly implemented information hiding so Python ticks a few more boxes for the industry.
Information hiding in a language that allows changing the values of small integers at runtime via ctypes is doomed anyway. And there are plenty of better languages that do it out of the box and in a straightforward manner.
I'm sorry but if you write Python functions/methods in camel case I can't take you seriously.
Just fyi, the author is https://en.wikipedia.org/wiki/Glyph_Lefkowitz, creator of Twisted.
Seems like this is even more reason to set a good example and follow the style guide.