I actually don’t like python type hints!
At my work we have a jit compiler that requires type hints under some conditions.
Aside from that, I avoid them as much as possible. The reason is that they are not really a part of the language, they violate the spirit of the language, and in high-usage parts of code they quickly become a complete mess.
For example a common failure mode in my work’s codebase is that some function will take something that is indexable by ints. The type could be anything, it could be List, Tuple, Dict[int, Any], torch.Size, torch.Tensor, nn.Sequential, np.ndarray, or a huge host of custom types! And you better believe that every single admissible type will eventually be fed to this function. Sometimes people will try to keep up, annotating it with a Union of the (growing) list of admissible types, but eventually the list will become silly and the function will earn a # pyre-ignore annotation. This defeats the whole point of the pointless exercise.
So, if the jit compiler needs the annotation I am happy to provide it, but otherwise I will proactively not provide any, and I will sometimes even delete existing annotations when they are devolving into silliness.
That's the same complaints people had about TypeScript in the beginning, when libraries such as Express used to accept a wide range of input options that would be a pain to express in types properly. If you look at where the ecosystem is now, though, you'll see proper type stubs, and most libraries get written in TS in the first place anyway. When editing TS code, you get auto-completion out of the box, even for deeply nested properties or conditional types. You can rely on types being what the compiler says they are, and runtime errors are a rarity now (in properly maintained code bases).
> The reason is that they are not really a part of the language, they violate the spirit of the language, and in high-usage parts of code they quickly become a complete mess.
I'll admit that this is what I hate Python, and it's probably this spirit of the language as you call it. I never really know what parameters a function takes. Library documentation often shows a few use cases, but doesn't really provide a reference; so I end up having to dig into the source code to figure it out on my own. Untyped and undocumented kwargs? Everywhere. I don't understand how someone could embrace so much flexibility that it becomes entirely undiscoverable for anyone but maintainers.
Because the flexibility has been a boon and not a problem. The problem only comes when you try to express everything in the type system, that is third party (the type checkers for it) and added on top.
It's a boon if the goal is to write code then go home. It's a loaded footgun if the goal is to compose a stack and run it in production within SLO.
Python type hints manage to largely preserve the flexibility while seriously increasing confidence in the correctness, and lack of crashing corner cases, of each component. There's really no good case against them at this point outside of one-off scripts. (And even there, I'd consider it good practice.)
As a side bonus, lack of familiarity with Python type hints is a clear no-hire signal, which saves a lot of time.
I think with types there is a risk of typing things too early or too strictly or types nudging one to go in a direction, that reduces the applicability and flexibility of the final outcome. Some things can be difficult to express in types and then people choose easier to type solutions, that are not as flexible and introduce more work later, when things need to change, due to that inflexibility or limited applicability.
People say this all the time, but I've never seen any data proving it's true. Should be rather easy too, I'm at a big company and different teams use different languages. The strictly typed languages do to have fewer defects, and those teams don't ship features any faster than the teams using loosely typed languages.
What I've experienced is that other factors make the biggest difference. Teams that write good tests, have good testing environments, good code review processes, good automation, etc tend to have fewer defects and higher velocity. Choice of programming language makes little to no difference.
>It's a boon if the goal is to write code then go home. It's a loaded footgun if the goal is to compose a stack and run it in production within SLO.
Never has been an issue in practice...
Did you forget /s at the end of this?
I work at big tech and the number of bad deploys and reverts I've seen go out due to getting types wrong is in the hundreds. Increased type safety would catch 99% of the reverts I've seen.
Also have fun depending on libraries 10 years old as no one likes upgrades over fear of renames.
Ops type here, I’ve got multiple stories where devs have screwed up with typing and it’s caused downstream problems.
> Because the flexibility has been a boon and not a problem
Well, you could say that the problem in this case was the lack of documentation, if you wanted. The type signature could be part of the documentation, from this point of view.
Let me give a kind-of-concrete example: one year I was working through a fast.ai course. They have a Python layer above the raw ML stuff. At the time, the library documentation was mediocre: the code worked, there were examples, and the course explained what was covered in the course. There were no type hints. It's free (gratis), I'm not complaining. However, once I tried making my own things, I constantly ran into questions about "can this function do X" and it was really hard to figure out whether my earlier code was wrong or whether the function was never intended to work with the X situation. In my case, type hints would have cleared up most of the problems.
> the lack of documentation
If the code base expects flexibility, trusting documentation is the last thing you'd want to do. I know some people live and die by the documentation, but that's just a bad idea when duck typing or composition is heavily used for instance, and documentation should be very minimal in the first place.
When a function takes a myriad of potential input, "can this function do X" is an answer you get by reading the function or the tests, not the prose on how it was intended 10 years ago or how some other random dev thinks it works.
Documentation doesn’t have to be an essay. A simple, automatically generated reference with proper types goes a long way to tell me „it can do that“ as opposed to „maybe it works lol“. That’s not the level of engineering quality I’m going for in my work.
This whole discussion is about how you might not want to be listing every single types a function accepts. I also kinda wonder how you automatically generate that for duck typing.
Generally using the Protocol[1] feature
This of course works with dunder methods and such. Also you can annotate with @runtime_checkable (also from typing) to make `isinstance`, etc work with it[1]: https://typing.python.org/en/latest/spec/protocol.html
You're then creating a Protocol for every single function that could rely on some duck typing.
Imagine one of your function just wants to move an iterator forward, and another just wants the current position. You're stuck with either requiring a full iterator interface when only part of it is needed or create one protocol for each function.
In day to day life that's dev time that doesn't come back as people are now spending time reading the protocol spaghetti instead of reading the function code.
I don't deny the usefulness of typing and interfaces in stuff like libraries and heavily used common components. But that's not most of your code in general.
For the collections case in particular, you can use the ABCs for collections that exist already[1]. There's probably in your use case that satisfies those. There's also similar things for the numeric tower[2]. SupportsGE/SupportsGT/etc should probably be in the stdlib but you can import them from typeshed like so
---In the abstract sense though, most code in general can't work with anything that quack()s or it would be incorrect to. The flip method on an penguin's flipper in a hypothetical animallib would probably have different implications than the flip method in a hypothetical lightswitchlib.
Or less by analogy, adding two numbers is semantically different than adding two tuples/str/bytes or what have you. It makes sense to consider the domain modeling of the inputs rather than just the absolute minimum viable to make it past the runtime method checks.
But failing that, there's always just Any if you legitimately want to allow any input (but this is costly as it effectively disables type checking for that variable) and is potentially an indication of some other issue.
[1]: https://docs.python.org/3.14/library/collections.abc.html
[2]: https://docs.python.org/3/library/numbers.html
> You're then creating a Protocol for every single function that could rely on some duck typing.
No, you are creating a Protocol (the kind of Python type) for every protocol (the descriptive thing the type represents) that is relied on for which an appropriate Protocol doesn’t already exist. Most protocols are used in more than one place, and many common ones are predefined in the typing module in the standard library.
Except Typescript embraces duck typing. You can say "accept any object with a quack() method", for example, and it'll accept an unexpected quacking parrot. It can even tell when two type definitions are close enough and merge them.
So does Python. They're called protocols. [0]
[0]: https://typing.python.org/en/latest/spec/protocol.html
> Except Typescript embraces duck typing.
So does Python:
https://typing.python.org/en/latest/spec/protocol.html
It's not duck typing if you have to declare the type...
Kind of, depends on the compiler configuration.
Doesn't Go also use structural typing?
I like Python a lot, and have been using it for personal projects since about 2010. It was only once I started working and encountering long-lived unfamiliar Python codebases regularly that I understood the benefits of type hints. It's not fun to have to trace through 5 or 6 different functions to try to figure out what type is being passed in or returned from something. It's even less fun to find out that someone made a mistake and it's actually two different incompatible things depending on the execution path.
That era of Python codebases were miserable to work in, and often ended up in the poorly though out "we don't know how this works and it has too many bugs, let's just rewrite it" category.
> It's not fun to have to trace through 5 or 6 different functions to try to figure out what type is being passed in or returned from something.
My position is that what is intended must be made clear between type hints and the docstring. Skipping this makes for difficult to read code and has no place in a professional setting in any non-trivial codebase.
This doesn't require type hints to achieve. :param and :rtype in the docstring are fine if type hints aren't present, or for complex cases, plain English in the docstring is usually better.
:param and :rtype are type hints, just type hints that cannot be validated by tooling and are guaranteed to go out of sync with the code eventually.
Proper type hints are typically very easy to add if the codebase is not a mess that passes things around far and wide with no validation. If it is, the problem is not with the type hints.
I agree, although I've found that correct and comprehensive use of the doctoring for this purpose has not existed in the environments I've worked in, or the open source codebases I have needed to understand. Something about type hinting makes people more likely to do it.
I am sorry, but whats wrong with doing something like, `print(type(var)); exit()` and just running it once instead of digging through 5-6 stack frames?
Sometimes a function's input or return type can vary depending on the execution path? Also, inserting print statements is often not practical when working on web backend software which is kind of a big thing nowadays. If you can run the service locally, which is not a given, dependencies get mocked out and there's no guarantee that your code path will execute or that the data flowing through it will be representative.
They don’t violate the spirit of the language. They are optional. They don’t change the behaviour at runtime.
Type annotations can seem pointless indeed if you are unwilling to learn how to use them properly. Using a giant union to type your (generic) function is indeed silly, you just have to make that function generic as explained in another comment or I guess remove the type hints
> They don’t violate the spirit of the language. They are optional.
That in itself violates the spirit of the language, IMO. “There should be one obvious way to do it”.
Well, precisely:
- There is one obvious way to provide type hints for your code, it’s to use the typing module provided by the language which also provides syntax support for it.
- You don’t have to use it because not all code has to be typed
- You can use formatted strings, but you don’t have to
- You can use comprehensions but you don’t have to
- You can use async io, but you don’t have to. But it’s the one obvious way to do it in python
The obvious way to annotate a generic function isn’t with a giant Union, it’s with duck typing using a Protocol + TypeVar. Once you known that, the obvious way is… pretty obvious.
The obvious way not be bothered with type hints because you don’t like them is not to use them!
Python is full of optional stuff, dataclasses, named tuples, meta programming, multiple ancestor inheritance. You dont have to use these features, but there are only one way to use them
> but there are only one way to use them
Optional nature of those features conflicts with this statement. As optionality means two ways already.
classes are optional in python, does that violate the spirit?
"There should only be one way to do it" has not really been a thing in Python for at least the last decade or longer. It was originally meant as a counterpoint to Perl's "there's more than one way to do it," to show that the Python developers put a priority on quality and depth of features rather than quantity.
But times change and these days, Python is a much larger language with a bigger community, and there is a lot more cross-pollination between languages as basic philosophical differences between the most popular languages steadily erode until they all do pretty much the same things, just with different syntax.
> "There should only be one way to do it" has not really been a thing in Python for at least the last decade or longer.
It never was a thing in Python, it is a misquote of the Zen of Python that apparently became popular as a reaction against the TMTOWTDI motto of the Perl community.
Not misquoted, paraphrased. I didn't feel like bothering to check the output of "import this" before posting.
the whole language violates this principle tbh, so it's very in spirit
Yeah that ship sailed some time before they added a third way to do templated string interpolation.
How so? There is one way to do it. If you want typing, you use type hints. You wouldn't say that, say, functions are unpythonic because you can either use functions or not use them, therefore there's two ways to do things, would you?
And Python failed at that decades ago. People push terribly complicated, unreadable code under the guise of Pythonic. I disagree with using Pythonic as reasoning for anything.
This is a popular misquote from the Zen of Python. The actual quote is “There should be one—and preferably only one—obvious way to do it.”
The misquote shifts the emphasis to uniqueness rather than having an obvious way to accomplish goals, and is probably a result of people disliking the “There is more than one way to do it” adage of Perl (and embraced by the Ruby community) looking to the Zen to find a banner for their opposing camp.
And Python failed at that decades ago. People push terribly complicated, unreadable code under the guise of Pythonic. I disagree with using Pythonic as reasoning for anything.
on that note, which is better, using `map()` or a generator expression?
Actually in Python it can. Since the type hints are accessible at runtime, library authors can for example change which values in kwargs are allowed based on the type of the argument.
So on the language level it doesn’t directly change the behavior, but it is possible to use the types to affect the way code works, which is unintuitive. I think it was a bad decision to allow this, and Python should have opted for a TypeScript style approach.
You can make it change the behaviour at runtime is different than it changes the behaviour at runtime I think?
Lots of very useful tooling such as dataclasses and framework like FastAPI rely on this and you're opinion is that it's a bad thing why?
In typescript the absence of type annotations reflection at runtime make it harder to implement things that people obviously want, example, interop between typescript and zod schemas. Zod resorts instead to have to hook in ts compiler to do these things.
I'm honestly not convinced Typescript is better in that particular area. What python has opted for is to add first class support for type annotations in the language (which Javascript might end up doing as well, there are proposals for this, but without the metadata at runtime). Having this metadata at runtime makes it possible to implement things like validation at runtime rather than having to write your types in two systems with or without codegen (if Python would have to resort to codegen to do this, like its necessary in typescript, I would personally find this less pythonic).
I think on the contrary it allows for building intuitive abstractions where typescript makes them harder to build?
Yeah, but then you get into the issues with when and where generic types are bound and narrowed, which can then make it more complicated, at which point one might be better off stepping back, redesigning, or letting go of perfect type hint coverage, for dynamic constructs, that one couldn't even write in another type safe language.
I don’t know anything about your jit compiler, but generally the value I get from type annotations has nothing to do with what they do at runtime. People get so confused about Python’s type annotations because they resemble type declarations in languages like C++ or Java. For the latter, types tell the compiler how to look up fields on, and methods that apply to, an object. Python is fine without that.
Python’s types are machine-checkable constraints on the behavior of your code.. Failing the type checker isn’t fatal, it just means you couldn’t express what you were doing in terms it could understand. Although this might mean you need to reconsider your decisions, it could just as well mean you’re doing something perfectly legitimate and the type checker doesn’t understand it. Poke a hole in the type checker using Any and go on with your day. To your example, there are several ways described in comments by me and others to write a succinct annotation, and this will catch cases where somebody tries to use a dict keyed with strings or something.
Anyway, you don’t have to burn a lot of mental energy on them, they cost next to nothing at runtime, they help document your function signatures, and they help flag inconsistent assumptions in your codebase even if they’re not airtight. What’s not to like?
So the type is anything that implements the index function ([], or __getitem__), I thnink that's a Sequence, similar to Iterable.
>from typing import Sequence
>def third(something: Sequence):
> return indexable[3]
however if all you are doing is just iterate over the thing, what you actually need is an Iterable
>from typing import Iterable
>def average(something:Iterable):
> for thing in something:
> ...
Statistically, the odds of a language being wrong, are much lower than the programmer being wrong. Not to say that there aren't valid critiques of python, but we must think of the creators of programming languages and their creations as the top of the field. If a 1400 chess elo player criticizes Magnus Carlsen's chess theory, it's more likely that the player is missing some theory rather than he found a hole in Carlsen's game, the player is better served by approaching a problem with the mentality that he is the problem, rather than the master.
> So the type is anything that implements the index function ([], or __getitem__), I thnink that's a Sequence
Sequence involves more than just __getitem__ with an int index, so if it really is anything int indexable, a lighter protocol with just that method will be more accurate, both ar conveying intent and at avoiding needing to evolve into an odd union type because you have something that a satisfies the function’s needs but not the originally-defined type.
> we must think of the creators of programming languages and their creations as the top of the field
The people at the top of the type-system-design field aren’t working on Python.
That is sort of ironic because the Pythonistas did not leave out any opportunity to criticize Java. Java was developed by world class experts like Gosling and attracted other type experts like Philip Wadler.
No world class expert is going to contribute to Python after 2020 anyway, since the slanderous and libelous behavior of the Steering Council and the selective curation of allowed information on PSF infrastructure makes the professional and reputational risk too high. Apart from the fact that Python is not an interesting language for language experts.
Google and Microsoft have already shut down several failed projects.
>"Guido: Java is a decent language," 1999
I get the idea that Python and Java went in opposite directions. But I'm not aware of any fight between both languages. I don't think that's a thing either.
Regarding stuff that happens in the 2020. Python was developed in the 90s, python 3 was launched in 2008. Besides some notable PEPs like type hints, WSGI, the rest of development are footnotes. The same goes for most languages (with perhaps the exception of the evergrowing C++), languages make strong bc guarantees and so the bulk of their innovation comes from the early years.
Whatever occurs in the 20th and 30th year of development is unlikely to be revolutionary or very significant. Especially ignoreable is the drama that might emerge in these discussions, slander, libel inter-language criticism?
Just mute that out. I've read some news about some communities like Ruby on Rails or Nix that become overtaken by people and discussions of political nature rather than development, they can just be ignored I think.
> Google and Microsoft have already shut down several failed projects
Could you elaborate on this?
Sure: Google fired the Python language team in 2024 that contained a couple of the worst politicians who were later involved in slandering Tim Peters.
Before that, Google moved heavily from Python to Go.
Microsoft fired the "Faster CPython Team" this year.
It’s unlikely those layoffs are related to that, but rather the industry at large and end of zirp. Those type of folks are common in bigtech companies as well.
For example the dart/flutter team was decimated as well.
>you better believe that every single admissible type will eventually be fed to this function
That's your problem right there. Why are random callers sending whatever different input types to that function?
That said, there are a few existing ways to define that property as a type, why not a protocol type "Indexable"?
>why not a protocol type
it was a sin that python's type system was initially released as a nominal type system. they should have been the target from day one.
being unable to just say "this takes anything that you can call .hello() and .world() on" was ridiculous, as that was part of the ethos of the dynamically typed python ecosystem. typechecking was generally frowned upon, with the idea that you should accept anything that fit the shape the receiving code required. it allowed you to trivially create resource wrappers and change behaviors by providing alternate objects to existing mechanisms. if you wanted to provide a fake file that read from memory instead of an actual file, it was simple and correct.
the lack of protocols made hell of these patterns for years.
I disagree. I think, if the decision was made today, it probably would have ended up being structural, but the fact that it isn't enables (but doesn't necessarily force) Python to be more correct than if it weren't (whereas forced structural typing has a certain ceiling of correctness).
Really it enabled the Python type system to work as well as it does, as opposed to TypeScript, where soundness is completely thrown out except for some things such as enums
Nominal typing enables you to write `def ft_to_m(x: Feet) -> Meters: and be relatively confident that you're going to get Feet as input and Meters as output (and if not, the caller who ignored your type annotations is okay with the broken pieces).
The use for protocols in Python in general I've found in practice to be limited (the biggest usefulness of them come from the iterable types), when dealing with code that's in a transitional period, or for better type annotations on callables (for example kwargs, etc).
TypeScript sacrificed soundness to make it easier to gradually type old JS code and to allow specific common patterns. There is no ceiling for correctness of structural typing bar naming conflicts.
>The use for protocols in Python in general I've found in practice to be limited (the biggest usefulness of them come from the iterable types)
Most Python's dunder methods make it so you can make "behave alike" objects for all kinds of behaviors, not just iterables
AFAIK, Python is missing a fully-featured up to date centralized documentation on how to use type annotations.
The current docs are "Microsoft-like", they have everything, spread through different pages, in different hierarchies, some of them wrong, and with nothing telling you what else exists.
> That's your problem right there. Why are random callers sending whatever different input types to that function?
Because it’s nice to reuse code. I’m not sure why anyone would think this is a design issue, especially in a language like Python where structural subtyping (duck typing) is the norm. If I wanted inheritance soup, I’d write Java.
Ironically, that’s support for structural subtyping is why Protocols exist. It’s too bad they aren’t better and the primary way to type Python code. It’s also too bad that TypedDict actively fought duck typing for years.
Why can’t you re-use it with limited types? If the types are too numerous/hard to maintain it seems like the same would apply to the runtime code.
Because it’s nice to reuse code. It’s virtually never the case that a function being compatible with too many types is an issue. The issue is sometimes that it isn’t clear what types will be compatible with a function, and people make mistakes.
Python’s type system is overall pretty weak, but with any static language at least one of the issues is that the type system can’t express all useful and safe constructs. This leads to poor code reuse and lots of boilerplate.
>It’s virtually never the case that a function being compatible with too many types is an issue
This kind of accidental compatibility is a source of many hard bugs. Things appear to work perfectly, then at some point it does something subtly different, until it blows up a month later
if the piece of code in question is so type independent, then either it should be generic or it's doing too much
Yes. It's not the type system that's broken, it's the design. Fix the design, and the type system works for you, not against you.
> Why are random callers sending whatever different input types to that function?
Probably because the actual type it takes is well-understood (and maybe even documented in informal terms) by the people making and using it, but they just don’t understand how to express it in the Python type system.
> For example a common failure mode in my work’s codebase is that some function will take something that is indexable by ints. The type could be anything, it could be List, Tuple, Dict[int, Any], torch.Size, torch.Tensor, nn.Sequential, np.ndarray, or a huge host of custom types! And you better believe that every single admissible type will eventually be fed to this function. Sometimes people will try to keep up, annotating it with a Union of the (growing) list of admissible types, but eventually the list will become silly and the function will earn a # pyre-ignore annotation. This defeats the whole point of the pointless exercise.
You are looking for protocols. A bit futzy to write once but for a heavily trafficked function it's woth it.
If your JIT compiler doesn't work well with protocols... sounds like a JIT problem not a Python typing problem
In my experience, the right tooling makes Python typing a big win. Modern IDEs give comprehensive real-time feedback on type errors, which is a big productivity boost and helps catch subtle bugs early (still nowhere near Rust, but valuable nonetheless). Push it too far though, and you end up with monsters like Callable[[Callable[P, Awaitable[T]]], TaskFunction[P, T]]. The art is knowing when to sprinkle types just enough to add clarity without clutter.
When you hit types like that type aliases come to the rescue; a type alias combined with a good docstring where the alias is used goes a long way
On the far end of this debate you end up with types like _RelationshipJoinConditionArgument which I'd argue is almost more useless than no typing at all. Some people claim it makes their IDE work better, but I don't use an IDE and I don't like the idea of doing extra work to make the tool happy. The opposite should be true.
You can use a Protocol type for that, makes a lot mote sense than nominal typing for typing use case.
Exactly, sounds like misuse of unions.
Although Python type hints are not expressive enough.
No idea about Python type system, but doesn't it have anything like this?
It does!
You can specify a protocol like this:
(Edit: formatting)The syntax is definitely harder to grasp but if the mechanism is there, I guess the parent poster's concern can be solved like that.
Although I understant that it might have been just a simplified example. Usually the "Real World" can get very complex.
> The syntax is definitely harder to grasp
Yes it is. I believe the reason is that this is all valid python while typescript is not valid javascript. Also, python's type annotations are available at runtime (eg. for introspection) while typescript types aren't.
That said, typescript static type system is clearly both more ergonomic and more powerful than Python's.
I feel pretty similarly on this. Python’s bolted on type system is very poor at encoding safe invariants common in the language. It’s a straight jacketed, Java-style OOP type system that’s a poor fit for many common Python patterns.
I would love it if it were better designed. It’s a real downer that you can’t check lots of Pythonic, concise code using it.
It sounds like that function is rightfully eligible to be ignored or to use the Any designation. To me that's why the system is handy. For functions that have specific inputs and outputs, it helps developers keep things straight and document code.
For broad things, write Any or skip it.
from typing import Protocol, TypeVar
T_co = TypeVar("T_co", covariant=True)
class Indexable(Protocol[T_co]): def __getitem__(self, i: int) -> T_co: ...
def f(x: Indexable[str]) -> None: print(x[0])
I am failing to format it proprely here, but you get the idea.
Just fyi: https://news.ycombinator.com/formatdoc
> Text after a blank line that is indented by two or more spaces is reproduced verbatim. (This is intended for code.)
If you'd want monospace you should indent the snippet with two or more spaces:
I give Rust a lot of points for putting control over covariance into the language without making anyone remember which one is covariance and which one is contravariance.
One of the things that makes typing an existing codebase difficult in Python is dealing with variance issues. It turns out people get these wrong all over the place in Python and their code ends up working by accident.
Generally it’s not worth trying to fix this stuff. The type signature is hell to write and ends up being super complex if you get it to work at all. Write a cast or Any, document why it’s probably ok in a comment, and move on with your life. Pick your battles.
Kotlin uses "in" and "out": https://kotlinlang.org/docs/generics.html
Co- means with. Contra- means against. There are lots of words with these prefixes you could use to remember (cooperate, contradict, etc.).
There is also bunch of prepackaged types, such as collections.abc.Sequence that could be used in this case.
Sequence does not cut it, since the op mentioned int indexed dictionaries. But yeah.
> Sequence[SupportsFloat] | Mapping[int,SupportsFloat]
This is really just the same mistake as the original expanding union, but with overly narrow abstract types instead of overly narrow concrete types. If it relies on “we can use indexing with an int and get out something whose type we don’t care about”, then its a Protocol with the following method:
More generally, even if there is a specific output type when indexing, or the output type of indexing can vary but in a way that impacts the output or other input types of the function, it is a protocol with a type parameter T and this method: It doesn’t need to be union of all possible concrete and/or abstract types that happen to satisfy that protocol, because it can be expressed succinctly and accurately in a single Protocol.As of Python 3.12, you don’t need separately declared TypeVars with explicit variance specifications, you can use the improved generic type parameter syntax and variance inference.
So, just:
is enough.> eventually the list will become silly and the function will earn a # pyre-ignore annotation. This defeats the whole point of the pointless exercise.
No, this is the great thing about gradual typing! You can use it to catch errors and provide IDE assistance in the 90% of cases where things have well-defined types, and then turn it off in the remaining 10% where it gets in the way.
Define a protocol[0] that declares it implements `__getitem__` and type annotate with that protocol. Whatever properties are needed inside the function can be described in other protocols.
These are similar to interfaces in C# or traits in Rust - you describe what the parameter _does_ instead of what it _is_.
[0]: https://typing.python.org/en/latest/spec/protocol.html
>The type could be anything, it could be List, Tuple, Dict[int, Any], torch.Size, torch.Tensor, nn.Sequential, np.ndarray, or a huge host of custom types!
That's not how you are supposed to use static typing? Python has "protocols" that allows for structural type checking which is intended for this exact problem.
Sounds like the ecosystem needs an "indexable" type annotation. Make it an "indexable<int>" for good measure.
Right, this was my thought.
Can’t you just use a typing.Protocol on __getitem__ here?
https://typing.python.org/en/latest/spec/protocol.html
Something like
Though maybe numpy slicing needs a bit more work to supportIndeed.
IMO, the trick to really enjoying python typing is to understand it on its own terms and really get comfortable with generics and protocols.
That being said, especially for library developers, the not-yet-existant intersection type [1] can prove particularly frustrating. For example, a very frequent pattern for me is writing a decorator that adds an attribute to a function or class, and then returns the original function or class. This is impossible to type hint correctly, and as a result, anywhere I need to access the attribute I end up writing a separate "intersectable" class and writing either a typeguard or calling cast to temporarily transform the decorated object to the intersectable type.
Also, the second you start to try and implement a library that uses runtime types, you've come to the part of the map where someone should have written HERE BE DRAGONS in big scary letters. So there's that too.
So it's not without its rough edges, and protocols and overloads can be a bit verbose, but by and large once you really learn it and get used to it, I personally find that even just the value of the annotations as documentation is useful enough to justify the added work adding them.
[1] https://github.com/python/typing/issues/213
Slicing is totally hintable as well.
Change the declaration to:
def __getitem__(self, i: int | slice)
Though to be honest I am more concerned about that function that accepts a wild variety of objects that seem to be from different domains...
I'd guess inside the function is a HUGE ladder of 'if isinstance()' to handle the various types and special processing needed. Which is totally reeking of code smell.
You explained some hyper niche instance where type hints should be ignored. 99% of the time, they are extremely helpful.
It's not even a niche instance, protocols solve their problem lol.
the issue of having multiple inputs able to be indexable by ints, is exactly why i prefer that type hints remain exactly as "hints" and not as mandated checks. my philosophy for type hints is that they are meant to make codebases easier to understand without getting into a debugger. their functional equivalence should be that of comments. it's a cleaner more concise way of describing a variable instead of using a full on docstring.
though maybe there's a path forward to give a variable a sort of "de-hint" in that in can be everything BUT this type(i.e. an argument can be any indexable type, except a string)
>though maybe there's a path forward to give a variable a sort of "de-hint" in that in can be everything BUT this type
I think this is called a negation type, and it acts like a logical NOT operator. I'd like it too, and I hear that it works well with union types (logical OR) and intersection types (logical AND) for specifying types precisely in a readable way.
Can't you define your own hint for "type that has __getitem__ taking int"?
The way I understand parent is that such a type would be too broad.
The bigger problem is that the type system expressed through hints in Python is not the type system Python is actually using. It's not even an approximation. You can express in the hint type system things that are nonsense in Python and write Python that is nonsense in the type system implied by hints.
The type system introduced through typing package and the hints is a tribute to the stupid fashion. But, also, there is no syntax and no formal definitions to describe Python's actual type system. Nor do I think it's a very good system, not to the point that it would be useful to formalize and study.
In Russian, there's an expression "like a saddle on a cow", I'm not sure what the equivalent in English would be. This describes a situation where someone is desperately trying to add a desirable feature to an exiting product that ultimately is not compatible with such a feature. This, in my mind, is the best description of the relationship between Python's actual type system and the one from typing package.
> In Russian, there's an expression "like a saddle on a cow", I'm not sure what the equivalent in English would be
“To fit a square peg into a round hole”
Close but not the same. In Russian, the expression implies an "upgrade", a failed attempt at improving something that either doesn't require improvement or cannot be improved in this particular way. This would be a typical example of how it's used: "I'm going to be a welder, I need this bachelor's degree like a saddle on a cow!".
"Lipstick on a pig"? Although that's quite more combative than the Russian phrase.
Yeah... this seems like it would fit the bill nicely. At least, this is the way I'd translate it if I had to. Just didn't think about it.
I like your point! I think the advantage in its light is this: People often use Python because it's convention in the domain, the project already uses it, or it's the language the rest of the team uses. So, you are perhaps violating the spirit, but that's OK. You are making the most of tools available. It's not the Platonic (Pythonic??) ideal, but good enough.
Isn't this supported by typing.SupportsIndex? https://docs.python.org/3/library/typing.html#typing.Support...
Mind you, I haven't used it before, but it feels very similar to the abstract Mapping types.
__index__ is not what you think it is
Oops, thanks for the correction. That's on me for drive-by commenting.
Why not do Indexable = Any and pass that? Even if it doesn't help your jit or the IDE, at least it is more explicit than
I mean, you can just... Not annotate something if creating the relevant type is a pain. Static analysis \= type hints, and even then...
Besides, there must be some behavior you expect from this object. You could make a type that reflects this: IntIndexable or something, with an int index method and whatever else you need.
This feels like an extremely weak argument. Just think of it as self-enforcing documentation that also benefits auto-complete; what's not to love? Having an IntIndexable type seems like a great idea in your use case.
> The reason is that they are not really a part of the language, they violate the spirit of the language
This is a good way of expressing my own frustration with bolting strong typing on languages that were never designed to have it. I hate that TypeScript has won out over JavaScript because of this - it’s ugly, clumsy, and boilerplatey - and I’d be even more disappointed to see the same thing happen to the likes of Python and Ruby.
My background is in strongly typed languages - first C++, then Java, and C# - so I don’t hate them or anything, but nowadays I’ve come to prefer languages that are more sparing and expressive with their syntax.
For this very specific example, isn't there something like "Indexable[int]"?
> something that is indexable by ints. > ...Dict[int, Any]...
If that is exactly what you want, then define a Protocol: from __future__ import annotations from typing import Protocol, TypeVar
Then you can call "first" with a list or a tuple or a numpy array, but it will fail if you give it a dict. There is also collections.abc.Sequence, which is a type that has .__getitem__(int), .__getitem__(slice), .__len__ and is iterable. There are a couple of other useful ones in collections.abc as well, including Mapping (which you can use to do Mapping[int, t], which may be of interest to you), Reversible, Callable, Sized, and Iterable.why not a protocol with getitem with an int arg?
This is like saying you don’t like nails because you don’t understand how to use a hammer though. Developers are not understanding how to use the hints properly which is causing you a personal headache. The hints aren’t bad, the programmers are untrained - the acknowledgement of this is the first step into a saner world.