I just returned to Python for the first time in a little while to collaborate on a side project and ran into a few tricky-to-debug errors that caused a fair bit of lost time. Know what the errors were?

In one case, I was iterating over the contents of what was supposed to be a list, but in some rare circumstances could instead be a string. Instead of throwing a type error, Python happily went along with it, and iterated over each character in the string individually. This threw a wrench into the works because the list being iterated over was patterns, and when you apply a single character as a pattern you of course match much more than you're expecting.

And another one, I typoed a variable name as groups_keys instead of group_keys (or vice-versa, I don't remember). Instead of just throwing an error, Python happily went along with it, used an uninitialized value, and then all the logic broke.

There's entire classes of errors you can cause yourself in Python that aren't possible in stronger, statically-typed languages. For a large project, I'd pick the old and boring Java over Python every time.

> And another one, I typoed a variable name as groups_keys instead of group_keys (or vice-versa, I don't remember). Instead of just throwing an error, Python happily went along with it, used an uninitialized value, and then all the logic broke.

This isn't what Python would do, if the variable was undefined Python will throw an error, so you must have defined it with this name or you're misremembering what have happened.

Point in the GP's favor: Fixing it would definitely not be a problem with an early-binding language! In fact, the nigh-impossibility of automated refactoring puts lie to the notion that late-binding languages are more "agile."

It's a design flaw, in the same way Python 2's allowing comparisons between different types was a flaw, e.g. "a" < 3 succeeds. Python 3 now, correctly, throws a TypeError because there's no sensible ordering between the two things.

(While I'm griping: Another design flaw is conflating iterables and iterators, which makes generators almost useless. Say a generator is pass to a function expecting an iterable. If the function uses it twice, the second time it silently returns nothing!)

> This isn't what Python would do, if the variable was undefined Python will throw an error

I think GP must have assigned to the name, in which case Python will create a lexically bound name.

Python's rules for naming can make perfect sense or be quite surprising:

    try:
        x = four()
    except Thing:
        x = 5
    print(x)  # 4 or 5

    for a in [1, 2, 3]:
        pass
    print(a)  # 3 ?!

  def my_func():
      group_keys = 'My thing'
      while group_keys == 'My Thing':
          if some_logic() == 42:
              groups_key = 'My other thing'  # typo here

Right off the top of my head, using type hints with a decent IDE or an assert statement would likely have caught the issue.

I'm not saying that Python doesn't have issues (all languages do), but I don't see the error noted above as any sort of deal breaker. On the other hand, if you're only ever going to use Python like a strongly typed language without taking any advantage of its dynamic characteristics, then I can see why it would seem as a total downgrade compared to languages like Java.

  group_keys = ...
  if not isinstance(group_keys, list):
    groups_keys = [ group_keys ]

However, here's a related but different python gotcha:

    if foo(a):
        v = list(bar(a))
    for i in v:
        print i

This is clearly a bug in the code, yet no static analysis tools I've tried have successfully discovered it. There's a bug in pylint that's been marked WONTFIX because it requires a wildly different implementation. At a language level, it feels weird that if blocks aren't a scope level for new variables. If you want to reference v outside the if loop, declare / assign it outside the loop first.

    unclear_type_thing = ...
    if isinstance(unclear_type_thing, list):
      group_keys = unclear_type_thing
    else:
      group_keys = [unclear_type_thing]

And I should add that the "right" way to do this would be to factor this out to a function:

    group_keys = coerce_to_list(...)

But is there a way to force immutable variables in Python? You can easily still end up in the same situation when you typo something (easy to do when plurals are involved), and then end up reassigning something when you meant to create a new variable.

Needless to say, I was not using Pycharm for this development, nor am I likely to install an entire IDE just for a small change I'm making on a random project. It's a non-trivial burden to configure and learn an entire IDE, vs just using what I already know (which is often just emacs).

any new tool chain.

It's hard to take complaints like this seriously.

Because those IDEs those solve problems, so that you can close tickets on the project at hand, without having to port it to the best language evar.

Not always. Mypy has experimental support for `Final[T]` [0], and attrs/dataclasses support final/frozen instances, but that's opt in on a per-argument basis.

[0]: https://mypy.readthedocs.io/en/latest/final_attrs.html

Uh. no. he typoed the reassignment, so it wouldn't re-assign the value.

> So, of three of the most popular dynamic languages, Python, Ruby, and Javascript, none of them would have helped you catch this kind of error at script-parsing time. So again, it seems like you have an irrational dislike for Python, all things considered.

Sure, but he's made it clear he likes Java. Fundamentally he's against dynamic typing, so of course he doesn't like any of the dynamic languages.

You're also assuming that I don't have the same problems with Ruby or JavaScript. I do. The exact same critique could be made of them as well, but they're not the subject of this thread; Python is.

The tradeoff is that dynamic languages are faster to develop, more concise, but more expensive in maintenance exactly because of issues like this. The data that I base this opinion on is an unpublished internal report from nearly a decade ago at Google quantifying costs of projects of different size in their different languages. Which was Java, C++, and Python. Python had the lowest initial development, and the highest maintenance costs. That is why Google then moved to Go as a replacement for Python. It was good for the same things that they used Python for, but being statically typed, its maintenance costs were lower.

To be clear - I'm not trying to say that Python is better in any objective way. Ultimately, I think people should use the tools they have available and prefer, to build what they want.

As I said, I'm a fan of dynamic languages. :-)

One of the top ways that startups fail is failing to build what they need to build quickly enough. Maintenance costs only matter if you succeed in the first place. Using dynamic languages is therefore a good fit.

But, even if you're not Google, if you're writing software and have the luxury of paying attention to the lifetime costs of the project up front, you should choose a statically typed language.

You could as well say "Just check if the object is a string" in the method, which would work but the point was rather that it is difficult to notice if you did not think about it. Compared to other languages that would crash or not compile instead.

I've had success validating such data against a schema, so I know it had consistent type structure before working with it.

But a strongly-typed language will catch such errors automatically (and for free) at compile time, even if you didn't anticipate the failure case.

Honestly this was a major attraction to python for me a decade plus ago as a student when I started learning--even when I used it irresponsibly. There are so many small tasks where you just kinda have to iterate over 100-1000 items that you're not worried about big-O or anything like that—you just want to iterate and work on a collection quickly for some task in the office.

I've been using python for about 13 years professionally and I wrote up a list of "things I wish python would fix but I think probably never will" and treating strings as iterable lists of characters was on there.

I've seen this bug multiple times and the fix is relatively easy - just to make strings by default non-iterable and use "string.chars" (or something) if you really want to iterate through the chars.

Nonetheless, I still love the language and wouldn't use anything else.

>Instead of just throwing an error, Python happily went along with it, used an uninitialized value, and then all the logic broke.

This one gets caught by linters - unfortunately 90% of most python linters spit out rule violations which aren't important which drowns out stuff like this in the noise.

What else is on your list? I'd be interested to see what other parts of Python you would wish to change.

* Implicitly casting strings, integers, dates, etc. to boolean (e.g. "if x" being true if x is a non empty string). Cause of more unexpected bugs that I can count, but would cause massive headaches if implemented and memories of the the 2-to-3 transition would scare anybody away from doing this I think.

* Treating booleans as integers (True + True = 2). Probably wouldn't cause that many headaches if implemented but everybody still seems to think it's a neat idea for some reason.

* Treating non-package dependencies of pip packages (e.g. C compilers, header files) as something that is either the package's problem or the OS's problem. Nobody looks at this problem and thinks "I should solve this".

So yeah, I totally agree with you, it'd be better if trying to iterate over a string were a flat-out error, and if you really want it, you should mean it. Though Python being dynamic still means that you'll only spot this error at runtime.

As for linters, how do they know if your intent was to reassign the value of an existing variable, or to define a new one? The language has no way to indicate which of these is intended.

I remember a bug I made using C#, where I wanted to delete a collection of directories recursively. I got mixed up into the various inner loops and ended up iterating over the characters like you. But C# allows implicit conversions of char to string so the compiler was OK with it, and since those where network drive directories (starting with "\\server\"), the first iteration started deleting recursively the directory "\", which in windows means the root directory of the active drive (c:\)... And SSDs are fast at deleting stuff.

This actually ruined type annotation in Python: you can't properly annotate a function to accept anything iterable except string.

https://github.com/python/mypy/issues/4334 is a FR to at least check it with mypy.

Python doesn't have uninitialized values, it throws NameError when you try to access a variable that hasn't been set. So I don't see how this could have happened.

Python code needs more testing, more run time type checking of function arguments than a statically typed language. If that's a deal-breaker then one shouldn't be using Python in the first place. What you gain though is some instant gratification, and the ability to get something off the ground quickly without spending time placating the type checker. Its great where your workflow involves lot of prototyping, exploration of the solution space and interactive use (ML comes to mind, but even there int32 vs int64 can byte, correction, bite). I see it as a trade off -- deferring one kind of work (ensuring type safety) over another. Hopefully that deferral is not forever. I like my type safety but sometimes I want that later.

What I typically do is once I am happy with a module and I do not need the extreme form of dynamism that Python offers (something that's frequently true) I take away that dynamism by compiling those parts with Cython.

It could be javascript.

The creator of a well-known alternative to Python has a single-letter email address, and regularly receives email generated by Python scripts with this exact bug (which means instead of sending an email to "user", sends an email to "u", "s", "e", and "r"). So I’ve heard.

(And nowadays, you can also use type hints to give you a warning for this kind of thing, e.g. your IDE/mypy will complain about passing a string where the function signature specified a List.)

Python is great for sharing ideas / concepts, but under some circumstances it seems irresponsible to choose it over other viable options like Go (if you use Python because it's easy), or C# (If you use Python because it's a 'safe' enterprise choice). (Ecosystem specific things aside at least)

If there's a code path that passes in a bare string instead of a list, and your logic breaks, then that code path should have a failing test case. However, type hints can provide another opportunity to catch this kind of mismatch before they even get committed.

> under some circumstances it seems irresponsible to choose it over other viable options like Go (if you use Python because it's easy)

This is probably true, but I think people tend to overuse this argument (i.e. use an overly broad set of "some circumstances"). I build fintech apps with Python, for example, and don't find any of these issues to be a problem. In my experience, if you implement sound engineering practices (thorough testing at unit, integration, and system levels, code review, clear domain models, good encapsulation of concerns, etc.), then the sort of errors that bite you are not ones that a type checker would help with. I agree that the worst Python code is probably far more unsound than the worst Go code, but I don't think that's the correct comparison; you should be comparing the Python and Go code that _you_ (or your team) would write.

I think it's easy to be dogmatic about this kind of thing; in practice most people are substituting personal preference for technical suitability. Sure, there are cases where the performance or correctness characteristics of a particular language make it more suitable than another. But for most software, then whatever your team is expert in is the best choice.

Moreover, using a strongly typed language doesn't magically make you invulnerable to invalid input. Unit tests are useful in every language.

There is no such thing in Python. You should get NameError if a name doesn't refer to any object.

  >>> def f():
  ...     name
  ... 
  >>> f()
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "<string>", line 2, in f
  NameError: name 'name' is not defined

There's a good document on this:

https://mypy.readthedocs.io/en/latest/existing_code.html

I've never seen a strict, type-annotated Python project out there in the wild, and I've seen a decent amount of them. A random non-primary-contributor isn't going to have much luck stepping into an established project and getting everyone to go along with adding annotations to the whole thing.

And if I were starting a project from scratch, rather than coercing the language to do something it wasn't really designed for, I'd just use a language that has first-class support for types directly in the compiler, like Java or Go.

It's a first language I worked with in my life that just clicked with my brain and doesn't just drain me.

I would take a Python job over a Java/C/C++/Go/Rust any day. There's some languages that could pull me away from Python (Nim, Crystal) but they're nowhere popular enough to move wholesale to them.

it's funny, I feel the exact opposite. I work on a team that maintains a digital catalog, and a lot of what we write is about taking in asset- and metadata files, asynchronously processing them, and then publishing that to a denormalized read-optimized data store. We often joke that we mostly take data from 'over here' and put it 'over there'.

All our stuff is in Java, and honestly, if you use Lombok to squeeze out the boilerplate, and a decent dependency injection framework like Guice or Dagger, modern Java really isn't so bad. Streams are clunky but they get the job done. We use Jackson a lot to serialize/deserialize Java pojos to JSON and XML, which is pretty seamless for us so far. The Optional class is again clunky, but it works well enough.

The thing for us though is that the problems we spend most time solving are just not really related to the language we write it in. The hard problems are much more around things like operations (cd/ci, metrics, alarms, canaries), performance (latency, load, etc.) and just the nuts and bolts of the business logic (what type should this new field be? what values can it take? how do we plumb this through to downstream system X owned by team Y? etc.)

I honestly wouldn't want to have to write this stuff in Python for a simple reason: I don't think I could live without static typing, which is a fantastic tool when you need to manage a large code base written by multiple people over multiple years. I can make a change in some package, do a dry-run compile of every system that uses it, and then see what needs updating. It gives me certain guarantees about data integrity right at compile time, which is super helpful when you're doing data conversion.

But hey, different jobs, different tools. Glad you found something you're happy with.

Programming in the large without type safety is a fool’s errand.

> But hey, different jobs, different tools.

Exactly. There’s a reason your kitchen drawer isn’t full of just sporks.

Lol. Right. No big system has ever been built in an untyped or weakly typed language. Well, except just about every bit of software we all use everyday. But it does seem like some small startups can't get by without it.

Many have built models of the Eiffel tower with toothpicks too, so?

You can still built things with inadequate tools: inadequate != prohibitive. You just have more problems going forward.

Which is exactly the lesson people who write large scale software have found.

What is this "just about every bit of software we all use everyday" that you wrote about as been written in weak types?

Most major software is still written in C/C++ (anything from operating systems, Photoshop, DAWs, NLEs, UNIX userland, MS and Open Office, databases, webservers, AAA games, what have you). One could use just that C/C++ software, and they'd have almost all bases covered.

The rest is e.g. Electron based software and online services. For the latter, most of the major ones (e.g. Gmail, Apple's iCloud services, Microsofts, online banks, online reservations, etc, etc) are not written in "weakly typed languages", only the client is.

And those that were initially written in a weakly typed language, e.g. Twitter with Ruby on Rails, others with Python, etc, have rewritten critical services (or entirely) to statically typed languages (e.g. Twitter went for Java/Scale, others for Go, etc).

And even for the client, most shops are now turning to Typescript (and FB to Flow) because they've found weakly typing is not good enough for large scale. So?

It's at best a colloquial term and it's misleading to non-technical management.

Strong typing means that types cannot be substituted for other types. In C, you can write `int x = "one"` and the char * (address of) "one" is automatically converted to an int, or in Javascript you can write 1 + "2" and a string "1" is automatically created; depending who you're talking to, either or both of these qualify as weak typing.

They're both spectrums, and commonly confused with each other.

When you say the type cannot change, that's ambiguous: do you mean the type of the value a variable holds, or the type of the value itself? In C (a statically typed language), "int x" means that x will always hold an int, but you can still assign a pointer to it, it just turns into an int (weak typing). In Python (a dynamically typed language), the variable "x" wouldn't have a type (so it could hold an int at one point and a string later), but the value it holds does, and because it's strongly typed, it would throw a type error if you attempted to use it in a place where it wanted a different type (eg, `1 + "2"` does not turn 1 into a string or "2" into an int).

Static typing was all the rage 20 years ago. C++ and Java were going to save us from the chaos of C. What people found was the vast bulk of software defects are not problems that can be detected by static typing.

Static typing just created a constraining, inflexible code base, that was no more reliable than C or smalltalk or lisp. Once your beautifully conceived collection of types were demolished by the cold hard reality of changing business requirements the type system actively worked against you.

Python and ruby and javascript started gaining traction, and at first it seemed crazy to use a language that didn't have a static type checker. But after people started using them they realized they just didn't have the kinds of bugs that a static type checker would catch anyway - because those types of bugs are caught by the dynamic type checker (something C doesn't have, and C++ only sort of kind of has) at run time when you write tests. And writing tests also caught all kinds of other logic bugs that didn't have anything to do with types. They were writing software faster and more reliably in dynamically typed langues than they ever could in the old statically typed languages.

Of course no language is a silver bullet, and writing software is still hard. Combine that with the fact that our industry has no sense of history, and a fair number of programmers today have only used dynamically typed languages, and you can see why the static typing fad is coming back around.

It seems intuitive that caching these type errors at compile time rather than run time will make for a more reliable system. But history tells us otherwise. Unless you just don't run your code before pushing it to production the dynamic type checker will catch it just as well when you run tests. And your types will drift away from the reality of the business requirements grinding development to a halt.

The static typing fad has a 5 year shelf life. Just enough time for managers to force a new generation of programmers to re-write all their code in typescript or whatever and learn it is just as unreliable, and much harder to work with.

Programming in the large without tests is a fool's errand. Type systems don't guarantee correctness.

(Sound) Type systems guarantee correctness for the invariants encoded as types. If it compiles, you know it doesn't have any type related errors at all. With more evolved type systems even your program's logic (or large parts of it) is guaranteed.

Tests just allow you to test random invariants about your program. If it compiles and your add() method works when passed 2, 2 and gives 4, it still might not work for 5, 5... (contrived example: imagine it with much more complex functions, though even a simple e.g. "one line" time conversion can have similar issues).

If you skimp on testing your system will be crap, but at least the type system can fool you into thinking otherwise because it still compiles.

Actually, if your type system is powerful enough, you don't need to test. That's the source of the "if it compiles, 99% of the time it works right" people mention about Haskell (and even more so languages like Idris etc).

Type systems are tests -- just formal and compiler-enforced, not ad-hoc "whatever I felt like testing" tests, like unit tests are.

From there on it's up to the power of the type system. But even a simple type system like Java's makes whole classes of tests irrelevant and automatically checked.

A programmer can also leverage a simpler type system to enforce invariants in hand crafted types -- e.g. your "executeSQL" function could be made to only accept a "SafeString" type, not a "string" type, and the SafeString type could be made to only be constructed by a method that properly escapes SQL strings and params. Or the same way an Optional type ensures no null dereferences.

Types only eliminate certain tests. You will always have system tests, acceptance tests and unit tests. One should use types to augment their system reliability.

Types will not catch logical errors in your code.

An effective use of a type system such as Haskell's Hindley-Milner can result in a vastly smaller surface area for possible problems and thus can cut a big number of otherwise mandatory unit tests off your todo list.

If you have to point to something like Idris I don't think you're making a real world argument yet.

I don't get why people try to use sophisticated types systems to prove software, when writing and maintaining tests is so superior, and funnier too!

Both static type systems and unit testing have their disadvantages. For static type systems, you sometimes need to bend backward to make it accept your code and it's not very useful before the code grows large enough. For unit tests, even if you have 100% test coverage, it doesn't mean that you're safe - underlying libraries may behave in unexpected ways and the test data input won't ever cover the whole range of values that the code expects to work. Integration tests have the same problem, the prepared input represents just a few cases, plus they are generally harder to run so they are run less frequently.

So, both tools are useful but they aren't solutions for all the problems in programming. Static type systems have the advantage of being checked without running any code, which should be much quicker than running the tests. Static type systems become more useful as you increase the precision of types and the amount of annotated code in the project. When used correctly, they provide certain guarantees about the code which you can rely on and they are used to restrict the domain (set of possible inputs) of type-checked procedures and classes. This means that you can write fewer unit tests because you don't have to worry about certain conditions which the type system guards against (static guarantee of something never being `null` is quite nice).

Anyway, I think that both static type systems and tests are great tools and they can and should be used together if you value the quality of the code you write. This is getting easier thanks to gradual type systems (optional type annotations like in Python or JS) which allow you to get some of the static guarantees without insisting on everything around being typed. With tests and mypy (in Python) you're much better off in terms of code quality than if you used just one of them. I see no reason not to use them both.

How large need a program to become, before the advantage of being allowed to write fishy code is counter-balanced by the types becoming untractable and the code impossible to refactor in any meaningful way?

This is a serious question. Some years ago, apparently Guido Van Rossum though 200 lines would be already quite an achievement [0]. Based on my own experience, I feel that 99 out of 100 errors thrown at me at compile time are valid and would have caused a crash at runtime (ie. when I do not expect it and have lost all the context of the code change). And I get about 50 such compilation errors in a day of work, so I guess I could write without the compiler safety net for about 10 minutes. That's my limit.

One could object that a 10 minutes program written in python can accomplish much more than a 10 minutes program written in Java. That's much certain! But then we are no longer comparing the merits of compile time vs runtime type checking, but two completely different languages. Of course it is easier to write a powerful/abstract language with runtime type checks, while writing a compiler for a powerful language is much harder. Still, since (and even before) python/perl/php were invented many powerful compiled languages have appeared thanks to PL research, that are almost as expressive as script languages. So it would be unfair to equate runtime type checking with lack of expressive power.

Now of course tests are important too. Compile time type checking does not contradict testing, like you made it sound somewhat in your message. Actually, if anything, it helps to test (because of test case generators based on type knowledge to exercice corner cases).

I'm sorry if all this sounds condescending. I am yet to decide whether I should allow myself to sound condescending as the only benefit of age :) But I'd not want to sound like I'm upset against anyone. Actually, I'm happy people have been using script languages since the 90s, for the same reason I have been happy that many smart people used Windows: my taste for independence gave me by chance a head start that I'm afraid would have been much tougher to get based on my intelligence alone.

And now that static type checking is fashionable again I'm both relieved and worried.

[0]: https://www.artima.com/intv/pyscaleP.html

I think it's better to measure the number of separate code entities (classes and functions and modules in Python) and how many different use-cases (ways of calling functions and object constructors) each entity is expected to cover... After converting to LOC, I'd say ~500 would be the limit. After that, it's a constant fight with TypeErrors, NameErrors, and AttributeErrors - it's just that everyone is already used to this, while not many know of any alternatives. Also, there are substantial differences between languages - in some 10 lines are enough to start complaining, while in some others I've seen and worked with ~2k loc code and it was manageable.

> many powerful compiled languages have appeared thanks to PL research, that are almost as expressive as script languages.

Yes, but on the other hand, some powerful static type systems for dynamic languages also appeared, and some of them are close to Haskell in terms of expressivity. The particular example here would be Typed Racket, which has a state of the art type system which is built on top of untyped Racket. It supports incrementally moving your untyped code to the typed one (whether a module is statically or dynamically typed is decided when module is created; as you can define many (sub)modules in a single file, you can just create a typed submodule, re-export everything that's inside, and move your code there one procedure at a time). Also, it automatically adds contracts based on static types, so that they still provide some guarantees when a typed function is imported and used in untyped code. There are many interesting papers on this, and TypedRacked is really worth looking into, if you have nothing against Lisps.

> Compile time type checking does not contradict testing, like you made it sound somewhat in your message.

Damn! I actually wanted to argue exactly this: that both tools are useful and both can be used together to cover their respective weaknesses. :) Looks like I need to work harder on my writing skills...

> I'm sorry if all this sounds condescending. I am yet to decide whether I should allow myself to sound condescending as the only benefit of age :)

Well, it didn't sound condescending to me, so no prob :) But, if you'd like an advice on this: please don't try to be condescending on the basis of age alone! It's totally ok to sound condescending if you have knowledge, experience and skill to back it up... Well, at least in my book :)

I never said you do not need tests nor that static typing is a panacea. In my view it's a necessary, but not sufficient condition, when programming in the large.

Anecdotally, I'm frequently enough bitten by type issues in JavaScript, but I can't recall very many in Python. Certainly not 15-38%, perhaps 1%.

Which furthers my point (for my set of circumstances): I find the majority of my bugs when I'm writing tests.

> a simple reason: I don't think I could live without static typing

The gradual type system for Python (mypy at the moment) is actually a very good tool. It's as expressive as C# easily, despite some limitations. It fully supports generics and protocols (interfaces or traits in other languages), it allows the user to control the variance of generic arguments, it supports pretty accurate type inference (although not as powerful as OCaml), and so on. Just set up a CI where one of the build steps is running mypy and make the build crash if there's an untyped and not type-inferrable statement anywhere. This is what I've been doing for a year already and it really helps with the maintenance of the projects and with development once the codebase becomes large enough.

This may be as good a chance as any to say this: gradual type systems are here to stay. It's been more than 10 years since the original paper (J. Siek paper was written in 2006; the PLT Scheme (Racket now) guys started working on what became Typed Racket around that time too) - as usual, the industry lags behind the research significantly, but it's bound to catch up at some point. Facebook's Flow and Mypy are the first large scale industrial applications (if I remember correctly) of the theory, but I believe it won't be long before similar functionality pops up all over the place.

While there's still much to be done (like deriving run-time constructs from static type annotations and preserving at least some of the benefits of static typing when interacting with untyped code), these type systems are already powerful tools, and the fact that they are "optional" isn't really a problem for bigger projects, where it can be enforced by the build process. Currently, the lack of type annotations in external libraries poses a problem, but the number of annotated ones is bound to grow because the static type system is an incredibly helpful tool if used correctly and consequently.

So, what I want to say is the distinction between statically and dynamically typed languages will continue to blur and, at some point, will become irrelevant. Especially when you notice that many statically typed langs started to also grow features from the "other side" like marking a variable `dynamic` and allowing the user to do whatever they want with it without complaining.

So, basically, if you go out of your way not to use Java as is, Java is not so bad for the task?

That's kind of my point, you may like other environments better, such as React/Node/NPM but that doesn't mean Python is a horror show.

I'm quite enjoying Go though.

I feel there are benefits for every language. I am just curious if super stable and scalable conditions can be met on python.

JP Morgan operates a ~30 million LOC Python platform for trading and analytics. (related talk: https://www.youtube.com/watch?v=ZYD9yyMh9Hk)

Yes, there are very, very large working python codebases in fields out there that demand correctness. I'm honestly getting tired of the static typing circlejerk that has entered the industry.

Not saying there is a problem, I’m sure some people like to have their throat taken out when the trade doesn’t execute at markdown price.

Anyways I don’t have a problem with that type of pressured enviorment. I’m moreso pointing out people’s need for comfort of solution rather then sustainability. Getting started is more difficult so many are turned away.

Also some of the computation python can do is very powerful and I would trust it if I was no risk besides myself going balls deep.

Python has been my main programming language since 2000, fwiw.

FORTRAN will give you super scalable conditions. I don't think you really meant that either... do you have a GPU cluster at work?

At the same time even when I used it heavily I never saw it as anything more than a scripting language to sit in front of some tool that was written in a language I couldn't be fucked to learn at that moment (numpy and scipy were used heavily throughout my college career).

If I'm being honest I don't understand how anyone could get as worked up about a language as the people in this thread have. At the end of the day most of us are still writing unportable imperative code that runs like shit. Maybe blaming language is how we cope with our own failure as engineers.

Sounds just about right ;)

Why do you group those languages like they’re similar but different from nim and crystal? They’re wildly different in terms of their target domain, runtime models, etc. Go and Java are general purpose application languages and the others are more suited for systems or performance critical applications.

Are you assuming everyone complaining here is young, and this is their first language? Consider that maybe they're complaining because they've used older languages they liked more.

Often, not having a feature is preferable to having a feature designed or implemented poorly.

That wasn't my point. My point is if you had to write Fortran IV using an IBM 32xx terminal you wouldn't be quite so hyperbolic about modern Python.

Unless you are claiming you would rather return to writing Fortran IV than use Python because you like Fortran IV better, in which case I'm very confused.

Python's competitors are Lisp, OCaml, Swift, C# etc.

I prefer at least Lisp and OCaml.

Nobody's going to bother out OCaml or Lisp for web development, data science, or OS scripting where Python is most often used.

Not even MIT teaches lisp anymore.

Lots of universities teach Scheme, particularly Racket. Although Python is more popular, even in that domain.

That's their loss.

If you had to write Python, would you also want to quit software engineering? Would you go back to Fortran instead of Python?

Of course you're allowed to hate Python but someone saying "every time I write python I want to quit software" is either extreme hyperbole, some tangentially related issue like depression, or just no language at all would make them happy enough.

Python 1.4 was an awesomely simple programming environment and I pretty much immediately fell in love with it. Then features were added. Now it is a whole home improvement store full of kitchen sinks.

I think that programming is a sort of theological process. Popular languages attract ideas. Unfortunately, in the case of Python, those ideas were not effectively filtered and now we have an expression of as many ideas as can possibly fit. The ultimate design by committee...

I suspect that the recent excitement about assignment expressions is really a kind of straw that broke the camels back. The problem isn't just this one feature, it's the sum of them.

It's funny, a lot of people hate on Elm for the exact opposite reasons: one person dictating the language's direction and removing features. I suppose a nice balance could be struck between the two ends of the spectrum.

I've used Python at the time (and up to now). It was a revelation compared to Perl, but it sucked compared to modern Python.

What exact features you have a problem with?

And of all the bugs I have written in recent memory, not one came down to a lack of static typing. They were due simply to logic errors, flawed assumptions, misunderstood requirements, and good old race conditions. The static typing zealots like to think if it compiles is must be perfect, however this is a mirage. Unit tests in Python can compensate quite well for lack of static typing.

I can't imagine anyone has ever tried to do engineering at scale (people wise) and did not find the value in static typing.

It's why startups eventually moved off RoR once they started scaling. It's why there is such a large push to type JavaScript (have you seen the rollbar article about the top 10 errors in JavaScript? All but one have to do with types: https://rollbar.com/blog/top-10-javascript-errors/), it's why Facebook created Hack, and outside of parentheses repulsion, it's probably why so few large projects have been written in a LISP or LISP descendant.

Python is great for small: small teams, small organizations, small projects with a few dedicated tasks, small scripting tasks. Most people aren't trying to take anything away from python here in the comments save a few irrational responses.

*again want to stress in my comment when I speak of scale I mean scaling people wise: more organizational structures in your company, more engineers, more collaboration between teams.

>why so few large projects have been written in a LISP or LISP descendant

The major dialect of Lisp, Common Lisp, is strongly typed, and many large projects have been written in it, for CAD/CAM, controlling a NASA spaceship, complete operating systems (Open Genera), the Mirai 3D graphics suite used for creating Gollum in "the lord of the rings", etc.

> 1. Uncaught TypeError: Cannot read property If you’re a JavaScript developer, you’ve probably seen this error more than you care to admit. This one occurs in Chrome when you read a property or call a method on an undefined object.

Does typing stop null object errors in JS, Java or C for that matter? No. You need to continually check for null objects in all langs I use including Python. It seems most of the bugs on that page are of a similar vein.

Python typing is quite similar to Flow, a JavaScript type checker.

I thought it was because of Ruby's poor performance characteristics.

I use Python 3 these days for a lot of stuff. It's pretty good.

These are better days. We all have complaints, but on the whole, things are not too bad.

I think for the most part that when nothing meets your expectations, it may be that your expectations need to be adjusted.

The winning thing python had that nothing else had at the time was it was social, it was readable, and there was generally only one way to do things: the right way. It no longer has the latter quality, and the preferred coding style in it seems to be java-ish OO-spaghetti, but it's still pretty good.

That said, these days, I resent every damn time I have to use it. It's eating data science, more or less because pandas and scikit is ... mostly good enough, and because unlike R it's .... mostly good enough to deploy in an enterprise application. But if you're working on the exploratory side of data science, Python is shit compared to R. Doesn't have the tooling, doesn't have native facilities, and is vastly more long winded. All the attempts to make Python more ... X ... are probably a mistake also. You're taking a beautifully simple tool and making it more exotic and complex. It's like trying to use Matlab to build webservers.

I can't help but to gently interject here. By 1997, I'd been programming for some time, and there certainly were people who consider Perl suitable for programming in the large, and there were certainly big projects so written.

While I disagree with the specific word 'nobody', I agree with the sentiment: Perl was widely considered to be only useful for 'small' things at the time. Widely, but not exclusively.

> There are two kind of languages: the ones everybody hates, and the ones nobody uses.

"There are only two kinds of languages: the ones people complain about and the ones nobody uses."

Bjarne Stroustrup's FAQ: Did you really say that?. Retrieved on 2007-11-15.

However, I never saw any complaints about Clojure package management in any topic about Clojure here.

It is not that bad, much like I also don't think Python packaging bad. Other ecosystems have better solutions, though.

I started to use Python quite recently and I really like it. It is a well-designed language with high-level abstractions that are really fun to use. I like the pervasive use of iterators, the 'everything is an object' philosophy, the minimalist syntax, the build-in datatypes...

That being said, I feel that the dynamic types show their limits when projects getter big. I use linters and static type annotations but I find refactoring very error-prone and there's a point where I don't really trust my programs.

—Richard W. Hamming[0]

I have rarely "chosen" to use Python at work, but it has never failed to get the job done.

[0] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2041981/

e.g. "Rule 1: Drop Modesty", "Rule 7: Believe and Doubt Your Hypothesis at the Same Time".

[0] - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2041981/

It's very easy to persuade the customers of your language that everything that goes wrong is their fault and not yours.

- Tony Hoare

Not to say I have any real complaints about Python.

Funny story, my first co-op job (Fortran IV), my boss made me fix a bug using punch cards so I'd appreciate why the codebase wasn't as nice as it might be. THAT was prespective.

Why did you convert the algorithm from Fortran to c seems a waste of time to me, unless it was a training exercise.

As if the only way to gauge quality is to forever compare to the technologies of the past. Python was a great improvement on its peers in its heyday, but people, technologies and philosophies have changed since its inception and there is nothing wrong with wanting something more. That's how progress is made.

Instead of evolving into something okay, Python is pretty much the same broken language as it’s always been, run by a guy who is openly hostile to PL theory, FP, and any major changes to Python whatsoever.

If you ever need/want to use Python, do yourself a favor and use Racket instead. Racket is better than Python in every single way (unless you are doing data science with Python, in which case fine keep using it).

Probably Go will be the next hotness in 5 years.

I think it will be difficult to grow a large ecosystem for a language with very poor FFI performance [0] in the long run. Golang's poor FFI performance is the number 1 reason I wouldn't use it for my own projects.

[0]: https://github.com/dyu/ffi-overhead

After using Go almost exclusively for about 18 months I have had to interface with existing C libraries exactly zero times.

With python, why is it so popular currently? For large part because of its very good data science and machine learning ecosystem. And why does that exist? Mostly because python libraries like numpy, theano, and scikit-learn were built on top of mature, high-performance C libraries like OpenBLAS, LAPACK, and Cuda.

I very much doubt that anything like scipy would exist if the developers had to reinvent the wheel of the underlying numbers libraries from scratch. C's been around a long-time. There's huge amounts of high-quality mature software that already exists in a C framework. A language's ability to easily "plug-in" to the C ecosystem is a major leg-up when it comes to bootstrapping its own comprehensive library ecosystem.

This is for example the reason why there won't be a large mathy/scientific ecosystem in golang.

For another example of a similar effect, Rust has great FFI. Yet I would expect over time it will be necessary for fewer and fewer things, because Rust is already roughly on par with C, and over time, a native Rust API will still be preferable to a C API wrapped with a Rust access layer. It will always have great C FFI, by its nature, but the percentage of projects that won't need it is already pretty high and probably only going up over time.

[1]: People seem to misinterpret this statement a lot, as if I'm saying Python is bad or something. No; it is simply this: Python performance is not very good at a very raw level. It is merely one characteristic of a language out of many, many relevant ones, not a full assessment of the language. Python has many other dimensions in which it has superior capabilities. It just pays for that on the performance dimension. (Whether that's an essential or an accidental tradeoff, well, ask me again in ten years; the programming language community seems to be in the process of working that out right now.)

Whether that's a good idea, and why that is, those would be entirely separate conversations. But it's just an observable fact that languages pick up native implementations of core functionality over time, subject to certain performance restrictions (e.g., I'm sure that if it wouldn't be unusably slow, Python would have a native-Python image library... it's just Python doesn't really have that option).

But those are the exceptions, and often you don't need a best-of-breed solution and may prefer the language-native one.

Again, I'm not theorizing about what could be here; I'm looking out in the world, where I see that most libraries tend to emerge out into a native version if the underlying language can possibly meet the basic requirements for performance and such. This is something that needs to be explained, not explained away.

Also... I love me some Haskell, and on a per capita basis the community is great, but if Rust's community isn't already several times larger and growing faster, I'd be stunned, just to pick one example. Haskell has some very interesting cases of best-of-breed libraries, but it doesn't exhibit the library profusion you get from sheer personpower. (Of course, it doesn't really have the problems you get when your libraries are generated by sheer personpower either.)

I'd love to see the numbers on that if you know where to find them.

That seems weird. Go has a pretty large (and growing) Data Science Community.

Lots of Math/Scientific stuff there, especially including many people from Python and Ruby backgrounds.

I think that's a fault of the devs, more than the language. In many cases (not all, of course) Go gives you multiple ways to achieve concurrent safety. Channels, being the big secondary. Yet it generally (in my experience) requires a very different implementation and in general has a lot of pitfalls. Over all I don't like Go these days, but I prefer it over Python (mostly due to at least having basic types)

I've switched everything to Rust though. Just as productive as Go (to me), with more tooling. Though, Rust will be much better in a couple years with some additional baking on new features (GATs, Futures, etc).

You should try type annotations, with mypy and/or PyCharm it helps finding bugs before you run the code. It also makes autocomplete and refactoring work correctly (I would say they work better in PyCharm than in GoLand)

1. the package offering the functionality I needed was much more complex than needed and my solution implementing that functionality was much simpler (because it implemented only what I needed) 2. was not the best quality, because it was written by someone who was not necessarily better than me, or didn't spent enough time understanding the problem he was trying to solve

Having said that generally popular packages are good quality, although even then #1 applies, if you need just small functionality of a specific package try implement it yourself, it might turn out that the problem was not as hard as you thought and because you're just implementing what you need it might be more elegant.

> Go attempts to combine the development speed of working in a dynamic language like Python with the performance and safety of a compiled language like C or C++.

https://techcrunch.com/2009/11/10/google-go-language/

All the languages you list are over 20 years old. I think the popularity of Go and Rust show that statically-typed languages are having a resurgence.

As a long-time C programmer (with significant bits of bash and perl on the side), I've really enjoyed learning Go. But the responsiveness to developing new language features has been _sooo sloooow_. We'll have to see in 10 years if this turns out to have been a wise decision or not.

I for one have started running TypeScript on my servers whenever I can, I love it. There's still room for improvement though.

I'm curious, do you mean compiling to JS and running it on the servers? Because I would love for the runtime itself to run TypeScript, something like Deno [0] but (maybe in the future) as mature as Node.js.

[0] https://deno.land/

Well, there’s WebAssembly…

You were saying about C/C++ package management?

It is a step forward, sure, but it's a far cry from Python's pip or Rust's Cargo or even Dlang's dub.

Also - you're moving the rhetorical goalpost. First you claimed there was no package management system, no you're complaining about the lack of default.

No package management for C++ is popular enough for C++ programmers to avoid all the hardworks of manually managing packages, because there are always some (or most) packages you rely on not in these package management systems.

pip, for all its shortcomings, does include almost all of the Python packages you ever need to use. That is a significant different in practice.