Python has a lot of problems that really slow down development, but they are all fixable.
The biggest issue, in my opinion, is in dependency management. Python has a horrible dependency management system, from top-to-bottom.
Why do I need to make a "virtual environment" to have separate dependencies, and then source it my shell?
Why do I need to manually add version numbers to a file?
Why isn't there any builtin way to automatically define a lock file (currently, most Python projects just don't even specify indirect dependency versions, many Python developers probably don't even realize this is an issue!!!!!)?
Why can't I parallelize dependency installation?
Why isn't there a builtin way to create a redistributable executable with all my dependencies?
Why do I need to have fresh copies of my dependencies, even if they are the same versions, in each virtual environment?
There is so much chaos, I've seen very few projects that actually have reproducible builds. Most people just cross their fingers and hope dependencies don't change, and they just "deal with" the horrible kludge that is a virtual environment.
We need official support for a modern package management system, from the Python org itself. Third party solutions don't cut it, because they just end up being incompatible with each other.
Example: if the Python interpreter knew just a little bit about dependencies, it could pull in the correct version from a global cache - no need to reinstall the same module over and over again, just use the shared copy. Imagine how many CPU cycles would be saved. No more need for special wrapper tools like "tox".
I've always seen it like this: Not everyone builds reproducible software with Python (or in general) and how you handle dependencies can vary. Python leaves it open how you do it: globally installed packages, local packages, or a mix of both.
In the end, it needs to find the import in the PYTHONPATH, so there's no magic involved, and there are multiple robust options to choose from.
So instead of bashing Python for not shoveling down an opinion on you, it's up to the developers to choose which tools they want to use.
If they don't choose one and are unable to freeze their dependencies, it's not a Python problem, but IMO lack of skill and seniority.
You can have both: provide a sane default for most users and allow people to roll their own.
The reason why Python gets extra criticism for this is because it likes to tell people that there should be one obvious way to do it and that it comes with batteries included yet it's dependency management system is just crap and doesn't follow that at all.
I agree. Most of the issues the parent mentions have been solved with poetry and pipenv.
And if you need "to create a redistributable executable with all your dependencies". You can either use pyinstaller [0] or nuitka [1] both of which are very actively maintained/developed and continually improving.
Anyone considering it for production usage should note that package installs in the current versions are much slower than pip or Pipenv. This might affect your CI/CD.
Could you give some details as to why it's better than other more commonly used tools (pip, venv, ...)?
Looking at the home page it's not immediately obvious to me. For example, the lock file it creates seems to be the equivalent of writing `pip freeze` to the requirements file. I see a quick mention of isolation at the end, it seems to use virtual environments, does it make it more seamless? What's the advantage over using virtualenv for example?
Python's dependency hell is what made me first look at Julia. I develop on Windows (someone has to :) ), and it was just impossible to get all of the numerical libraries like pydstool, scipy, FEniCS, Daedalus, etc. playing nicely together... so I gave Julia a try. And now the only time I have issues getting a package to run are Julia packages which have a Python dependency. Python is a good language, but having everything in one language and binary-free is just a blessing for getting code to run on someone else's computer.
I've had a good experience with pip-tools (https://github.com/jazzband/pip-tools/) which takes a requirements.in with loosely-pinned dependencies and writes your requirements.txt with the exact versions including transitive dependencies.
"If two of your dependencies are demanding overlapping versions of a library, pip will not necessarily install a version of this library that satisfies both requirements" e.g. https://github.com/pypa/pip/issues/2775
This is what I've always done. Develop using a few dependencies, freeze, continue development with reproducible builds. It has always included the sub-dependencies in the list so, as far as I can tell, this works great for that case...
It catches way too much. IPython, black and the testing libraries are _not_ a part of my actual dependencies and shouldn't be installed in production. A good UI for a dependency manager at the very least distinguishes between dev and production context, and ideally lets me define custom contexts.
saddened to see this poorly constructed comment berating python at the top of this thread. the author seems to have some personal issues with the language given the generally frustrated tone of the comment. the entire comment could have just been 1 line "We need official support for a modern package management system, from the Python org itself." which would be consumed as constructive feedback by all readers with the right context. but somehow the author chooses to "vent" adding unnecessary drama to something that does not get in the way of writing high quality production grade python apps (general purpose, web, ai or otherwise)
there is no language that is devoid of shortcomings - so to any new (<3 yrs exp) python users, please ignore the above comment entirely as it has no bearing on anything practical that you are doing/will do. and all experienced python users know that there are ways to work around the shortcomings listed here and beyond.
> the author seems to have some personal issues with the language given the generally frustrated tone of the comment
What "personal issues" do you think the author has? The frustrated tone comes from the frustrations the author explicitly outlines; unless you think this shouldn't be so, you are turning this into an ad-hom.
> the entire comment could have just been 1 line "We need official support for a modern package management system, from the Python org itself."
Why? because you don't appreciate the detail on why we need such a thing? These issues certainly get in the way of producing production apps; not in the sense that they make it impossible, but they make the process harder and slower than it needs to be.
I actually quite liked that post. I use python maybe once a year or less, and don't enjoy the experience. That post distinguished some of the details which in my rare usage I see simply as a gloopy mess.
It is funny - half of the real desire/need for containers comes back to these sorts of issue with both node and Python. And then they bring in their own different challenges.
I have been programming with node for the last 3 years and I never had any dependency issues with node (at least for 3rd party dependencies). I cannot say that with python that requires using some tool be it docker or virtualenv to isolate them from the already installed ones.
Node's dependency managers npm/yarn just copy the versioned dependencies from their cache folder into the local node_modules folder and remove transitive dependencies duplicates when possible by flattening them into node_modules.
Having such a basic part of a programming language be awful is inexcusable. It's not just that it takes a lot of time; even if it took no extra time, you're still wasting extra space on your computer, risking breakage on external updates, and compromising security because you can't even tell what code you're running.
Last time I gave it a go, I found it was pretty strongly welded to virtualenv, rather than using Python's own (much less problematic) venv. I came away less than enthused as a result.
(To be fair, modifying it to use venv is... non-trivial).
Pipenv has a good approach to management, similar to npm, but the implementation is buggy. It gained popularity by being recommended very early on in the official python documentation while being misleadingly advertised as production-ready.
If you look at the issues section in its github repo, you'll see that there are some pretty basic bugs there which are very annoying or disruptive. Moreover it seems the author has almost left the boat and a handful of contributors have to tidy things up.
Just to illustrate my point. I think a package manager that takes a few minutes to install a single tiny package, or don't prevent you from adding non-existing packages (e.g. spelling mistake), or doesn't let you install a new package without trying to upgrade all other packages isn't really production-ready. These are known issues since November last year.
Excuse me, but as a long time Python user I have to disagree. I started using Rust two years ago and Rust’s dependency managment is easily the best thing I ever saw (keep in mind that I didn’t see everything, so there is a chance there are better things out there).
The project-/dependency-manager Cargo¹ is more “pythonic” than anything Python ever came up with and where others mumbled ”dependency hell is everywhere” the Rust people seem to have thought something like: ”there must be a way to do this properly”.
The whole thing gives me hope for Python, but angers me everytime I start a new python project. Poetry is good, but there should be an official way to do this.
It saddens me to see, that some people seem to just have given up on dependency managment alltogether and declared it unsolvable (which it is not, and probably never has been).
Rust's cargo, JS's yarn and the grand daddy of them all Ruby's bundler address all these issues. Even newer versions of Gradle support a workflow where you specify the versions you know you want and just on everything else, including transitive dependencies, down.
Python is uniquely ill-suited for dependency management compared to many other languages. For some reason dependencies are installed into the interpreter itself (I know what I just said is very imprecise/inaccurate but I think it gets the point across).
In JS, which also has a single interpreter installed across the system (or multiple if you use nvm), the packages aren't installed "directly" into the interpreter, which removes the need for things like virtual-envs, thus making life a lot easier. I wish Python did something like this.
That being said, pipenv is making things easier. However, I think pipenv is a workaround more fundamental problems.
It's not a "model," but if you're able to 1) use fewer dependencies 2) use stable dependencies 3) use dependencies with fewer dependencies, it helps with dependency hell. I've even made commits to projects to reduce their dependency count.
I have found that I would rather code my own versions of some libraries so I have control over it. Even if there is some extra long term maintenance and some up front dev costs, it's paid off already a number of times.
I agree that builtin tools suck for dependency management.
However a lot of the issues that you mentioned (such as lock file and transitive dependencies) can be handled by pipenv, which should be the default package manager
Python was a scripting language. All those problems are caused by people using it like something it isn't. Python has way outlived it's usefulness and it's about time we move on to something better.
The virtualenv thing just galls me. Sure, pipenv aped rbenv - appropriately, I might add - but until they supplant virtualenv as the recommended way to have separate environments, I'll pass.
.NETs solution to this was the project file, a configuration file that lists the compiler version, framework version, and dependencies (now including NuGet packages and their versions).
> The biggest issue, in my opinion, is in dependency management. Python has a horrible dependency management system, from top-to-bottom.
I agree, although a lot of it has to do that there's so much misinformation about the web, and many articles recommending bad solutions. This is because python went through many packaging solutions. IMO the setuptools one is the one that's most common and available by default. It has a weakness though, it started with people writing setup.py file and defining all parameters there. Because setup.py is actually a python program it encourages you to write it as a program and that creates issues, setuptools though for a wile had a declarative way to declare packages using setup.cfg file, you should use that and your setup.py should contain nothing more than a call to setup().
> Why do I need to make a "virtual environment" to have separate dependencies, and then source it my shell?
Because chances are that your application A uses different versions than application B. Yes this could be solved by allowing python to keep multiple versions of the same packages, but if virtualenv is bothering you you would like to count on system package manager to keep care of that, and rpm, deb don't offer this functionality by default. So you would once again have to use some kind of virtualenv like environment that's disconnected from the system packages.
> Why do I need to manually add version numbers to a file?
You don't have to, this is one of the things that there's a lot of misinformation about how to package application. You should create setup.py/cfg and declare your immediate dependencies, then you can optionally provide version _ranges_ that are acceptable.
I highly recommend to install pip-tools and use pip-compile to generate requirements.txt, that file then works like a lock file and it is essentially picking the latest versions within restrictions in setup.cfg
> Why isn't there any builtin way to automatically define a lock file (currently, most Python projects just don't even specify indirect dependency versions, many Python developers probably don't even realize this is an issue!!!!!)?
Because Python is old (it's older than Java) it wasn't a thing in the past.
> Why can't I parallelize dependency installation?
Not sure I understand this one. yum, apt-get etc don't parallelize either because it's prone to errors? TBH I never though of this as an issue, because python packages are relatively small and it installs quickly. The longest part was always downloading dependencies, but caching solves that.
> Why isn't there a builtin way to create a redistributable executable with all my dependencies?
Some people are claiming that python has a kitchen sink and that made it more complex, you're claiming it should have even more things built in, I don't see a problem, there are several solutions to package it as an executable. Also it is a difficult problem to solve, because Python also works on almost all platforms including Windows and OS X.
> Why do I need to have fresh copies of my dependencies, even if they are the same versions, in each virtual environment?
You don't you can install your dependencies in system directory and configure virtualenv to see these packages as well, I prefer though to have it completly isolated from the system.
> There is so much chaos, I've seen very few projects that actually have reproducible builds. Most people just cross their fingers and hope dependencies don't change, and they just "deal with" the horrible kludge that is a virtual environment.
Not sure what to say, it works predictable to me and I actually really like virtualenv
> We need official support for a modern package management system, from the Python org itself. Third party solutions don't cut it, because they just end up being incompatible with each other.
setuptools with declarative setup.cfg is IMO very close there.
> Example: if the Python interpreter knew just a little bit about dependencies, it could pull in the correct version from a global cache - no need to reinstall the same module over and over again, just use the shared copy. Imagine how many CPU cycles would be saved. No more need for special wrapper tools like "tox".
There is a global cache already and pip utilizes it even withing an virtualenv. I actually never needed to use tox myself. I think most of your problems is that there are a lot of bad information about how to package a python app. Sadly even the page from PPA belongs there.
It's not that bad if you use the right tools. The two main options are an all-in-one solution like poetry or pipenv, and an ensemble of tools like pyenv, virtualenvwrapper, versioneer and pip-tools. I prefer the latter because it feels more like the Unix way.
Why should Python have some "official" method to do this? Flexibility is a strength, not a weakness. Nobody ever suggests that C should have some official package manager. Instead the developers build a build system for their project. After a while every project seems to get its own unique requirements so trying to use a cookie-cutter system seems pointless.
In general, leaving such things open leads to a proliferation of different 'solutions', as multiple people try to solve the issue... leading to the additional confusion and cognitive load of trying to find a single solution which suits your use-case and works, when often none of them are perfect.
Sometimes a 'benign dictator' single approach has benefits...
The virtual env is really the thing that has stopped me from using python. It's a lovely language but the tooling around it needs a lot of help. I'm sure it will get there though. I mean if the js folks can do it, certainly python can.
Remember NOT to jump into Python for your new product if don't know Python. If you are developing for a young startup, have time crunch, then stick to what you know.
IF you do not have a language, or know Python a bit, then pick Python. Here are some of the reasons why I stick to Python (young startup/web APIs):
- OOPs is not too strict (might give a headache to some folks)
- Mixins, lambda, decorators, comprehensions - Pythonic ways make me feel productive easily
- Create a data Model, drop into a shell, import and try things
- Can do that on a live server
- Do the same with Controllers, or anything else actually
- really nothing fancy needed
- Command line processing, SSH, OS integration, etc. has so many great libs
- Python Dict someone looks like JSON (this is purely accidental but useful)
- Debugger support even in free IDE like PyCharm Community Ed is great
- Integration to a world of services is so easy, even ones you do not commonly see
- Documentation - many libs have consistent structure and that helps a LOT
- Really large community, perhaps only smaller than Java
- The even larger group of people using Python in all sorts of domains from Biotech to OS scripts
What I would like improved in the language would be an even longer list. Every language has a list like that, but when you are focused on being scrappy and building a product, yet making sure that software quality does not take a big hit, Python helps a lot.
> Remember NOT to jump into Python for your new product if don't know Python. If you are developing for a young startup, have time crunch, then stick to what you know.
Are you saying this as a general maxim (don't try to learn a new tech under pressure) or because of characteristics specific to Python, that make it worse in such a situation than any other language/ecosystem?
I know it's only a one off anecdote but two developers in my team who previously didn't know python used it for a small time-critical project and it was a brilliant success.
The main point I am trying to make is that not all of us are language experts. I absolutely know the vital role that language experts play and that we need to solve the issues. But companies need to build in the meanwhile. Python is easier to learn, this is why it is such a popular language in Universities.
I am not a high IQ person to even grip some of the nitty-gritty underpinnings of a language. But should that stop me from building a product?
In the end, I need a language that is easy to pick-up, be productive, has its heart in the right place. A young Python programmer can web scrap easily or plug into Ansible or so many other things. If you know of another language that would make more practical sense and still be easy to pick-up, I would switch.
Reading this got me thinking and I wonder if other people feel like me about this, so I'm going to share it. This is not serious, but not entirely unserious...
I try to be a good sport about it, but every time I write python I want to quit software engineering. It makes me angry how little it values my time. It does little for my soured disposition that folks then vehemently lecture me about the hours saved by future barely-trained developers who will ostensibly have to come and work with my code. Every moment working with python (and that infernal pep-8 linter insisting 80 characters is a sensible standard in 2019) increases my burnout by 100x.
I try to remind myself that we're trying to make the industry less exclusive and more welcoming to new developers and "old" isn't necessarily "good" (in fact, probably the opposite), but damn I just don't understand it.
It used to be that I could focus on other languages (Erlang, Nemerle, F#, Haskell, Ocaml, even C++) and sort of balm myself. But now, I can't even overcome the sinking feeling as I read the Julia statistics book that I'm going to be dragged back to Python kicking and screaming in the morning, so why even bother?
And frustratingly: it's one of the few languages with decent linear algebra libraries. And that means it's one of the few languages with good ML and statistics support. So it's very hard not to use it because when you want to cobble together something like a Bayesian model things like PyMC or Edward actually give you performance that's obnoxiously difficult to reproduce.
This is what the industry wants and evidently a lot of people are okay with it, but to me it's misery and I can't work out why people seem to like it so much.
I am about to hit a decade of python experience. I work with it daily, all my major codebases are written in it.
I hate it.
It has an ok object system which is possibly its only redeeming quality. I found Racket about 4 years ago, and any new project that I work on will be in Racket or CL.
I could go on at length about the happy path mentality of python and the apologists who are too ignorant of other people's use cases to acknowledge that maybe there might be some shortcomings.
The syntactic affordances are awful and you can footgun yourself in stupidily simple ways when refactoring code between scopes. Python isn't really one language it is more like 4, one for each scope and type of assignment or argument passing, and all the sytactic sugar just makes it worse.
Not to mention that packaging is still braindead. I actually learned the Gentoo ebuild system because pip was so unspeakably awful the moment you stepped away from the happy path.
Blub blub blub, someone help I'm drowning in mediocrity.
I am about to hit two decades, with lot of pauses. I now use it professionally, but it was still my favorite language a decade ago (then I fell in love with Common Lisp and lately Haskell).
I think you need to look at it historically. Against other languages circa 2000, Python was a huge win. Easy to write (compared to C), consistent (compared to PHP), minimalist (compared to Perl), multi-paradigm and playing well with C (compared to Java), with a great standard library (compared to Lisp).
Today, the landscape is very different. Lot of languages took hints from and got influenced by Python (including Racket I am sure). Functional languages have taken off. Type inference is a standard feature.
History is always changing. There is some potential for a next Python, but we don't know what it will look like yet. I suspect it will be functional, but I don't think it will be in the Lisp family. It probably won't happen until after Rust and/or Julia get lot more adopted. Anyway, just like C, Python will be with us for decades to come, for better or worse.
Two decade Perl programmer who's been using a bunch of Python recently. I don't hate it, but I've not been getting any particularly complicated use-cases. I do miss Moose a lot, and I miss Bread::Board and I miss DBIx::Class, which SQLAlchemy does not make up for. The weird scoping didn't take too long to get right, although string interpolation still takes me too much thinking about.
What I am missing from Perl is library documentation and MetaCPAN. It seems that the go-to solution is that you're meant to create some kind of damn mini-site for any even vaguely complicated project you do using Sphinx, which seems bizarro land. Also the _requests_ documentation looks like it was written by Wes Anderson and wtf I hate it. Also I hate that all libraries have cute rather than descriptive names; yes there's some of that in Perl, but it feels like less. Bah humbug. Other than that it's fine.
Thanks for sharing, it certainly makes me feel better about my troubles with accepting python. And "the happy path" might be a state you spend a /minority/ of your time in, depending on project. So first you spend 10% of your time to fix the 80% of easy features, but the tricky stuff can be more awful than it should be.
And packages. I thought I was just stupid, but thankfully there was more to it.
> Is it the language, or someone forcing you to use that linter and settings?
It's definitely both.
Preface in true internet style: these are just opinions and you may not share them. That's fine.
I really don't like Python as a language. I don't like its total lack of composability. I don't like its over-reliance on a very dated vision of OO. I don't like how its list comprehensions aren't generic. I don't like how it farms all its data structures out to C. I don't like how it uses cruddy one line lambdas and forces me to name things that are already named by the things I'm passing it to.
And also the linter just exacerbates these things, because the linter is just a crystallized slice of a kind of software engineering culture I really don't like.
I'm not the person you're responding to but what they say resonates with me as a Python developer.
I believe Python is quite possibly the best language for a few things
* Exploratory programming - such as what data scientists do
* Writing programs that will never grow beyond 150LOC, or roughly what fits on a screen + one page down
When I have those two constraints met I am almost always choosing Python.
Here are some problems I face on codebases as they scale up:
* Python conventions lead to what I consider bad code. Devs will often choose things like 'patch' over dependency injection, and I have seen on multiple occasions projects derided for providing DI based interfaces - "oh, why would you write code that looks like Java? This is Python".
There's a lot of death by a thousand cuts. Keyword args in functions are abused a lot, because they're "easy", ability to patch at runtime means it's often "easy" to just write what looks like a simple interface, but it's then harder to test. Inheritance is "easy" and often leads to complex code where behaviors are very, very non-local.
Dynamic types mean that people are often a little too clever with their data. I've seen APIs that return effectively `Union[NoneType, Item, List[Item]]` for absolutely no semantic reason, without type annotations, meaning that if you assumed a list or none came back you were missing the single Item case. The implementation actually internally always got a list back from the underlying query but decided to special case the single item case... why? I see this sort of thing a bit, and other languages punish you for it (by forcing you to express the more complex type).
* I find myself more and more leveraging threads these days. I did this often with C++, and all the time in my Rust code. Python punishes you for threading. The GIL makes things feel atomic when they aren't, reduces you to concurrency, all while paying the cost of an OS thread. Threading primitives are also quite weak, imo, and multiprocessing is a dead end.
And, really, Python is anti-optimization, which is a good and bad thing, but it's a bit extreme.
* Implicit exceptions everywhere. I see 'raise Exception' defensively placed in every Python codebase because you never know when a piece of code will fail. I see a lot of reliance on 'retry decorators' that just catch everything because you never know if an error is transient or persistent.
The common "don't use exceptions for control flow" is broken right off the bat with StopIteration. I just think error handling in Python is god awful, really.
* Mypy is awesome, but feels like a bit of a hack. The type system is great, and in theory it would solve many problems, but coverage is quite awful in open source projects and the type errors I get are useless. I actually only use mypy to make my IDE behave, I don't run it myself, because my types are actually technically unsound and the errors are too painful to work with (and honestly the type system is quite complex and hard to work with, not to mention the bugs).
There are lots of other smaller issues, such as my distaste for pip, py2/3 breakage, lack of RAII, the insane reliance on C all over the place, I think syntax is bad in a lot of places (lambdas, whitespace in general), etc but these are probably my main sticking points with using the language for large projects.
Again, Python is actually my favorite language for specific tasks, but I really think it's best optimized for small codebases.
1) No pattern matching. In 2019, this is just unacceptable. Pattern matching is so fundamental to good FP style that without it, it becomes almost impossible to write good and clean code.
2) Statement vs expression distinction. There’s no need for this and it just crippled the language. Why can’t I use a conditional inside an assignment? Why can’t I nest conditions inside other functions? It makes no sense and is stupid and inelegant.
3) Related to 2), why do I need to use a return statement? Just return the last expression like many other (better) languages
4) Bad and limited data structures. In Python all you get are arrays, hashmaps, sets. And only sets have an immutable version. This is unacceptable. Python claims to be “batteries included” but if you look at the Racket standard library it has like 20x more stuff and it’s all 100x better designed. In Scala you get in the standard library support for Lists, Stacks, Queues, TreeMaps, TreeSets, LinearMaps, LinearSet, Vectors, ListBuffers, etc.
5) Embarrassing performance. Python is so slow it’s shameful. I wrote a compiler and some interpreters in college and I honestly think I could create a similar language 10x faster than Python. Sometimes you need to trade off performance and power, but that’s not even the case with Python: it’s an order of magnitude slower than other interpreted languages (like Racket).
6) Missing or inadequate FP constructs. Why are lambdas different from a normal function? Why are they so crippled? Why do they have a different conditional syntax? The only sort of FP stuff Python has is reduce/filter/map. What about flatMap, scanr, scanl, foldl, foldr? Or why doesn’t Python have flatten? All of these are very useful and common operations and Python just makes everyone write the same code over and over again.
7) No monads. Monads can be used for exceptions, futures, lists, and more. Having to manually write some try catch thing is unseemly and worse than the monadic Haskell or Scala approach.
8) No threads and no real single process concurrency. Despite Python being used a lot, no one really seems to care about it. How can such a problem not be solved after over 20 years? It’s shameful and makes me wonder about the skill of Guido. There’s no reason why Python couldn’t have turned into something beautiful like Racket, but instead it has been held back by this grumpy old guy who is stuck in the past.
9) Others might not have a problem with this, but I detest Python’s anything can happen dynamic typing. It makes reasoning about code difficult and it makes editor introspections almost impossible. If I can’t know the exact type of every variable and the methods attached to it, it hampers my thinking a lot. I use Python for data science and if I could just have a language that was compiled and had static typing I would be 3x as productive.
Let me conclude by saying there currently is one good reason to use Python: if the domain is ML/DL/quant/data science Python is still the undisputed king. The libraries for Python are world class: scipy, sklearn, pandas, cvxpy, pytorch, Kerala, etc.
But Julia is catching up very fast and the people I have talked to are getting ready to ditch Python in 2-3 years for Julia. I don’t think I’ve encountered anyone who didn’t prefer Julia to Python.
> and that infernal pep-8 linter insisting 80 characters is a sensible standard in 2019
I don't know why 80 characters is a problem. I don't use the linter but I enforce this rule religiously with a couple of exceptions (long strings comes to mind). It forces me to think heavily about the structure of the code I'm writing. If I'm nesting so deeply, something has gone wrong. If I've got a ton of chained methods or really lengthy variables, it forces me to rethink them.
This also has the advantage of being able to put 4 files next to each other with full visibility. Vertical space is infinite, horizontal space isn't. It's probably a good idea to use it.
It's also awesome if you have to do code reviews on a laptop or don't have a massive screen available.
That said, we usually just go with autoformatting via black, which is 120 by default. No more hassle manually formatting code to be pep8-compliant. Just have black run as a commit hook, which is super easy via pre-commit [0]. And you can run the pre-commit checks during CI to catch situations where somebody forgot to install the hooks or discovered `--no-verify`.
Can't really imagine developing Python without Black any more.
All I know is that with 80-char width I can have 2 files side-by-side on a 15" MBP along with the dir-tree on the left in an editor like PyCharm or VSCode and fully see both files wo wrapping. It helps my productivity immensely.
Same deal when it comes to reviewing PRs in GitHub. Wrapping just interrupts flow for me.
I feel the complete opposite. I really enjoy working with python over any other language. R does linear models and time series better and matlab has its charm, but overall I prefer python. Python is so easy to read and quick to program in. I am so glad I am not in the Java/C++ world anymore, but I know people in different roles have to deal with different issues.
> I really enjoy working with python over any other language.
I assume you mean, "over any other language I have tried" ?
As someone with a mathematical background myself, I am always surprised at how many data scientists and quants are ignoring more mathematically principled languages like F#, OCaml and Haskell.
Well, yeah compared to R and matlab, I am willing to believe Python excels, but the person you are replying to is probably not doing data science, so he has options besides the 3 just mentioned.
Re. linting, I'd highly recommend Black with whatever line length you want - it'll reliably reformat your code, and once you lose the urge to reformat while typing it's fantastic. It's like deleting a bunch of useless mental code. And the code reviews are even better: include a linting step (ideally with isort) in CI and you can avoid 95% of formatting comments.
100% this. I just switched to using Black recently and not having to ever fix a lint issue again has been life-changing. Use Black with pre-commit (https://pre-commit.com) and never look back.
Over-sensitive individuals are hard to please and often unhappy. That's more of a personality flaw than a flaw with the current state of software engineering.
If there's one thing wrong with our profession is a lack of ethics and accreditation - we're essentially letting random people build critical infrastructure and utilities.
We don't have a tooling problem, in fact we have too many tools.
I see so many people (especially on HN) fixating on tools, dissecting programming languages, editors and libraries into their most minute parts and then inevitably finding faults in them, disavowing them and swearing that other tools X, Y and Z are purer and worthier.
If you want to stop hating software and improve your burn out, stop caring about irrelevant stuff.
Is that supposed to help someone? I fail to see how telling someone "just ignore the stuff that irks you that you have to spend 40+ hours a week dealing with. You are overly sensitive and your concerns are irrelevant" helps anyone. Even if it was true, it was delivered in such a ham-fisted manner that I can't imagine anyone taking it to heart.
>If there's one thing wrong with our profession is a lack of ethics and accreditation - we're essentially letting random people build critical infrastructure and utilities.
Oversensitive? Is the contractor who chooses a Dewalt or Ridgid over a Ryobi for daily work "caring about irrelevant stuff"? A drill is a drill right? Why is it different for us in software?
> Over-sensitive individuals are hard to please and often unhappy. That's more of a personality flaw than a flaw with the current state of software engineering.
Ah yes. Just remove the part of myself that got me into software as a child, and proceed robotically.
> If there's one thing wrong with our profession is a lack of ethics and accreditation - we're essentially letting random people build critical infrastructure and utilities.
Spot on. Leftpad (and perhaps the whole js ecosystem) are good examples.
I know I’m responding to opinion but: developer productivity isn’t one of the pitfalls of Python.
Yes I agree that if you’re using Python for a large scale project involving lots of developers its not the best; but that’s because it doesn’t have a good type system.
You can’t work out why people like it so much because of this misconception. The languages that you gave as examples most definitely do _not_ value your productivity, it values correctness as enforced by a type system and refactoring needed for large projects.
I am more productive in a language with an expressive type system (e.g. Haskell) than one without. Thinking about types not only guides me towards a working solution quickly, but the checker catches all my minor mistakes.
In Haskell, you can actually defer all type errors to runtime if you want to. But I have never felt this makes me more productive.
The bigger the team, the more distributed the engineers and the bigger the codebase, the more productivity is lost by using scripting languages. I find it infuriating because it is so frustrating and it definitely does not feel productive.
For a lone hacker, its the other way around. Compare e.g. to Golang's "programming at scale" rationale.
The way I think about it is that Python is a strong local optimum in a space that has a massively better optimal solution really close by. But it's nearly impossible to get most people's algorithms to find the real optimum because Python's suboptimal solution is "good enough". And the whole software industry (and in some ways, by extension, all of humanity ... to be over melodramatic) is suffering for it.
> And the whole software industry (and in some ways, by extension, all of humanity ... to be over melodramatic) is suffering for it.
I don't think the biggest services built with Python (think Instagram, Dropbox, etc.) have more consumer-facing issues than services written in other languages.
If you're talking only about developers, fine, however I also think most Python developers like the language. For me it seems that Python has strong vocal critics, that show well in places like HN, however it is not representative of the majority.
So I really don't think Python is making the humanity suffer, for any good measure of suffering.
There is great saying that Python is the second-best language for everything and it might be true. Where Python excels is that you have good-enough libraries and support almost everything. All other languages have some pain points where they shine very brightly in some areas but they have very bad or non-existing libraries and support to other areas.
```--max-line-length=120``` passed to flake8 will switch it to 120. Use another number for a different length.
Things like max line length should be something your org or your fellow contributors decide on, not something dictated to you no matter what the language.
A corporate point of view is than any programmer should be interchangeable with the minimum amount of fuss. I understand why someone building an injury organization has a responsibility to think about the future.
But I confess that sometimes I feel very demoralized when an organization implicitly tells me that my years is study and my ongoing self-education and practice is all meanless Because in principle someone fresh out of college should be able to take over my project with the two week handoff and a few docs.
i fully agree with you. it has a stubbornness about it and not in a good way like the stubbornness you might find in lisp, scheme, and sml derivatives. i have had to write python on only a few side projects, but it was miserable, as you say. not only is the language all over the place and embarrassingly slow, the ecosystem is as well. i tried getting an application someone left when they left running on windows 10, and it was basically a null effort without a complete rewrite, upgrade to 3.x, and upgrading/replacing GUI libraries.
if i had to write python as a day job, i would quit. i have said it before, but python is the new c++, helping people everywhere write brittle, unmaintainable code.
I'm sympathetic that Python is relatively not a great language, but IMHO an 80-character line limit is quite reasonable. It's easier to read on smaller screens, easier to view side-by-side diffs, and tends to force you to break up your code more.
That said, this shouldn't be a lint, it should just be enforced by a formatting tool as a format-on-save setting. It just destroys all the wasted arguments about formatting and the wasted time trying to manually line up code.
I'd also add that while perhaps a bit on the pessimistic side, I tend to view the 80 char rule / limit not as an ancient hardware limitation of monitors, but as a limitation of our eyes and visual processing circuitry. There is a reason why newspapers and well laid-out websites don't have 300 char width lines. Those are physically harder to read, whether we want to admit it or not, as our eyes lose track of the flow to the next line.
I'm all for decreasing unnecessary cognitive load, there should be quite enough of that without us adding more by accident.
It's hypothetical so I can't say for sure, but I did work a lot with the JVM and I used Clojure (and I'd probably use Kotlin now as well) and I didn't feel as burnt out as I do now.
Yeah, I'm a bit sick of Python. We have some internal utilities written in it and while developers could manage all the dependencies and everything, we kept having problems with getting everything working on the machines of our electronics techs.
Gradually pretty much everything has been rewritten in C++ (with Qt GUI framework). Way easier to be able to have a setup program (using free InnoSetup) that just extracts the eight or so DLLs required in the same folder as the EXE and that's it.
We just use Python for a bit of prototyping and data crunching here and there now.
> Way easier to be able to have a setup program (using free InnoSetup) that just extracts the eight or so DLLs required in the same folder as the EXE and that's it.
Thanks for posting. I've wondered if it's just me or does anyone else want to leave software engineering because of Python's dominance. There's no arguing against it either because as this thread shows, "I like it for my use case, look at these libraries that let you get X and Y done SOOOO easy, so it must be great for everything."
It is a language, better than MatLab/R/SPSS, which come before it.
I honestly don't think it is that bad. And they are many people don't care from a programming language perspective, they need to just finish the functionality.
> It is a language, better than MatLab/R/SPSS, which come before it.
I don't think so. I think R is a lot more expressive and not really any harder to read. It might have a steeper learning curve, but it's not so bad that I think that actually matters.
I look at developer surveys sometimes when I'm trying to decide what to learn next. According to the 2018 stack overflow survey, 68% of python users would like to use it again in the future [1].
The surveys never tell me why though. What do people like or dislike about python? I know it has a lot of libraries people love (scikit-learn, tensorflow come to mind)
Those surveys are often targeting mosstly newer devs who know no better because they did not have the time to validate their opinion against real world.
Its not thir fault tho, you can only do so much in limited time, thats why expertize requires years.
I would rather wish Golang eat the world than Python, just because the practicality of python becomes questionable when performance is key.
In my previous startup in India, I trained unskilled personnel to become decent python developers to work on our product; everything was fine till the product grew exponentially and we had to optimise every nook and corner of it to better serve the customers and save on server bills.
So we had to optimise our application with Cython to meet the demands. So, when training someone to code if we use Python as a language; we should follow up with the disclaimer "You will learn to code faster than most other languages, but you cannot use this knowledge to build something at scale (i.e at-least when budget is of concern, when you are not instagram)".
In comparison, Golang excels in both form and function. It is just as easy to teach/learn as python and doesn't need the aforementioned disclaimer. Web services are a breeze to write and is built for scale.
I understand that there are criticisms against the language design of Go, some are valid and most are just because it was made by Google but none questioning the scalability of Go applications.
Many of my complaints about Python are valid for Go, except that Go makes even more perilous decisions than Python for error handling (and the community kinda gleefully embraces it).
But at least Go is a lot faster and has real concurrency AND parallelism, so it's definitely better than Python.
I’m really glad to hear someone else voice these frustrations. I’ve really tried to embrace Python and everyone tells me how great it is. I feel like a failure as a developer but I dislike it so much I quit a job, that switched to Python as the primary language, and took a year off from development. I’m pretty much a polyglot as far as languages go but Python riles me.
Python is the simplest mainstream language yet still reasonably powerful. Some folks don’t like simple and I’ve found it’s better in the long term to find those that do.
>that infernal pep-8 linter insisting 80 characters is a sensible standard in 2019
So change the limit or disable that check. If someone is keeping you from doing that they're the one who's insisting on 80 characters, not the language. Who uses any linter without making some changes to its default settings?
I was smart and lucky enough to switch from Python to Clojure and from Javascript to Clojurescript. I am not even sure anymore what have I liked about Python back then. I know for sure - those who really like Python probably haven't given a heartfelt try to Clojure, Haskell or Racket.
(and that infernal pep-8 linter insisting 80 characters is a sensible standard in 2019) increases my burnout by 100x
I've yet to encounter a python linter where you can't pick and choose which rules to ignore. This is a first one to go. Annoying PEP for sure, but https://pypi.org/project/black/ almost completely eliminates your issue.
There is a lot of hype and "they are using it too" mentality as well I think. I am working on control software which, if it crashes or even takes a tiny bit too long to issue a command, could cause the company big financial loss. I was forced to do it in python "because everyone else is using python".
This is about as sensible to me as saying it is a good idea because most trains are longer than they are wide. Who cares about those things, we use computers that nearly all have 4:3 or 16:~10 displays (or wider), and many of use use more than one of them.
it's funny because a few people ended up using python as a prototyping tool to find better approaches to then rewrite in cpp or else. Somehow in that regard it saved them tons of time.
You can configure Pep8/Flake8 to ignore subset of rules. You might wanna look into that. Also, automatically executing it might be your preference, or not.
What I find worse about Python than other languages is the lack of tooling and a relatively small collection of libs - many of which are half way done. In a last assignment, we decided Python is at most a hobby language. Python is great for machine learning because most research was conducting using this language, and as such, tooling is available. I would use it at most as an API exposing ML but that's pretty much it.
> In a last assignment, we decided Python is at most a hobby language.
I don't know if you are just trolling, but this is the silliest, most detached from reality thing I've read online today. Python was key in building numerous massive public companies like Instagram and Dropbox. It's one of the most popular and widely used programming languages on the planet for everything from APIs to desktop clients to data pipelines. It had a lot of early popularity in the Silicon Valley start-up scene in the early 2000s, even pre-dating the Ruby on Rails web dev trend.
The guy who founded the company that runs this very website wrote about the draw of Python 15 years ago [1] at which point it was already widely used in certain niches. This is before any deep learning libraries existed. I remember first playing with Python around 1999 or so.
> I would use it at most as an API exposing ML but that's pretty much it.
I don't know how old you are or how long you've been in the industry, but the ML thing is a "second act" for Python. Deep learning grew up in a time and place where Python was a good fit which put Python in the right place to benefit. But Python had already lived a long and full life before any of that happened.
It's fine if you don't like Python or don't think it is a good fit for a project, but claiming it is a "hobby language" with a "lack of tooling and a relatively small collection of libs" is a good way to get laughed out of the room. It has one of the largest and most diverse libraries. And as far as tooling, Python is one of the most popular languages for implementing tooling. Check out ansible.
> There are plenty of packages for data science. For building APIs and web stuff usually there aren't. Python is nowhere near NodeJS for instance.
This has got to be a troll, right? Just some of the most popular web frameworks: Django, TurboGears, web2py, Pylons, Zope, Bottle, CherryPy, Flask, Hug, Pyramid, Sanic... Lots of huge websites were originally built with Python like Instagram, Reddit and YouTube. Of course they mature into complex distributed architectures using all kinds of languages, but it's all about the right tool for the job.
If only its package management were as easy as its syntax...
I wish pip worked the same way as npm: -g flag installs it globally, otherwise it creates a local "python_modules" folder I can delete at any time. And optionally I can save the dependency versioning info to some package.json...
Instead, pip is a nightmarish experience where it fails half the time and I have no idea where anything is being installed to and I'm not sure if I'm supposed to use sudo or not and I'm not sure if I'm supposed to use pip or pip3, etc.
2. For each project you want to work on, create a venv. Yes, there are tools for this, but the base venv tool is totally fine. (venv should be included in your Python, but a few distributors like Debian put it in a separate package - install it from them if needed.) Use python3 -m venv ~/some/directory to create a venv. From here on out
3. As a first step, upgrade pip: ~/some/directory/bin/pip install -U pip.
4. Install things with ~/some/directory/bin/pip install.
5. Run Python with ~/some/directory/bin/python.
Slightly advanced move: make a requirements.txt file (you can use .../bin/pip freeze as a starting point) and use .../pip install -r requirements.txt. That way, if you get any sort of package resolution error, you can just delete your venv and make a new one. (Downloads are cached, so this isn't super annoying to do.)
A "project" can either be actual Python development, or just a place to install some Python programs and run them out of the resulting bin/.
(Edit: Yes, I know about activate, see the replies below for why I don't recommend it. With these rules, you get to say "Never ever type pip, only .../bin/pip", which is a good safety measure.)
Herein lies my problem. If I want to start a Node project I run `npm init` and then `npm install --save` to my heart's content. If I somehow manage to mess it up I just delete node_modules/ and install again.
If I want to start a Python project I have to set up venv and remember to put relative paths in front of every command or else it'll run the system version. Sounds simple, but it's still something to always remember.
1. pip install --user and sudo pip install are fine actually, they will not interfere with venv, they can co-exist just fine.
2. yes
3. probably do "source bin/activate" first, then run 'pip install -U pip'
4. just run pip install whatever, no need the full PATH
5. just run python directly, no need the full PATH
6. run 'deactivate' when you're done for now, 'source bin/activate' when you want to continue/resume sometime later
in fact I like this better than node_modules, the venv layout of bin/include/lib is more natural comparing to the odd name of "node_modules" in my opinion, and I don't need npx etc to run commands under node_modules either, all are taken care by venv with its 'activate' script
Python 3 comes with it as "python -m venv". Once in the virtualenv, you don't have to worry about the various pip forms and effects, you can just pio install.
You can get fancier than that of course, but that's what works with stock python, on all OS.
I haven't seriously JavaScript'd in a couple of years but my problem with it then was different versions of node or npm. Nice thing about python virtual environments is that problem never exists (can make environments with whatever version you want).
Exactly. NPM gets a lot of hate but lockfiles and local install by default is great. The default mode should not be global installation. Also imo virtual environments aren't amazing. Having some mode you have to remember to flip on that changes essentially the semantics of your pip commands seems a little brittle. Tools that work on top of pip and venv like Pipenv or Poetry seem a lot better.
This isn't even the start of the problems with pip and pypi. If I install pylibtiff, it embeds a copy of an old libtiff which is binary incompatible with the libraries on my system. I have to build it from source, then it works just fine. But I can't inflict this level of brokenness on my end users.
This applies to many packages embedding C libraries, including numpy, hdf5 and all the rest. There has been minimal thought put into binary compatibility here, meaning that it's a lottery if it works today, or will break in the future.
I couldn't agree more with this. I was forced into doing some UI coding and although I could never full embrace js, the package management aspects (esp having the sane default of not installing packages globally) were definitely superior to python.
I feel uncomfortable with the fact that people feel a third-party solution is the best way to solve this mess. It can also get messy when packages installed with pip, pip3, conda and apt are all entangled with one another in various ways.
Holy Crap! What a lot of irrational, hyperbolic hate for Python.
I think everybody should spend their first couple of years working in Fortran IV on IBM TSO/ISPF. No dependency management because you had to write everything yourself. Or maybe [edit: early 90's] C or C++ development where dependency management meant getting packages off a Usenet archive, uudecoding and compiling them yourself after tweaking the configure script.
I'm not saying Python is perfect, but if it's causing your burnout/destroying your love of programming/ruining software development you seriously need some perspective.
I just returned to Python for the first time in a little while to collaborate on a side project and ran into a few tricky-to-debug errors that caused a fair bit of lost time. Know what the errors were?
In one case, I was iterating over the contents of what was supposed to be a list, but in some rare circumstances could instead be a string. Instead of throwing a type error, Python happily went along with it, and iterated over each character in the string individually. This threw a wrench into the works because the list being iterated over was patterns, and when you apply a single character as a pattern you of course match much more than you're expecting.
And another one, I typoed a variable name as groups_keys instead of group_keys (or vice-versa, I don't remember). Instead of just throwing an error, Python happily went along with it, used an uninitialized value, and then all the logic broke.
There's entire classes of errors you can cause yourself in Python that aren't possible in stronger, statically-typed languages. For a large project, I'd pick the old and boring Java over Python every time.
Python is a dynamic language, that's what dynamic languages do, you don't have a type checker but have greater flexibility, but you don't have to settle on that, you can actually use mypy and annotate types and get best out of both worlds.
> And another one, I typoed a variable name as groups_keys instead of group_keys (or vice-versa, I don't remember). Instead of just throwing an error, Python happily went along with it, used an uninitialized value, and then all the logic broke.
This isn't what Python would do, if the variable was undefined Python will throw an error, so you must have defined it with this name or you're misremembering what have happened.
Your 2nd error isn't possible in Python, so I'm not sure what you did there. Regarding the first, sure, it is a bug that was annoying to catch. But, having an `Iterable` interface in Python is also really neat and useful if used responsibly. If you're programming regularly in Python, you are accustomed to the tradeoffs that come with a dynamic programming language and no static types, and you can still avoid issues like the one above.
Right off the top of my head, using type hints with a decent IDE or an assert statement would likely have caught the issue.
I'm not saying that Python doesn't have issues (all languages do), but I don't see the error noted above as any sort of deal breaker. On the other hand, if you're only ever going to use Python like a strongly typed language without taking any advantage of its dynamic characteristics, then I can see why it would seem as a total downgrade compared to languages like Java.
>In one case, I was iterating over the contents of what was supposed to be a list, but in some rare circumstances could instead be a string. Instead of throwing a type error, Python happily went along with it, and iterated over each character in the string individually.
I've been using python for about 13 years professionally and I wrote up a list of "things I wish python would fix but I think probably never will" and treating strings as iterable lists of characters was on there.
I've seen this bug multiple times and the fix is relatively easy - just to make strings by default non-iterable and use "string.chars" (or something) if you really want to iterate through the chars.
Nonetheless, I still love the language and wouldn't use anything else.
>Instead of just throwing an error, Python happily went along with it, used an uninitialized value, and then all the logic broke.
This one gets caught by linters - unfortunately 90% of most python linters spit out rule violations which aren't important which drowns out stuff like this in the noise.
For your first error, you can do some foot-shooting with a statically typed language too.
I remember a bug I made using C#, where I wanted to delete a collection of directories recursively. I got mixed up into the various inner loops and ended up iterating over the characters like you. But C# allows implicit conversions of char to string so the compiler was OK with it, and since those where network drive directories (starting with "\\server\"), the first iteration started deleting recursively the directory "\", which in windows means the root directory of the active drive (c:\)... And SSDs are fast at deleting stuff.
> And another one, I typoed a variable name as groups_keys instead of group_keys (or vice-versa, I don't remember). Instead of just throwing an error, Python happily went along with it, used an uninitialized value, and then all the logic broke.
Python doesn't have uninitialized values, it throws NameError when you try to access a variable that hasn't been set. So I don't see how this could have happened.
Well this is anything but a new complaint. I would assume a user who has worked in Python for some modest amount of time to have made peace with this. One works in Python knowing that this can and will happen (well one does have linter on steroid like mypy now to counter these).
Python code needs more testing, more run time type checking of function arguments than a statically typed language. If that's a deal-breaker then one shouldn't be using Python in the first place. What you gain though is some instant gratification, and the ability to get something off the ground quickly without spending time placating the type checker. Its great where your workflow involves lot of prototyping, exploration of the solution space and interactive use (ML comes to mind, but even there int32 vs int64 can byte, correction, bite). I see it as a trade off -- deferring one kind of work (ensuring type safety) over another. Hopefully that deferral is not forever. I like my type safety but sometimes I want that later.
What I typically do is once I am happy with a module and I do not need the extreme form of dynamism that Python offers (something that's frequently true) I take away that dynamism by compiling those parts with Cython.
> In one case, I was iterating over the contents of what was supposed to be a list, but in some rare circumstances could instead be a string.
The creator of a well-known alternative to Python has a single-letter email address, and regularly receives email generated by Python scripts with this exact bug (which means instead of sending an email to "user", sends an email to "u", "s", "e", and "r"). So I’ve heard.
In my CS program, we learned Python as a convenient way to sketch a program. We also learned C++ for speed and OCAML for those functional feels. A programming language is a tool, Python has some great use cases mostly focused around ease-of-programming.
The bugs you describe should both be easy to catch with unit tests. It sounds like the problem is not that you're using Python, it's that your project lacks tests. Sure, you can typo this sort of thing; but it should be apparent within seconds when your tests go red.
(And nowadays, you can also use type hints to give you a warning for this kind of thing, e.g. your IDE/mypy will complain about passing a string where the function signature specified a List.)
> Python happily went along with it, used an uninitialized value
There is no such thing in Python. You should get NameError if a name doesn't refer to any object.
>>> def f():
... name
...
>>> f()
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "<string>", line 2, in f
NameError: name 'name' is not defined
Agreed. I really don't understand all these buckets filth being poured on Python in this thread.
It's a first language I worked with in my life that just clicked with my brain and doesn't just drain me.
I would take a Python job over a Java/C/C++/Go/Rust any day.
There's some languages that could pull me away from Python (Nim, Crystal) but they're nowhere popular enough to move wholesale to them.
> I would take a Python job over a Java/C/C++/Go/Rust any day
it's funny, I feel the exact opposite. I work on a team that maintains a digital catalog, and a lot of what we write is about taking in asset- and metadata files, asynchronously processing them, and then publishing that to a denormalized read-optimized data store. We often joke that we mostly take data from 'over here' and put it 'over there'.
All our stuff is in Java, and honestly, if you use Lombok to squeeze out the boilerplate, and a decent dependency injection framework like Guice or Dagger, modern Java really isn't so bad. Streams are clunky but they get the job done. We use Jackson a lot to serialize/deserialize Java pojos to JSON and XML, which is pretty seamless for us so far. The Optional class is again clunky, but it works well enough.
The thing for us though is that the problems we spend most time solving are just not really related to the language we write it in. The hard problems are much more around things like operations (cd/ci, metrics, alarms, canaries), performance (latency, load, etc.) and just the nuts and bolts of the business logic (what type should this new field be? what values can it take? how do we plumb this through to downstream system X owned by team Y? etc.)
I honestly wouldn't want to have to write this stuff in Python for a simple reason: I don't think I could live without static typing, which is a fantastic tool when you need to manage a large code base written by multiple people over multiple years. I can make a change in some package, do a dry-run compile of every system that uses it, and then see what needs updating. It gives me certain guarantees about data integrity right at compile time, which is super helpful when you're doing data conversion.
But hey, different jobs, different tools. Glad you found something you're happy with.
I've worked in so many languages and environments in my career and Django/python/virtualenv has to be one of the least painful. I tried Rails which is very similar but feels "inside out", a good friend of mine loves Rails and hates Django and has the exact same feeling about Django.
That's kind of my point, you may like other environments better, such as React/Node/NPM but that doesn't mean Python is a horror show.
Python is the first language that clicked with my brain as well and in college I often used it to prototype homework algorithms before translating them into the language I needed to actually submit my work in. I have nothing but love for python as a language.
At the same time even when I used it heavily I never saw it as anything more than a scripting language to sit in front of some tool that was written in a language I couldn't be fucked to learn at that moment (numpy and scipy were used heavily throughout my college career).
If I'm being honest I don't understand how anyone could get as worked up about a language as the people in this thread have. At the end of the day most of us are still writing unportable imperative code that runs like shit. Maybe blaming language is how we cope with our own failure as engineers.
Python is great; I picked it up back in the early 2.xx days. My main problem with it is the string handling/conversion code is brittle and the breakage of backward compatibility. But it's overall a great language.
And some people feel the opposite. I’m glad Python works for you. I had the same click-with-my-brain feeling with Ruby, whereas I find working with Python to be draining and demoralizing.
> I would take a Python job over a Java/C/C++/Go/Rust any day.
Why do you group those languages like they’re similar but different from nim and crystal? They’re wildly different in terms of their target domain, runtime models, etc. Go and Java are general purpose application languages and the others are more suited for systems or performance critical applications.
I had to learn Fortran IV for my first job. Am I allowed to hate Python?
Are you assuming everyone complaining here is young, and this is their first language? Consider that maybe they're complaining because they've used older languages they liked more.
Often, not having a feature is preferable to having a feature designed or implemented poorly.
Sure! I dislike all sorts of languages and environments.
That wasn't my point. My point is if you had to write Fortran IV using an IBM 32xx terminal you wouldn't be quite so hyperbolic about modern Python.
Unless you are claiming you would rather return to writing Fortran IV than use Python because you like Fortran IV better, in which case I'm very confused.
There's a difference between hating Python, and saying (I'm guessing is the comment that spurred this one) this: "I try to be a good sport about it, but every time I write python I want to quit software engineering.", like a top-level comment below says.
If you had to write Python, would you also want to quit software engineering? Would you go back to Fortran instead of Python?
Of course you're allowed to hate Python but someone saying "every time I write python I want to quit software" is either extreme hyperbole, some tangentially related issue like depression, or just no language at all would make them happy enough.
Python 1.4 was an awesomely simple programming environment and I pretty much immediately fell in love with it. Then features were added. Now it is a whole home improvement store full of kitchen sinks.
I think that programming is a sort of theological process. Popular languages attract ideas. Unfortunately, in the case of Python, those ideas were not effectively filtered and now we have an expression of as many ideas as can possibly fit. The ultimate design by committee...
I suspect that the recent excitement about assignment expressions is really a kind of straw that broke the camels back. The problem isn't just this one feature, it's the sum of them.
>I think that programming is a sort of theological process. Popular languages attract ideas. Unfortunately, in the case of Python, those ideas were not effectively filtered and now we have an expression of as many ideas as can possibly fit. The ultimate design by committee...
It's funny, a lot of people hate on Elm for the exact opposite reasons: one person dictating the language's direction and removing features. I suppose a nice balance could be struck between the two ends of the spectrum.
>Python 1.4 was an awesomely simple programming environment and I pretty much immediately fell in love with it. Then features were added. Now it is a whole home improvement store full of kitchen sinks.
I've used Python at the time (and up to now). It was a revelation compared to Perl, but it sucked compared to modern Python.
I think it's pretty disappointing that most of the top comments don't talk about the interview with Guido himself over the history of Python. Tangentially related discussion is one of the appeals of HN but I think it's a bit out of control here.
Well in just glad this is the top comment, as Python really is taking over the world for a reason.
And of all the bugs I have written in recent memory, not one came down to a lack of static typing. They were due simply to logic errors, flawed assumptions, misunderstood requirements, and good old race conditions. The static typing zealots like to think if it compiles is must be perfect, however this is a mirage. Unit tests in Python can compensate quite well for lack of static typing.
I cut my teeth in FORTRAN IV (on RSX-11M). I lived through the archie days of uuencoded fragments to build my C environment, supplemented by DECUS tapes. Those were good old days.
I use Python 3 these days for a lot of stuff. It's pretty good.
These are better days. We all have complaints, but on the whole, things are not too bad.
I think for the most part that when nothing meets your expectations, it may be that your expectations need to be adjusted.
People don't realize what a revelation Python 1.x was back in the day. Around 97 or so I was tasked with porting a giant mathematica program for calculation diffraction grating efficiencies into something which ran open source (there was no free mathematica engine for running scripts pack then). Back then, that meant either C or Fortran. Sure, stuff like Perl existed; nobody thought of it as a real interpreter that could be used to construct complicated things any more than Awk was. When I realized Hugunin over at Livermore had done lapack extensions for Python (whatever numpy was called back then) ... well this massive job was done in a week and worked the first time.
The winning thing python had that nothing else had at the time was it was social, it was readable, and there was generally only one way to do things: the right way. It no longer has the latter quality, and the preferred coding style in it seems to be java-ish OO-spaghetti, but it's still pretty good.
That said, these days, I resent every damn time I have to use it. It's eating data science, more or less because pandas and scikit is ... mostly good enough, and because unlike R it's .... mostly good enough to deploy in an enterprise application. But if you're working on the exploratory side of data science, Python is shit compared to R. Doesn't have the tooling, doesn't have native facilities, and is vastly more long winded. All the attempts to make Python more ... X ... are probably a mistake also. You're taking a beautifully simple tool and making it more exotic and complex. It's like trying to use Matlab to build webservers.
> ... Perl existed; nobody thought of it as a real interpreter that could be used to construct complicated things any more than Awk was ...
I can't help but to gently interject here. By 1997, I'd been programming for some time, and there certainly were people who consider Perl suitable for programming in the large, and there were certainly big projects so written.
While I disagree with the specific word 'nobody', I agree with the sentiment: Perl was widely considered to be only useful for 'small' things at the time. Widely, but not exclusively.
I'm a somewhat older programmer, and I've worked with a variety of languages (C, OCaml, C++, Scheme, Go, Java...). I think all of them are great in their own way and there's a lot to be learned with all of them.
I started to use Python quite recently and I really like it. It is a well-designed language with high-level abstractions that are really fun to use. I like the pervasive use of iterators, the 'everything is an object' philosophy, the minimalist syntax, the build-in datatypes...
That being said, I feel that the dynamic types show their limits when projects getter big. I use linters and static type annotations but I find refactoring very error-prone and there's a point where I don't really trust my programs.
I always thought that sentiment to be too broadly applied. A good craftsman should be able to make use of the tools he is given, but nonetheless not shirk his duty to improve upon them.
Not to say I have any real complaints about Python.
I don't think Python is a bad language but that quote is a pretty ridiculous response to criticism when the discussion on this article is a far cry from people blaming failures on Python/Python tooling.
No, not at all, I'm saying if you had been a C programmer in the 90's you would have some perspective on some of the complaints and comments about python in 2019.
Why would you rationally compare something that exists and lives today, in the form it has today - with something from early 90s? In what world is that an objective comparison?
Back in my day we had to walk 20 miles to get to school -- uphill both ways!
As if the only way to gauge quality is to forever compare to the technologies of the past. Python was a great improvement on its peers in its heyday, but people, technologies and philosophies have changed since its inception and there is nothing wrong with wanting something more. That's how progress is made.
I find programming in Python dull and a little mind numbing. Sure, maybe Python was an okay choice 20 years ago, but the world has left it behind at this point. It’s lack of functional programming constructs, very few data structures in the stdlib, terrible performance, worthless statement vs expression semantics, no multi threading, no macros, etc.
Instead of evolving into something okay, Python is pretty much the same broken language as it’s always been, run by a guy who is openly hostile to PL theory, FP, and any major changes to Python whatsoever.
If you ever need/want to use Python, do yourself a favor and use Racket instead. Racket is better than Python in every single way (unless you are doing data science with Python, in which case fine keep using it).
Isn't it also just as much about Python is having it's day, granted a day long in the waiting but many langs go through this (Ruby, PHP) and then it tapers off and the next language has it's day.
> Probably Go will be the next hotness in 5 years.
I think it will be difficult to grow a large ecosystem for a language with very poor FFI performance [0] in the long run. Golang's poor FFI performance is the number 1 reason I wouldn't use it for my own projects.
Actually, Golang isn't so great. Try to change something lower level, for example in their socket implementation. Also, it's trying to promise a sane concurrency and all code I've seen use mutex all over the place.
Yes and no. Go has its applications, but is not built to replace python but rather to be a better c. That overlaps with the "easy code" part and not much else. Glue code, application scripting, etc. are python's strengths, and I don't see those going away.
Maybe, I love Go but I'm not sure it's the next "hot" thing, all the recent "hot" languages have been scripting languages PHP, Ruby, Python, JS ... I'd say JS and Python are currently jockeying for that position.
Yeh I recall working on MAP/Reduce using Fortran/ PL1G (1980's) we had to build an entire suite of JCL programs to mange every thing including build / deploy.
I don't agree at all. Your comment basically says: "It used to suck badly. So don't complain it sucks now.". I think dynamic typing is the bane of good software and we as an industry should try to actively discourage new code to be written in dynamically typed languages.
That's not to say that Python doesn't have it's place. But I see it more as a programming language for small utilities no more then 2k loc in length.
No, what I'm saying is that it used to suck really badly and we survived just fine, so I find all the over-wrought hand wringing about the havoc and burnout caused by Python's flaws hyperbolic.
To understand the hate, you have to realise that no one likes being forced into using a particular technology. Especially one that is more of a lowest common denominator and ignores much of the progress in programming language research over the last 40-50 years (e.g. expressive static type systems, and Python still markets itself as "strongly typed").
> and Python still markets itself as "strongly typed"
What's wrong with that? It is strongly typed (doesn't coerce everything like Javascript) just not statically typed (objects have types but variables don't).
I go with what helps me get work done. I have a feeling many people are the same. Python lets me be productive in a crazy variety of tasks and mostly gets out of my way when I do so.
Why do you think that Python should be compared to Fortran? Why not Rust or Julia? If we compared everything to worse humanity would not progress at all. I have spent 10 years on Python and I sympathize with some of the criticism below here pretty much. I wish Rust or Julia will replace it for data crunching soon.
I love python as a scratch pad for playing around with code, but I don’t think I would put anything into production written in it. It’s just too hard to debug and maintain and deploy once it gets to even a medium amount of complexity.
Interactive python and Jupiter notebooks are an absolute joy to work with though.
A hundred thousand years from now, when the Terran Empire's Dyson Swarms are ubiquitous throughout the Orion Arm, the relativistic generation-ships of the Andromeda Colonization Fleet have set out on their multi-million-year journey across the intergalactic deeps, and the World Computers housing the Great Intelligences serve the daily needs of quadrillions of citizens, there will still be job ads for COBOL programmers.
What about pkgconf + autoconf? Might not be the grand unified package manager you're conditioned to look for, but works well and is pretty much universally used. Besides, C and C++ aren't about language ecosystem lock-in, but about creating standardized artifacts (shared libs, static libs, and executables) designed to work and link well in a polyglot environment (well C++ maybe less so with its symbol mangling/typesafe linkage).
I don't think its hyperbolic. If so many people are complaining then we have a issue. I have tried to learn Python but every time I did, some things kept turning me off.
- First indentation was a issue for me but I looked past it and went ahead to give another go at Python.
- Even the best in class IDE suffer to give any kind of insight into python code.
- Two versions: At my workplace we use Python 2. I prefer to learn newer version but don't have choice. When Python 3 came out they should have initiated deprecation of Python 2 but that does not seem to be the case. Tonnes of libraries are still in Python 2. They should have maintain language backward compatible instead of fragmenting the entire community.
- So many ways to do a thing. This is subjective but I need a language which gives predictable performance for given piece of code. Go does this. There's usually one way of doing things and only one way. No need to know nooks and crannies of the language nor there is such a thing in Go
You’re probably aware of this, but at the odd chance you aren’t, Python 2 end of life is 2020. You really should be moving to Python 3.
> Go does this.
There is a lot of love for Go and Rust on HN, but unless your region of the world is significantly different than mine, then chances are that there won’t be a Go or Rust job in your lifetime. I’ve only ever seen one of them mentioned in a job opening for my entire country, and that was as “nice to know” for a c++ job at Google.
I’m sure Rust and Go are truly excellent languages, but that doesn’t really matter for most people, if they never manage to see any adoption outside of Silicon Valley.
Complaining isn't the problem, yes there are package management problems, a virtualenv package solves most of them. I'm not saying there isn't. Python 2 really stopped being a problem for me last year, I haven't hit an issue for a while.
I'm talking about the comments saying it was causing their burnout "x100" and other such hyperbolic statements.
According to people who work with a lot of programmes teaching coding, the reason python is so appealing to new coders is the syntax. Particularly the lack of braces, the indents, the nice keywords, that make reading code much easier to a newcomer.
Having taught both python and JavaScript, I can tell you that the former is far far less confusing to newcomers than the latter. Imagine explaining the problem with equality comparison or prototypes in JS to someone who just learnt what an IF statement is.
The reason I agree that python will dominate programming for decades to come is the batteries included and pythonic approach that enables people to do cool things with just a few lines of code. As a result of the above, the ecosystem has reached critical mass. For most tasks there’s a library that does that, and it’s good, and it’s easy to use, and the developer of the library usually doesn’t expect you to understand their opinionated philosophy to use it.
I love the python convention of putting a minimal example on the homepage of a library.
You’re not going to get your average developer to appreciate the beauty of Lisp or the value of static typing. They want their ML to work, or their graphics to display, that’s all. Ends not means, almost entirely, for most people who aren’t full-time programmers.
Let’s not hate on a language that opens up programming to the world, unless we want to be gatekeepers. If you want to maintain an air of superiority, just learn Lisp or Haskell or C++ and write code that’s “deeper” into the tech labyrinth. Programming will probably specialise into “end user programming” and “Service and system programmers”. Embrace the democratisation of coding.
Since most of my professional experience is with C like syntax languages, I don't get how the lack of braces is an advantages, quite the opposite in my limited experience with Python.
Braces define scope unequivocally, they are easy to visually parse and don't care whether you are using tabs or spaces or even if you, loud gasp, mix them. Furthermore, you can copy and paste, say for loop from a method to another method and it will work. In Python you might have to faff about with spaces (I'm sure there must be IDEs that solve this problem but it was no fun on vim or even notepad++)
Absolutely agree. maybe its just cos I am coming from languages with braces (java) but yeah I find the lack of braces + whitespace having meaning (tabs versus spaces for example) _incredibly_ frustrating
I also don't like the fact it is dynamically typed. It makes reading soure code so much harder. E.g What is the type of this thing that is being passed as a function arg?
So does whitespace, otherwise Python wouldn’t work as a programming language
> they are easy to visually parse
only if you indent properly
> Furthermore, you can copy and paste, say for loop from a method to another method and it will work. In Python you might have to faff about with spaces (I'm sure there must be IDEs that solve this problem but it was no fun on vim or even notepad++)
Shift+V
}
< or >
Hit . as often as needed
Too difficult?
Not to mention you will run into exactly the same problem when copy pasting C unless you like randomly indented code
Two things: 1. For people not used to do programming, braces in text are usually significant - having them everywhere, and even start and end on different lines, are confusing. 2. For people not using standard US keyboard braces are often an SHIFT or ALT command, that makes it wildly different to program uninterrupted.
The part that Python really bothers me, as someone who had write Python for almost 10 years, and right now doing it professionally, though personally I don't consider myself a Python developer rather than a Python senior user, is actually obvious to nail down, and frustratingly, difficult to handle.
SLOW. The sin of all. I don't want to make other arguments with people who just come to repeat 'Oh you can make it fast, you are doing it the wrong way'. Sorry, I have heard that 100 times, now is sounds like bug-as-feature excuse to me.
Sorry, but Python is just slow. All the stuff people might bring up is just trying everything to bypass Python itself. Multiprocessing is hell to work with, buggy and unpredictable. Cython is OK but it is not Python.
Python's slowness is like cancer, and terminally so. Down deep, it is because its overtly flexible/loose type system, the never-going-away GIL, and prematurely exposing of C-API makes it almost impossible to optimize without breaking the existing compatibility.
And most importantly, I don't think it is solvable, truth is the patches are so integrated, people already forget what problem they tried to fix in the first place and just live with it.
But it will haunt Python in long term though, people will be struggling with Python in performant production until they start reimagining a world that without Python. Maybe a better Python in that sense.
Same here. I've used Python daily or almost daily for the last ~10 years (thankfully I've used other languages too).
I've architected reliable systems on top of it but it was obvious to me almost from the get-go that Python is a terrible language. I credit my previous years of programming in Lisp that trained my mind to see through the Python mirage.
One issue is that veneer of Python user friendliness that lures people, usually those who are new to programming or don't have extensive experience with _multiple_ languages, in.
The major issue for me is that Python lacks internal consistency and is based on a inverted-design (meaning, no design really) that accumulated over the years and stinks. This makes it really easy for people to use Python and write unmaintenable code but also very hard for them to find out how to write good code. The hallmark of a terrible language is making good code hard to write.
As far as specifics, I think the module system is a joke, the packaging even worse, the REPL is not really a REPL (in the Lisp sense), ctypes is bad and has 100 different ways you can hang yourself with, there are tons of gotchas in the language and standard library that if you haven't memorized [usually by running into them], they will bite you hard, the GIL ties my hands, iterator/generator obsession makes for really bad code messes if people take it seriously and start poorly designing their APIs around it, absence of proper lexical scope, statements are not expressions and led to gross hacks like PEP 572... I could keep going for days.
I honestly think you've just worked on bad projects. Python isn't the perfect language but the quality of the available libraries makes it awesome. The individual quality of projects in any language will always vary.
Python's success is richly deserved. The fact that it has been a long time coming (Python is almost 30 years old!) is an indication of the amount of sheer effort involved, and should give heart for all those working on building up the next generation of languages.
Since this forum is likely to have some folks looking towards the future, I found interesting a couple of talks by Python veteran Armin Ronacher [1, 2]. Watching that presentation makes me wonder whether I should think of this in the context of "Worse is better". That said, Python's runtime introspection is such a killer feature that it's difficult to explain how empowering it feels for someone who is relatively new to programming.
Readit News
The biggest issue, in my opinion, is in dependency management. Python has a horrible dependency management system, from top-to-bottom.
Why do I need to make a "virtual environment" to have separate dependencies, and then source it my shell?
Why do I need to manually add version numbers to a file?
Why isn't there any builtin way to automatically define a lock file (currently, most Python projects just don't even specify indirect dependency versions, many Python developers probably don't even realize this is an issue!!!!!)?
Why can't I parallelize dependency installation?
Why isn't there a builtin way to create a redistributable executable with all my dependencies?
Why do I need to have fresh copies of my dependencies, even if they are the same versions, in each virtual environment?
There is so much chaos, I've seen very few projects that actually have reproducible builds. Most people just cross their fingers and hope dependencies don't change, and they just "deal with" the horrible kludge that is a virtual environment.
We need official support for a modern package management system, from the Python org itself. Third party solutions don't cut it, because they just end up being incompatible with each other.
Example: if the Python interpreter knew just a little bit about dependencies, it could pull in the correct version from a global cache - no need to reinstall the same module over and over again, just use the shared copy. Imagine how many CPU cycles would be saved. No more need for special wrapper tools like "tox".
In the end, it needs to find the import in the PYTHONPATH, so there's no magic involved, and there are multiple robust options to choose from.
So instead of bashing Python for not shoveling down an opinion on you, it's up to the developers to choose which tools they want to use.
If they don't choose one and are unable to freeze their dependencies, it's not a Python problem, but IMO lack of skill and seniority.
The reason why Python gets extra criticism for this is because it likes to tell people that there should be one obvious way to do it and that it comes with batteries included yet it's dependency management system is just crap and doesn't follow that at all.
Are you saying “There’s more than one way to do it”?
It's good. Projects should use it.
Ah goddamnit.
868 lines, including os.rmtree calls and stuff.
Also installable via pip, but... "not recommended", and:
And if you need "to create a redistributable executable with all your dependencies". You can either use pyinstaller [0] or nuitka [1] both of which are very actively maintained/developed and continually improving.
[0]: https://github.com/pyinstaller/pyinstaller [1]: https://github.com/Nuitka/Nuitka
Anyone considering it for production usage should note that package installs in the current versions are much slower than pip or Pipenv. This might affect your CI/CD.
Looking at the home page it's not immediately obvious to me. For example, the lock file it creates seems to be the equivalent of writing `pip freeze` to the requirements file. I see a quick mention of isolation at the end, it seems to use virtual environments, does it make it more seamless? What's the advantage over using virtualenv for example?
Is there anything wrong with pip freeze > requirements.txt and then pip install -r requirements.txt ? This would install the exact versions
Then bar releases version 0.2.2
So your deps want bar 0.2.1, but foo now wants bar 0.2.2
This breaks your pip install.
EDIT: there are a few other gotchas (please respond to this post if you know of any more)
e.g. from https://medium.com/knerd/the-nine-circles-of-python-dependen...
"If two of your dependencies are demanding overlapping versions of a library, pip will not necessarily install a version of this library that satisfies both requirements" e.g. https://github.com/pypa/pip/issues/2775
there is no language that is devoid of shortcomings - so to any new (<3 yrs exp) python users, please ignore the above comment entirely as it has no bearing on anything practical that you are doing/will do. and all experienced python users know that there are ways to work around the shortcomings listed here and beyond.
this is my psa for the python community!
I personally would consider this to be a strictly worse comment because it does not go into detail about what’s lacking like the parent comment does.
>there is no language that is devoid of shortcomings
So we should just silence all criticism like a dictatorship?
>as it has no bearing on anything practical that you are doing/will do.
Are you saying no python user has to deploy their application ever? If that's what you really mean then your comment is just pure trolling.
What "personal issues" do you think the author has? The frustrated tone comes from the frustrations the author explicitly outlines; unless you think this shouldn't be so, you are turning this into an ad-hom.
> the entire comment could have just been 1 line "We need official support for a modern package management system, from the Python org itself."
Why? because you don't appreciate the detail on why we need such a thing? These issues certainly get in the way of producing production apps; not in the sense that they make it impossible, but they make the process harder and slower than it needs to be.
Node's dependency managers npm/yarn just copy the versioned dependencies from their cache folder into the local node_modules folder and remove transitive dependencies duplicates when possible by flattening them into node_modules.
Once your project is setup, dependency management is what you do once in two weeks perhaps. Rest is just writing code.
Deleted Comment
(To be fair, modifying it to use venv is... non-trivial).
If you look at the issues section in its github repo, you'll see that there are some pretty basic bugs there which are very annoying or disruptive. Moreover it seems the author has almost left the boat and a handful of contributors have to tidy things up.
Just to illustrate my point. I think a package manager that takes a few minutes to install a single tiny package, or don't prevent you from adding non-existing packages (e.g. spelling mistake), or doesn't let you install a new package without trying to upgrade all other packages isn't really production-ready. These are known issues since November last year.
"
Let's take an example:
will fail with this error: while Poetry will get you the right set of packages"
Dependency hell is everywhere.
Excuse me, but as a long time Python user I have to disagree. I started using Rust two years ago and Rust’s dependency managment is easily the best thing I ever saw (keep in mind that I didn’t see everything, so there is a chance there are better things out there).
The project-/dependency-manager Cargo¹ is more “pythonic” than anything Python ever came up with and where others mumbled ”dependency hell is everywhere” the Rust people seem to have thought something like: ”there must be a way to do this properly”.
The whole thing gives me hope for Python, but angers me everytime I start a new python project. Poetry is good, but there should be an official way to do this.
It saddens me to see, that some people seem to just have given up on dependency managment alltogether and declared it unsolvable (which it is not, and probably never has been).
¹: https://doc.rust-lang.org/cargo/index.html
In JS, which also has a single interpreter installed across the system (or multiple if you use nvm), the packages aren't installed "directly" into the interpreter, which removes the need for things like virtual-envs, thus making life a lot easier. I wish Python did something like this.
That being said, pipenv is making things easier. However, I think pipenv is a workaround more fundamental problems.
I have found that I would rather code my own versions of some libraries so I have control over it. Even if there is some extra long term maintenance and some up front dev costs, it's paid off already a number of times.
Use Nix.
> nix-shell -p python3Packages.numpy python3Packages.my_important_package
It solves every problem you quoted before.
Yup, I love Python over my current language in my job(JS/TS).
But I really dislike handling conflicts using pip, requirements.txt and virtualenv.
So much so that I will take JS node_modules over it.
However a lot of the issues that you mentioned (such as lock file and transitive dependencies) can be handled by pipenv, which should be the default package manager
I seems to have some neat functionality wrt dep handling (and I'd never really heard of it before).
There is and it's called docker. The other issues could indeed be fixed with something like poetry.
I agree, although a lot of it has to do that there's so much misinformation about the web, and many articles recommending bad solutions. This is because python went through many packaging solutions. IMO the setuptools one is the one that's most common and available by default. It has a weakness though, it started with people writing setup.py file and defining all parameters there. Because setup.py is actually a python program it encourages you to write it as a program and that creates issues, setuptools though for a wile had a declarative way to declare packages using setup.cfg file, you should use that and your setup.py should contain nothing more than a call to setup().
> Why do I need to make a "virtual environment" to have separate dependencies, and then source it my shell?
Because chances are that your application A uses different versions than application B. Yes this could be solved by allowing python to keep multiple versions of the same packages, but if virtualenv is bothering you you would like to count on system package manager to keep care of that, and rpm, deb don't offer this functionality by default. So you would once again have to use some kind of virtualenv like environment that's disconnected from the system packages.
> Why do I need to manually add version numbers to a file?
You don't have to, this is one of the things that there's a lot of misinformation about how to package application. You should create setup.py/cfg and declare your immediate dependencies, then you can optionally provide version _ranges_ that are acceptable.
I highly recommend to install pip-tools and use pip-compile to generate requirements.txt, that file then works like a lock file and it is essentially picking the latest versions within restrictions in setup.cfg
> Why isn't there any builtin way to automatically define a lock file (currently, most Python projects just don't even specify indirect dependency versions, many Python developers probably don't even realize this is an issue!!!!!)?
Because Python is old (it's older than Java) it wasn't a thing in the past.
> Why can't I parallelize dependency installation?
Not sure I understand this one. yum, apt-get etc don't parallelize either because it's prone to errors? TBH I never though of this as an issue, because python packages are relatively small and it installs quickly. The longest part was always downloading dependencies, but caching solves that.
> Why isn't there a builtin way to create a redistributable executable with all my dependencies?
Some people are claiming that python has a kitchen sink and that made it more complex, you're claiming it should have even more things built in, I don't see a problem, there are several solutions to package it as an executable. Also it is a difficult problem to solve, because Python also works on almost all platforms including Windows and OS X.
> Why do I need to have fresh copies of my dependencies, even if they are the same versions, in each virtual environment?
You don't you can install your dependencies in system directory and configure virtualenv to see these packages as well, I prefer though to have it completly isolated from the system.
> There is so much chaos, I've seen very few projects that actually have reproducible builds. Most people just cross their fingers and hope dependencies don't change, and they just "deal with" the horrible kludge that is a virtual environment.
Not sure what to say, it works predictable to me and I actually really like virtualenv
> We need official support for a modern package management system, from the Python org itself. Third party solutions don't cut it, because they just end up being incompatible with each other.
setuptools with declarative setup.cfg is IMO very close there.
> Example: if the Python interpreter knew just a little bit about dependencies, it could pull in the correct version from a global cache - no need to reinstall the same module over and over again, just use the shared copy. Imagine how many CPU cycles would be saved. No more need for special wrapper tools like "tox".
There is a global cache already and pip utilizes it even withing an virtualenv. I actually never needed to use tox myself. I think most of your problems is that there are a lot of bad information about how to package a python app. Sadly even the page from PPA belongs there.
I think people should start with this: https://setuptools.readthedocs.io/en/latest/setuptools.html#...
Yes it still has some of the problems you mentioned, but it fixes some others.
Deleted Comment
Deleted Comment
Why should Python have some "official" method to do this? Flexibility is a strength, not a weakness. Nobody ever suggests that C should have some official package manager. Instead the developers build a build system for their project. After a while every project seems to get its own unique requirements so trying to use a cookie-cutter system seems pointless.
"There should be one-- and preferably only one --obvious way to do it.": https://www.python.org/dev/peps/pep-0020/
Sometimes a 'benign dictator' single approach has benefits...
IF you do not have a language, or know Python a bit, then pick Python. Here are some of the reasons why I stick to Python (young startup/web APIs):
What I would like improved in the language would be an even longer list. Every language has a list like that, but when you are focused on being scrappy and building a product, yet making sure that software quality does not take a big hit, Python helps a lot.Are you saying this as a general maxim (don't try to learn a new tech under pressure) or because of characteristics specific to Python, that make it worse in such a situation than any other language/ecosystem?
I am not a high IQ person to even grip some of the nitty-gritty underpinnings of a language. But should that stop me from building a product?
In the end, I need a language that is easy to pick-up, be productive, has its heart in the right place. A young Python programmer can web scrap easily or plug into Ansible or so many other things. If you know of another language that would make more practical sense and still be easy to pick-up, I would switch.
Differences in languages themselves, I'm sure you can get used to and is far easier to cope with than lack of decent lib/tool/googleability.
I try to be a good sport about it, but every time I write python I want to quit software engineering. It makes me angry how little it values my time. It does little for my soured disposition that folks then vehemently lecture me about the hours saved by future barely-trained developers who will ostensibly have to come and work with my code. Every moment working with python (and that infernal pep-8 linter insisting 80 characters is a sensible standard in 2019) increases my burnout by 100x.
I try to remind myself that we're trying to make the industry less exclusive and more welcoming to new developers and "old" isn't necessarily "good" (in fact, probably the opposite), but damn I just don't understand it.
It used to be that I could focus on other languages (Erlang, Nemerle, F#, Haskell, Ocaml, even C++) and sort of balm myself. But now, I can't even overcome the sinking feeling as I read the Julia statistics book that I'm going to be dragged back to Python kicking and screaming in the morning, so why even bother?
And frustratingly: it's one of the few languages with decent linear algebra libraries. And that means it's one of the few languages with good ML and statistics support. So it's very hard not to use it because when you want to cobble together something like a Bayesian model things like PyMC or Edward actually give you performance that's obnoxiously difficult to reproduce.
This is what the industry wants and evidently a lot of people are okay with it, but to me it's misery and I can't work out why people seem to like it so much.
I hate it.
It has an ok object system which is possibly its only redeeming quality. I found Racket about 4 years ago, and any new project that I work on will be in Racket or CL.
I could go on at length about the happy path mentality of python and the apologists who are too ignorant of other people's use cases to acknowledge that maybe there might be some shortcomings.
The syntactic affordances are awful and you can footgun yourself in stupidily simple ways when refactoring code between scopes. Python isn't really one language it is more like 4, one for each scope and type of assignment or argument passing, and all the sytactic sugar just makes it worse.
Not to mention that packaging is still braindead. I actually learned the Gentoo ebuild system because pip was so unspeakably awful the moment you stepped away from the happy path.
Blub blub blub, someone help I'm drowning in mediocrity.
I think you need to look at it historically. Against other languages circa 2000, Python was a huge win. Easy to write (compared to C), consistent (compared to PHP), minimalist (compared to Perl), multi-paradigm and playing well with C (compared to Java), with a great standard library (compared to Lisp).
Today, the landscape is very different. Lot of languages took hints from and got influenced by Python (including Racket I am sure). Functional languages have taken off. Type inference is a standard feature.
History is always changing. There is some potential for a next Python, but we don't know what it will look like yet. I suspect it will be functional, but I don't think it will be in the Lisp family. It probably won't happen until after Rust and/or Julia get lot more adopted. Anyway, just like C, Python will be with us for decades to come, for better or worse.
What I am missing from Perl is library documentation and MetaCPAN. It seems that the go-to solution is that you're meant to create some kind of damn mini-site for any even vaguely complicated project you do using Sphinx, which seems bizarro land. Also the _requests_ documentation looks like it was written by Wes Anderson and wtf I hate it. Also I hate that all libraries have cute rather than descriptive names; yes there's some of that in Perl, but it feels like less. Bah humbug. Other than that it's fine.
And packages. I thought I was just stupid, but thankfully there was more to it.
Can you be more specific about how it increases your burnout? Is it the language, or someone forcing you to use that linter and settings?
It's definitely both.
Preface in true internet style: these are just opinions and you may not share them. That's fine.
I really don't like Python as a language. I don't like its total lack of composability. I don't like its over-reliance on a very dated vision of OO. I don't like how its list comprehensions aren't generic. I don't like how it farms all its data structures out to C. I don't like how it uses cruddy one line lambdas and forces me to name things that are already named by the things I'm passing it to.
And also the linter just exacerbates these things, because the linter is just a crystallized slice of a kind of software engineering culture I really don't like.
I believe Python is quite possibly the best language for a few things
* Exploratory programming - such as what data scientists do
* Writing programs that will never grow beyond 150LOC, or roughly what fits on a screen + one page down
When I have those two constraints met I am almost always choosing Python.
Here are some problems I face on codebases as they scale up:
* Python conventions lead to what I consider bad code. Devs will often choose things like 'patch' over dependency injection, and I have seen on multiple occasions projects derided for providing DI based interfaces - "oh, why would you write code that looks like Java? This is Python".
There's a lot of death by a thousand cuts. Keyword args in functions are abused a lot, because they're "easy", ability to patch at runtime means it's often "easy" to just write what looks like a simple interface, but it's then harder to test. Inheritance is "easy" and often leads to complex code where behaviors are very, very non-local.
Dynamic types mean that people are often a little too clever with their data. I've seen APIs that return effectively `Union[NoneType, Item, List[Item]]` for absolutely no semantic reason, without type annotations, meaning that if you assumed a list or none came back you were missing the single Item case. The implementation actually internally always got a list back from the underlying query but decided to special case the single item case... why? I see this sort of thing a bit, and other languages punish you for it (by forcing you to express the more complex type).
* I find myself more and more leveraging threads these days. I did this often with C++, and all the time in my Rust code. Python punishes you for threading. The GIL makes things feel atomic when they aren't, reduces you to concurrency, all while paying the cost of an OS thread. Threading primitives are also quite weak, imo, and multiprocessing is a dead end.
And, really, Python is anti-optimization, which is a good and bad thing, but it's a bit extreme.
* Implicit exceptions everywhere. I see 'raise Exception' defensively placed in every Python codebase because you never know when a piece of code will fail. I see a lot of reliance on 'retry decorators' that just catch everything because you never know if an error is transient or persistent.
The common "don't use exceptions for control flow" is broken right off the bat with StopIteration. I just think error handling in Python is god awful, really.
* Mypy is awesome, but feels like a bit of a hack. The type system is great, and in theory it would solve many problems, but coverage is quite awful in open source projects and the type errors I get are useless. I actually only use mypy to make my IDE behave, I don't run it myself, because my types are actually technically unsound and the errors are too painful to work with (and honestly the type system is quite complex and hard to work with, not to mention the bugs).
There are lots of other smaller issues, such as my distaste for pip, py2/3 breakage, lack of RAII, the insane reliance on C all over the place, I think syntax is bad in a lot of places (lambdas, whitespace in general), etc but these are probably my main sticking points with using the language for large projects.
Again, Python is actually my favorite language for specific tasks, but I really think it's best optimized for small codebases.
1) No pattern matching. In 2019, this is just unacceptable. Pattern matching is so fundamental to good FP style that without it, it becomes almost impossible to write good and clean code.
2) Statement vs expression distinction. There’s no need for this and it just crippled the language. Why can’t I use a conditional inside an assignment? Why can’t I nest conditions inside other functions? It makes no sense and is stupid and inelegant.
3) Related to 2), why do I need to use a return statement? Just return the last expression like many other (better) languages
4) Bad and limited data structures. In Python all you get are arrays, hashmaps, sets. And only sets have an immutable version. This is unacceptable. Python claims to be “batteries included” but if you look at the Racket standard library it has like 20x more stuff and it’s all 100x better designed. In Scala you get in the standard library support for Lists, Stacks, Queues, TreeMaps, TreeSets, LinearMaps, LinearSet, Vectors, ListBuffers, etc.
5) Embarrassing performance. Python is so slow it’s shameful. I wrote a compiler and some interpreters in college and I honestly think I could create a similar language 10x faster than Python. Sometimes you need to trade off performance and power, but that’s not even the case with Python: it’s an order of magnitude slower than other interpreted languages (like Racket).
6) Missing or inadequate FP constructs. Why are lambdas different from a normal function? Why are they so crippled? Why do they have a different conditional syntax? The only sort of FP stuff Python has is reduce/filter/map. What about flatMap, scanr, scanl, foldl, foldr? Or why doesn’t Python have flatten? All of these are very useful and common operations and Python just makes everyone write the same code over and over again.
7) No monads. Monads can be used for exceptions, futures, lists, and more. Having to manually write some try catch thing is unseemly and worse than the monadic Haskell or Scala approach.
8) No threads and no real single process concurrency. Despite Python being used a lot, no one really seems to care about it. How can such a problem not be solved after over 20 years? It’s shameful and makes me wonder about the skill of Guido. There’s no reason why Python couldn’t have turned into something beautiful like Racket, but instead it has been held back by this grumpy old guy who is stuck in the past.
9) Others might not have a problem with this, but I detest Python’s anything can happen dynamic typing. It makes reasoning about code difficult and it makes editor introspections almost impossible. If I can’t know the exact type of every variable and the methods attached to it, it hampers my thinking a lot. I use Python for data science and if I could just have a language that was compiled and had static typing I would be 3x as productive.
Let me conclude by saying there currently is one good reason to use Python: if the domain is ML/DL/quant/data science Python is still the undisputed king. The libraries for Python are world class: scipy, sklearn, pandas, cvxpy, pytorch, Kerala, etc.
But Julia is catching up very fast and the people I have talked to are getting ready to ditch Python in 2-3 years for Julia. I don’t think I’ve encountered anyone who didn’t prefer Julia to Python.
I don't know why 80 characters is a problem. I don't use the linter but I enforce this rule religiously with a couple of exceptions (long strings comes to mind). It forces me to think heavily about the structure of the code I'm writing. If I'm nesting so deeply, something has gone wrong. If I've got a ton of chained methods or really lengthy variables, it forces me to rethink them.
This also has the advantage of being able to put 4 files next to each other with full visibility. Vertical space is infinite, horizontal space isn't. It's probably a good idea to use it.
That said, we usually just go with autoformatting via black, which is 120 by default. No more hassle manually formatting code to be pep8-compliant. Just have black run as a commit hook, which is super easy via pre-commit [0]. And you can run the pre-commit checks during CI to catch situations where somebody forgot to install the hooks or discovered `--no-verify`.
Can't really imagine developing Python without Black any more.
[0] https://pre-commit.com/
Same deal when it comes to reviewing PRs in GitHub. Wrapping just interrupts flow for me.
I feel the complete opposite. I really enjoy working with python over any other language. R does linear models and time series better and matlab has its charm, but overall I prefer python. Python is so easy to read and quick to program in. I am so glad I am not in the Java/C++ world anymore, but I know people in different roles have to deal with different issues.
I assume you mean, "over any other language I have tried" ?
As someone with a mathematical background myself, I am always surprised at how many data scientists and quants are ignoring more mathematically principled languages like F#, OCaml and Haskell.
If there's one thing wrong with our profession is a lack of ethics and accreditation - we're essentially letting random people build critical infrastructure and utilities.
We don't have a tooling problem, in fact we have too many tools.
I see so many people (especially on HN) fixating on tools, dissecting programming languages, editors and libraries into their most minute parts and then inevitably finding faults in them, disavowing them and swearing that other tools X, Y and Z are purer and worthier.
If you want to stop hating software and improve your burn out, stop caring about irrelevant stuff.
https://ncees.org/ncees-discontinuing-pe-software-engineerin...
There was a license for a professional software engineer; they're discontinuing it, apparently due to lack of demand.
If folks wanted to get a "software people taking the FE" study group together, I'd join up.
Ah yes. Just remove the part of myself that got me into software as a child, and proceed robotically.
Why didn't I think of that?
Spot on. Leftpad (and perhaps the whole js ecosystem) are good examples.
Alternatively, if you believe the differences between languages are irrelevant why do you not program everything in assembler?
Yes I agree that if you’re using Python for a large scale project involving lots of developers its not the best; but that’s because it doesn’t have a good type system.
You can’t work out why people like it so much because of this misconception. The languages that you gave as examples most definitely do _not_ value your productivity, it values correctness as enforced by a type system and refactoring needed for large projects.
For a lone hacker, its the other way around. Compare e.g. to Golang's "programming at scale" rationale.
Deleted Comment
The way I think about it is that Python is a strong local optimum in a space that has a massively better optimal solution really close by. But it's nearly impossible to get most people's algorithms to find the real optimum because Python's suboptimal solution is "good enough". And the whole software industry (and in some ways, by extension, all of humanity ... to be over melodramatic) is suffering for it.
I don't think the biggest services built with Python (think Instagram, Dropbox, etc.) have more consumer-facing issues than services written in other languages.
If you're talking only about developers, fine, however I also think most Python developers like the language. For me it seems that Python has strong vocal critics, that show well in places like HN, however it is not representative of the majority.
So I really don't think Python is making the humanity suffer, for any good measure of suffering.
Things like max line length should be something your org or your fellow contributors decide on, not something dictated to you no matter what the language.
It seems you are really complaining about having to write code which is maintainable by others than yourself?
A corporate point of view is than any programmer should be interchangeable with the minimum amount of fuss. I understand why someone building an injury organization has a responsibility to think about the future.
But I confess that sometimes I feel very demoralized when an organization implicitly tells me that my years is study and my ongoing self-education and practice is all meanless Because in principle someone fresh out of college should be able to take over my project with the two week handoff and a few docs.
Pylint and mypy and black are way more useful.
Maybe there are good alternatives but Flake8 starts from PEP8 until you tell it otherwise.
if i had to write python as a day job, i would quit. i have said it before, but python is the new c++, helping people everywhere write brittle, unmaintainable code.
That said, this shouldn't be a lint, it should just be enforced by a formatting tool as a format-on-save setting. It just destroys all the wasted arguments about formatting and the wasted time trying to manually line up code.
I'm all for decreasing unnecessary cognitive load, there should be quite enough of that without us adding more by accident.
I can imagine other people's code in Python frustrating myself, but if it is only my own code base then Python is a rewarding language for me.
Gradually pretty much everything has been rewritten in C++ (with Qt GUI framework). Way easier to be able to have a setup program (using free InnoSetup) that just extracts the eight or so DLLs required in the same folder as the EXE and that's it.
We just use Python for a bit of prototyping and data crunching here and there now.
You can do the same with Python.
use an auto-linter - black is what most seem to be using
and stop using flake8/pylint directly, try prospector with sensible --sensitivity
Yes, I know I'm in the minority these days. :/
It is a language, better than MatLab/R/SPSS, which come before it.
I honestly don't think it is that bad. And they are many people don't care from a programming language perspective, they need to just finish the functionality.
I don't think so. I think R is a lot more expressive and not really any harder to read. It might have a steeper learning curve, but it's not so bad that I think that actually matters.
The surveys never tell me why though. What do people like or dislike about python? I know it has a lot of libraries people love (scikit-learn, tensorflow come to mind)
[1] https://insights.stackoverflow.com/survey/2018/#most-loved-d...
Its not thir fault tho, you can only do so much in limited time, thats why expertize requires years.
In my previous startup in India, I trained unskilled personnel to become decent python developers to work on our product; everything was fine till the product grew exponentially and we had to optimise every nook and corner of it to better serve the customers and save on server bills.
So we had to optimise our application with Cython to meet the demands. So, when training someone to code if we use Python as a language; we should follow up with the disclaimer "You will learn to code faster than most other languages, but you cannot use this knowledge to build something at scale (i.e at-least when budget is of concern, when you are not instagram)".
In comparison, Golang excels in both form and function. It is just as easy to teach/learn as python and doesn't need the aforementioned disclaimer. Web services are a breeze to write and is built for scale.
I understand that there are criticisms against the language design of Go, some are valid and most are just because it was made by Google but none questioning the scalability of Go applications.
But at least Go is a lot faster and has real concurrency AND parallelism, so it's definitely better than Python.
> why people seem to like it so much?
> cobble together something like a Bayesian model things like PyMC or Edward actually give you performance that's obnoxiously difficult to reproduce
And this, across many domains
So change the limit or disable that check. If someone is keeping you from doing that they're the one who's insisting on 80 characters, not the language. Who uses any linter without making some changes to its default settings?
I've yet to encounter a python linter where you can't pick and choose which rules to ignore. This is a first one to go. Annoying PEP for sure, but https://pypi.org/project/black/ almost completely eliminates your issue.
Deleted Comment
It's a very sensible standard. There's a reason most books are taller than they are wide. The fact that we have bigger screens now doesn't change it.
Deleted Comment
This comment is more highly rated than some links I've submitted that made it to the front page.
Deleted Comment
In web dev andam data science I have yet to see a language with as many libraries for useful stuff.
Now, if what you're looking for is high performance and precise memory management, sure, the language will never give you that.
What language has as many easily available libraries? Java, .net and npm come to mind as the only comparable ecosystems.
I don't know if you are just trolling, but this is the silliest, most detached from reality thing I've read online today. Python was key in building numerous massive public companies like Instagram and Dropbox. It's one of the most popular and widely used programming languages on the planet for everything from APIs to desktop clients to data pipelines. It had a lot of early popularity in the Silicon Valley start-up scene in the early 2000s, even pre-dating the Ruby on Rails web dev trend.
The guy who founded the company that runs this very website wrote about the draw of Python 15 years ago [1] at which point it was already widely used in certain niches. This is before any deep learning libraries existed. I remember first playing with Python around 1999 or so.
> I would use it at most as an API exposing ML but that's pretty much it.
I don't know how old you are or how long you've been in the industry, but the ML thing is a "second act" for Python. Deep learning grew up in a time and place where Python was a good fit which put Python in the right place to benefit. But Python had already lived a long and full life before any of that happened.
It's fine if you don't like Python or don't think it is a good fit for a project, but claiming it is a "hobby language" with a "lack of tooling and a relatively small collection of libs" is a good way to get laughed out of the room. It has one of the largest and most diverse libraries. And as far as tooling, Python is one of the most popular languages for implementing tooling. Check out ansible.
> There are plenty of packages for data science. For building APIs and web stuff usually there aren't. Python is nowhere near NodeJS for instance.
This has got to be a troll, right? Just some of the most popular web frameworks: Django, TurboGears, web2py, Pylons, Zope, Bottle, CherryPy, Flask, Hug, Pyramid, Sanic... Lots of huge websites were originally built with Python like Instagram, Reddit and YouTube. Of course they mature into complex distributed architectures using all kinds of languages, but it's all about the right tool for the job.
[1] http://www.paulgraham.com/pypar.html
I wish pip worked the same way as npm: -g flag installs it globally, otherwise it creates a local "python_modules" folder I can delete at any time. And optionally I can save the dependency versioning info to some package.json...
Instead, pip is a nightmarish experience where it fails half the time and I have no idea where anything is being installed to and I'm not sure if I'm supposed to use sudo or not and I'm not sure if I'm supposed to use pip or pip3, etc.
1. Don't install anything globally. Don't pip install --user, definitely don't sudo pip install.
2. For each project you want to work on, create a venv. Yes, there are tools for this, but the base venv tool is totally fine. (venv should be included in your Python, but a few distributors like Debian put it in a separate package - install it from them if needed.) Use python3 -m venv ~/some/directory to create a venv. From here on out
3. As a first step, upgrade pip: ~/some/directory/bin/pip install -U pip.
4. Install things with ~/some/directory/bin/pip install.
5. Run Python with ~/some/directory/bin/python.
Slightly advanced move: make a requirements.txt file (you can use .../bin/pip freeze as a starting point) and use .../pip install -r requirements.txt. That way, if you get any sort of package resolution error, you can just delete your venv and make a new one. (Downloads are cached, so this isn't super annoying to do.)
A "project" can either be actual Python development, or just a place to install some Python programs and run them out of the resulting bin/.
(Edit: Yes, I know about activate, see the replies below for why I don't recommend it. With these rules, you get to say "Never ever type pip, only .../bin/pip", which is a good safety measure.)
If I want to start a Python project I have to set up venv and remember to put relative paths in front of every command or else it'll run the system version. Sounds simple, but it's still something to always remember.
You won't be able to install with pip unless you are in a virtualenv.
And you don't need to keep typing out the full directory name. When done with the environment
$ deactivate
Python 3 comes with it as "python -m venv". Once in the virtualenv, you don't have to worry about the various pip forms and effects, you can just pio install.
You can get fancier than that of course, but that's what works with stock python, on all OS.
It really isn’t that difficult, it’s just different. Different always seems wrong, at first.
This applies to many packages embedding C libraries, including numpy, hdf5 and all the rest. There has been minimal thought put into binary compatibility here, meaning that it's a lottery if it works today, or will break in the future.
As someone who has to look after a bunch of servers used by researchers, conda is the biggest cause of things failing for everyone.
conda installs stuff all over the place and its very easy to install conflicting stuff in the same place.
I just wish that either venv was automatic, or at least the second thing you learn in python.
Deleted Comment
I think everybody should spend their first couple of years working in Fortran IV on IBM TSO/ISPF. No dependency management because you had to write everything yourself. Or maybe [edit: early 90's] C or C++ development where dependency management meant getting packages off a Usenet archive, uudecoding and compiling them yourself after tweaking the configure script.
I'm not saying Python is perfect, but if it's causing your burnout/destroying your love of programming/ruining software development you seriously need some perspective.
I just returned to Python for the first time in a little while to collaborate on a side project and ran into a few tricky-to-debug errors that caused a fair bit of lost time. Know what the errors were?
In one case, I was iterating over the contents of what was supposed to be a list, but in some rare circumstances could instead be a string. Instead of throwing a type error, Python happily went along with it, and iterated over each character in the string individually. This threw a wrench into the works because the list being iterated over was patterns, and when you apply a single character as a pattern you of course match much more than you're expecting.
And another one, I typoed a variable name as groups_keys instead of group_keys (or vice-versa, I don't remember). Instead of just throwing an error, Python happily went along with it, used an uninitialized value, and then all the logic broke.
There's entire classes of errors you can cause yourself in Python that aren't possible in stronger, statically-typed languages. For a large project, I'd pick the old and boring Java over Python every time.
> And another one, I typoed a variable name as groups_keys instead of group_keys (or vice-versa, I don't remember). Instead of just throwing an error, Python happily went along with it, used an uninitialized value, and then all the logic broke.
This isn't what Python would do, if the variable was undefined Python will throw an error, so you must have defined it with this name or you're misremembering what have happened.
Right off the top of my head, using type hints with a decent IDE or an assert statement would likely have caught the issue.
I'm not saying that Python doesn't have issues (all languages do), but I don't see the error noted above as any sort of deal breaker. On the other hand, if you're only ever going to use Python like a strongly typed language without taking any advantage of its dynamic characteristics, then I can see why it would seem as a total downgrade compared to languages like Java.
I've been using python for about 13 years professionally and I wrote up a list of "things I wish python would fix but I think probably never will" and treating strings as iterable lists of characters was on there.
I've seen this bug multiple times and the fix is relatively easy - just to make strings by default non-iterable and use "string.chars" (or something) if you really want to iterate through the chars.
Nonetheless, I still love the language and wouldn't use anything else.
>Instead of just throwing an error, Python happily went along with it, used an uninitialized value, and then all the logic broke.
This one gets caught by linters - unfortunately 90% of most python linters spit out rule violations which aren't important which drowns out stuff like this in the noise.
I remember a bug I made using C#, where I wanted to delete a collection of directories recursively. I got mixed up into the various inner loops and ended up iterating over the characters like you. But C# allows implicit conversions of char to string so the compiler was OK with it, and since those where network drive directories (starting with "\\server\"), the first iteration started deleting recursively the directory "\", which in windows means the root directory of the active drive (c:\)... And SSDs are fast at deleting stuff.
This actually ruined type annotation in Python: you can't properly annotate a function to accept anything iterable except string.
https://github.com/python/mypy/issues/4334 is a FR to at least check it with mypy.
Python doesn't have uninitialized values, it throws NameError when you try to access a variable that hasn't been set. So I don't see how this could have happened.
Python code needs more testing, more run time type checking of function arguments than a statically typed language. If that's a deal-breaker then one shouldn't be using Python in the first place. What you gain though is some instant gratification, and the ability to get something off the ground quickly without spending time placating the type checker. Its great where your workflow involves lot of prototyping, exploration of the solution space and interactive use (ML comes to mind, but even there int32 vs int64 can byte, correction, bite). I see it as a trade off -- deferring one kind of work (ensuring type safety) over another. Hopefully that deferral is not forever. I like my type safety but sometimes I want that later.
What I typically do is once I am happy with a module and I do not need the extreme form of dynamism that Python offers (something that's frequently true) I take away that dynamism by compiling those parts with Cython.
It could be javascript.
The creator of a well-known alternative to Python has a single-letter email address, and regularly receives email generated by Python scripts with this exact bug (which means instead of sending an email to "user", sends an email to "u", "s", "e", and "r"). So I’ve heard.
(And nowadays, you can also use type hints to give you a warning for this kind of thing, e.g. your IDE/mypy will complain about passing a string where the function signature specified a List.)
There is no such thing in Python. You should get NameError if a name doesn't refer to any object.
Deleted Comment
It's a first language I worked with in my life that just clicked with my brain and doesn't just drain me.
I would take a Python job over a Java/C/C++/Go/Rust any day. There's some languages that could pull me away from Python (Nim, Crystal) but they're nowhere popular enough to move wholesale to them.
it's funny, I feel the exact opposite. I work on a team that maintains a digital catalog, and a lot of what we write is about taking in asset- and metadata files, asynchronously processing them, and then publishing that to a denormalized read-optimized data store. We often joke that we mostly take data from 'over here' and put it 'over there'.
All our stuff is in Java, and honestly, if you use Lombok to squeeze out the boilerplate, and a decent dependency injection framework like Guice or Dagger, modern Java really isn't so bad. Streams are clunky but they get the job done. We use Jackson a lot to serialize/deserialize Java pojos to JSON and XML, which is pretty seamless for us so far. The Optional class is again clunky, but it works well enough.
The thing for us though is that the problems we spend most time solving are just not really related to the language we write it in. The hard problems are much more around things like operations (cd/ci, metrics, alarms, canaries), performance (latency, load, etc.) and just the nuts and bolts of the business logic (what type should this new field be? what values can it take? how do we plumb this through to downstream system X owned by team Y? etc.)
I honestly wouldn't want to have to write this stuff in Python for a simple reason: I don't think I could live without static typing, which is a fantastic tool when you need to manage a large code base written by multiple people over multiple years. I can make a change in some package, do a dry-run compile of every system that uses it, and then see what needs updating. It gives me certain guarantees about data integrity right at compile time, which is super helpful when you're doing data conversion.
But hey, different jobs, different tools. Glad you found something you're happy with.
That's kind of my point, you may like other environments better, such as React/Node/NPM but that doesn't mean Python is a horror show.
I'm quite enjoying Go though.
At the same time even when I used it heavily I never saw it as anything more than a scripting language to sit in front of some tool that was written in a language I couldn't be fucked to learn at that moment (numpy and scipy were used heavily throughout my college career).
If I'm being honest I don't understand how anyone could get as worked up about a language as the people in this thread have. At the end of the day most of us are still writing unportable imperative code that runs like shit. Maybe blaming language is how we cope with our own failure as engineers.
Why do you group those languages like they’re similar but different from nim and crystal? They’re wildly different in terms of their target domain, runtime models, etc. Go and Java are general purpose application languages and the others are more suited for systems or performance critical applications.
Are you assuming everyone complaining here is young, and this is their first language? Consider that maybe they're complaining because they've used older languages they liked more.
Often, not having a feature is preferable to having a feature designed or implemented poorly.
That wasn't my point. My point is if you had to write Fortran IV using an IBM 32xx terminal you wouldn't be quite so hyperbolic about modern Python.
Unless you are claiming you would rather return to writing Fortran IV than use Python because you like Fortran IV better, in which case I'm very confused.
If you had to write Python, would you also want to quit software engineering? Would you go back to Fortran instead of Python?
Of course you're allowed to hate Python but someone saying "every time I write python I want to quit software" is either extreme hyperbole, some tangentially related issue like depression, or just no language at all would make them happy enough.
Python 1.4 was an awesomely simple programming environment and I pretty much immediately fell in love with it. Then features were added. Now it is a whole home improvement store full of kitchen sinks.
I think that programming is a sort of theological process. Popular languages attract ideas. Unfortunately, in the case of Python, those ideas were not effectively filtered and now we have an expression of as many ideas as can possibly fit. The ultimate design by committee...
I suspect that the recent excitement about assignment expressions is really a kind of straw that broke the camels back. The problem isn't just this one feature, it's the sum of them.
It's funny, a lot of people hate on Elm for the exact opposite reasons: one person dictating the language's direction and removing features. I suppose a nice balance could be struck between the two ends of the spectrum.
I've used Python at the time (and up to now). It was a revelation compared to Perl, but it sucked compared to modern Python.
What exact features you have a problem with?
And of all the bugs I have written in recent memory, not one came down to a lack of static typing. They were due simply to logic errors, flawed assumptions, misunderstood requirements, and good old race conditions. The static typing zealots like to think if it compiles is must be perfect, however this is a mirage. Unit tests in Python can compensate quite well for lack of static typing.
I use Python 3 these days for a lot of stuff. It's pretty good.
These are better days. We all have complaints, but on the whole, things are not too bad.
I think for the most part that when nothing meets your expectations, it may be that your expectations need to be adjusted.
The winning thing python had that nothing else had at the time was it was social, it was readable, and there was generally only one way to do things: the right way. It no longer has the latter quality, and the preferred coding style in it seems to be java-ish OO-spaghetti, but it's still pretty good.
That said, these days, I resent every damn time I have to use it. It's eating data science, more or less because pandas and scikit is ... mostly good enough, and because unlike R it's .... mostly good enough to deploy in an enterprise application. But if you're working on the exploratory side of data science, Python is shit compared to R. Doesn't have the tooling, doesn't have native facilities, and is vastly more long winded. All the attempts to make Python more ... X ... are probably a mistake also. You're taking a beautifully simple tool and making it more exotic and complex. It's like trying to use Matlab to build webservers.
I can't help but to gently interject here. By 1997, I'd been programming for some time, and there certainly were people who consider Perl suitable for programming in the large, and there were certainly big projects so written.
While I disagree with the specific word 'nobody', I agree with the sentiment: Perl was widely considered to be only useful for 'small' things at the time. Widely, but not exclusively.
> There are two kind of languages: the ones everybody hates, and the ones nobody uses.
"There are only two kinds of languages: the ones people complain about and the ones nobody uses."
Bjarne Stroustrup's FAQ: Did you really say that?. Retrieved on 2007-11-15.
I started to use Python quite recently and I really like it. It is a well-designed language with high-level abstractions that are really fun to use. I like the pervasive use of iterators, the 'everything is an object' philosophy, the minimalist syntax, the build-in datatypes...
That being said, I feel that the dynamic types show their limits when projects getter big. I use linters and static type annotations but I find refactoring very error-prone and there's a point where I don't really trust my programs.
—Richard W. Hamming[0]
I have rarely "chosen" to use Python at work, but it has never failed to get the job done.
[0] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2041981/
e.g. "Rule 1: Drop Modesty", "Rule 7: Believe and Doubt Your Hypothesis at the Same Time".
[0] - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2041981/
It's very easy to persuade the customers of your language that everything that goes wrong is their fault and not yours.
- Tony Hoare
Not to say I have any real complaints about Python.
As if the only way to gauge quality is to forever compare to the technologies of the past. Python was a great improvement on its peers in its heyday, but people, technologies and philosophies have changed since its inception and there is nothing wrong with wanting something more. That's how progress is made.
Instead of evolving into something okay, Python is pretty much the same broken language as it’s always been, run by a guy who is openly hostile to PL theory, FP, and any major changes to Python whatsoever.
If you ever need/want to use Python, do yourself a favor and use Racket instead. Racket is better than Python in every single way (unless you are doing data science with Python, in which case fine keep using it).
Probably Go will be the next hotness in 5 years.
I think it will be difficult to grow a large ecosystem for a language with very poor FFI performance [0] in the long run. Golang's poor FFI performance is the number 1 reason I wouldn't use it for my own projects.
[0]: https://github.com/dyu/ffi-overhead
Deleted Comment
Deleted Comment
That's not to say that Python doesn't have it's place. But I see it more as a programming language for small utilities no more then 2k loc in length.
Deleted Comment
Interactive python and Jupiter notebooks are an absolute joy to work with though.
Oooh look, there's a hammer over there with a modern handle. I want that. I don't like wood anymore.
You were saying about C/C++ package management?
It is a step forward, sure, but it's a far cry from Python's pip or Rust's Cargo or even Dlang's dub.
- First indentation was a issue for me but I looked past it and went ahead to give another go at Python.
- Even the best in class IDE suffer to give any kind of insight into python code.
- Two versions: At my workplace we use Python 2. I prefer to learn newer version but don't have choice. When Python 3 came out they should have initiated deprecation of Python 2 but that does not seem to be the case. Tonnes of libraries are still in Python 2. They should have maintain language backward compatible instead of fragmenting the entire community.
- So many ways to do a thing. This is subjective but I need a language which gives predictable performance for given piece of code. Go does this. There's usually one way of doing things and only one way. No need to know nooks and crannies of the language nor there is such a thing in Go
You’re probably aware of this, but at the odd chance you aren’t, Python 2 end of life is 2020. You really should be moving to Python 3.
> Go does this.
There is a lot of love for Go and Rust on HN, but unless your region of the world is significantly different than mine, then chances are that there won’t be a Go or Rust job in your lifetime. I’ve only ever seen one of them mentioned in a job opening for my entire country, and that was as “nice to know” for a c++ job at Google.
I’m sure Rust and Go are truly excellent languages, but that doesn’t really matter for most people, if they never manage to see any adoption outside of Silicon Valley.
I’d say deprecation has been initiated.
I'm talking about the comments saying it was causing their burnout "x100" and other such hyperbolic statements.
Having taught both python and JavaScript, I can tell you that the former is far far less confusing to newcomers than the latter. Imagine explaining the problem with equality comparison or prototypes in JS to someone who just learnt what an IF statement is.
The reason I agree that python will dominate programming for decades to come is the batteries included and pythonic approach that enables people to do cool things with just a few lines of code. As a result of the above, the ecosystem has reached critical mass. For most tasks there’s a library that does that, and it’s good, and it’s easy to use, and the developer of the library usually doesn’t expect you to understand their opinionated philosophy to use it.
I love the python convention of putting a minimal example on the homepage of a library.
You’re not going to get your average developer to appreciate the beauty of Lisp or the value of static typing. They want their ML to work, or their graphics to display, that’s all. Ends not means, almost entirely, for most people who aren’t full-time programmers.
Let’s not hate on a language that opens up programming to the world, unless we want to be gatekeepers. If you want to maintain an air of superiority, just learn Lisp or Haskell or C++ and write code that’s “deeper” into the tech labyrinth. Programming will probably specialise into “end user programming” and “Service and system programmers”. Embrace the democratisation of coding.
Since most of my professional experience is with C like syntax languages, I don't get how the lack of braces is an advantages, quite the opposite in my limited experience with Python.
Braces define scope unequivocally, they are easy to visually parse and don't care whether you are using tabs or spaces or even if you, loud gasp, mix them. Furthermore, you can copy and paste, say for loop from a method to another method and it will work. In Python you might have to faff about with spaces (I'm sure there must be IDEs that solve this problem but it was no fun on vim or even notepad++)
I also don't like the fact it is dynamically typed. It makes reading soure code so much harder. E.g What is the type of this thing that is being passed as a function arg?
So does whitespace, otherwise Python wouldn’t work as a programming language
> they are easy to visually parse
only if you indent properly
> Furthermore, you can copy and paste, say for loop from a method to another method and it will work. In Python you might have to faff about with spaces (I'm sure there must be IDEs that solve this problem but it was no fun on vim or even notepad++)
Shift+V
}
< or >
Hit . as often as needed
Too difficult?
Not to mention you will run into exactly the same problem when copy pasting C unless you like randomly indented code
And for many who are. Maybe not on our main projects, but at least for helper tools, side projects etc.
SLOW. The sin of all. I don't want to make other arguments with people who just come to repeat 'Oh you can make it fast, you are doing it the wrong way'. Sorry, I have heard that 100 times, now is sounds like bug-as-feature excuse to me.
Sorry, but Python is just slow. All the stuff people might bring up is just trying everything to bypass Python itself. Multiprocessing is hell to work with, buggy and unpredictable. Cython is OK but it is not Python.
Python's slowness is like cancer, and terminally so. Down deep, it is because its overtly flexible/loose type system, the never-going-away GIL, and prematurely exposing of C-API makes it almost impossible to optimize without breaking the existing compatibility.
And most importantly, I don't think it is solvable, truth is the patches are so integrated, people already forget what problem they tried to fix in the first place and just live with it.
But it will haunt Python in long term though, people will be struggling with Python in performant production until they start reimagining a world that without Python. Maybe a better Python in that sense.
I've architected reliable systems on top of it but it was obvious to me almost from the get-go that Python is a terrible language. I credit my previous years of programming in Lisp that trained my mind to see through the Python mirage.
One issue is that veneer of Python user friendliness that lures people, usually those who are new to programming or don't have extensive experience with _multiple_ languages, in.
The major issue for me is that Python lacks internal consistency and is based on a inverted-design (meaning, no design really) that accumulated over the years and stinks. This makes it really easy for people to use Python and write unmaintenable code but also very hard for them to find out how to write good code. The hallmark of a terrible language is making good code hard to write.
As far as specifics, I think the module system is a joke, the packaging even worse, the REPL is not really a REPL (in the Lisp sense), ctypes is bad and has 100 different ways you can hang yourself with, there are tons of gotchas in the language and standard library that if you haven't memorized [usually by running into them], they will bite you hard, the GIL ties my hands, iterator/generator obsession makes for really bad code messes if people take it seriously and start poorly designing their APIs around it, absence of proper lexical scope, statements are not expressions and led to gross hacks like PEP 572... I could keep going for days.
Python is really the PHP of the 2010 generation.
Since this forum is likely to have some folks looking towards the future, I found interesting a couple of talks by Python veteran Armin Ronacher [1, 2]. Watching that presentation makes me wonder whether I should think of this in the context of "Worse is better". That said, Python's runtime introspection is such a killer feature that it's difficult to explain how empowering it feels for someone who is relatively new to programming.
[1]: How Python was Shaped by leaky Internals -- https://www.youtube.com/watch?v=qCGofLIzX6g
[2]: A Python for Future Generations -- https://www.youtube.com/watch?v=IeSu_odkI5I