I've grow to highly respect the Django project for its good documentation, its healthy consideration for backwards compatibility, security, steady improvements and all round goodness.
Interestingly, I have the exact opposite view on Django.
I hate their API and overall architecture, which I find to be the result of glueing features on top of features for many years.
The internal code also is just like that: looks like every single method is riddled with out-of-band conditionals, which is the result of a community that prefers to hack things to work, instead of rethinking/refactoring.
I'd prefer to hold judgement on the internals of Django, however as a developer using the API, I also think it's a pleasure to use. There is a very high level of consistency, similar patterns used throughout, and the architecture results in a project structure that I can explain to a new developer who has never used Django before in minutes. I find few frameworks really scale in terms of structure and consistency, and working on a large web application, Django has proven to be very valuable for us.
I'm curious, what's an example of a framework that you think has a better architecture, and in what context are you evaluating this?
Could you give a description of some subsystem that has been architectured in this way, and then provide a concrete example of some methods that implement this pattern and why it is bad?
We are users of Django, and it helps us to deliver projects, quickly. Interested to know how you think things could be improved.
Not sure why you hate it? It is one thing to say Django is bad at x and then specify x and other thing is to say I hate Django because it its internals are shit and shoot your points like a machine gun. The second approach is not going to be helpful to anyone.
I honestly think Django was a victim of it's own success. It started as a python (explicit is better than implicit) framework, but with a ruby-on-rails (convention over configuration) mentality. It became too successful too fast and it was too late to change things without breaking things. I remember it had all sorts of double import issues, a very limited (compared to jinja2) template engine and a very limited (compared to SQL Alchemy) ORM. It also coupled everything with everything, so it made it very hard to fix those issues. So as people grew into their projects, django got in the way.
Most things were hidden in the framework lazily loaded so when you had some weird issues you didn't know where to look. I found it hard to digest that I couln't write a simple script that accesses the database without having to go through the entire autodiscovery process, hence incurring a 2-3 second delay on every execution. We also stopped upgrading at some point, because by the time django started supporting our special needs (especially around user handling), we already hacked around it, and upgrading would have been near impossible. The core developers probably knew all of these and they did more and more "de-magic-ification" with every version, and things are much better now, but we are stuck on an old version.
In my opinion the user (where user is a web developer) facing layer of Django is among the best I've ever worked with. I will agree with you that the internals are very messy and hard to read and work with.
The overall architecture is sound though. Tight coupling of core components, loose coupling of non-core components. Keep in mind that it's an "opinionated" framework though, so they do safely assume that if you're using django you want to be using its core components as a bundle, and will work with them as designed.
I'd say if you hate it, you should propose the changes and how to refactor. There's only so much people can do and they certainly will benefit from first-time contributors who may have fresh ideas. Otherwise this is just plain complain and nothing else.
> its healthy consideration for backwards compatibility
Is this a joke? They break backwards compatibility with every minor version, wasting tens of thousands of man-hours all over the world - time that, if we're honest, is not exactly billable.
Most people don't upgrade because of this and keep on using vulnerable versions. It's good that they are trying to kill the project. It saves newbies from stepping into the tar pit.
For the most part, they think they things through a lot and most importantly they document breaking changes. Their approach is an absolute dream compared to, say, updating xcode/iOS apps. It's a total shit show at Apple.
Whilst your menacing tone is not really in tune with HN's guidelines (and you'd be wise to edit it and soften it up I think), I think this is an important point to discuss. I don't have many Django projects, but even I have experienced weird, sometimes intermittent API breakages on minor version bumps that I've had to spend hours debugging, only to find that Django upstream changed the way some function behaved.
I think the project is venerable and I have respect for it in general. It's one of the most accessible major open-source projects for newcomers; like I said, I'm not a heavy user and I've already submitted a couple of patches and had them merged. But issues like this one are worth pointing out.
Is there a deficiency in the test suite? Is the open and accepting nature of the community that I just praised the cause of this issue, because it leads to a bias against thorough vetting? I really don't know enough about the project to know, but it'd be nice if they figured it out.
It was pretty clear for multiple years that Django would eventually migrate to python3 only. Even regarding library support, I've started going python3-only with my Django projects at the beginning of 2015 and had practically zero problems, as only some esoteric packages were not supporting python3 (and then I wouldn't have used them for client projects anyway, because esoteric).
I still have one customer with a Django/python2 code base. When migrating to Django 1.11 in 2017, that project gets security updates until at least 2020. Plenty of time to gradually transition to py3. I honestly don't see the problem.
You're complaining about not being able to use the latest major version of a library when you refuse to change to the corresponding latest version of the language? So you want to stay cutting edge here but not there?
That's illogical and inconsistent. they're providing an LTS release for you. Get over it
Well it has been on the roadmap for around 3 years, and they will have support for Python 2 in active maintenance for the next ~3 years, so I think this is actually fantastic support from the core team.
This call has been made a while back, and it makes perfect sense. Python 2 is slowly being EOL'd and if you're starting a brand new Django project there's no reason on earth you should choose Python 2 anymore.
Sure legacy projects still need support and for that they get the 1.11 LTS, but otherwise it's really time to move on.
Is really not that hard to make C extension support Python 3, you're either using an extension that is no longer maintained, or its author(s) are stubborn, which is a different kind of problem.
Making code compatible with python2.7 onward means that you can't use any new features from 3+ and the code is plain ugly, I don't know if you wrote 2/3 compatible code, it is not as enjoyable to do.
I think Django's approach where the LTS will still work on 2.7 (and LTS is for 3 years, which is until python2 itself stop being maintained) is fair.
You still have 3 years to fix things and if you want to use latest Django and be cutting edge, you probably should use latest Python as well.
>Python 2 is slowly being EOL'd and if you're starting a brand new Django project there's no reason on earth you should choose Python 2 anymore.
Except that there are still some critical packages that aren't on Python 3 yet. Not to mention a lot of functionality breaks even if the libraries do exist, which means you have to code things up quite differently sometimes.
That argument has grown stale. If the devs of your favorite package are still too lazy to port to Py3 there are bigger issues with the existing code you should be concerned about. 2to3 fixes most everything other than a few incompatible changes to the stdlib API.
>> and if you're starting a brand new Django project there's no reason on earth you should choose Python 2 anymore.
> Why on earth will you go to Python 3 at huge rewriting costs?
If you're starting a brand new Django project you're not rewriting anything. If you're rewriting something, it's not really new.
My company is a Django shop and while we have no intention of upgrading Python 2.x codebases to 3, we do start all our new projects on Python 3. It just doesn't make sense not to do it.
Why do you have to rewrite all the code in your company if you are starting a brand new Django project? Seemingly the brand new means it is separate from currently existing projects.
I'm glad they are making a clean break from Python 2 and I hope this pushes other projects in the ecosystem to fix those remaining libraries without Python 3 support. It does get a bit frustrating when things break between Django releases, but they have a good system of deprecating things for a couple of releases beforehand. And at the end of the day, Django is for people who want to build websites, not life support machines... and I think they're doing a decent job of striking a balance between breakage and stagnation.
I have a Python 2.7 project that has been running smoothly for many years now and I'm having trouble finding a reason to upgrade to Python 3. The project uses the unicode type to represent all strings, and encodes/decodes as necessary (usually to UTF-8) when doing I/O. I haven't really had any of the Unicode handling problems that people seem to complain about in Python 2.
Can someone explain what benefit I would actually gain from upgrading to Python 3 if I'm already "handling Unicode properly" in Python 2? So far it still seems rather minimal at the moment, and the risk of breaking something during the upgrade process (either in my own code or in one of my dependencies) doesn't seem like it's worth the effort.
You gain access to continued language support in 2020. New features involving strings will have less risk of bugs. The "range" function is more memory efficient. Integer division automatically floors, reducing bug risk. Dictionaries with guaranteed ordering. Thousands separator in string formatting. Bit length on integers. Combinations with replacement on itertools. New, faster I/O library and faster json. Concurrent futures module. Ability to define stable ABI for extensions. New CLI option parsing module. Dictionary-based logging configuration. Index and count on ranges. Barrier synchronization for threads. Faster sorting via internal upgrade to Timsort. Async I/O. Support for spawn and forkserver in multiprocessing. Child context in multuprocessing. Has collision cost reduced. Significantly faster startup. Type hints. Faster directory traversal. Faster regular expression parsing. Faster I/O for bytes. Faster dumps. Reduced method memory usage through better caching. Dramatically less memory usage by random. Faster string manipulation. Faster propert calls. Formatted string literals (interpolated strings). Asynchronous generators. Asynchronous comprehensions.
How much faster might your code run just by upgrading to Python 3? How much memory might you save?
I don't think you're suppose to depend on the ordering of dictionaries. It's an implementation detail which might get changed, although it wont actually ever be changed because people will come to depend on it.
You forgot functools.lru_cache. I've found in some Python 2.7 projects you can speed things up with a hacky dictionary-cache, but the proper functools.lru_cache gives better results and is far more flexible.
Like many database-backed apps performance is currently limited by I/O, and memory is limited based on the size of the data sets I load (machine learning). A faster, lower memory Python does sound nice but I'm not sure how much effect it would actually have in real life.
What py3 does is it enforces the things that py2 only suggests as far as unicode goes. That means on py2 you can go for the proper handling and if you did do everything right, then py3 migration should be almost boring.
The difference is when you didn't handle all the cases correctly. In that situation Py2 will silently do either the right, or the wrong thing and you'll never know. Py3 will likely throw an extension and tell you where the issue is.
So if you're handling lots of unicode text and think you've handled all the string correctly already - that's the reason to move. Now you'll be sure it's correct. The problem with py2 wasn't so much that it was handling anything badly - it was that the default, easy way was incorrect and just waiting to blow up in peoples' faces. (or maybe just silently corrupt something)
I'm fairly sure I'm handling strings correctly under Python 2. What I'm not sure of is the risk of something breaking if I upgrade to Python 3, especially when it comes to upgrading external dependencies. Even if it's just non-backwards compatible API changes, it's more risk to deal with.
It has always seemed to me that Python 3 was mainly a fix on the philosophy of handling strings, but that it didn't offered a clear practical advantage for programmers already handling strings with care. I don't think there is a practical reason to upgrade to Python 3 in terms of language design. The reason will be in term of survival as the community seems to be willing to follow the Python 3 movement and official support for Python 2 ends in 2020.
There are a lot of reasons to upgrade! There is so much more useful stuff in python 3!
Even if you think you would not use those features, other libraries you may use might benefit a lot from it.
A few features: async/await, lists (and others) use iterators, no var leaking in list comprehensions, super().my_method() instead of super(MyClass, self).my_method(), class MyClass: instead of class MyClass(object):, improved exception handling, required arguments, ", ".join(["etc"]* 1000)
Besides that: a much improved standard library, although that technically not is language design.
That's pretty much true of any django release... you can stay on it if you so choose to. It will get security updates/bug fixes for a bit and then they will stop coming. You're free to stay where you are if you want.
It turns out that most developers have a desire to move to the next version if it's not too hard. There's still COBOL programmers out there too and that's perfectly fine.
Django has made the process as smooth as it can be. You can upgrade to python3 while maintaining your Django version. Then update to the next Django version as a separate step. It's fine to have waited until now. You can keep waiting if you want but it's getting to the point where you should really just do it. It's not so bad.
We switched and python2 --> python3 was bumpier and more work than most Django updates we've done (we've done pretty much every one since 1.0) but it was still entirely reasonable. We're much happier now.
Well, there are some new nifty things, e.g. not having to write u in front of every string or a more pleasant subprocess API or the fact that dicts in 3.6 keep their insertion order which makes debugging a bit easier. It adds up.
But the big one is that Python 2.7 will go away at some point.
Good. I've been getting into python a bit because i have an interest in datascience. I'm mostly a Java dev. I have to say the python2/3 divide is a real turn off. Many of the science libs want to use seem to be in 2.7 with no signs of moving.
Is there any hope of other libraries and OSes doing the same thing?
I used to work on a GUI app in Python. I ported it to Python 3, then switched OSes for various reasons. 5 years on, on Ubuntu Xenial (so new I can't even use it in Travis, but that's a separate whine), I install pykdeuic4 and it's using Python 2. So I've basically abandoned that project for 5 years now, because every time I looked at it I thought "surely Python 3 will be here in a few months, I don't want to port backwards to Python 2".
(Serious question: is there a PPA or anything I can use to get these things for Python 3? I need PyQwt as well as PyKDE)
At least the science libraries have a better reason for staying. They often require bigger fixes of the modules compiled from other languages. Still, it's good to see larger projects moving on.
I am data scientist who uses both pythons and the only lib I had issues with was the first release of tensorflow (and that was fixed very soon). Otherwise I find it hard to believe that there exist any maintained libraries that do not support both versions.
I do a lot of data science/ML work and it is almost exclusively done with Python3 so I'm not sure what libraries you are referring to as I haven't had any issues.
A whole pile of people complaining about upgrading Django highlights two things to me:
Not enough people are using tests. A decent set of tests make upgrade super easy. The upgrade documentation is decent so you just spend 20 minutes upgrading broken things until it all works again.
People pick the wrong version. I've seen people develop and even deploy on -dev and it makes me cry inside because they'll need to track Django changes in realtime or near enough. Pick an LTS release and you get up to three years on that version with security and data-loss upgrades and no API changes.
Is anyone going to talk about what this means for Python and Django? I read the first 30-40 comments and they are all about off topic stuff related to Django, but still the core premise is the committed move to Python 3.x going forward.
What do people think of that?! I'm a newer dev and I'd really really love to hear what people think of that and what it means for the future rather than side conversations about how bad their API is, how good it is, how good their Docs are and how bad they are.... Blah blah.
Please!! This community is filled with some of the most brilliant minds and I for one don't want to miss out on this chance to hear what people think of this change.
Please please don't reply that you disagree with my POV. That's irrelevant, but please do if you are interested in the initial topic. I'd be be very excited to hear your thoughts.
First, this is a good thing for the community. The ecosystem has been pretty well prepared for 3.x adoption for a while, but we just haven't done it. Still, when Django switched its default docs to use 3.x instead of 2.x, it noticeably increased adoption of 3.x. (Source: Kenneth Reitz on "Talk Python to Me" episode #6.) By pushing on with 3.x, Django is doing its part to drag the rest of us forward with it.
Second, this is necessary. Support for Python 2.x is supposed to end in 2020, per Guido's keynote at PyCon 2016, so Django is going to have to get in line in ~3 years one way or the other. A major version increment is a great time to introduce such a breaking change.
So ... "what this means" is that Django is doing what it has to do, which happens to coincide with the interests of the community at large. shrug I'm glad it's happening, but there shouldn't be a whole lot of drama or hand-wringing here.
Readit News
https://www.djangoproject.com/weblog/2015/jun/25/roadmap/
I've grow to highly respect the Django project for its good documentation, its healthy consideration for backwards compatibility, security, steady improvements and all round goodness.
I hate their API and overall architecture, which I find to be the result of glueing features on top of features for many years. The internal code also is just like that: looks like every single method is riddled with out-of-band conditionals, which is the result of a community that prefers to hack things to work, instead of rethinking/refactoring.
I'm curious, what's an example of a framework that you think has a better architecture, and in what context are you evaluating this?
And breaking backwards compatibility
Django is not RoR. Their users rely on being able to upgrade seamlessly.
The overall Api makes sense, there are some rough corners (yes Sites, I'm talking about you) the docs are ok once you get the hang of them
We are users of Django, and it helps us to deliver projects, quickly. Interested to know how you think things could be improved.
The overall architecture is sound though. Tight coupling of core components, loose coupling of non-core components. Keep in mind that it's an "opinionated" framework though, so they do safely assume that if you're using django you want to be using its core components as a bundle, and will work with them as designed.
A project I was really impressed with the internals of is Celery (was expecting the worst having seen Django).
Deleted Comment
Is this a joke? They break backwards compatibility with every minor version, wasting tens of thousands of man-hours all over the world - time that, if we're honest, is not exactly billable.
Most people don't upgrade because of this and keep on using vulnerable versions. It's good that they are trying to kill the project. It saves newbies from stepping into the tar pit.
For the most part, they think they things through a lot and most importantly they document breaking changes. Their approach is an absolute dream compared to, say, updating xcode/iOS apps. It's a total shit show at Apple.
I think the project is venerable and I have respect for it in general. It's one of the most accessible major open-source projects for newcomers; like I said, I'm not a heavy user and I've already submitted a couple of patches and had them merged. But issues like this one are worth pointing out.
Is there a deficiency in the test suite? Is the open and accepting nature of the community that I just praised the cause of this issue, because it leads to a bias against thorough vetting? I really don't know enough about the project to know, but it'd be nice if they figured it out.
Like screwing the huge majority of users with Python 2.x Django projects and having them update or be left behind?
I still have one customer with a Django/python2 code base. When migrating to Django 1.11 in 2017, that project gets security updates until at least 2020. Plenty of time to gradually transition to py3. I honestly don't see the problem.
That's illogical and inconsistent. they're providing an LTS release for you. Get over it
I think this is smart and will help them focus on the road forward. Django 1.11 is an LTS release, so legacy stuff can stick with it and be just fine.
Python 2.7 will EOL in 2020. But I guess even then some luddites will complain how they will have been left hanging
Start your migration now.
Sure legacy projects still need support and for that they get the 1.11 LTS, but otherwise it's really time to move on.
One possible solution would be to offload this work into a celery worker which can call python2 as necessary.
Making code compatible with python2.7 onward means that you can't use any new features from 3+ and the code is plain ugly, I don't know if you wrote 2/3 compatible code, it is not as enjoyable to do.
I think Django's approach where the LTS will still work on 2.7 (and LTS is for 3 years, which is until python2 itself stop being maintained) is fair.
You still have 3 years to fix things and if you want to use latest Django and be cutting edge, you probably should use latest Python as well.
Except that there are still some critical packages that aren't on Python 3 yet. Not to mention a lot of functionality breaks even if the libraries do exist, which means you have to code things up quite differently sometimes.
http://py3readiness.org/
2) Let's see some proof of your argument, because many packages and platforms are happily supporting Python 3 now. Put your money where your mouth is.
https://python3wos.appspot.com/
almost everything on the list has gone green now
[citation needed]
Pretty much all critical packages now work on 3, as witnessed by the Wall of Superpowers.
Maybe some django-specific lib is still 2-only? In which case, rewrites/forks should be relatively trivial and I'm even happy to have a look myself.
How about millions of lines of code in your company in Python 2, and several Python 2 based services and websites?
Why on earth will you go to Python 3 at huge rewriting costs? To get some fancy syntactic sugar and improved unicode?
If you're starting a brand new Django project you're not rewriting anything. If you're rewriting something, it's not really new.
My company is a Django shop and while we have no intention of upgrading Python 2.x codebases to 3, we do start all our new projects on Python 3. It just doesn't make sense not to do it.
Can someone explain what benefit I would actually gain from upgrading to Python 3 if I'm already "handling Unicode properly" in Python 2? So far it still seems rather minimal at the moment, and the risk of breaking something during the upgrade process (either in my own code or in one of my dependencies) doesn't seem like it's worth the effort.
How much faster might your code run just by upgrading to Python 3? How much memory might you save?
I don't think you're suppose to depend on the ordering of dictionaries. It's an implementation detail which might get changed, although it wont actually ever be changed because people will come to depend on it.
The difference is when you didn't handle all the cases correctly. In that situation Py2 will silently do either the right, or the wrong thing and you'll never know. Py3 will likely throw an extension and tell you where the issue is.
So if you're handling lots of unicode text and think you've handled all the string correctly already - that's the reason to move. Now you'll be sure it's correct. The problem with py2 wasn't so much that it was handling anything badly - it was that the default, easy way was incorrect and just waiting to blow up in peoples' faces. (or maybe just silently corrupt something)
Even if you think you would not use those features, other libraries you may use might benefit a lot from it.
A few features: async/await, lists (and others) use iterators, no var leaking in list comprehensions, super().my_method() instead of super(MyClass, self).my_method(), class MyClass: instead of class MyClass(object):, improved exception handling, required arguments, ", ".join(["etc"]* 1000)
Besides that: a much improved standard library, although that technically not is language design.
It turns out that most developers have a desire to move to the next version if it's not too hard. There's still COBOL programmers out there too and that's perfectly fine.
Django has made the process as smooth as it can be. You can upgrade to python3 while maintaining your Django version. Then update to the next Django version as a separate step. It's fine to have waited until now. You can keep waiting if you want but it's getting to the point where you should really just do it. It's not so bad.
We switched and python2 --> python3 was bumpier and more work than most Django updates we've done (we've done pretty much every one since 1.0) but it was still entirely reasonable. We're much happier now.
But the big one is that Python 2.7 will go away at some point.
If you don't need any of the new features and fixes, don't, that's perfectly fine.
I believe this already exists: https://docs.python.org/2.7/library/logging.config.html#logg...
The most important scientific libraries have pledged to drop support before 2020, and are all python3-ready
http://www.python3statement.org/
I used to work on a GUI app in Python. I ported it to Python 3, then switched OSes for various reasons. 5 years on, on Ubuntu Xenial (so new I can't even use it in Travis, but that's a separate whine), I install pykdeuic4 and it's using Python 2. So I've basically abandoned that project for 5 years now, because every time I looked at it I thought "surely Python 3 will be here in a few months, I don't want to port backwards to Python 2".
(Serious question: is there a PPA or anything I can use to get these things for Python 3? I need PyQwt as well as PyKDE)
Not numpy, scipy, tensorflow...
Not enough people are using tests. A decent set of tests make upgrade super easy. The upgrade documentation is decent so you just spend 20 minutes upgrading broken things until it all works again.
People pick the wrong version. I've seen people develop and even deploy on -dev and it makes me cry inside because they'll need to track Django changes in realtime or near enough. Pick an LTS release and you get up to three years on that version with security and data-loss upgrades and no API changes.
What do people think of that?! I'm a newer dev and I'd really really love to hear what people think of that and what it means for the future rather than side conversations about how bad their API is, how good it is, how good their Docs are and how bad they are.... Blah blah.
Please!! This community is filled with some of the most brilliant minds and I for one don't want to miss out on this chance to hear what people think of this change.
Please please don't reply that you disagree with my POV. That's irrelevant, but please do if you are interested in the initial topic. I'd be be very excited to hear your thoughts.
So Django moving to Python 3.X Go :)
Second, this is necessary. Support for Python 2.x is supposed to end in 2020, per Guido's keynote at PyCon 2016, so Django is going to have to get in line in ~3 years one way or the other. A major version increment is a great time to introduce such a breaking change.
So ... "what this means" is that Django is doing what it has to do, which happens to coincide with the interests of the community at large. shrug I'm glad it's happening, but there shouldn't be a whole lot of drama or hand-wringing here.
[1] https://patch-diff.githubusercontent.com/raw/django/django/p...