Is Python a Good Choice for Entreprise Projects?
A few weeks ago, one of my followers, Morteza, reached out and asked me the following:
I develop projects mostly with Python, but I am scared that Python is not a good choice for enterprise projects. In many cases, I've encountered a situation where Python performance was not sufficient, like thread spawning and so on, and as you know, the GIL supports one thread at the time.
Some friends told me to try to use Java, C++ or even Go for enterprise projects instead of Python. I see many job boards that require Python just for testing, QA or some small projects. I feel that Python is a small gun for showing my experiences and that I'd have to choose an alternative language.
As you are advanced and professional in many topics especially in Python, I'd need your advice. Is Python good enough for enterprise systems? Or should I choose an alternative language which fills the gaps that exist in Python?
If you follow me for a long time, you know I've been doing Python for more than ten years now and even wrote two books about it. So while I'm obviously biased, and before writing a reply, I would also like to take a step back and reassure you, dear reader, that I've used plenty of other programming languages those last 20 years: Perl, C, PHP, Lua, Lisp, Java, etc. I've built tiny to big projects with some of them, and I consider that Lisp is the best programming language. 😅 Therefore, I like to think that I'm not overly partial.
To reply to Morteza, I would say that you first need to acknowledge that a language itself is not slow or fast. English is not faster than French; however, some French people speak faster than English people.
So then, yes, CPython, the chief implementation of the Python programming language has some limitations: the GIL (Global Interpreter Lock) as Morteza says, is the most significant parallelism limiter. The rest of the language is being optimized regularly, and you can follow the work done in each Python version to see where this is going. CPython gets faster on each minor version.
On the other hand, don't think that Go or Java are miracles: they both have their limitations. For example, you can read this compelling presentation from Ben Bangert at Mozilla entitled "From Python to Go and back again". Ben explains some of the limitations that he encountered while switching to Go.
I'm sure you can find problems and limitations with the Java Virtual Machine too.
In Scaling Python, I wrote a few chapters covering the GIL and how you can circumvent its limitation. If you write widely scalable applications, the GIL is not such a big deal, as you need, anyway, to spread the load across multiple servers, not only on several processors.
There are tons of companies running Python applications at large scale, e.g. Instagram, Google and YouTube, Dropbox or PayPal.
Therefore, no, Python is not only for QA applications, no more than Java is only good for browser applets nor Go is for devops or whatever.
They all are different languages that approach problems from different angles. Depending on your mindset and on the solution that you want to implement, some might appear better equipped than others. Their virtual machines or compilers are marvelous, but also have their limitations and shortcomings that you need to be aware of so you can avoid falling into a big trap.
Of course, another approach is to remove all those issues by going down a layer and use a lower level language, e.g. C or C++. That'll remove those limitations for sure: no Python GIL, no Go resources leaking, no JVM startup slowness, etc. However, it'll add a ton of extra work and problems that YOU will have to solve – puzzles that are already resolved by higher-level languages. That's a matter of trade-offs: do you want to write a blazingly fast program in 10 years or do you want to write a decently fast program in 1 year? 😏
In the end, picking a language is not only a matter of performance but also a concern of support, community, and ecosystem. Picking battle-tested languages like Python and Java is the assurance of reliability and trustworthiness, while selecting a younger language like Rust might be an exciting ride. Doing some "reality check" is always worth considering before choosing a language. If you wanted to write an application that uses, e.g., AMQP and HTTP/2, are you sure that there are libraries providing those features and that are broadly used and supported? Or are you ready to commit time to maintain them yourself?
Again, Python is pretty solid here. Considering the extensive practice it has, there are tons of generously used libraries for everything you could ever need. The community is large and the ecosystem is flourishing.
In the end, I do think that yes, Python is a terrific choice for any enterprise projects, and considering the number of existing projects it counts, I'm not the only one thinking that way.
Feel free to share your experience – or even projects – in the comments section below!
