State of the software world: mid-2017

One of the things about the software world is that things are constantly changing, and we can feel that change, but can we quantify it?

I’m a big fan of Google Trends, so let’s use it to identify some (mostly arbitrary) software trends and see how they compare to our intuitions.

Let’s start with a few selected programming languages from the past 5 years:

Note the fact that Javascript and Python have overtaken PHP, and somewhere in 2015 Javascript became the most popular open source language out of the ones listed. However, if we zoom out and add one more language, we can see a broader trend:

Namely, that Java has dominated for the past decade+ but has been on a long slow decline (along with PHP) for as far back as the data goes. There are probably a few reasons for each, none of which are particularly original: Java for much of its history has been overly verbose and “feels” heavyweight, because of relatively long boot times. It’s use in the enterprise has branded it an “enterprise” language.

PHP, meanwhile, benefited from being lightweight enough that it could be run on shared hosting with decent-ish security and sandboxing, as well as being loosely-typed enough and amazingly simple to deploy that it was the hobbyist platform of choice pretty much since the dawn of its existence.

But the lack of good package management plagued the Java and PHP ecosystems. These aren’t problems for Ruby, Python, or Javascript, and I personally believe they made a huge difference: people could benefit from, and add incremental value to, the ecosystem very easily, instead of siloing solutions to the same problems in repositories everywhere.

But back to the charts. If we look only at the search terms (as opposed to the topic name in any language), we see a slightly different interpretation:

Mainly, one where Ruby, PHP, and Python have more relative volume overall (but with unchanged slopes) –  perhaps a quirk of the overly aggressive categorization of searches. Which graph you trust depends on how often you think people search the term “ruby” or “python” without intending to search for the programming languages by those names.

Since I can only chart 5 terms at once, I will superimpose C++ on as the 6th line (in orange) for the curious:

It turns out C++ is the most popular search term, and has held up quite well compared to Java’s trendline. Since most software is eventually C++ or C, this is not extremely surprising, even if it is a lazy observation.

Next up is the slow death of coffeescript next to the skyrocketing popularity of es6:

Not much deep introspection needed here – Coffeescript paved the way for ES6 and was never meant to be its successor. I personally think its conciseness and tolerance leads to sloppy and sometimes buggy code. What’s more surprising to me is this:

…In which Typescript interest seems to be outpacing that of ES6. I like the concept of Typescript. It’s nice to see some respect for type safety again (all you hooligans that grew up knowing nothing but NodeJs will come around eventually).

I think this isn’t a perfect comparison, though: including “Javascript” in this chart would be meaningless, for example, because ES6 represents a subset of “Javascript” proper, whose search volume would dominate all the others. Typescript/Coffeescript are languages of their own, even if they compile to JS. However, I still find the trends interesting and somewhat unexpected. This however makes sense in light of how popular AngularJS is:

AngularJS seems to be healthy, whereas EmberJS is many times smaller and has stopped growing. I’ve thrown in jQuery and React (both libraries, not frameworks) because they both fit into the larger story: jQuery’s mission today is less relevant because modern browsers behave well and are standards compliant. This has in turn allowed React (as a library, and as a concept) to take off in the front-end.

And here, a few web servers from the past 13 years:

No surprise here: declining share of Apache and IIS is being eaten by Nginx and Node.js, with Node.js in the lead by far. Overall volumes are also down over the years, as fewer people manage their own servers. The success of Nginx at the expense of Apache doesn’t surprise me – I myself have seen good performance increases from the switch to an evented model. And Microsoft long ago lost the war for web server market share, where the open HTTP protocol and the proprietary .NET ecosystem hindered adoption by both repelling users looking for free alternatives and those drawn to open source stacks.

A similar story plays out when looking at web frameworks:

Microsoft’s stack is the only one to have lost significant market share, with CakePHP dwindling from “small” to “insignificant”. (note I couldn’t search Express.js or the like; I used Node.JS in its place). We should expect .NET and IIS adoption to move in tandem, and they do.

Finally, my favorite:

MySQL has dominated for many of the same reasons PHP has dominated for a long time: ease of setup and use, even at the expense of lax / insecure defaults, or unpure/technically inferior architecture and design.

MySQL’s relative decline is probably to the benefit of the many NoSQL solutions available today – many of which aren’t even listed.

There are other forces at play in this chart. Cassandra is almost certainly benefiting from overloaded search terms. I’m pretty sure it’s less popular than MongoDB, and it has always been difficult to deploy compared to the others (another success story along the lines of “build it fast and easy, then worry about making it reliable”). Heroku has probably done more than any other company on the planet at increasing PostgreSQL adoption.

The growing popularity of Redis does not surprise me – it has always been high value to offer “a block of memory in the cloud.”

What other trends are interesting to chart?

1 thought on “State of the software world: mid-2017”

  1. Would it be also relevant to check out cloud services keywords like AWS, Azure etc., as indeed that there are lesser people manage their own servers nowadays (although it’s this is more like a market share survey)?

    Another trend I’m curious to look into would be the “stacks” (i.e. MEAN, LAMP, Java + Spring, Python + Django etc.)?

    Thank you for sharing this insightful and interesting article, Peter. As a newbie of the industry I found it helpful for my decision on the investment of my time studying certain knowledge 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *