Kubernetes is great, but it’s been a 7 year distraction
The snow globe is shook, 3 major problems are solved, & now it's back to developers. Plus, reconsidering potatoes in omelettes, the lifespan of S&P 500 companies, DevOps is for the rich, & links.
Kubernetes is great, but it’s been a 7 year distraction
Opening question
Or, here if that’s your thing.
Shaking the snow globe
When you watch the making of kubernetes documentary, you’ll notice a strange character. Time and time again Amazon is referred to directly or indirectly. But, Amazon’s spokesperson in the documentary is a silent, cinematic drone shot of an Amazon building. (This is my memory, at least. I apologize if I am dead wrong. I was lazy and didn’t take the 60 minutes to go back and verify this. If I am wrong, just forget this opening story-time. That documentary is great, by the way. I wish there were a lot more like it.)
“How do [we] change things up — how do we shake the snow globe in a way that may not be all about Google, but at least gives Google a fighting chance to be able to start grabbing some of these customers, and to start being that balance against the dominance that AWS had at the time.”
I assume the filmmakers asked Amazon to participate, and it would have made historic sense for someone(s) from Amazon to do so. But, I can see their PR people being like “Hard pass. Why would we talk about how we were defeated, and by Google of all competitors?”
Not showing up turned out well, story wise: in that documentary, Amazon is a looming, powerful figure of Cold War stature. Whether they’re the Soviets or the Americans is a Rorschach test for the viewers. Either way, you have the opposing force needed for good tension and a push to conflict-driven plot in a fun story.
Like OpenStack before it (boy, there’s a tale!) kubernetes was trying to open up public cloud. It was trying to defeat Amazon and the other closed cloud services (that existed at the time, RIP those glorious contenders) and vendors. It’s won: not by current marketshare by a long shot, but my mindshare and desire. It’s won the future.
A soft landing is all but guaranteed
There is a lot of kubernetes out there, but Gartner estimates that just 5% of “enterprise applications” are in in “container environment” (in 2022):
By 2025, 15% of enterprise applications will run in a container environment, up from 5% in 2022, hampered by application backlog, technical debt, skills availability and cultural change.
—From a recent “Gartner Forecast Analysis: Container Management (Software and Cloud Services), Worldwide.”
(These numbers don’t seem to have moved much since 2020. Even the “hampered” wording is the same! So maybe the above is bad note-taking on my part. If so, we still have the next survey to make the point…)
An older survey from the CNCF measured marketshare by developers using Kubernetes. It estimated that 21% of developers worldwide used kubernetes (5.6m out of 26.8m by my estimates of how they came up with this).
I prefer measuring by workload/application, so I pay closer attention to marketshare models like the Gartner estimate. But, do whatever you like…
However you estimate it, kubernetes is about to cross the enterprise chasm, or is in mid-jump. I think it’ll land safely. And why not? It’s proven to make delicious chicken and waffle fries and people want it: our surveys show sustained and increasing interest each year.
Right back where we started
However, I’ve started to wonder if this has all been a good idea. The answer to that doesn’t matter, because it is what we have now. And, as I tell my kids when they get obsessed over past things, since we don’t have a time machine, there’s no use worrying about it.
Buuuuuuut….indulge me in a little armchair history pondering.
The idea of commoditizing, standardizing, and opening up IaaS is great. There’s at least two other things Kubernetes is standardizing: first, the management interface and operations practices for cloud (“systems management,” as us olds call it), and second, a distributed application architecture. These are things that J2EE and .Net did to great success and benefit, that the Spring Framework did and does, and even rowdy communities like rails. It’s a failing and a blind spot of mine that I don’t know any newer examples, but…moving on.
Those three things have historically been externally messy, costly, and just a general pain in the ass. Having everyone decide to, more or less, just follow the same model and use, as the kubernetes people like to put, the same “API,” in an incredible benefit. It makes it all totally worth it.
Moving pixels on the screen
Despite that, application developers seem to be particularly left-out here. I’m not sure I’ve ever seen a survey that says “application developers love kubernetes!”
As a former application developer, I know they must love learning about it and tinkering with it: few things are more of a delightful distraction to an application developer than spelunking down the stack. But if you went to an enterprise developer and told them they needed to put up with Kubernetes and work on getting those new features into their actuary apps by the end of the week, I think you’d get a different reaction.
This is an old discussion, and the Kubernetes community rightly says that kuberntes is not for application developers. They keep telling us this.
Here is the feeling, the intuition I keep returning to. I have no way to quantify it, so I want to hear how others are thinking through it. It is this: despite best intentions, Kubernetes distracted all of us from making application developer’s lives better.
While it’s solved huge problems in the infrastructure space, what kubernetes gives application developers is a great set of “primitives.” But only that. This leads to a problem, as Stephan O’Grady described two years ago:
Fragmentation makes it impossible for vendors to natively supply the requisite components for a fully integrated toolchain. That does not change the reality, however, that developers are forced to borrow time from writing code and redirect it towards managing the issues associated with highly complex, multi-factor developer toolchains held together in places by duct tape and baling wire. This, then, is the developer experience gap. The same market that offers developers any infrastructure primitive they could possibly want is simultaneously telling them that piecing them together is a developer’s problem.
(Here’s some slightly newer commentary from RedMonk-land.)
Meanwhile
There’ve been winners in the application developer space over this time. Microsoft/GitHub have benefited from filling that hole (after reversing their multi-decade closed stack, mono-vendor, you have to pay for it approach), and people like GitLabs seem to have taken the opportunity to fill the hole too. And most intriguingly, we have the fresh face of “platform engineering” coming in to rewrite and take over from the declared “dead” DevOps.
Platform engineering vs. DevOps
The last, platform engineering, is intriguing, and the second half of CY2022 was a crazy time for those of us who live in the DevOps world. By my reckoning, the idea of internal developer portals (IDP) grew from creating a post-Atlassian developer intranet to gobble responsibility for the entire stack (down to just above kubernetes, I think, but sometimes it chows down on all that yaml too). This was DevOps’s job, so you can see why you wouldn’t sit the two camps at the same table at your wedding.
Platform engineering is trying to solve the developer experience gap, to make kubernetes useable and useful for application developers. The various ways to do this are usually about removing the need for application developers to directly interact with kubernetes.
Let me shortcut what could be a meandering point: platform engineering is about building a PaaS.
What’s new is that platform engineering is also about building and integrating together the tools and developer collaboration stuff (hence, “post-Atlassian”), inner sourcing, and “service catalogs” to use older terms. Let’s use another old term for this stuff: Software Development Life Cycle (SDLC), also Application Lifeycle Management (ALM). These were not always concerns of PaaS. If “platform engineering” == (PaaS + SDLC), this is great stuff!
Once infrastructure people start thinking about application developers as customers, and start finding the bottlenecks developers have, and start working on removing them, you end up with something that looks like this definition of platform engineering. There’s a lot of great, proven best practices and “journeys” here that we can draw on here instead of slogging through re-discovering them.
However, my gut feel, is that this is exactly where we were in 2015/2016. We were just getting into this idea of solving PaaS and crawling into SDLC. Then kubernetes happened, and we gleefully rappelled back down to infrastructure layer for seven years.
I don’t know. This all a big claim and I don’t know how to analyze it. Maybe you do and will be kind enough to tell me.
Afterthoughts
Which is to say, other things to think about but I needed to get on with my day:
I think the three things kubernetes standardizes (an API for infrastructure, standardizing systems management, and an architecture for distributed applications) are worth whatever “distraction” it takes, hopefully took.
Open source and “open” continue to be weird-forces in this kind of thinking. As ever, the mental gumbofication of lock-in, open source, and buyers wanting everything to be free goos up the gears. We as a community need to figure out how we let those forces guide our thinking, and grease the gears instead.
It’s likely that the world of infrastructure and the world of application development are just too different to operate in the same way or be the same group, both on the vendor/cloud side and the user/customer side. I don’t think the DevOps community ever came to terms with this and, in fact saying that there are different groups is a huge, career-limiting taboo in that community, and will get you a stern talking to. “Bi-modal,” anyone?
I mean, yeah, you could say I’m biased because I come from the PaaS world. But, I’m not really looking for a schadenfreude snack here. I try not to have King Canute’s Syndrome.
And now, back to my usual nonsense.
Wastebook
I have been opposed to potatoes in omelettes for awhile. Being a child of breakfast tacos and Tex-Mex, I see potatoes as filler. But, I tried it this morning: thinly sliced medallions previously roasted and then heated in the microwave. This means they were slightly crispy, well cooked, but now soft, almost like a dumpling (of the chicken ‘n’ dumplin’s type). Would potato again.
“Jazz up your Google Slides.” 🎺
Although the chart here can seem more like a slide-stunt, we all used to use it to instill a sense of urgency. The chart indicates that the average lifespan of public companies listed in the S&P 500 is below 30 years. The chart hasn’t gotten better over the years. The chart was used as a rhetorical stepping point for getting to the importance of doing digital transformation back in the mid-2010’s. If we believe the chart, every company is susceptible to this short life cycle, so it’s not surprising to see previously disruptor companies like Twitter, Peloton, and Netflix struggle 20 to 30 years into their founding. As the chart(s) show, a lifespan of 30 years for a company is not typical and there are only a few exceptions. Therefore, a tech company struggling is more expected than you’d think. {{{I used ChatGPT to edit this chunk of text down from a voice transcript from Drafts app, and then did a little fixing up. Pretty good.}}}
So far, using iA Presenter is…interesting. It’s an opinionated presentation tool. It enforces one very specific type of presentation: the big words, one word per slide, big pictures style. I do not like watching this style of presentation and I do not like giving it, Sam I Am. But I seem to be the only one in the world like this, so I’m interested in seeing what I’ve been missing out in all these years. Maybe if I focus on that style, but avoid doing the whole 300 slides deck thing it’ll work for me. I don’t know if I need closed captioning for my presentations at the rate of one slide per word.
Relevant to your interests
Yes! And… - “How to be effective in the theatre of work.”
Platform Product Management Versus Platform Engineering - The bottleneck is product management skills // ‘So this leads me to my thesis: “Platform engineering” is not the point and not the right topic of focus. While engineering excellence will always be needed, it’s not the constraint. The real problem is managing platforms as products in the other three dimensions.’
Understanding Business Insider’s pivots - Only try for a conversion when you have interesting and unique content, not just factual coverage/status updates. // “My bet is that Insider looked at its analytics and found that political news had a particularly low conversion rate — not surprising since it’s geared toward a more general interest audience. Washington, DC is already saturated with coverage, so it’s hard to mourn the loss of a single bureau. Why reinvent the wheel when someone already built a better version?”
What is the Future of the PaaS Term? - From back in 2016. This makes a good case that 2016 was the year we got distracted from making the lives of application developers better and focused on infrastructure teams, that is, making Kubernetes usable and featuredfull. // This is what got me thinking about the above.
DevOps is for the Rich - …platform groups are for the middle-class?
Midjourney: a delftware vase depicting the return of a Dutch ship to a busy harbor, very detailed, only blue color transparent background.Logoff
Logoff
Don’t worry, I still have that work I need to do that I mentioned yesterday. But, it will be fine. I have the chance to talk with a big organization about this platform stuff tomorrow, which will be fun.
Here is something I have been trying to do to take care of myself psychologically, or whatever. In my line of work, worrying that you both know everything and convince people people about it is a huge stress. It is not possible to know everything. But, we all feel like we should. Instead, I have been trying to do two things: (1) tell people what I do know, even if it’s not enough, and, then (2) ask them what they know and listen closely. This is what’s known as “having a conversation,” but that is not as easy to orchestrate as it sounds when you work for a software vendor. I’m also never sure if most people really want to “have a conversation,” but I’m also working on being a more optimistic person.
Anyhow, I will be in Ghent at cfgmgmtcamp on Monday, and at the speaker’s dinner on Sunday night. Hopefully I’ll get to catch-up with people I haven’t seen in awhile. There’s so many interesting things in computers to talk about nowadays. I’ve got some fun content to put together for an internal meeting on Wednesday in London that will hopefully generate some notes for the wunderkammer here. Maybe even some “conversations.”
See y’all next time!