Coté Memo #16 - Outsourcing & DevOps, Lovecraft, 38% DevOps penetration
Fall is finally coming to Austin, which means it's nice and cool. With a long lull in travel, I've been working on a second edition of my "cloud native journey" PDF. See a fragment of it below, on outsourcing. Meanwhile, I'm reading through the DevOps Handbook to write a review.
Outsourcing and DevOps, it's a problem
An excerpt from the second edition of my cloud native journey booklet. This is from the second section where I cover the common questions and "barriers" to doing DevOps/Agile/cloud native/whatever you want to call it. As I get more of the text done, I'll post more, if not the whole document for review
One of the most common cloud native inhibitors I hear people complain about in large organizations (esp. government and Western European organizations) is outsourcing. We all know the story: around five or ten years ago, a new CIO came to the organization and achieved huge cost savings by outsourcing much of IT, including developers. It was great for several years, so great that the CIO got an even better job at a new company. In the present day, these organizations find themselves with a huge bill and results they usually don't like. And in the present, in the era of transient advantage, when creating custom written software is mission critical, these organizations have little to no ability to do produce high quality software.
"Government contractors" often are mythologized to be the worse of all outsourcers. This mini-story from Steven Levy is good example of what typically goes wrong with large outsourcing deals. The US Federal government had been trying to digitize the process for green card renewals, but it wasn't going well:
Steven Levy: Do you have a metric that shows the difference between that immigration program before and after your small team revamped it?
Haley Van Dyck: Well, the metric before was that the integration system was entirely paper-based. To actually apply to the system it cost about $400 per application, it took end user fees, it took about six months, and by the end, your paper application had traveled the globe no less than six times. Literally traveled the globe as we mailed the physical papers from processing center to processing center. This transformation process existed well before we showed up [but wasn’t succeeding]. There was actually a billion dollar contract that was out at one point to start this modernization process. At the end of the five-year contract, which included [an additional] two-year requirement-gathering phase, zero code was delivered that worked. They were in the second year of another five-year long contract when we showed up. And I-90 is the first functioning release that has existed on this project in almost seven years. [After the interview, a government spokesperson clarified that the first contract was a seven-year deal for $1.2 billion, and the result was “behind schedule and slower than paper.”]
Steven Levy: What did I-90 cost when you folks did it?
Haley Van Dyck: The salaries of five people.
Mikey Dickerson: They worked with an existing organization, but what we added to the project was five people. Not even measurable against a $1 billion contract.
While the scale of this project may not exactly fit what you've experienced, the general shape and pattern of this story matches what many large organizations tell me is wrong with contractors...which they're forced to rely on because of past outsourcing decisions that no longer seem like good ideas. It's little wonder that in a recent study, over 75% of senior executives said they want to replace their "legacy" outsourcers because those providers are so unwilling to change to new models.
Among many others, Citi provides one example of addressing this need to change how outsourcing and offshoring is done in large organizations. Currently 80% of Citi's developers are contractors, while 20% are employees. Citi wants to invert this model by the end of 2017 and are shooting for 80% of developers being employees with only 20% remaining contractors.Their hope is to drive a culture of ownership, leading to better product level thinking, creating greater alignment to the business’s needs. That is: to become more innovative.
Further, while remote work is possible, Citi has found that having co-located workers pays off: They're reducing 26 different development locations down to just 4. Going through the process of co-locating team and working with stakeholders is driving a 57% increase in the speed of delivering their software.
Again, what color badge someone wears won't prevent them from becoming cloud native, but the traditional relationships with outsourcers will probably be an inhibitor. There's no way around it: you have to start insourcing more, or have such a special relationship with your "outsourcer" that it's virtually insourcing.
That special relationship essentially means having outsourcers follow the core principals of your cloud native approach. For example, one of the most useful requirements you should have is that the outsourcers work on the same code base, use the same build pipeline, and follow the same integration requirements that your insourced teams use. As Gary Gruver puts it:
Ensuring the different [outsourcing] vendors don’t go off track and create code that won’t work together in production is critical and a well-designed deployment pipeline is a critical tool for coordinating development across different outsourced organizations in traditional tightly coupled systems. For these types of outsourced organization, DevOps principles while different are almost more important than they are for companies that do all their development internally.
One of the key enabling principals of the cloud native approach is reducing as much variability up and down the stack: automating as much as possible, all using the same code base that's integrated multiple times a day, and other practices strip out variability that causes errors and slow-downs as different processes "synch-up." By the very nature of being from another company, contracted to augment your own staff, outsources typically create a "silo" that must be continually synched-up, introducing variability. Clearly, eliminating that variability is top of the list if you're doing any kind of outsourcing.
Arguably, the "lower levels" of your cloud platform could be outsourced since there are clearly defined contracts between the application and platform layers. However, care must be taken to avoid introducing scarcity into the system. At The Home Depot, for example, after several rounds of trying to speed up the release cycle, managers found that product teams were still not doing releases frequently. Upon investigating why, they found that teams were charged in internal money for each release, making teams leery of blowing out budgets when they released frequently. This sort of a la carte fee schedule occurs in outsourcing arrangements and create negative incentives that will inhibit the rapid release cycles, slowing down the innovation engine for your business.
Coté Content
Pivotal Conversations #36: Managing employee experience and building trust, cloud native HR with Joe Militello — soundcloud.com
We talk with Pivotal's head of HR about doing agile HR.
Pivotal Conversations #37: Microservices governance with Spring Cloud Contract, guest Marcin Grzejszczak — soundcloud.com
Marcin Grzejszczak returns to talk with me about consumer-driven contracts in Spring. Essentially, it's a way to better manage evolving APIs in services heavy architectures, you know, like how microservices would be used in large organizations. As the title suggests, I think it's a new method of governance that enterprise architect types could start using to, wel, govern.
Pencils and pens with Bridget Kromhout — soundcloud.com
Check out Bridget and I talking about what we do, tech evangalism. And travel bags, and other vagaries.
Links
38% of enterprises now using DevOps, Gartner
“Gartner estimates by 2021, more than 50% of established corporations will be leveraging lean startup techniques at the business level to increase the pace and success of business transformation.”
6 DevOps columns: government, compliance, ROI, management — cote.io
Last year I wrote some DevOps-themed columns for FierceDevOps. It's near impossible to find them anymore, so I moved them over the Medium. They're listed here
Bet You Didn’t Know You Could Get Into This Much Trouble Using Calculus: “The Dreams in the Witch House” — www.tor.com
I've been re-reading HP Lovecraft's stories. Like most good fiction, it's a completely different experience 10 years later. I didn't realize the first time that all the monsters are actually aliens. These commentaries from two authors at Tor are awesome to read afterwards. As an English major I eat that kind of shit up.
There Is No Spotify Model — www.infoq.com
Two things to take away from this:
1. You can rely on extracting best practices from just a single point of time at a company. You have to look over many years and see what works per context. Cf. The Halo Effect.
2. The best process methodology is one that continually morphs and changes to the needs of the organization and the people you have at hand.