There was an interesting discussion / Q&A in Spiceworks recently with a leading voice in the DevOps community. What's your view on DevOps? Is it something that you are actively employing, considering, or not interested in? Why?
We recently sat down with Scott Alan Miller (aka SAM), a DevOps veteran, to better understand the whys and wherefores of the DevOps movement. Read on for SAM's perspective, including the notion that the practice of DevOps is actually as old as software itself . . .
Let's start with your personal definition of DevOps.
It's the intersection of where developers become their own administrators. Traditional enterprise architecture is that developers are on one side, they write their code, they throw it over the wall, and system administrators deploy it. The developers can never touch the servers. They can never be involved in the production server world. And likewise, the systems administrators can never touch the code – which is important because a lot of big businesses require separation of duties, and they are generally two very unique skill sets.
If you go to Microsoft, they are developers. They make Office, they make Windows, they make SQL Server, they make Exchange, and you, the systems administrator, buy that package and deploy it. You don't write the code, and they don't deploy it for you. It's a very clear-cut distinction as to the two roles. But when you get into Office 365, that's being manned by a DevOps team – people who to some degree are writing the software are also involved in the administration of it. Modern cloud/hosted environments are pushing DevOps simply because we're seeing at large scale where people who write the code are also the people who run the code.
What are some of the challenges of doing DevOps?
The really big one is that the vast majority of programmers understand nothing of systems administration, and even more so, the average systems administrator knows nothing of software development. So finding people who actually are skilled at both is incredibly unlikely. And so you have a combination of struggling to find people to hire, then the people you do hire are in very high demand and so are being hunted constantly, and they demand a much higher price because they are carrying two of the top-paying skills together. It's a very expensive, very tiny pool of people.
And then you're requiring those people to be flexible and keep up on all the technologies that you're using. Maybe you're a great Ruby and Solaris DevOps, and suddenly the shop you're working in has to make a change and they're going to Node.js and Linux. And now you have to be a fully qualified Node developer and also a fully qualified Linux systems administrator. It's very difficult, even for someone who's able to get to the point of being DevOps, to be able to maintain the flexibility needed by an organization to make those kind of moves.
So what kind of organizations stand to benefit from DevOps? Who's it good for?
Small shops have typically always done DevOps; they just didn't talk about it in those terms. My company, NTG, started off as a DevOps cloud shop in the '90s. We did all these things that people now talk about as the hot new things. We just didn't have all those titles yet. We were doing hosted software as a service for hospitals in a DevOps way.
The large-scale administration needs of an organization – that's what leads you to problems when you try to implement an all-DevOps environment. You can't use DevOps people to be your desktop admins, for example, or to be your Active Directory people, or your web server people. And because they typically can't play all the roles, you now have unstructured IT mixed in with your structured IT. These big environments that are pushing hard for DevOps are getting burned as they realize that it's very expensive to do and very hard to maintain, and they're not getting the value in many cases.
It's a person-by-person, shop-by-shop question based on what they need to do. If you're Microsoft, the Office 365 group needs to be DevOps in order to be as good as they can be. But they also have teams of dedicated non-development people who provide the security oversight, the networking oversight – all those pieces. Then they have other parts of the organization, like the Microsoft Office team, which I guarantee is 100% not DevOps. I think successful companies learn to blend it where it makes sense and avoid it where it doesn't.
In cases where it makes sense, what are the main benefits?
The biggest benefits really come from the tight stack integration, where you have a single person or small group of people who have a complete picture of what's going on. This is what kills larger organizations that have heavy separation of duties – the right hand doesn't know what the left hand is doing. The systems people say, "Well, it's not a systems problem," and they pass it off to the network team, who say, "It's not a network problem," and they pass it off to the storage team. No one takes ownership, and no one has a complete view of an operation from the moment it leaves the disk until the moment it goes out onto the network and hits an end user. Everybody just has their one little slice of it. Even if companies are very good at not doing this, they still have to deal with the "one department versus another department" interface issue. With DevOps, you're able to at least get pretty close, and if you get really extreme DevOps groups, you do get people who are writing code and doing storage, networking, servers – everything the whole way up.
It looks like there's very little in the way of industry consistency around DevOps – not just in the tools and what they do but in the terms people use to describe different parts of the process.
That's true, and this is because DevOps is actually so old. Systems going back to the '60s and '70s were all DevOps. DevOps is where IT started. And it was only once you got mature enough and big enough that there was an ops organization, then you started getting developers who were like, "I don't have to know how the system works anymore." They got to work at an abstracted level. Originally we didn't think of it in those terms, but there's a huge portion of the industry that has had DevOps since day one.
When we started doing DevOps, we didn't say, "This is weird." It wasn't like we were doing something special. You just expected that everybody worked together, and that the developers leaned toward development, and the systems people leaned toward systems, but we all worked on the code, we all worked on the servers. All the terms that are coming up now – DevOps as an example – are attempting to apply labels to something that has always been there. So it's very difficult to get consensus on the new labels.
Things like continuous integration – that's fairly new, but a lot of people strongly associate it with DevOps. Continuous integration is a great idea, but the vast majority of DevOps people don't do that. And now we're starting to see lots of people talking about tools like Chef and CFEngine and SaltStack and Puppet as being the tools of the DevOps trade, and I don't think that's true. They're very important, and it's really important to talk about them because they're big market-shakers right now, but I think the vast majority of DevOps people don't use them. And they're really designed around cloud computing architecture – elastic compute clouds – and they don't apply to traditional computing architecture.
Looking around at people in the Spiceworks community, I've never run into anybody and said to them, "You know what you need? You need Puppet. You have three servers. You need a framework for managing those. You need to have your architecture as code." Of course not. To log in once and update a package? But a lot of people talking DevOps, even with a handful of servers, will jump to, "Oh, you need to use these really heavyweight tools and learn all this stuff and code it all out." And you're like, "But I only have one server! What does this do for me?" I've come from environments where we were managing tens of thousands of servers, and we still didn't use those tools because the servers weren't alike. We had 20,000 servers, but each one was, as we like to say, a unique snowflake. You just log in and administer it like a normal person.
Since we're talking about tools, an awful lot of free and not-free tools have to be glued together to create a unified DevOps process, but I'm also seeing more and more "DevOps platforms" or toolkits that glue all those pieces together for you in one stack. What do you think is the trend out in the real world?
My impression is that people are really heavily gluing things together, and I think that part of it is just the DevOps mentality. Pure systems administrators look at the world and say, "I need a tool that does that," because their job is to use tools. Developers look at the world and say, "I can make a tool that does that," because their job is to make tools. Two fundamentally different roles. One uses what the other one makes. So when you have a DevOps person, I think that the dev portion of their brain has a tendency to take over there, and that's not good or bad – it's just how it is. They tend to lean toward making tools instead of acquiring tools whenever possible. Even with tools like Chef, for example, you just write everything in Ruby.
What role do you see Spiceworks playing in the DevOps community?
Bearing in mind that the Spiceworks community itself is larger than the entire DevOps community, this will be the destination to discuss the tools. I have friends who do Puppet on Linux but aren't developers. They're real systems administrators, but they use those tools. And Red Hat is rolling Puppet out now with Satellite, so Puppet is being rolled out at a huge pace to big organizations that run Red Hat, but nobody knows what to do with it, because they're all like, "What's Puppet? What are these tools? How do they get used?"
All your normal crowd in SMB have no reason to use them. At NTG, Chef is used, but only for certain environments. NTG has some environments with large numbers of identical or nearly identical machines, and other environments that are snowflakes, and one's on cloud and one's not. And so they're managed in two different ways. So you'll get lots of discussion about how this doesn't make sense for anybody, when in reality it does make sense for people – just not for those people.
Some sysadmins are worried that in a DevOps world, they won't be needed. It's the whole argument of "Cloud is taking my job away, and now DevOps is taking my job away." Do you see any validity in those fears?
I really don't. IT and development are people whose job is managing change. We manage change for the environment, for the company. We constantly train, we watch for change coming down the pike, and we deliver the ability to handle change for an organization. You want to keep up, you want to keep your email working, you want to keep your website up-to-date, you want to keep your ERP working, you need to change and evolve at that level. And so your IT department, even in traditional companies, is the fast and limber one. It's human nature to experience fear when change is coming, thinking that any change of the status quo could knock us down. But on average we are the people who get the most benefit from change because of all the jobs that are coming. All those cloud jobs, all those DevOps jobs? They have to come from a pool of people, and we're that only pool.