The cloud, the componentization of software and demands for continuous delivery are all combining to profoundly...
change applications. Nowhere are these changes more significant than in the area of deployment and operations lifecycles, where what was once called "DevOps" seems to be morphing into something called "orchestration."
Developers need to understand these trends in order to ensure development and operations continue to evolve in synchrony. This requires that developers understand the differences between DevOps and orchestration, rethink their development and operations strategies, and stay aware of important emerging trends.
What orchestration really means
The evolution of applications from development to deployment has always been a marriage of multiple practices. There are four distinct sets of tools and practices involved in applications. Version control and development management bind the development processes together and link them back to enterprise architecture models. DevOps tools then link the development processes to deployment and application lifecycle management.
The term "orchestration" has confused many developers and architects because it's not always clear exactly what changes are occurring. Orchestration describes the broad process of applying software automation techniques to, potentially, all of the four practice areas involved in application development and operations. It modifies the tools that are already largely automated, DevOps in particular -- so much that orchestration is sometimes seen purely as an evolution to DevOps. It also adds an automation dimension that cuts across much of the rest of the development-deployment spectrum.
The common driver for orchestration is the need for greater agility and personalization in applications. This has manifested itself in mobile-based, front-end processes; mobile backend as a service and microservice-based applications. Combining these trends with virtual resources and cloud computing creates two dimensions of variables. This is why automation is needed to speed processes and reduce errors.
The new development and deployment structure
It's best to visualize modern development and deployment as a two-layer structure. The top is the Agile or continuous delivery layer of software development that unites development processes and is now embodied in application lifecycle management. The bottom is the modernized form of DevOps, designed for highly componentized ("microservicized," if you will) applications and fully virtualized resources.
The most important point in making this new structure work is to ensure that the ALM tools are fully integrated with DevOps tooling throughout the application lifecycle. There have always been benefits to this kind of integration. But if continuous delivery and innovation through software automation is the goal, there is no other path to success. In the modern application model, continuous delivery efforts and Agile development must be directed both at the services themselves and at the DevOps frameworks that deploy and sustain them. That's what orchestration defines.
Processes become like services
The notion that continuous delivery and DevOps have to integrate doesn't seem insightful to many. After all, DevOps was always meant to define the relationship between development and operations. The problem is that in a world of cloud and Agile development DevOps processes have to be considered almost as coequal to services or components. This kind of relationship is already visible in the toolkits from integrated software vendors like IBM, Microsoft and Oracle as well as in open source form from Red Hat.
With continuous development and DevOps integration, the DevOps models or recipes associated with each ALM phase have to be designed following the ALM processes. Then it can be codified in DevOps language -- declarative or imperative, as appropriate. When development changes are made to an application, component or service, the changes not only have to be tested in terms of application functionality, security and compliance, but also in how they impact the integration between ALM and DevOps.
The tight coupling of development, ALM and DevOps demanded by continuous delivery has changed DevOps already. The two most popular tools, the imperative Chef and declarative Puppet, have both evolved to support modular declarations of resources. This in turn supports virtualization and modular definitions of component and service deployments to facilitate reuse and modularized deployment. Second-generation DevOps tooling like Ansible or SaltStack have supported these capabilities from the start.
There aren't many API issues or integration challenges associated with linking DevOps to continuous development and ALM, but there are definitely ways to ensure that critical things don't get overlooked.
The magic word is "modularize." DevOps models or scripts should always be built from component- or service-level elements upward to applications. Anywhere that the ALM phases require a mix of models, the models should be linked in a simple way to the production deployment models.
Another critical consideration is event control in lifecycle management. Without event control, resource processes can't signal conditions to lifecycle management and deployment processes to allow for automatic handling.
Event-handling in a DevOps tool is critical if it's to be used in orchestration. Generally, DevOps tooling that supports the emerging "Infrastructure-as-Code" model (as both Chef and Puppet do) will handle events, but check any other tools for compatibility with event-driven automation.
Keep an eye on the future
It's possible that the concept of orchestration as the automation of everything related to continuous delivery and Agile development will eventually subsume all the current tools into one continuous lifecycle-flow process. In fact, it seems that this is the only outcome that can really ensure that future software development practices can keep pace with business.
Vendors who offer all the right tools have the best chance of quickly assembling an "orchestrated" combination of them, so keep an eye on these players even if you rely on other tools. At the very least, they'll help frame your vision of an orchestrated future.
Brush up on best practices for building apps using cloud tools
What's the right language for microservice orchestration?
A guide to understanding your cloud computing tools