Database DevOps: Extending the Power of DevOps for Database Management

Organizations and teams that have successfully implemented DevOps culture in their work processes realizing its true value.

Its benefits?

• It bridges the gap between development and operations teams
• Enables automation
• Accelerates delivery
• Lowers security vulnerabilities
• Improves software performance.

Both developers and operations teams reap the fruits of speed and innovation. So why leave database administrators out of it?

Database administrators are responsible for ensuring an application’s availability, its performance, and incorporating changes that developers usually introduce to modify an application. But it seems like DBAs are facing a lot of heat by being outside of the entire DevOps pipeline.

Now the question is, what is the need to incorporate DevOps for the Database?

And if you are here solely for that reason, then this article is here to unpack all the intricacies that are there about database DevOps. Get to know about the concept, the process, the challenges developers, DBA, and IT managers usually face, and some popular solutions you can adopt to improve application delivery.

Database DevOps is the ‘second age of DevOps’ operations that organizations are slowly adopting. It is all about incorporating database code changes into the DevOps processes. Instead of waiting for DBAs to review code changes within the application at the very end, it gets the same done in the earlier stages by incorporating the right tools and teamwork.

The irony…?

Organizations implementing DevOps into their work processes are blatantly ignoring the changes that database administrators have to make right before an app is ready for production. As a result, despite the quick releases with a unified development and operations team, DBAs usually face the brunt of deploying late updates into production.

No matter how many application features or releases are done in a day, its deployment would ultimately be slow till the time DBAs do not verify the code changes. A primary cause of this behavior is the pre-DevOps culture where the database was the stronghold. It is the developer’s ignorance about the database changes and their dependency that usually creates the database bottleneck challenges.

With the increasing scale of database changes, it is natural to expect DBAs to be under constant pressure. They need to standardize the app development and database development process together.

The solution is to conjoin the database work process under a unified DevOps process. It’ll ensure that upcoming changes in the application code are coordinated with changes in the database code as well.

Now you might be wondering, what would that process look like?

The Database DevOps process consists of 3 main tasks ( of course, closely similar to implementing DevOps for application development).

• Centralizing Source Control:

A centralized source control system would have all the database codes stored in one repository for everyone’s access. Keeping static data, config files, and scripts in a unified source controlled environment makes it easy to synchronize and deploy changes on time, and roll them back in case of any issues.

• Synchronizing with CI/CD Pipelines:

Database operations can be automated alongside application releases for synchronizing with the continuous integration, continuous deployment, and continuous delivery processes. This would allow database administrators and operators to verify and test the code changes while integrating the application code as per the existing database dependencies.

• Continuous Testing and Monitoring:

Lastly, a centralized system coupled with CI/CD approaches makes it possible to check for flaws and security issues in the database at the early stages of development. Developers and administrators can identify and address the problems the moment a new build is ready and compiled in the repository.

Whenever I ask decision-makers if they have implemented DevOps for their database management processes, the reactions are not always positive.

I’m not surprised by that reaction. I understand the apprehension about handling databases. It primarily stems from the fact that they know the complexities of databases (or, should I say, only heard about it). But being developers, they feel they have the privilege of getting away from the complexities of databases.

It would be your loss if you conform to this idea of privilege.

A veracious DevOps demands a unifying team collaboration, irrespective of team members’ positions. Everyone has to collaborate!

To understand the real problem, let us first understand the job of a database administrator.

A 2019 survey mentions that approximately 57% of application changes require a change in the database code. This means that writing SQL codes, which is not exactly a favorite task for developers. A dreadful job means bad code, and a lousy code involves security risks. The job is ultimately pushed down to database administrators, who are the real heroes for reviewing all code, deploy database changes, and ensuring that the application runs flawlessly.

As organizations adapt to DevOps practices, DBAs face increasing pressure of delivering the database changes faster. Given that database management practices have always been done in the traditional manner, i.e., changing, verifying, and updating database codes weekly/monthly and manually, it makes deployment difficult. On top of this, there is also a mounting concern that speed will take over application quality, eventually affecting the application’s performance in the long run.

On a more technical side, database management is all about maintaining the state of the existing database and protecting the data despite the code changes. In the worst-case scenario, DBAs don’t even have the option to roll back a faulty application version after changing codes on the database – data restoration is time-consuming and very expensive. This is one of the reasons DBAs work carefully and follow methodological approaches to ensure that there are minimal application risks.

Overcoming this challenge is necessary if you want to keep up with the demanding market growth and competitors.

The solution is to enable DevOps for database management.

Extending the DevOps culture for database management can be equally, or even more, advantageous as it is for the software development process. More importantly, it can eliminate the bottleneck of slow data reviews and code changes in the database.

Let’s take a look at the benefits brought about by database DevOps for various roles.

For a CEO, the area of interest is the investment and profitability that Database DevOps can get for the organization.

• Faster time to market a product at a lowered cost.
• High-quality and secured products that consumers/users engage in.
• Improvement in operational efficiency.
• A reliable and stable infrastructure and product delivery cycle.
• Ensuring higher business efficiency with high customer satisfaction.

Going down the management ladder, the CIO is the next person who looks toward improving the IT processes within the entire organization. Their focus of interest is on quality service and product delivery.

• Improvement in operational support.
• Quick fixes and improved security for quality delivery.
• Automation and cross-skilling for a stable work process.
• Improved flexibility, agility, and collaboration in the team.
• Creating an environment for innovation and research.

Down the road, it is the IT managers and technical leads who are directly responsible for overseeing the management processes, bridging the gap in the managerial hierarchy and DBAs. Their concern is ensuring that the team or department gives out the best output for business growth.

• Keeping a check on the metrics and producing products on time.
• Reduced time, cost, and volume of defects on the product.
• Improved application performance, speed, and user engagement.
• Lowered risks, security issues, and failure rates.
• Maximum utilization of the required resources.

Lastly, it is the database administrator and manager who is directly taking the hits in the warzone. Their concern is to ensure that the code changes are verified, tested, and monitored, keeping in mind that it does not affect the application’s performance and availability.

• Shorter iterations for better data management.
• Consistent builds, testing, deployment, and release processes through version control.
• Keeping on track with application CI/CD processes for faster releases.
• Automating database provisioning and configuration management.
• Improved communication and collaboration with development and operations teams.

How you implement database DevOps for your work processes determine its success and the business benefits your organization would receive in the long run. While every organization would have its own implementation process, there are some standard step-by-step guidelines you can keep in mind.

• Understanding your business requirements

Assess the current state of your software development environment, its life cycle, the current database structure, and the SQL features that are already implemented and in use. Keep all your stakeholders in the loop to understand their vision, requirements, and understand what changes they are expecting to see.

• Creating a database project

At this stage, expect to have a number of compilation and referencing errors in the project. Resolve all these challenges one by one, identify and eliminate the outdated codes, and customize solutions as per your business requirements.

• Integrating the source code

Add your project to a source code repository, create the new branches that are required, and prepare for the new database development methodology.

• Implementing CI/CD processes

In this step, you won’t have to create a new CI/CD pipeline from scratch. Instead, you can integrate the new database project into the existing CI/CD processes.

• Testing the database

In the last step, create an automated unit testing framework for your database DevOps testing processes. Integrate this automation into the CI/CD pipeline to accelerate the entire DevOps process.

Initiating DevOps for your database is a path that is full of sticks and stones.
But fret not, if organizations across the globe are managing to accomplish this feat, so can you!

Let us look at some of the typical challenges organizations face while implementing Database DevOps.

Traditional development and operations tooling were never meant for database management purposes, therefore, leaving database admins out of the growth spectrum. I’m not saying that you can’t use the same tools, oftentimes, you would end up customizing services manually by yourself.

For instance, your ORMs and migration tools might function well on a system with SQLite database, but might fail on a production service. This might be because of scaling and loading issues where schema changes on a large database running on load might create an outage, or the tool might have internal dependencies and assumptions.

Database automation is the first step to glory, and this is the reason HP introduced Orchestrated Datacenter in 2014. It is an automation and cloud management solution that incorporates several open-source techs like Chef, Hadoop, and Openstack. The solution optimized their data center operations to meet the infrastructure and product delivery requirements by eliminating the repetitions and duplications while processing codes.

Utilizing a production database is one of the best ways to ensure that the codes and data are working properly as expected. But the problem usually arises when developers don’t have direct access to production data unless it’s encrypted and masked. While this ensures that all the data is secured, the process becomes a bottleneck when developers have to depend on someone from IT to implement the configurations.

The process is usually long, tedious, and time-consuming. Moreover, there are high chances that wrong specs are provided by the IT, even though the developer asked for something else. The worst part comes when the application is ready for release, but database changes make the application codes non-functional.

Etsy is a fantastic example of a company that embraced DevOps in the truest sense when they extended the process for their database as well. In order to save the time and cost it took for data provisioning, they shifted their database over to a cloud platform that allowed the entire team to function independently. This shift saved their computing energy by up to 50% and reduced costs by 42%.

Data persistence is a huge challenge that all developers would agree on, whenever changes are done to the application code. It is not easy to roll back an upgrade or a code from the database’s end. It is expensive, time-consuming, and can cause a colossal outage if not appropriately handled. This challenge is usually a result of inefficient tooling and improper company culture or processes that makes it difficult to process database changes along with the CI/CD pipeline.

Netflix utilized a polyglot persistence approach while leveraging their microservices application tiers stateless to deal with the data persistence issue that came with database DevOps practices. This solution allocated the data to the application’s relevant persistence tier and allowed developers to choose the appropriate data store for their use case. This made reading and writing data over a persistent database flexible and seamless.

Many developers and decision-makers assume that if there’s DevOps, there’s definitely going to be a microservices architecture. It is not entirely correct! Tightly-coupled architecture often poses a challenge for database administrators, especially when it is a centralized database.

The obvious solution is to shift to a microservices architecture where each service would interact with a well-defined database of its own. Services directly interacting with a centralized database can increase the load over the database, eventually slowing down the entire application. For developers and database admins, a microservices architecture would avoid data outages, degradation, and resource exhaustion.

When Adobe transitioned to delivering its products as cloud-based services, they became conscious of their data utilization, cloud usage, and how they would scale up. To meet the changing market demands of fast service delivery, they moved their Creative Cloud, Marketing Cloud, and Document cloud over to a microservices architecture. This step improved the speed at which they delivered their services, managed fixes quickly over the database, and eliminated security concerns while making changes.