Exploring the VMware Software Defined Datacenter (SDDC) - Part 8 - Data Services Manager

Welcome back to the VMware SDDC series for Part 8. 

The previous parts can be found here and cover the core components of the VMware Cloud Foundation (VCF) architecture around Storage and Networking, and the core elements of Compute and Operations. 

Parts 8 and 9 will cover some of the innovations that VMware by Broadcom is building onto the VCF foundation. These will be Private AI, Data Services Manager and Live Recovery. 

In this part, we will explore VMware Data Services Manager and, more importantly, why we are seeing the rise of Open-source Database platforms. 

Revolutionising Data Management in the Age of Open Source 

The database landscape is shifting with the adoption of open source, which is becoming mainstream in many enterprise organisations. As teams tackle the demands of varied transformation projects, the need for innovative, robust, and flexible data management solutions has never been more critical. Many organisations have seen a significant surge over the past decade; recent studies show that open-source databases can account for over 50% of deployments. Factors like cost savings, flexibility, and robust features drive this trend. 

Notably, a 2023 survey by Percona revealed that 79% of enterprises are running at least one open-source database in production; a significant increase from 55% in 2018. 

So why are enterprises considering open-source databases over the established Oracle and Microsoft SQL Server? In an evolving landscape of macroeconomic pressures, decision-makers are continually weighing the benefits of open source in general and databases specifically against the traditional proprietary options. Several compelling reasons can drive these decisions.   

Cost Efficiency: Open-source databases often have a lower total cost of ownership than proprietary databases. 

Flexibility and Customisation: Open-source databases offer more flexibility, allowing developers to tailor the software to meet their needs. 

Community and Innovation: The vibrant open-source community allows rapid innovation and continuous improvement. Enterprises benefit from the collective expertise of a global network of developers, which ensures the databases are regularly updated with the latest features. 

Avoiding Vendor Lock-in: Open-source solutions allow developers to choose and switch vendors without being tied to a single provider. This independence mitigates the risks associated with vendor lock-in, such as price hikes. 

Performance and Scalability: Most open-source databases are built to handle high-performance requirements and can scale efficiently to accommodate growing data needs. Technologies like PostgreSQL and MySQL have proven track records in large-scale, mission-critical applications. 

The opposing argument proposes a challenge of a lack of support services and enterprise features to enable day-two operations and consistent service levels in production environments. VMware is looking to augment the open-source platforms with Data Services Manager. When we consider that 25% of all vSphere workloads are data services-based, it’s a significant area of focus for VMware. 

The Open-Source Challenge 

Exploring the counter-position to the open-source movement a little more, we can see four areas that could be cause for concern. These issues are more prevalent in enterprise organisations that may have more stringent compliance, security, and operational rules to comply with. However, we must note that these are the same organisations that will need the speed and flexibility that open source can unlock. As always, it’s a balance.  

Operational Risk: A lack of a mature toolset and processes built around decades of proprietary database solutions could leave open-source solutions vulnerable to challenges like cyber risk. 

IT Operations: A lack of time, skills, and SLA-backed support could put traditional Infra teams at risk when providing support to production environments. 

Database Management: Consistent tooling and operating procedures can be lacking for provisioning, patching, and upgrading open-source environments, which could lead to more operational risk. 

Business Operations: Open source's strengths of flexibility and optionality must be balanced against operational needs around standardisation (to allow consistent operations), high availability, backup, restore, and cyber defence. The lack of standard operating processes could lead to significant risk. 

We now have a compelling list of advantages for moving towards an open-source strategy, but a conflicting list of challenges that any COO would appreciate having to manage. This is why VMware has developed Data Services Manager (DSM) 2.0, which helps organisations balance the outcomes and risks and deliver the best of both worlds. 

The VMware Data Services Manager (DSM)   

When we look at the database space, we have two distinct camps to consider: the traditional commercial offerings (DB2, SQL, Oracle, SAP HANA) and the modern open-source offerings (MySQL, MongoDB, PostgreSQL). VMware knows there is no value in developing solutions for the traditional offerings; the ecosystem is mature and proven. There is a significant opportunity in the open-source space to provide a management and deployment tooling framework and enterprise-grade support services. This is the focus of DSM, and for those who may have tested v1.0 of DSM, this is a new solution learned from those challenges.  

When we look at the Data Services Strategy from the VCF division at VMware, we can see that the mission is to deliver a suite of outcomes that can support traditional, modern, and AI application development. The VCF platform will provide consistent consumption, monitoring, and protection services for all the data services shown in the image below. For selected data services (open-source database is the focus of this article), VMware also provides enhanced management, templated hardening, deployment, and enterprise-grade certified support services. 

The goal is to provide that middle ground between the challenges and opportunities outlined above. Organisations can unlock the entire open-source potential by consuming a suite of data services that can simplify the deployment, and enhance day-two operations through monitoring and data protection.  

When we overlay this against the two database categories, we can see the elements in green provided by VMware and how they differ by database type.  

Taking a different view, we can see how the solution will help everyone in the lifecycle, from IT to the database admin and app developer. The aim is to simplify the end-to-end lifecycle while introducing standard operating procedures to ensure compliance. 

DSM offers six benefits to organisations looking to start or mature an open-source database strategy.  

• Unified Management Interface: DSM provides a single pane of glass for managing multiple database instances across different environments. This unified approach simplifies operations and reduces the administrative burden on IT teams. 

• Automated Deployment: With DSM, enterprises can automate the deployment of databases, reducing the time and effort required to set up new instances. This feature is precious in dynamic environments where rapid scaling is necessary. 

• Scalability: DSM's scalable architecture allows organisations to manage growing data volumes efficiently. It supports horizontal and vertical scaling, ensuring that database performance remains optimal as demand increases. 

• Monitoring and Analytics: DSM includes robust tracking and analytics tools that provide real-time insights into database performance. These tools help IT teams identify and address issues before they impact operations. 

• Security and Compliance: DSM prioritises security, which includes advanced security features such as encryption, access controls, and audit logging. Additionally, DSM helps organisations comply with industry regulations. 

• Support for Hybrid and Multi-cloud Environments: DSM is designed to work seamlessly across on-premises, hybrid, and multi-cloud environments. This flexibility ensures enterprises can choose the deployment model that best fits their needs. 

DSM supports all this on top of VMware Cloud Foundation 5.2 for PostgreSQL and MySQL as part of the 2.0.3 release from June 2024. The plan is to bring Google AlloyDB Omni to the platform, currently in tech preview. 

Summary 

Firstly, DSM is now GA and accessible for customers running VCF 5.2! This means you can start testing the technology for no additional cost. 

Secondly, we would love to explore the open-source pros and cons of your organisation and understand how your journey is progressing - the above is just one approach to balancing the benefits and challenges.  

Regardless of your maturity level, we have a growing trend, so we must incorporate it into the longer-term strategic plans for IT service provision.