Team Autonomy: Transition to Microservice Architecture
Numerous articles and books discuss microservice architecture, suggesting that all modern applications should already be defined as microservices. However, as you are aware, architecture always involves making trade-offs…
In the dynamic world of software development, microservices architecture emerges as a robust solution to tackle the challenges faced by modern applications. With many articles and books promoting its use, the attractiveness of microservices is clear. However, among this wealth of information, one crucial aspect stands out — the independence of development teams.
As we navigate the complexities of transitioning from large, integrated systems to agile microservices, this article emphasizes the pivotal importance of team autonomy. The concept of microservices, praised for its advantages, requires a deep understanding of trade-offs and strategic planning, especially in nurturing autonomy within development teams.
In the multitude of resources supporting microservices, it’s essential to recognize that making architectural decisions inherently involves making choices. In the early stages of this journey, determining the need for additional complexity becomes a nuanced task. A fundamental question arises: Is it more important to promptly test an idea today with the intention of gradually evolving the architecture as we gain a clearer vision of future business needs?
This article aims to delve into the intricacies of the transition, highlighting the vital role of intentional decision-making and careful planning in empowering development teams with autonomy. Through an exploration of key steps, patterns, and potential challenges, our goal is to illuminate the path toward the successful adoption of microservices, with a specific focus on enhancing team autonomy in the ever-evolving landscape of software architecture.
The Paradox of Success
A blessing for business, yet a hurdle for engineering
This story reflects a common pattern of success in today’s applications. On one side, these applications achieve significant popularity and profitability. On the other side, the development teams face a dilemma — they struggle to find time to update and improve their applications due to their overwhelming success in the business world. The continuous demand from customers for new features makes it difficult to focus on architectural enhancements. While it’s a triumph in the business realm, it presents challenges from an engineering perspective.
Typically, these projects feature millions of lines of code and intricate architectures with interconnected modules. Managing such codebases often involves coordinating multiple teams. As a result, seemingly simple tasks become challenging when numerous engineers are involved. For example, updating the web framework to the latest version requires extensive coordination, reaching out to various engineers, understanding their ongoing projects, and anticipating potential impacts. The consequence is a gradual slowdown of these projects as communication demands increase.
During these crucial moments, it becomes essential to consider a strategic shift in the architectural landscape. Certain metrics can guide such decisions, such as evaluating the number of lines of code per developer and examining the cyclomatic complexity of the codebase. This introspective approach can act as a catalyst for a much-needed turnaround, providing an opportunity to revamp strategies and bring about a positive change in the current architectural direction.
Defining Subsystems
Subsystems define the path to autonomy
A crucial initial step in shifting to microservices involves defining subsystems, which can be one of the most complex aspects of the project. Initially, monolithic systems display intricate connections among all elements, with each component closely linked to others. To address this, strategic decoupling can be achieved by specifying distinct services for an internet store, such as:
- User Authorization / User Authentication
- Search
- Customer Feedback and Rating System
- Billing and Payments
and so on.
When carrying out this separation, it is essential to focus on creating vertical pillars within the application, avoiding horizontal divisions like Web UI, API, and Database. Although these layers still need separation, managing them under one cohesive team can be beneficial for achieving autonomy. This approach ensures that the subsystems align with the business logic and functionalities, promoting a more streamlined and effective transition to a microservices architecture.
The Strangler Pattern
It allows systems to evolve gracefully, much like nature’s strangler figs replacing host trees.
As teams define the optimal boundaries and prepare to start the process of breaking down monoliths into independent services, an effective and gradual transition approach is the Strangler Pattern. This software design and architectural pattern, introduced by the esteemed software engineer and author Martin Fowler in his book “Patterns of Enterprise Application Architecture” gets its name from the behavior of certain plants, like strangler figs, that slowly envelop and replace their host trees in nature.
In the realm of software development, the Strangler Pattern serves as a method for the gradual and non-disruptive modernization of legacy systems. It tackles the challenges posed by aging monolithic applications — typically challenging to maintain, inflexible, and lacking modern technological capabilities.
One effective strategy for transitioning applications towards a more robust state involves using routing at the infrastructure level, employing tools such as a reverse proxy like NGINX or an Application Load Balancer within the context of Public Cloud Providers like AWS. With a simple configuration, these tools enable the initiation of the first steps in the transition process, allowing for a methodical and initiative-driven transformation.
Escaping the Shared Database Antipattern
Shared databases hinder autonomy
A critical point to highlight during the shift to microservices is the importance of avoiding the antipattern of utilizing a shared database among services. To illustrate this, let’s step back and consider a scenario where multiple services, each supported by different teams, opt for a shared database. This approach unintentionally creates a lack of ownership over the storage system. From an engineering perspective, a database essentially acts as an interface on top of a hard drive. Examining the internal workings of database engines reveals that it’s not mysterious; instead, it represents a state-of-the-art implementation of algorithms working on the hard drive and utilizing other resources like memory for caching database pages.
Using a shared hard drive poses a significant challenge to team autonomy. One team might inadvertently exceed usage limits in the Cloud, creating an imbalance. More critically, in the context of databases, a new pattern related to database requests introduced by one team, combined with a missing index, could lead to performance degradation for all other teams. Such a scenario contradicts the essence of autonomy, which is a core objective of adopting microservices. The primary goals are to enhance team autonomy and improve system reliability. Hence, I strongly recommend avoiding the use of shared databases as it represents an antipattern in the microservices paradigm.
Embracing Messaging for Autonomy
Messaging systems help microservices communicate easily and independently.
While the increased autonomy within our teams is undoubtedly a significant improvement, it’s crucial to recognize that achieving complete isolation of applications and domains is often an idealistic scenario. In many real-world cases, connections between different entities persist, and initially, it can be challenging to reimagine everything. In such instances, I recommend exploring messaging communication as a viable solution between various services. Popular options include RabbitMQ, Kafka, or AWS SNS and AWS SQS, or their equivalents in the case of public cloud providers.
Utilizing messaging communication allows for the establishment of connections between different services by transmitting necessary data. This approach introduces a level of decoupling to the system that enables teams to maintain a higher degree of autonomy. It facilitates a transition to a new echelon of service reliability, allowing for seamless communication and data sharing while still preserving the autonomy gained in the microservices framework.
Conclusion
In summary, moving from big software systems to microservices is a crucial journey for modern development teams. While the benefits are clear — more freedom, better reliability, and increased scalability — there are challenges along the way.
In this article, we’ve discussed different strategies, like the Strangler Pattern, and emphasized avoiding problems like shared databases. Each step in the transition needs careful planning.
We’ve also stressed the need to recognize real-world complexities where full isolation might not be possible. In such cases, using messaging communication between services can be a practical solution, balancing autonomy with the reality of interconnected systems.
The move to microservices isn’t just a technical change — it’s a strategic shift for the whole organization. By following best practices, staying flexible, and promoting teamwork, teams can successfully navigate this journey and unleash the full power of microservices for innovation and delivering value to customers.
Related Articles
Building Autonomy for Software Engineering Teams
Team Autonomy: Organizational Anti-Patterns
Want to Connect?
Yury Fedoseev's LinkedIn
Vancouver FullStack Developers Group