There are several benefits to using microservices. For one, because these smaller applications are not dependent on the same coding language, the developers can use the programming language that they are most familiar with. That helps developers come up with a program faster with lower costs and fewer bugs. The previous monolith could be transformed using the microservies pattern into the following: Advantages of Microservices Evolutionary Design. One of the biggest advantages of the microservices pattern is that it does not require you to rewrite your whole application from the ground up.
Posted on March 17, 2016
We are in the midst of an application development and IT system management revolution driven by the cloud. Fast, agile, inexpensive, and massively scalable infrastructure, offered fully self-service and with pay-as-you-go billing, is improving operational efficiency and enabling faster-time-to-value across industries. The emergence of containers, with their fast startup, standardized application packaging, and isolation model, is further contributing to efficiency and agility.
However, many companies are finding that making their applications highly available, scalable and agile is still challenging. Competitive business pressures demand that applications continuously evolve, adding new features and functionality while remaining available 24x7. For example, it is no longer acceptable for a bank website to have a maintenance window, whereas even a few years ago it was the norm. Similarly, an e-commerce site that’s down for even a short time will drive customers to one of many competitors that can serve them at that moment. Failure to meet these demands can mean the difference between staying relevant and losing business.
These business realities are driving developers to adopt an application architecture model called “microservices,” a term popularized by James Lewis and Martin Fowler. In this post, I'll talk about how and why a microservices architecture can help with application development and lifecycle tasks, and describe the capabilities that platforms can provide to support those architectures. Then I’ll list some of the platforms commonly used by developers as the foundation for their microservice based applications that Azure supports, and finally, I’ll briefly describe our microservice application platform, called Service Fabric, that provides comprehensive support for microservices lifecycle management out of the box.
Monolithic application models
For decades, the cost, time, and complexity of provisioning new hardware, whether physical or virtual, has strongly influenced application development. These factors are more pronounced when those applications are mission-critical, since high uptime requires highly available infrastructure, including expensive hardware such as SANs and hardware load-balancers. Because IT infrastructure is static, applications were written to be statically sized and designed for specific hardware, even when virtualized. Even when applications were decomposed to minimize overall hardware requirements and offer some level of agility and independent scaling, it was commonly into the classic three-tier model, with web, business logic and data tiers, as shown in the figure below. However, each tier was still its own monolith, implementing diverse functions that were combined into a single package deployed onto hardware pre-scaled for peak loads. When load caused an application to outgrow its hardware, the answer was typically to “scale up,” or upgrade the application’s hardware to add capacity, in order to avoid datacenter reconfiguration and software re-architecture.
Figure 1. Three-Tier Monolithic Application
The monolithic application model was a natural result of the limitations of infrastructure agility, but it resulted in inefficiencies. Because static infrastructure and long development cycles meant that there was little advantage to decomposing applications beyond a few tiers, developers created tight coupling between unrelated application services within tiers. A change to any application service, even small ones, required its entire tier to be retested and redeployed. A simple update could have unforeseen effects on the rest of the tier, making changes risky and lengthening development cycles to allow for more rigorous testing. Their dependence on statically-assigned resources and highly-available hardware made applications susceptible to variations in load and hardware performance, which could push them outside their standard operating zone and cause their performance to severely degrade. An outright hardware failure could send the entire application into a tailspin.
Finally, another challenge facing monolithic applications that took advantage of a tiered approach was delivering fast performance with data stored in the backend tier. A typical approach was to introduce intermediate caches to buffer against the inefficiencies caused by separating compute and data, but that raised costs by adding unused hardware resources, and it created additional development and update complexities.
Figure 2. Three-Tier Monolithic Application with Caches
Microservices architecture
While there are simple and limited scale applications for which a monolithic architecture still makes sense, microservices are a different approach to application development and deployment, one that is perfectly suited to the agility, scale and reliability requirements of many modern cloud applications. A microservices application is decomposed into independent components called “microservices,” that work in concert to deliver the application’s overall functionality. The term “microservice” emphasizes the fact that applications should be composed of services small enough to truly reflect independent concerns such that each microservice implements a single function. Moreover, each has well-defined contracts (API contracts) – typically RESTful - for other microservices to communicate and share data with it. Microservices must also be able to version and update independently of each other. This loose coupling is what supports the rapid and reliable evolution of an application. Figure 3 shows how a monolithic application might be broken into different microservices.
Figure 3. Breaking the Monolith into Microservices
Microservice-based applications also enable the separation of the application from the underlying infrastructure on which it runs. Unlike monolithic applications where developers declare resource requirements to IT, microservices declare their resource requirements to a distributed software system known as a “cluster manager,” that “schedules,” or places, them onto machines assigned to the cluster in order to maximize the cluster’s overall resource utilization while honoring each microservice’s requirements for high availability and data replication, as shown in Figure 4. Because microservices are commonly packaged as containers and many usually fit within a single server or virtual machine, their deployment is fast and they can be densely packed to minimize the cluster’s scale requirements.
Figure 4. Cluster of Servers with Deployed Microservices
With this model, microservice scale-out can be nearly instantaneous, allowing an application to adapt to changing loads. Their loose coupling also means that microservices can scale independently. For example, the public endpoint HTTP listener, one microservice in the Web-facing functionality of an application, might be the only microservice of an application that scales out to handle some additional incoming traffic.
The independent, distributed nature of microservice-based applications also enables rolling updates, where only a subset of the instances of a single microservice will update at any given time. If a problem is detected, a buggy update can be “rolled back,” or undone, before all instances update with the faulty code or configuration. If the update system is automated, integration with Continuous Integration (CI) and Continuous Delivery (CD) pipelines allow developers to safely and frequently evolve the application without fear of impacting availability.
While the classic model for application scalability is to have a load-balanced, stateless tier with a shared external datastore or database to store persistent state, stateful microservices can achieve higher performance, lower latency, massive scale and maintain developer agility for service updates. Stateful microservices manage persistent data, usually storing it locally on the servers on which they are placed to avoid the overhead of network access and complexity of cross-service operations. This enables the fastest possible processing and can eliminate the need for caches. Further, in order to manage data sizes and transfer throughputs beyond that which a single server can support, scalable stateful microservices partition data among their instances and implement schema versioning so that clients see a consistent version even during updates, regardless of which microservice instance they communicate with.
Microservice application platforms
Our experience internally running cloud-scale Microsoft services like Bing, Cortana and Intune have given us a deep firsthand understanding of the complexities associated with designing, developing, and deploying large-scale applications at cloud scale. Frequent updates to large-scale, always on applications is a challenge, no matter how well designed the application. Just dropping microservices into virtual machines or even containers does not enable the full potential of the microservices approach, that requires a microservices application platform with DevOps-focused tooling.
A full-featured microservices application platform provides all the previously stated microservices architecture benefits of cost efficiency, scalability and 24x7 availability. It also takes these a step further. As mentioned earlier, it should choreograph safe and reliable upgrades using an extensible health model and automatic rollback. It also helps microservices to discover each other by providing a naming service, and monitors and maintains their health. For example, when scaling or healing, a microservices platform communicates updated placement information to other microservices via the naming service so that they can quickly establish or re-establish communication.
To keep microservices healthy, the platform automatically moves instances to healthy VMs or servers when the software or hardware on which they are running fails or must be restarted for upgrades. In addition, a microservices platform must be deployable in private and public clouds. This is necessary to support hybrid scenarios where workloads burst from a private cloud to a public one, and to enable public cloud dev/test with production deployment in a private cloud. Supporting multiple clouds also addresses concerns about vendor lock-in when choosing an application platform, separating the platform from the infrastructure.
Figure 5. Microservices Platform
This section briefly describes several popular platforms on which developers are building and deploying microservices applications today. All of the below can run on Azure infrastructure, giving you choice based upon your requirements.
Docker Swarm and Docker Compose
The standard packaging format and resource isolation of Docker containers have made them a natural fit with microservices architectures. Docker Compose defines an application model that supports multiple Docker-packaged microservices, and Docker Swarm serves as a cluster manager across a set of infrastructure that exposes the same protocol that a single-node Docker installation does, so that it works with the broad Docker tooling ecosystem. The Azure Container Service supports both Docker Swarm and Docker Compose. More details running this on Azure can be found here.
Kubernetes
Kubernetes is an open-source system for automating deployment, operations, and scaling of containerized applications. It groups containers that make up an application into logical units for easy management and discovery. Originally developed by Google, it builds on their experiences running large scale services such as Search and Gmail. Even some traditional PaaS solutions are merging with Kubernetes, like Apprenda. More details running Kubernetes on Azure can be found here.
Mesosphere DCOS, with Apache Mesos and Marathon
Microsoft and Mesosphere have partnered to bring open source components of the Mesosphere Datacenter Operating System (DCOS), including Apache Mesos and Marathon, to Azure. The Mesosphere DCOS, powered by Mesos, is a scalable cluster manager that includes Mesosphere’s Marathon, a production-grade container orchestration tool. It is available as part of the Azure Container Service. There is also an enterprise version available from Mesosphere that runs on Azure. The Mesosphere DCOS provides microservices platform capabilities including service discovery, load-balancing, health-checks, placement constraints, and metrics aggregation. Finally, Mesosphere offers a library of certified services that provide additional capabilities, like Kafka, Chronos, Cassandra, Spark and others, which can all be installed with a single command. More details running this on Azure can be found here.
OpenShift
OpenShift by Red Hat is a platform-as-a-service offering that leverages Docker container-based packaging for deploying container orchestration and compute management capabilities for Kubernetes, enabling users to run containerized JBoss Middleware, multiple programming languages, databases and other application runtimes. OpenShift Enterprise 3 offers a devops experience that enables developers to automate the application build and deployment process within a secure, enterprise-grade application infrastructure. With recent support for Red Hat Enterprise Linux images in Azure, OpenShift is supported in Azure. Look for some documentation on this topic shortly.
Pivotal Cloud Foundry
Pivotal Cloud Foundry enables microservice architectures by combining workflow and container scheduling from Cloud Foundry with integrations for microservice patterns like service discovery, client side load balancing, circuit breakers and distributed tracing, utilizing Spring Cloud and NetflixOSS. Pivotal Cloud Foundry supports ongoing microservice operations via deployment and service management capabilities such as autoscaling, blue-green updates, health monitoring, application metrics, streaming logs and more. Learn more about running Pivotal Cloud Foundry on Azure here.
Service Fabric
To support our own internal evolution from on-premises to cloud and from monolithic to microservice-based applications, we developed Service Fabric over ten years ago. Service Fabric powers many of our hyper-scale cloud services, including SQL DB, DocDB, Intune, Cortana and Skype for Business, as well as many internal Azure infrastructure services. We’ve taken the exact same technology and released Service Fabric as-a-service on Azure, and the standalone SDK will enable the deployment of Service Fabric applications in on-premises clusters and on other clouds. In its initial public release, Service Fabric runs on Windows with any .NET language and Linux and Java support are under development. Service Fabric has built-in support for lifecycle management, hybrid deployments, and 24x7 availability with integrated development experiences using Visual Studio. The platform offers extensible health models for both the infrastructure and microservices to enable automated health-based upgrade and automatic rollback, simplifying DevOps. Furthermore, Service Fabric supports both stateless and stateful microservices with leadership election to support data consistency and a state replication framework that supports transactions for stateful data guarantees. More details running this on Azure can be found here.
Conclusion
The computing world has changed forever with the advent of the cloud. Cloud gives developers access to infrastructure instantly, cheaply, and at nearly infinite scales. The agility of cloud and high availability and constant agility demands of modern business have strained monolithic architectures and resulted in the rise of microservices-based applications. With a comprehensive microservices platform, developers can create applications that support massive scale with high performance, high availability, cost effectiveness and independent lifecycle management, across public clouds and private clouds. Microservices are an application revolution powered by the cloud.
Check out the video on below where I get to talk Microservices with Channel9’s Seth Juarez.
by Jake Lumetta
My good friend Darby Frey recently kicked off a greenfield project after assuming his new role as Sr. Platform Engineering Lead of Gamut. Despite starting out with monolith at his previous company Belly, he discovered that — in the right circumstances — starting with a monolith isn’t always the best way to go.
At Belly, Darby and his team broke down their monolith into a fairly large microservices architecture. They managed to get it to a good place, but only after months of trials and tribulations migrating to microservices.
With this experience fresh in his mind, he approached his new project at Gamut a bit more cautious of microservices.
“I was firmly a member of Team Monolith. [I thought] let’s build a single application and just pull things apart later if we start to feel pain,” he said.
While this was a greenfield project, Darby’s team was small, and he had aggressive timelines, so on the surface, a monolith seemed like the obvious choice.
“[But with this new project], I was anxious to not repeat the mistakes of the past.”
And with that, he found himself faced with a decision we all struggle with: should we start with a monolith or microservices, and how do we decide?
Conventional Wisdom About Monoliths
Many startups begin with a monolith, because its very structure lends itself well to small teams and offers other advantages like less operational overhead. Also, monoliths often just have one massive code base. Server side application logic, front end client side logic, background jobs, etc, are all defined in the same code base. This means that if developers want to make any changes or updates, they need to build and deploy the entire stack all at once.
A monolith isn’t a dated architecture that we need to leave in the past. In certain circumstances, a monolith is ideal. I spoke to Steven Czerwinski, Head of Engineering at Scaylr and former Google employee, to better understand this.
“Even though we had had these positive experiences of using microservices at Google, we [at Scaylr] went [for a monolith] route because having one monolithic server means less work for us as two engineers,” he explained. This was back in the early beginnings of Scalyr.
In other words, because his team was small, a unified application was more manageable in comparison to splitting everything up into microservices.
The Alternative of Microservices
Zachary Crockett, CTO of Particle, told me this during an interview:
“When discussing microservices, people tend to focus on one end of that spectrum: many tiny applications passing too many messages to each other. At the other end of the spectrum you have a giant monolith doing too many things. For any real system, there are many possible service oriented architectures between those two extremes.”
The microservice architectural style is an approach to developing a single application as a suite of small services. Each runs in its own process and communicates with lightweight mechanisms, often an HTTP resource API.
These services are built around business capabilities and are independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and may use different data storage technologies.
Microservices do have distinct advantages:
- Better Organization: Microservice architectures are typically better organized. Each microservice has a very specific job, and it is not concerned with the jobs of other components.
- Decoupled: Decoupled services are also easier to recompose and reconfigure to serve the purposes of different apps (for example, serving both the web clients and public API). They also allow for fast, independent delivery of individual parts within a larger, integrated system.
- Performance: Under the right circumstances, microservices can also have performance advantages depending on how they’re organized. It’s possible to isolate hot services and scale them independently of the rest of the app.
- Fewer Mistakes: Microservices enable parallel development by establishing a hard-to-cross boundary between different parts of your system. By doing this, you make it hard — or at least harder — to do the wrong thing: namely, connecting parts that shouldn’t be connected, and coupling too tightly those that need to be connected.
But they also create unique challenges:
- Cross-cutting Concerns Across Each Service: As you’re building a new microservice architecture, you’re likely to discover lots of cross-cutting concerns that you did not anticipate at design time. You’ll either need to incur the overhead of separate modules for each cross-cutting concern (i.e. testing), or encapsulate cross-cutting concerns in another service layer that all traffic gets routed through. Eventually, even monolithic architectures tend to route traffic through an outer service layer for cross-cutting concerns, but with a monolithic architecture, it’s possible to delay the cost of that work until the project is much more mature.
- Higher Operational Overhead: Microservices are frequently deployed on their own virtual machines or containers, causing a proliferation of VM wrangling work. These tasks are frequently automated with container fleet management tools.
Which Model Is Right For You?
Interviews with dozens of CTOs taught us that one approach does not fit all. Their experiences serve as a useful rubric for you to decide between a monolith and microservices. Answering these questions should guide you in the right direction:
Are You In Familiar Territory?
Darby and his team at Gamut were able to delve directly into Microservices because he had experience with eCommerce platforms, and his company had a wealth of knowledge concerning the needs and demands of their customers. If he had been traveling down an unknown path, a monolith may have actually been the safer option.
Similarly, startups are often born out of pains experienced at previous companies. In those scenarios, sometimes it’s quite clear that scaling is going to be a primary requirement, especially in infrastructure-based services like cloud log management.
Is Your Team Prepared?
Does your team have experience with microservices? What if you quadruple the size of your team within the next year — are microservices ideal for that situation? Evaluating these dimensions of your team is crucial to the success of your project.
Julien Lemoine, CTO at Algolia, chimed in on this point:
“ We have always started with a microservices approach. The main goal was to be able to use different technology to build our service, for two big reasons:
1) We want to use the best tool for each service. Our search API is highly optimized at the lowest level and C++ is the perfect language for that. That said, using C++ for everything is a waste of productivity, especially to build a dashboard!
2) The want the best talents and using only one technology would limit our options. This is why we have different languages in the company, Go is less perfect than C++ when you want to optimize everything at the millisecond level but is the perfect language when performance is still key (processing of logs where we process several terabytes of logs per day, using ruby or python would be a waste of CPU).”
If your team is prepared, starting with microservices is wise as it allows you to get used to the rhythm of developing in a microservice environment, right from the start.
How’s Your Infrastructure?
In reality, you’re going to need cloud-based infrastructure to make microservices work for your project. David Strauss, CTO of Pantheon, explained it thus:
“[Previously], you would want to start with a monolith because you wanted to deploy one database server. The idea of having to set up a database server for every single microservice and then scale out was a mammoth task. Only a huge, tech-savvy organization could do that.
Whereas today with services like Google Cloud and Amazon AWS, you have many options for deploying tiny things without needing to own the persistence layer for each one.”
Evaluate The Business Risk
You may think microservices is the “right” way to go as a tech-savvy startup with high ambitions. But microservices pose a business risk. David Strauss explained:
“A lot of teams overbuild their project initially; everyone wants to think their startup will be the next unicorn and that they should, therefore, build everything with microservices or some other hyper-scalable infrastructure. But that’s usually wrong, almost all the time.”
One example of this from his early days at Pantheon was a system that was limited to a single VM. They thought it would be a month or two until they’d be forced to scale it. It ended up being over a year — and they ended up scaling it a completely different way than they had anticipated.
He went on to say that, in these cases, the areas that you thought you needed to scale are probably not the parts that will need to scale first. This results in misplaced effort even for the systems that will need to scale.
Context is Key — Rules of Thumb For Choosing an Architecture
We’ve taken hours of interviews with successful CTO’s and distilled them into general guidelines to follow when choosing your service architecture.
When To Start With A Monolith
Here are some scenarios that indicate that you should start your next project using monolithic architecture.
- Your Team Is At Founding Stage: Your team is small, between 2–5 members, and is thus unable to tackle a broader and high-overhead microservices architecture.
- You’re Building An Unproven Product or Proof of Concept: Are you building an unproven product in the market? If it’s a new idea, it’s likely going to pivot and evolve over time, so a monolith is ideal to allow for rapid product iteration. Same applies to a proof of concept where your goal is just to learn as much as possible as quickly as possible, even if you end up throwing it away.
- You Have No Microservices Experience: If your team has no prior experience with microservices, unless you can justify taking the risk of learning “on the fly” at such an early stage, it’s likely another sign you should stick to a monolith to start.
When To Start With Microservices
Here are some scenarios that indicate that you should start your next project using microservices:
- You Need Quick, Independent Service Delivery: Microservices allow for fast, independent delivery of individual parts within a larger, integrated system. Note that, depending on your team size, it can take time to see service delivery gains versus starting with monolith.
- A Piece of Your Platform Needs to Be Extremely Efficient: If your business is doing intensive processing of petabytes of log volume, you’ll likely want to build that service out in a very efficient language (i.e. C++) while your user dashboard may be built in Ruby on Rails.
- You Plan To Grow Your Team: Starting with microservices gets your team used to developing in separate small services from the beginning. And having teams separated by service boundaries makes it much easier to scale up your team when you need to without introducing exponential complexity.
Choose the Architecture You Can Comfortably Live With
Microservices has become an increasingly popular service architecture, but it’s essential to understand whether it’s the best fit for your project. Your own context, evaluated against the above considerations, is the key to deciding if you should start with monolith or microservices.
Jake is the CEO of ButterCMS. For more content like this, follow @ButterCMS on Twitter and subscribe to our blog.