Transition to Cloud Native Architectures
It doesn't mean just moving to the cloud..
Cloud native is currently a focal point of discussion within the IT sphere, sparking considerable debate. Moving to the cloud marks the beginning of updating the IT setup.
Moving digital assets from on-premises servers to cloud-based platforms is known as cloud migration. This transition allows organizations to take advantage of cloud computing's scalability, flexibility, and cost-effectiveness. Once the migration is done, the company sets the stage for a newer, more flexible, and safer IT setup. But after the initial move, many companies hit a snag and slow down. The real power of using the cloud comes from updating applications and how they're built, bringing out some really good advantages. Often, companies find it hard to explain why updating is valuable and how it helps the business.
Fundamentals around Cloud Native Architectures
Typically, cloud-native architecture comprises five key building blocks:
Microservices - Microservices are used to divide big, monolithic applications into smaller, independent services. Each service focuses on a specific function or capability, allowing for independent development, deployment, and scalability. This approach promotes agility by enabling teams to iterate rapidly, update deployments frequently, and scale specific parts of the application without impacting the entire system.
Containers - Containers are like little packages that hold all the things an application needs to run, like code, libraries, and settings. They make it easy to move applications around different environments, from development to testing to production. Platforms like Docker help with creating, managing, and organizing these containers.
Orchestration - Orchestration involves automating the deployment, scaling, and management of containerized applications. Container orchestration platforms, such as Kubernetes, allow organizations to manage large-scale container deployments efficiently. They provide features like automated scaling, service discovery, load balancing, and health monitoring, enabling applications to run reliably and at scale in dynamic environments.
Serverless - Serverless computing simplifies infrastructure management, freeing up developers to concentrate on coding. In a serverless framework, applications are divided into functions that are activated by events from outside sources like HTTP requests, database updates, or file uploads. Services like AWS Lambda, Azure Functions, and Google Cloud Functions provide serverless platforms that scale and maintain infrastructure automatically, billing users solely for the resources used during operation.
DevOps - DevOps is all about working together, communicating effectively, and using automation to make things run smoothly between the development and operations teams. By getting rid of barriers and promoting a sense of shared duty, DevOps makes it easier to deliver software quickly and consistently. Key DevOps activities involve constant integration, ongoing delivery, treating infrastructure as code, automating tests, and keeping an eye on performance. Embracing DevOps ideas can help businesses simplify their development processes, speed up product launches, and raise the bar for software excellence and reliability.
Why Companies Need to Switch to Cloud Native?
Leveraging the Cloud Native model offers numerous advantages when building new applications instead of overhauling traditional ones. The primary goal remains consistent: swiftly and efficiently delivering innovative solutions to both new and existing users who demand improved experiences and fresh functionalities. Utilizing the appropriate orchestration, structures, and specialized technologies facilitates meeting these demands.
Consider these five compelling arguments in favor of adopting the Cloud Native model:
Swift Market Entry: Cloud-native development facilitates the rapid deployment of new features, enabling immediate user access. This approach encourages efficiency and flexibility, directly stimulating business growth by allowing scalable resources to meet user demand promptly. Continuous integration and self-service options enable companies to be more responsive and relevant with app updates, enhancing customer satisfaction and loyalty.
Dynamic Performance Enhancement: The cloud-native model enables agile delivery of performance enhancements in response to changing customer expectations. Flexibility to make real-time changes ensures a higher likelihood of meeting customer expectations on the first attempt. Continuous delivery of even minor alterations enhances user experience and satisfaction.
Competitive Edge & Customer Satisfaction: Embracing cloud-native technologies ensures a consistently positive customer experience while staying competitive. Greater capabilities, such as integrating artificial intelligence or blockchain services, may even position companies ahead of competitors.
Efficient Modernization & Downtime Minimization: The cloud-native model eliminates the need for extensive modifications to entire applications, saving resources and minimizing downtime. Efficient modernization options, such as containerization on a public cloud, streamline deployment and facilitate ongoing modernization efforts without disrupting operations.
Enhanced Collaboration & Integration: Adopting the cloud-native model fosters collaboration with other companies, cloud providers, and related applications. This approach facilitates the development of microservices architecture, allowing for seamless collaboration between devops teams and partners to deliver integrative features and functionality.
Widespread adoption of the cloud-native model is evident across various industries, including government, blockchain consortiums, financial services, and health research. Embracing this model opens opportunities for collaboration and partnership on future projects.
Real-Life Use Cases and Business Outcomes
Skycanner
Skyscanner’s cloud architecture
Skyscanner's decision to fully migrate to AWS was largely driven by the need to refresh its data center hardware. Faced with the choice of reinvesting in hardware or migrating to the cloud under a tight deadline, they chose the latter. The migration involved five global data centers, each hosting over 7,000 virtual machines and more than 300 services owned by various engineering teams within Skyscanner, making it a highly complex undertaking.
Skyscanner operates under a decentralized approach, where product-aligned teams are responsible for their own services. Each team formulated and executed its migration plan, guided by project roadmaps outlining milestones and deadlines.
While most teams initially aimed for a cloud-native rewrite, it became clear that this approach wasn't feasible for some applications. Having flexible roadmaps allowed teams to pivot to alternative strategies, such as rehosting, when necessary.
Two Skyscanner applications highlight different approaches to cloud-native maturity post-migration:
The rewrite of Skyscanner's Flight Stack demonstrates a strategic development of cloud-native principles over time, albeit with a lengthy process spanning over two years.
The re-platforming of Skyscanner's translation tool, Strings-as-a-Service, was a simpler process but still presented challenges. In-flight decisions were necessary to ensure a swift and smooth migration, enabling seamless operation post-migration.
Netflix
Netflix’s cloud architecture
In August 2008, Netflix started moving its operations to the cloud following a significant database issue that caused DVD deliveries to stop for three days. Understanding the importance of avoiding potential system failures, Netflix opted for Amazon Web Services (AWS) due to its flexible scaling and wide range of services. By 2015, the majority of Netflix's systems, including those serving customers directly, had made the transition to the cloud. It wasn't until January 2016, after seven years of dedicated effort, that Netflix completed its cloud migration and shut down its remaining data center bits.
The transition to the cloud brought several benefits to Netflix. With eight times as many streaming members as in 2008, and viewing growing significantly, the cloud's elasticity enabled rapid expansion to over 130 new countries in 2016. Leveraging AWS's global infrastructure, Netflix dynamically scales and shifts its global capacity to ensure an optimal streaming experience for members worldwide.
Netflix relies on the cloud for all scalable computing and storage needs, including business logic, distributed databases, recommendations, and content delivery through Netflix Open Connect. Despite initial challenges, the cloud has significantly increased service availability, approaching the desired goal of four nines of uptime. By embracing redundancy and graceful degradation principles, Netflix minimizes the impact of failures on member experience.
While cost reduction wasn't the primary driver for cloud migration, Netflix's cloud costs per streaming start are a fraction of those in data centers, thanks to cloud elasticity and economies of scale. However, completing the migration took seven years of hard work and involved rebuilding technology and fundamentally changing company operations. The shift to a cloud-native approach, with hundreds of microservices and NoSQL databases, has positioned Netflix for continued growth and innovation.
As Netflix continues to evolve its streaming technology, it aims to address challenges and collaborate with industry experts through initiatives like Netflix Open Source.
Shopify
Shopify’s cloud architecture
Shopify aimed to enhance its capabilities to meet growing customer demands by leveraging container-based cloud benefits for better support. Transitioning from self-managed data centers to a cloud-based infrastructure, they sought predictable builds, robust rollbacks, and elimination of configuration drift.
Utilizing Google Cloud, developing a "Shop Mover" database migration tool, and employing Docker containers and Kubernetes, Shopify transformed its data center to bolster customer support for online shops, achieving all objectives. This transition provides Shopify customers with increasingly scalable and resilient applications, ensuring enhanced consistency, reliability, and version control.
Immutable infrastructure significantly enhances deployments by minimizing or eliminating configuration drift, ensuring easier, simpler, and more consistent deployments.
Scalability is crucial for dynamic e-commerce businesses like Shopify, enabling servers to flexibly meet fluctuating demand without disrupting commerce operations. This flexibility is vital for handling flash sales events, such as those hosted by Kylie Cosmetics on Shopify's platform.
Spotify
Spotify’s cloud architecture
Recognizing the limitations of their in-house data centers in provisioning and maintenance, Spotify aimed to enhance innovation by transitioning to Google Cloud Platform (GCP) in 2015. Their goal was to streamline operations, minimize disruption to product development, and simplify the hybrid operation.
Over two years of preparation, Spotify formed a dedicated migration team to oversee the transition to GCP. The migration was divided into two phases: services and data, each taking a year. Engineering teams migrated services to the cloud in focused sprints, while data migration offered options for "forklifting" or rewriting. This migration increased scalability and freed up developers to innovate.
Stakeholder buy-in is essential. By involving engineers in the vision, Spotify gained enthusiastic advocates for the migration.
Thorough preparation is crucial. Spotify's dedicated team carefully researched cloud strategies, mapped dependencies, and collaborated with Google to identify the right solutions.
Focus and dedication yield significant benefits. Spotify's dedicated team ensured alignment and minimized disruption to product development by keeping everyone focused and aware of past experiences and lessons learned.
Overall, the transition to cloud-native architectures on AWS represents a significant shift in IT strategy, fostering innovation and scalability. Real-world examples from companies like Skyscanner, Netflix, Shopify, and Spotify highlight the tangible benefits of this approach, including enhanced service availability, scalability, and cost efficiency. Successfully migrating systems demands careful preparation, stakeholder support, and committed implementation. As companies progress in their digital transformation journey, adopting cloud-native infrastructures is crucial for fostering creativity and staying ahead in the competitive market.
References
Journey to being Cloud-Native – How and where should you start? | Amazon Web Services. (2021, January 15). Amazon Web Services. https://aws.amazon.com/blogs/apn/journey-to-being-cloud-native-how-and-where-should-you-start/
Completing the Netflix Cloud Migration - about Netflix. (n.d.). About Netflix. https://about.netflix.com/en/news/completing-the-netflix-cloud-migration
E-Commerce with Shopify – Amazon Web Services. (n.d.). Amazon Web Services, Inc. https://aws.amazon.com/eventbridge/integrations/shopify/
Engineering, S. (2020, July 6). Spotify’s event delivery – The Road to the Cloud (Part I) - Spotify Engineering. Spotify Engineering. https://engineering.atspotify.com/2016/02/spotifys-event-delivery-the-road-to-the-cloud-part-i/