Leaping to the clouds
Cloud architectures and costs
Leaping to the clouds:
- 24 August, 2021
- Article - Software Best Practices
Compiling a cloud services comparison is a daunting task in the rapidly-evolving cloud computing landscape. In the first two parts of Entelect’s “Leaping to the Clouds” series, we unpacked the benefits of migrating to a cloud platform, how to choose between the big three providers, and the possibilities and risks associated with each platform.
In this segment we’re using a case study to compare each cloud platform’s major features, as well as one of the more important considerations – cost.
A case study common to enterprises
Resilience, ease-of-change, and cost efficiency is the goal.
As a hypothetical case study, let’s assume we’re engineering an online market place offering that lives along-side existing offerings of an existing large enterprise. The e-commerce storefront will sell books which demands detailed catalogues of the products available for customers to browse and order. We won’t dive into the detailed functional requirements for the solution, but think of any modern e-commerce store that you’ve used. We will be focusing on the technical components of the solution, and concern ourselves with the assumptions made for traffic, handling load, processing requests, and storing data. Resilience, ease-of-change, and cost efficiency is the goal.
Assumptions | |
|---|---|
Detail | Qty |
Monthly active users | 200000 |
Book orders per month | 50000 |
Emails per month | 150000 |
PDF’s per month | 50000 |
PDF size (KiB) | 80 |
Product listing (per customer) | 200 |
Product image size (KiB) | 32 |
High load configuration (Hours) | 150 |
Standard load configuration (Hours) | 300 |
Low load configuration (Hours) | 270 |
Products | 5000000 |
Totals | |
Comic image storage (GiB) | 152,59 |
PDF Docs (GiB) | 3,81 |
Storage Fetch | 40000000 |
Data transfer (GiB) | 1220,70 |
After analysing the existing technical landscape the enterprise, including the existing data stores, business functions, and requirements for the storefront, we understand that there’s a need for various cloud paradigms to be leveraged to achieve the overall solution.
Storefront components
The build of the e-commerce store functionality will live inside of a Platform-as-a-Service (PaaS) solution that can automatically scale when demand is high and will be scaled down during off-peak times – assuming the store is only servicing customers in a single time zone, otherwise traffic will fluctuate based on regional usage worldwide. The PaaS solution has an auto-scale configuration in place for high demand periods to horizontally scale the solution when demand is high. It also has a manual scale configuration to vertically scale the hardware to save costs during low usage time periods.
Data components
The Database-as-a-Service (DbaaS) offering provided by the various cloud providers use different technologies (Microsoft SQL for Azure, MySQL for AWS and Google Cloud). This can have implications in the team skills required and should be a consideration when choosing these services. Choosing a single database vendor like Microsoft SQL also has its own implications in additional licensing fees when hosted on AWS and Google Cloud.
On-demand functions components
The system will also be sending emails in the form of order notifications, and bulk marketing emails to customers when new products become available that they might be interesting in. The solution should be able to scale rapidly to support these requirements.
To cater for this burst-like scaling of individual functions, such as recommendation engine calculations or emails, a Functions-as-a-Service (FaaS) solution will be put in place to optimise cost and system responsiveness. The FaaS solution automatically scales based on demand and no additional configuration is needed.
Legacy line-of-business system component
We also have an existing warehouse line-of-business system that has traditionally been hosted on-premises. A new solution for the warehouse is currently being built, but in the meantime a lift and shift approach with Infrastructure-as-a-Service (IaaS) approach will be taken for the current warehousing system by provisioning virtual machines in the cloud to run the software and its database. Usage does not fluctuate in this solution, as it is internal facing. If required, manual intervention is needed to scale.
HA and DR must be architected as part of the solution, and take into consideration the business needs.
Other concerns
HA and DR must be architected as part of the solution, and take into consideration the business needs.
Apart from the PaaS, FaaS, and IaaS solutions, we also need to consider other components for the architecture such as the SMPT server sending the emails, the software synchronising the new e-commerce database with the warehouse database, and the automation software to scale our PaaS offering. Although the cloud providers can provide solutions to tackle these problems, we opted to exclude them from the final considerations because of the deep context required on the specific business scenarios. For example, Office 365 and MailChimp might be in place for transactional email and marketing email, respectively.
Werner Vogles, VP and CTO of AWS, said “Everything fails all the time”. There is truth in this statement, and something that is often overlooked with cloud deployments, the fallacy being “I’m deploying to cloud, there’s no need for disaster recovery planning”. With cloud deployments, there are new inputs that contribute towards high availability and cloud deployments do not automatically mean your workload is protected against disaster events. The strategy for handling high availability (HA) and disaster recovery (DR) has a direct implication to cost, as a simple approach such as restore from backups means low cost but a long time to recover, whereas multi-region solutions have a high cost with practically zero time to recover. HA and DR must be architected as part of the solution, and take into consideration the business needs to ensure minimal downtime.
An Azure approach
Storefront components
The Azure solution will make use of Azure’s app services for the PaaS hosting of the e-commerce store. This provides us with both horizontal and vertical scaling capabilities which allows us to optimise cost, system responsiveness, and availability based on traffic demand.
Horizontal and vertical scaling capabilities allows us to optimise cost, system responsiveness, and availability based on traffic demand.
Data components
To store images of the books, invoice PDFs, and other blob data required by the solution we will be making use of Azure storage accounts. Azure storage accounts automatically scale to meet traffic demand, high availability and regional replications is on by default, and additional lifecycle policies can be used to move lower usage blob data to cheaper storage tiers to save on costs.
On-demand functions components
For the FaaS component we are making use of Azure functions. These are easy to integration with the event queueing mechanism, and it perfectly fits the burst style nature of email sending by rapidly scaling from zero instances to as many as needed to process messages from the queue as quickly as possible.
Legacy line-of-business system component
To help smooth the transition and reduce the impact to existing business process, a common strategy is “lift-and-shift”. This means that the existing applications and data are moved to cloud with minimal or no modification. For the IaaS solution we are making use of Azure’s virtual machines that provide us with machines with the exact hardware specifications that we need for our software and it includes basic maintenance functions such as backups, replication, the ability to perform critical system updates from a single management plane within Azure.
Something to keep in mind with IaaS is that although it has a lower base-cost than PaaS or FaaS, it does have an additional management and operational cost in that skilled personnel are required to ensure that the operating system and relevant software on the machine itself is regularly maintained and monitored.
Networking components
To ensure we provide our customers with the best experience we will use Azure Front Door. Front Door acts as a content delivery network (CDN) to ensure that customers get served data quickly by a data centre closest to them. It additionally acts as our firewall to protect our services from command OWASP attacks and exploits, as well ensuring our website traffic is served over a secured SSL connection. Lastly, Azure Front Door will also act as the global load balancer for our solution to ensure traffic is routed the closest available servers to serve the request.
Azure | |||
|---|---|---|---|
Services | Cloud Services | USD $ per month | |
Web Hosting (PaaS) | Azur App Service | $370,53 | |
Functions (FaaS) | Azure Functions | $0,00 | |
Legacy Warehouse Software (IaaS) | Azure VM | $160,00 | |
Legacy DB (IaaS) | Azure VM | $319,00 | |
SQL Database (DBaaS) | Azure SQL Database | $304,00 | |
Blob Storage (SaaS) | Azure Blob Storage | $23,00 | |
Event Queue (SaaS) | Azure Service Bus | $15,00 | |
Front Door | Azure Front Door | $280,00 | |
Totals | $1 471,53 | ||
* Some services that appear in our architecture, may not appear on the cloud providers’ calculators due to the calculator limitations.
An AWS approach
Storefront components
AWS provides various tools and services for hosting, each with a degree of autonomy. To simplify the web hosting, Elastic Beanstalk is used as a platform service to reduce the complexity of capacity provisioning, load balancing, scaling, and health monitoring. Elastic Beanstalk allows you to quickly deploy and manage applications in AWS, while it manages the infrastructure. The service itself is free to use, and you pay for the resources deployed to manage your site – in this case the EC2 and load balancer.
Data components
The data requirements are split into two categories: files (images, invoices, books) and transactional commerce data (customers, address, orders, products). For general files, AWS S3 will be used as it is low cost and highly available. Perfect for static resources in an e-commerce solution. Transactional data is stored in Amazon Aurora for MySQL, as it is well-balanced in terms of performance, cost, and simplicity. It does come at higher cost when compared to other database solutions, but the higher premium brings various automations such as failovers, back-ups, hardware installation, and read-replicas in the need for improved performance.
On-demand functions components
A cost optimising approach to handling sporadic and short running requests such as emailing order updates, is to utilise serverless compute. AWS provides Lambdas, which can run operations based of various events such as message events or API requests and are costed against the time it takes to run. It is priced by the number requests executed and at the time it takes to run. AWS includes 1 million free requests per month for 400 000 seconds (approximately 110 hours). In Lambdas, you do not provision hardware or maintain scaling as these are self-managed.
Legacy line-of-business system component
Most cloud migration projects include the need to utilise legacy applications on virtual machines and the lift-and-shift approach is common. AWS EC2 is used to provision virtual machines in AWS, which allows the legacy applications to be deployed in a similar or replica environment. Additionally, AWS provides a number tools to simplify the migration such as VM Import/Export.
Most cloud migration projects include the need to utilise legacy.
Networking components
Site performance and security is among the top concerns for most e-commerce stores. CloudFront from AWS is a CDN (content distribution network) and WAF (web application firewall). As a CDN, it can cache commonly accessed resources at locations closer to your customers, this makes web site pages loads faster for static resources. As a WAF, it can protect your website from the most common internet threats e.g., DDoS, SQL Injection and JS injection.
AWS | |||
|---|---|---|---|
Services | Cloud Services | USD $ per month | |
Web Hosting (PaaS) | AWS EC2 | $142,00 | |
Functions (FaaS) | Lambda | $5,00 | |
Legacy Warehouse Software (IaaS) | AWS EC2 | $104,00 | |
Legacy DB (IaaS) | AWS EC2 | $170,00 | |
SQL Database (DBaaS) | Amazon Aurora | $740,00 | |
Blob Storage (SaaS) | AWS S3 | $26,00 | |
Event Queue (SaaS) | Amazon SQS | $2,00 | |
Front Door | Elastic Load Balancing | $196,00 | |
Route53 | $53,90 | ||
CloudFront | $109,83 | ||
Totals | $1 549,11 | ||
A Google Cloud approach
Data components
To store images of the books, invoice PDFs, and other files / binary large object (BLOB) data required by the solution we will use Google Cloud Storage. Cloud storage can automatically scale to meet traffic demand, has high availability and replication options, and rules can be configured to move lower usage blobs to cheaper storage tiers to save on costs.
For the FaaS component we are making use of Google Cloud Functions. Cloud functions have built in support for event queueing mechanisms, and it perfectly fits the burst style nature of order placement and email sending by rapidly scaling from zero instances to as many as needed to process messages from the queue as quickly as possible.
Legacy line-of-business system component
For the legacy line-of-business system, we will use Google Compute Engine. Compute Engine provisions machines with the exact hardware specifications that we need for our warehousing software and it includes basic maintenance functions such as backups, replication, the ability to perform critical system updates. As with the other cloud providers, Compute Engine has a lower base cost than PaaS or Faas does have an additional management and operational cost in that skilled personnel are required to ensure that the operating system and relevant software on the machine itself is regularly maintained and monitored.
Networking components
To access the e-commerce store over the internet, we will use Google Cloud Load Balancing to ensure we have a highly available load balancer that can always route requests to running services and to make sure users are accessing our services from a data centre close to them. To ensure our services are protected from security threats we will use Google Cloud Armor as firewall for all requests entering the system.
Google Cloud | |||
|---|---|---|---|
Services | Cloud Services | USD $ per month | |
Web Hosting (PaaS) | App Engine | $223,00 | |
Functions (FaaS) | Cloud Functions | $6,3 | |
Legacy Warehouse Software (IaaS) | Compute Engine | $0,00 | |
Legacy DB (IaaS) | Compute Engine | $744,00 | |
SQL Database (DBaaS) | Cloud SQL (MySQL) | $191,00 | |
Blob Storage (SaaS) | Cloud Storage | $25,00 | |
Event Queue (SaaS) | Pub/Sub | $0,14 | |
Front Door | Cloud Load Balancing + Cloud Armor | $50,00 | |
Totals | $1 373,44 | ||
Weighing up options
When comparing cloud providers, there a number of variables and considerations to examine, from a business objective, technical, and operational perspective. In our introduction to cloud platforms segment, we mentioned the importance of strategy, data regulation concerns, and regional availability when choosing a cloud provider. In our possibilities and risks segment, we provided an overview of the features and risks for each provider, and from the above deep-dive case study, we can see that the material way a solution manifests itself can be similar but also vastly different in each platform.
AWS, Azure, and Google Cloud all have comparable costing for the e-commerce solution discussed in the case study. Google Cloud comes in as the least expensive at $1 373 per month, Azure at $1 471 per month, and AWS at the most expensive at $1 549 per month.
Azure | AWS | Google Cloud | ||||
|---|---|---|---|---|---|---|
Services | Cloud Services | USD $ per month | Cloud Services | USD $ per month | Cloud Services | USD $ per month |
Web Hosting (PaaS) | Azur App Service | $370,53 | AWS EC2 | $142,00 | App Engine | $223,00 |
Functions (FaaS) | Azure Functions | $0,00 | Lambda | $5,00 | Cloud Functions | $6,3 |
Legacy Warehouse Software (IaaS) | Azure VM | $160,00 | AWS EC2 | $104,00 | Compute Engine | $0,00 |
Legacy DB (IaaS) | Azure VM | $319,00 | AWS EC2 | $170,00 | Compute Engine | $744,00 |
SQL Database (DBaaS) | Azure SQL Database | $304,00 | Amazon Aurora | $740,00 | Cloud SQL (MySQL) | $191,00 |
Blob Storage (SaaS) | Azure Blob Storage | $23,00 | AWS S3 | $26,00 | Cloud Storage | $25,00 |
Event Queue (SaaS) | Azure Service Bus | $15,00 | Amazon SQS | $2,00 | Pub/Sub | $0,14 |
Front Door | Azure Front Door | $280,00 | Elastic Load Balancing | $196,00 | Cloud Load Balancing + Cloud Armor | $50,00 |
Route53 | $53,90 | |||||
CloudFront | $109,83 | |||||
Totals | $1 471,53 | $1 549,11 | $1 373,44 | |||
Each cloud platform has a multitude of options for supporting different solution architectures and optimising costs. Although moving to the cloud seems like a technical endeavour, architecting useful and long-lasting solutions involves deep understanding of the business objectives, functional operations of the business, and complete enterprise technical landscape.
Existing software solutions can work in a cloud environment; either through a lift-and-shift approach, or private networks available with all cloud providers. Furthermore, existing software-as-a-service (SaaS) solutions being leveraged by the business can work in harmony with solutions on the cloud.
If a cloud initiative derails or overshoots, it’s more than likely that the right skillsets were not leveraged for needs analysis, architecture design and implementation.
So if the cloud platforms are similar in offerings, features, and availability, what other factors should be taken into consideration? Skills, teams, and an effective way of working.
If a cloud initiative derails or overshoots, it’s more than likely that the right skill-sets where not leveraged for needs analysis, architecture design and implementation, the right teams were not assembled, and the right ways-of-working with business was not established.
We will explore the skills, teams, and ways-of-working concerns in cloud projects in our next and final segment.
This article was the third part of “Leaping to the Clouds” – a series where Entelect unpacks common cloud questions and concerns for enterprises across industries.
Create or refine your cloud strategy
Get in touch with us to create a cloud strategy or revisit yours with a complimentary 4-hour lightning workshop.
Previous
Possibilities and risks
