David Ross's Blog Random thoughts of a coder

Environments should be destroyed each night

3. December 2020 02:14 by David Ross in

A big driver for enterprises to migrate to the Cloud are the cost and reliability benefits of having infrastructure that auto-scales. Cloud costs are broken across CPU usage, disk and I/O, network traffic and the Cloud services that the solution utilises. For most VMs and PaaS services if you shut down the component CPU and networking costs are zero, but you still pay for the storage. This general rule unfortunately does not apply for Azure App Services where you continue to pay until they are deleted.

Hence the biggest cost optimisation is when an environment is either partially or completely deleted when not in use. Thus, each morning test environments are provisioned and each night they are decommissioned. For this to occur the entire process needs to be fully automated by the CI/CD process executing scripts that interact with Cloud APIs.

When designing a cost optimised infrastructure solution, you need to:

  • Ensure that the Cloud components can be created and destroyed by executing code from the CI/CD process in a controlled and managed way
  • Determine which components can be destroyed as this will naturally include all components that can be built from source
  • Determine what databases and files need to persist across environment shutdowns
  • Determine whether production databases and files can be restored into test environments as opposed to persisting the environments own data
  • Determine the schedule for the environments to be up

For Infrastructure as Code to be effective it needs:

  • All infrastructure scripts need to be stored in the same source repository as the application that it deploys so that application functionality is linked to the scripts that do the deployment
  • To share scripts between projects a mono-repo or GIT subtrees can be used
  • The CI/CD orchestration code should preferably be in a YML like format as opposed to being “entered” in the tooling so that it can tracked and managed using Push Requests
  • The branching approach to ensure that the CI/CD process is able to deploy to all environments especially production at all times.  Often the CI/CD steps will change when a new feature is added to the solution and its better to have a branching approach as opposed to utilising backwards compatibility
  • To take all environment specific configuration as command line parameters
  • Environment configuration items should be stored in source control and the scripts should grab the parameters that are specific for the environment

Environment configuration items that need to be applied during deployment:

  • Firewall rules
  • Server and service sizes if they vary between environments
  • Active Directory and Azure AD group names for role management
  • As passwords should NEVER be stored in source control the team should look to using the Cloud providers inbuilt identity management capabilities to allow for services to access resources such as databases

Modern Cloud development with PaaS

3. December 2020 00:29 by David Ross in

Over the last few years, I have been lucky enough to work on several so called “Cloud First”, “Shift Left” or “Cloud 2.0” projects for different clients. What they all had in common is their heavy usage of Platform as a Service and/or Function as a Service technologies.

As the innovation and technology changes in the Cloud space is so rapid it is vital to understand what the client is hoping to achieve from its Cloud journey. For this reason I usually start with the following diagram below by Paul Kerrison (https://www.paulkerrison.co.uk/random/pizza-as-a-service-2-0) to show just how different these modern architectures are at the infrastructure level and how those differences ripple through the makeup of the team and their responsibilities.

clip_image002

Docker based solutions are more complex than those that utilise technologies built by the Cloud providers. Hence, I recommend that that teams utilise Docker if their solution needs to be able to cross Cloud providers or when components need to run within an organisation’s own data centre.

The primary driver for using Platform as a Service (PaaS) or Function as a Service (FaaS) is to leverage the Cloud providers extraordinary levels of automation in the areas of software defined networking, firewalls, patching, scaling, managing backups, replication and cross region/data centre high availability.

Often the easiest and first component to move to PaaS is the database with both Azure SQL and Amazon RDS being robust mature technologies. Although they are more expensive than a self-managed solution when the additional labour costs are included in the analysis the total ROI is often far lower. Further we have had great success using the serverless configuration of SQL Azure which automatically scales the database based on load. For systems that are idle at night the serverless costs can be 60% lower than using a static server sizing.

The next component to move to PaaS is the web server. Often there will need to be some minor code changes required as part of the migration. Issues can occur when the website connects to local or remote file storage which need to be replaced with a cloud storage solution. Or when its running under an integrated security context which will need to be moved to a Cloud security API such as Azure’s Managed Identity or a simple username and password combination.

Next scheduled jobs that are kicked off by CRON or Windows Schedular are converted to AWS Lambda or Azure Functions. This can lead to the removal of all Virtual Machines from the solution.

The removal of VMs enables a simplified networking and security model. Where instead of locking down the subnets that host VMs the firewall rules are lifted into the application layer where IP Address filtering and role management are combined to provide a richer security model.

These changes lead to a different team makeup as well. The environment support team will naturally reduce in size as that function as been outsourced. There are entire classes of problems that effect environment support staff that are eliminated by Cloud such as those caused by differences between environments. The Cloud provider will ensure that their PaaS and FaaS components are at the same patch level.

The next change is driven by Infrastructure as Code where Cloud resources are created via scripting against APIs as opposed to being built manually. This innovation is turning Infrastructure Architects into programmers and programmers into environment experts. This cross skilling is extremely beneficial to the team. I would recommend that pair programming techniques are utilised as we have found that there is often lots of code duplication and poor error handling in code written by Infrastructure Architects. Meanwhile infrastructure code written by developers may not have firewalls configured correctly or can be oversized.