David Ross's Blog Random thoughts of a coder

A big driver for enterprises to migrate to the Cloud are the cost and reliability benefits of having infrastructure that auto-scales. Cloud costs are broken across CPU usage, disk and I/O, network traffic and the Cloud services that the solution utilises. For most VMs and PaaS services if you shut down the component CPU and networking costs are zero, but you still pay for the storage. This general rule unfortunately does not apply for Azure App Services where you continue to pay until they are deleted.

Hence the biggest cost optimisation is when an environment is either partially or completely deleted when not in use. Thus, each morning test environments are provisioned and each night they are decommissioned. For this to occur the entire process needs to be fully automated by the CI/CD process executing scripts that interact with Cloud APIs.

When designing a cost optimised infrastructure solution, you need to:

• Ensure that the Cloud components can be created and destroyed by executing code from the CI/CD process in a controlled and managed way
• Determine which components can be destroyed as this will naturally include all components that can be built from source
• Determine what databases and files need to persist across environment shutdowns
• Determine whether production databases and files can be restored into test environments as opposed to persisting the environments own data
• Determine the schedule for the environments to be up

For Infrastructure as Code to be effective it needs:

• All infrastructure scripts need to be stored in the same source repository as the application that it deploys so that application functionality is linked to the scripts that do the deployment
• To share scripts between projects a mono-repo or GIT subtrees can be used
• The CI/CD orchestration code should preferably be in a YML like format as opposed to being “entered” in the tooling so that it can tracked and managed using Push Requests
• The branching approach to ensure that the CI/CD process is able to deploy to all environments especially production at all times.  Often the CI/CD steps will change when a new feature is added to the solution and its better to have a branching approach as opposed to utilising backwards compatibility
• To take all environment specific configuration as command line parameters
• Environment configuration items should be stored in source control and the scripts should grab the parameters that are specific for the environment

Environment configuration items that need to be applied during deployment:

• Firewall rules
• Server and service sizes if they vary between environments
• Active Directory and Azure AD group names for role management
• As passwords should NEVER be stored in source control the team should look to using the Cloud providers inbuilt identity management capabilities to allow for services to access resources such as databases