Continuous Integration & Delivery with Microsoft Azure & GitHub – Best Practices

Are you ready?

The developer’s perspective

We will use the Vincent Driessen’s branching model in order to control the way new software is released. Let’s explain in a nutshell how this works:

The tester’s perspective

This is where Microsoft Azure deployment slots and Azure CLI 2.0 come into the scene. Testers don’t have to be aware about the configuration for all these, all they need to know in order to deploy release candidates in staging or the production environment, is the name of the release or hotfix branch. Deployment and slot swapping between the staging and the production environment will happen using two Azure CLI 2.0 commands.

Continuous Integration & Delivery Architecture

All requirements that we have listed before can be achieved using the architecture shown in the above image. There are three platforms used, each one for different reason. Microsoft Azure, is where we have the deployment slots for our application. Assuming we have a .NET Core Web application that also uses an SQL Azure database, that would result in an Azure App Service with two deployment slots, the dev and the staging. The default website is considered as the production slot. Each of these slots have their application settings and of course a separated database connection string. The connection string setting will be a per slot setting so they don’t swap when we swap staging and production slots (more on this later..) As far as for the source control, there is a develop branch, from which all the new feature branches are created. The develop branch has a fixed Webhook hooked to the App Service dev slot. This means that each time we push to develop branch, changes are reflected on the dev slot (build/re-deploy).

When we wish to ship a new software release, we create a new release branch with the next version (tag) number. We deploy the new release candidate branch on the staging slot using Azure CLI 2.0. What this will do, is delete any Webhook existed on the Staging slot and dynamically create on demand, a new one hooked to the new release branch. Testers can either test the new features using the staging slot’s settings or swap with preview the staging and production slots in order to test the release candidate using the production settings. Till the release candidate branch passes all the tests, any push to that branch would result in a new build/deploy cycle on the staging slot. When all the tests pass, testers finalize the swap to production. The release candidate branch can now be merged to the develop and master and finally deleted. The same applies for any hotfix branch that is being created from the master branch.

Any push to develop / release-version / hotfix-version or feature branches or pull requests, should trigger an Appveyor build task, that should detect if the build and any tests succeeded. This is very important, because can detect bugs before reaching the production environment and the end users.

Setup the CI/CD environment

This section will describe exactly the steps I took in order to implement the requirements we have set for Continuous integration & Delivery using Microsoft Azure and Git. Let’s begin. I started by creating an empty ASP.NET Core Web Application (no database yet) and initialize a Git repository so that I can push it up to GitHub. I made sure to create a develop branch from master before pushing it. The repository I used for the tutorial is this one. As shown from the architecture diagram, there is build/test task running on AppVeyor each time we push code to certain branches. So this is what I needed to setup next. The steps to make this work are quite easy. First, I signed in to Appveyor with my GitHub account, pressed the NEW PROJECT button and selected the azure-github-ci-cd repository.

AppVeyor requires you to add an appveyor.yml file at the root of your repository, a file that defines what tasks do you want it to run when something is pushed to your repository. There are a lot of things you can configure in AppVeyor but let’s stick with the basics, which are the build and test tasks. Here is the appveyor.yml file I created.

# branches to build

# configuration for "master" branch
# build in Release mode and deploy to Azure
-
  branches:
    only:
      - master
      - /release/.*/
      - /hotfix/.*/
      - /bugfix/.*/
  image: Visual Studio 2017
  environment:
    DOTNET_CLI_TELEMETRY_OPTOUT: true
    DOTNET_SKIP_FIRST_TIME_EXPERIENCE: true
  version: 1.0.{build}
  configuration: Release
  platform: Any CPU
  skip_branch_with_pr: false
  before_build:
    - cmd: dotnet restore
  test_script:
    - cmd: dotnet test C:\projects\azure-github-ci-cd\azure-github-ci-cd.Tests\azure-github-ci-cd.Tests.csproj  --configuration Release

# configuration for all branches starting from "dev-"
# build in Debug mode and deploy locally for testing
-
  branches:
    only:
      - develop
      - /feature/.*/
  image: Visual Studio 2017
  environment:
    DOTNET_CLI_TELEMETRY_OPTOUT: true
    DOTNET_SKIP_FIRST_TIME_EXPERIENCE: true
  version: 1.0.{build}
  configuration: Debug
  platform: Any CPU
  skip_branch_with_pr: false
  before_build:
    - cmd: dotnet restore
  test_script:
    - cmd: dotnet test C:\projects\azure-github-ci-cd\azure-github-ci-cd.Tests\azure-github-ci-cd.Tests.csproj --configuration Debug

For master, release/.*/, hotfix/.*/ and bugfix/.*/ branches I want Appveyor to build and test the solution in Release mode while the develop and feature/.*/ branches may run in Debug. More over I made sure the tasks run on pull requests as well. With this configuration file, each time we push to those branches a build/test task is running on AppVeyor. The final result can be shown using badges like this:

[![Build status](https://ci.appveyor.com/api/projects/status/github/chsakell/azure-github-ci-cd?branch=master&svg=true)](https://ci.appveyor.com/project/chsakell/azure-github-ci-cd/branch/master)

The next step is to create the AppService up on Microsoft Azure Portal and the required slots as well. I named the App Service ms-azure-github and added two slots, dev and staging. The default instance will be used as the production slot. For all these slots, I added the following App setting, to make sure that Git operations will be used without problems.

SCM_USE_LIBGIT2SHARP_REPOSITORY 0

What this setting will do, is ensure to use git.exe instead of libgit2sharp for git operations. Otherwise you may get errors such as this one. I didn’t add any database yet, I did later on, when I wanted to create a new feature for my app (more on this on the example section..). At the moment, the only thing left was to add a Webhook between the develop branch and the dev slot. This is very important step, because you are going to authorize Azure to access your GitHub account. Doing this directly from the Azure Portal will help you run related commands from the Azure CLI 2.0 without any authentication issues. To do this, I selected the dev slot, clicked the deployment options, connected to my GitHub account, select the ms-azure-git-ci-cd repository and finally the develop branch. Azure instantly fired up the first build and I could see the dev slot online.

Example: Run a full release cycle (new feature / release / hotfix)

This is the section where we will see in action a full release cycle. A full release cycle will show us how to use and mix together, all the tools, platforms and Git operations in order to automate the release process. We ‘ll start developing a new feature, publish it and open a pull request,merge the feature and create a release branch for testing. We will deploy the release candidate on the staging slot using Azure CLI 2.0 and apply a swap with preview with the production slot. After finishing and ship the feature in production, which by the way will be related to adding database features, we will run another full cycle where will have to make a hotfix directly from the master branch. Before starting though, I would like to introduce you some Git extensions that will help us simplify the process when applying the git-flow branching model we mentioned on the start of the post. The extension is named git-flow cheatsheet and I encourage you to spend 5 minutes to read and see how easy is to use it. As a Windows user I installed wget and cygwin before installing the git-flow-cheatsheet extension. I made sure to add the relative paths to the System path to work from the command line.

You certainly don’t have to work with git-flow-cheatsheet, you can just use the usual git commands on the way you are used to, that’s fine. I used the utility for this tutorial to emphasize the git-flow branching model and that’s all. Having the extension installed I run the following git-flow command at the root of my git repository.

git flow init

I left the default options but you could pick your own, for example you can set the name of the releases or hotfix branches etc..

Add a new Feature

Let’s say we have a new task where we need to add entity framework and also show a page with users. The users data will come from the database. The developer that is assigned to complete the task (always happy to..), starts by creating a new feature branch directly from the develop one. With git-flow-cheatsheet this is done by typing:

git flow feature start add-entity-framework

SQL Azure Database

Since this task is related to database, we need to prepare the database environment first, that is create 3 databases, one for each environment: dev, staging, production. We also need to add the connection string settings per slot, in each App Service slot, pointing to the respective database. Creating database on Microsoft Azure is quite easy. You select SQL Databases, click add and fill the required fields.

As you can see I have named the databases in the same way I named the slots. If you click on a database and select Properties, you can find the connection string.

Make sure to set this connection string to the respective App Service slot in the Application Settings connection strings.

It’s very important to check the Slot setting option. Now that you have the SQL Databases set, you need to create the schema of course. For start, I added the Entity Framework model, created the connection string pointing to the dev database, in the appsettings.json file.

{
  "ConnectionStrings": {
    "DefaultConnection": "<your-azure-sql-connection-string-here>"
  },
  "Logging": {
    "IncludeScopes": false,
    "LogLevel": {
      "Default": "Warning"
    }
  }
}

I run Entity Framework Core migrations from Visual Studio 2017 and when it was time to update the Azure database, Visual Studio asked me to sign in with my Microsoft Azure account and also add my local IP to the relative Firewall Rules. I run the same Update-Database command three times, for the three different connection strings. I also added some mock users to all databases to ensure that connections work as intended. When the add-entity-framework feature was ready for review, I published (pushed) it up to GitHub. Other developers could also contribute to that feature.

As soon as I pushed the branch, AppVeyor run its tasks and send me the result on my personal address. Apparently there was an error occurred..

I signed in to AppVeyor and checked the build for details..

After fixing the error while still working on the add-entity-framework branch, the new push triggered a new successful build.

Now I was ready to open a Pull Request to merge the new feature branch the develop one.



Since the feature branch passed the review and all the build/test tasks, I merged it with the develop and then delete it by running the git-flow command:

git flow feature finish add-entity-framework

When I pushed the develop branch to origin, the related slot took all the changes have been made on the add-entity-framework feature.


By the way, at the same time there was another (fiction) developer working on another feature following the same process and when opened a pull request, AppVeyor build/test tasks failed.


He fixed the issue, merged the new feature in to develop and delete that branch.

Ship a new Release

It is about time to ship a new release, where we can see all the new features have been added to develop branch. We ‘ll deploy the new features through a new branch named release/{next-version} where {next-version} is the next tag/release of your software, such as 1.0.0, 1.2.3 etc.. Of course you can make your own convensions for naming your tags. To get the next version simply run git tag. I run the command and since the latest tag was 1.0.1 I named the next release release/1.0.2. I did that using the git-flow-cheatsheet command:

git flow release start 1.0.2

Based on the convensions I accepted during the git-flow init, this created a new branch named release/1.0.2 directly from the develop branch. I also published that branch up to origin using the command:

git flow release publish 1.0.2

It is very important to push the release branch up to GitHub because the next step is to sync the staging slot with that branch. At this point the developer may inform the tester that a new release candidate with the version 1.0.2 is ready for testing. The tester may sync the release/1.0.2 branch with a single Azure CLI 2.0 command but first he/she has to have it installed. You can install Azure CLI 2.0 from here. The first time you are going to use Azure CLI you have to login with your Microsoft Azure credentials. In order to do this, open a Powershell and run the az login command. After a successful login you will be able to access your Azure resources from the command line. As we described before, you could add a Webhook to the staging slot through the Azure portal in the same way we did for the develop branch and the dev slot. But since the staging slot is going to continuously “host” different release candidate branches, we need to use a more robust way and this is why we use Azure CLI. We want a Powershell script that accepts as a paremeter the release candidate branch name (e.g. 1.0.2) and when runs, it removes any previous branch/Webhook bound and add the new one. Doing this would result a new build and deployment on the staging slot, having all the new release features we want to test. Awesome right? The script is this one:

$branch=$args[0]
$repo = '<your repo url here>'
$resourceGroup = '<resource-group name here>'

az webapp deployment source delete --name ms-azure-github --resource-group ms-azure-githubRG --slot staging
az webapp deployment source config --name ms-azure-github --repo-url $repo --resource-group $resourceGroup --branch $branch --slot stagingwer

# example
# .\staging-deploy.ps1 "release/1.0.2"

You should replace your repository url and the Azure Resource Group under which the App Service is bound. Let’s see what I run.


As you can see, Azure instantly fired up a new build on the staging slot, and here is the result after the successful build..

You can find the script here. Testers at this point can test the new release features on a staging environment (staging database, application settings, etc..). If they find a bug, the developer can commit and push the changes on the release branch as it is. Since there is a Webhook to that branch, the staging slot will trigger a new deployment each time you push changes to the branch.

A very good practice for testing new features is to test what impact would the new features have in the production environment. But can we do this without affecting the actual production environment? Yes we can, thanks to the Swap with Preview feature. What this does in a nutshell, is apply production application settings on the staging slot. The very first time, you may haven’t deployed anything to production yet, but remember that we have already set all the application settings, such as connection strings. So we run a swap with review, production settings are applied in to the staging slot and we test the release with those settings. When we ensure that everything is OK, we complete the swap. If we aren’t, we “reset” the swap, that is apply the default staging settings to the staging slot, in other words revert the settings. We want to do all that stuff in a robust way as well and this is why we will use an Azure CLI 2.0 command again.

$action = $args[0] # preview, reset, swap
$resourceGroup = '<resource-group name here>'
$webapp = '<webapp name here>'
$slot = 'staging'

az webapp deployment slot swap --name $webapp --resource-group $resourceGroup --slot $slot --action $action

# example
# .\swap-slots.ps1 "preview"
# .\swap-slots.ps1 "swap"
# .\swap-slots.ps1 "reset"

You can find the script here. The script have 3 options to pass, preview to apply the production settings into the staging slot, swap to complete the swap which will result to deploy exactly what the staging slot has in to production and reset which resets the staging slot’s settings. You can read more about swap here.

After completing the swap to production we can finally see the release version deployed on the production slot.

A full release cycle has been completed and we can merge now the release branch to both the develop and master. Then we can safely delete it from both local and origin. The command to do it using the git-flow-cheatsheet is this:

git flow release finish 1.0.2

Hotfix

There are times (like a lot..) that you have to immediately push a hotfix in production. The way we do this using git-flow branching model is create a new hotfix branch from master and follow the same process we did for a release candidate branch. The naming though may vary, for example if you want to make a hotfix to release 1.0.2 then the hotfix branch may be named hotfix/1.0.2-1 and so on. Using git-flow-cheatsheet simply run this command:

git flow hotfix start <release-tag-version-#>

Push/publish the branch up to origin and sync with the staging slot as we did with a release branch.

Hotfix branches trigger AppVeyor build tasks as well..

Test the hotfix in staging, apply the swap with preview with the production and complete the swap when ready. When you are ready, finish/delete the hotfix branch, that is merge it with both the master and develop and then deleted it.

git flow hotfix finish VERSION

Discussion

Let’s review what we’ have done in this post. We saw how to use and setup Microsoft Azure, AppVeyor and Azure CLI continuously work with GitHub and more specifically the Git-Flow branching model. The more important points to remember is that a single push to develop branch where all the features are merged, automatically re-deploys the dev-slot up on Azure so that developers have a first view of what is currently happening. We also saw how easy and quick is to dynamically deploy a new release candidate or a hotfix up to the staging slot and swap it with the production. As far as the status of your builds, AppVeyor with its badges and email notifications will capture any failed builds or tests during the push or pull requests.

In case you find my blog’s content interesting, register your email to receive notifications of new posts and follow chsakell’s Blog on its Facebook or Twitter accounts.

Facebook	Twitter	GitHub
.NET Web Application Development by Chris S.