I’ve recently open sourced a set of build scripts for .NET development on Github to assist you in automating compilation, running tests, creating release artifacts and deploying those artifacts:
In this article I describe how to use them on your workstation, in your Continuous Integration/Delivery server and what parts still need a bit more attention.
What are the goals?
- To be able to set up a new project with as little effort as possible to avoid having any projects that live without CI, because of the lack of time to set it up. To reduce setup time the scripts favor convention over configuration.
- To be able to run all actions equally well from your workstation as from the build server.
What does it automate for you?
The scripts are continuously improving, currently they do quite a lot:
Compilation
- Versions all your compiled assemblies with parameter BuildVersion, which needs a major.minor.build.revision (1.5.13.859 for example) version number.
- For both MVC and WebForms applications it compiles them using AspNetCompiler to check for errors in your views.
- Outputs your generated assemblies to a set directory, so solutions can reference assemblies generated by another solution within the same repository.
Testing
- Runs tests created with both NUnit and Machine.Specification.
- Runs JMeter tests.
Creating release artifacts
- Minifies CSS and Javascript grouped per directory with YUI Compressor and versions the filenames to avoid any caching issues after deploy.
- Compiles Flash with Flex, outputs it to your web project and versions the filename.
- Applies config transforms for any config file (where the original is called [a-z]+.config and the transform [a-z]+.[a-z]+.config).
- Creates artifacts for each environment you have a config transform for.
- Updates filenames with 1.0.0.0 in them to reflect the current build version to avoid caching issues and updates all references to them in css and js.
- Precompiles asp.net websites.
- Removes any files that shouldn’t be part of the release.
- Optimizes pngs.
- Optimizes jpegs.
Deploying release artifacts
- Deploys websites using MSDeploy.
- Deploys database projects.
- Deploys data-tier-applications (supported by Azure).
- Deploys Azure packages to staging, then swaps to production and removing staging afterwards; which of these steps is executed can be parameterized.
Various
- Delete Azure instances.
- Analyze code using FXCop.
Which actions are taken for what projects is automatically handled by sticking to the project naming conventions. More about these below.
Read on to learn how the build files are structured, how to use them and how to configure them in your favorite CI server.
Structure
The project has the following MSBuild files:
- properties-convention-based.msbuild
- properties-repository-specific.msbuild
- targets.msbuild
- targets-repository-specific.msbuild
The first and third are the ones that continuously get improved so are best left untouched after you first start using them except for updating them to the newest version; the second and fourth allow you to override any defaults as defined in the other two files and have no need to be updated to the newest version.
The other scripts used from targets.msbuild are
- AzureDelete.ps1
- AzureDeploy.ps1
- DacDeploy.ps1
If you inspect properties-convention-based.msbuild you can see that the directory structure as used in the repository is defined there and that the following conventions for project names are used:
- *.Website.csproj
- *.WebService.csproj
- *.Console.csproj
- *.Service.csproj
- *.Database.dbproj
- *.Worker.csproj
- *.Azure.ccproj
- *.UnitTests.csproj
- *.IntegrationTests.csproj
For the location of your css and javascript within a Website project the following conventions are used:
- static\css
- static\js
Within these directories the contents of each sub-directory will be merged and minified. In the case of css the resulting file is stored in the sub-directory (so all relative paths stay intact), for javascript it’s stored in the static/js directory.
Any of the conventions can be overridden, this is best done in the properties-repository-specific.msbuild file, so you can still easily update properties-convention-based.msbuild and targets.msbuild to the newest version.
Getting started
To get this stuff on the road the following steps need to be taken:
- Make sure you have Visual Studio 2010 SP1 installed.
- Fork https://github.com/JorritSalverda/msbuild-by-convention or download as a zip and extract to your repository.
- Get at least the following tools and add their files in the respective /Build/Tools/ sub-directory:
- Compile – MSBuild.Community.Tasks.v1.2.0.306
- Compile - MSBuild.ExtensionPack.Binaries.v4.0.2.0
- Run unit/integration tests – NUnit-2.6.0.12051
- Run unit/integration tests - Machine.Specifications.0.5.6.0
- Create release - Yahoo.Yui.Compressor.v1.6.0.2
- Create release - optipng-0.7.1-win32
- Create release - Jpegtran
- Create release - VS2010WebPublishing
- Create release - Windows.Azure.Tools.v1.6
- Deploy vs2010 studio database projects - Vs2010DbCmd
- Deploy data-tier applications - SqlPowershell-10.50.1600.1
- Deploy websites via MSDeploy - MsDeploy.v2
- Deploy websites to Azure – install Azure powershell commandlets.
- Run Jmeter tests - apache-jmeter-2.6
- Analyze code quality - FxCop.v10.0
I’ll soon check which of these tools can be included in the repository without being in conflict with their license in order to avoid step 3 and further.
Deploying to Azure and deploying Data-Tier applications both use Powershell scripts. As these aren’t signed they would normally be blocked for execution. To enable unsigned scripts to be executed start the Powershell and run the following command:
set-executionpolicy unrestricted
This does put your machine at risk, but it’s currently the only way I know how to enable the execution of those scripts.
Running the scripts
Okay, after all this work it is time to really get started. From a visual studio command line navigate to the /Build/Scripts/ directory and use the following commands.
To compile all solutions in the /Source/Csharp/ directory and subdirectories perform the following command:
msbuild.exe targets.msbuild /t:Build
To run all unit test projects:
msbuild.exe targets.msbuild /t:RunUnitTests
To create release artifacts for all your projects in the /Build/Releases/ directory:
msbuild.exe targets.msbuild /t:Release
To run all integration test projects:
msbuild.exe targets.msbuild /t:RunIntegrationTests
To deploy VS2010 database projects or Data-Tier Applications:
msbuild.exe targets.msbuild /t:Deploy /p:ReleaseConfiguration=”PROD” /p:ProjectToDeploy=”Projectname.Database” /p:DeployServer=”database-server-name” /p:DeployTargetName=”database-name” /p:DeployUsername=”db-admin-name” /p:DeployPassword=”db-admin-password”
To deploy a website with MSDeploy to an IIS server:
msbuild.exe targets.msbuild /t:Deploy /p:ReleaseConfiguration=”ACC” /p:ProjectToDeploy=”Projectname.Website” /p:DeployServer=”iis-server-name” /p:DeployTargetName=”iis-website-name” /p:DeployUsername=”deploy-user-name” /p:DeployPassword=”deploy-user-password”
To deploy an Azure web role or worker role:
msbuild.exe targets.msbuild /t:Deploy /p:DeployEnvironment=”PROD” /p:ProjectToDeploy=”Projectname.Azure” /p:AzureSubscriptionID=”subscription-id” /p:AzureCertificatePassword=”management-certificate-password” /p:AzureHostedServiceName=”hosted-service-name” /p:AzureStorageAccountName=”storage-account-name” /p:AzureSwapToProductionAfterDeploy=”True” /p:AzureRemoveStagingAfterSwap=”True” /p:AzureDisallowMultipleActiveInstances=”False”
To enable deployments to Azure you need to store a management certificate at /Build/Scripts/AzureManagementCertificate.pfx. The RunUnitTests and Release targets depend on the Build target being executed; the Deploy targets depend on the Release target. With these separate commands you can easily set up a pipeline within your CI server.
Setting up a Continuous Delivery pipeline
A possible pipeline set up can look like the one pictured below.
To implement this you would call the following targets and actions:
1 Compile
- target: Build
2 Run unit tests
- target: RunUnitTests
3 Create release
- target: Release
- save Release artifacts from directory /Build/Releases/
4 Deploy to INTEGRATION + run integration tests
- fetch Release artifacts
- target: Deploy for database
- target: Deploy for website (using MSDeploy or deploy to Azure)
- target: RunIntegrationTests
- target: RunJMeterTests
5 Deploy to ACCEPTANCE
- fetch Release artifacts
- target: Deploy for database
- target: Deploy for website (using MSDeploy or deploy to Azure)
6 Deploy to PRODUCTION
- fetch Release artifacts
- target: Deploy for database
- target: Deploy for website (using MSDeploy or deploy to Azure)
When you set this up in Teamcity you would like to make sure the separate steps have a snapshot dependency on the previous step, the same version number is used for every step (use %dep… as version number, after setting up the snapshot dependency) and the steps that use the Release artifacts have an artifact dependency to step 3.
In Bamboo you have no such thing as a snapshot dependency, so in that case I save the whole working copy as an artifact in step 1 and have every following step import this artifact. From step 3 and further it also has a dependency on the Release artifact.
Where Teamcity and CruiseControl automatically pass their version numbers to the scripts in Bamboo you have to do this explicitly by adding the following parameter:
/p:BuildVersion=1.0.0.${bamboo.buildNumber}
What needs to be improved?
Of course there are still a lot of improvements on the wish list:
- Reduce the ‘getting started’ to 2 steps by adding the necessary /Build/Tools to the repository where their respective licenses allow for that.
- Remove the need for Visual Studio and SQL Server to be installed on a build agent.
- Support Robocopy.
- Handle first time deploy to Azure, not needing a first manual deploy.
- Deploy certificates to Azure.
- Add Selenium testing.
- Make it more extendable.
- Determine code coverage.
Let me know if any parts of the setup are unclear so I can document them into more depth. I can also provide you with the /Build/Tools directory zipped up, until I’ve figured that out, license-wise.
Cheers, Jorrit
