Docker Fundamentals

This tutorial will explain the fundamentals of Docker and start you with some basic usage.

What is Docker?

Docker is open source software to pack, ship and run any application as a lightweight container. Containers are completely hardware and platform independent so you don’t have to worry about whether what you are creating will run everywhere.

In the past virtual machines have been used to accomplish many if these same goals. However, Docker containers are smaller and have far less overhead than VMs. VMs are not portable as different VM runtime environments are very different. Docker containers are extremely portable. Finally, VMs were not built with software developers in mind; they contain no concept of versioning, and logging/monitoring is very difficult. Docker images, on the other hand, are built from layers that can be version controlled. Docker has logging functionality readily available for use.

You might be wondering what could go into a “container”. Well, anything! You can isolate pieces of your system into separate containers. You could potentially have a container for nginx, a container for MongoDB, and one for Redis. Containers are very easy to setup. Major projects like nginx, MongoDB, and Redis all offer free Docker images for you to use; you can install and run any of these containers with just one shell command. This is much easier than using a virtual machine (even with something like Vagrant).

Lets Go!

Docker Fundamentals

Deployment Through the Ages

Vanessa Love just posted this intriguing little snippet on Twitter:

//platform.twitter.com/widgets.js

And I got halfway through sticking some notes into the Google doc, and then thought actually this might make a fun blog post. So here’s how deployment has evolved over the 14 years since I first took over the hallowed mantle of webmaster@spotlight.com.

2003: Beyond Compare (maybe?)

The whole site was classic ASP – no compilation, no build process, all connection credentials and other settings were managed as application variables in the global.asa file. On a good day, I’d get code running on my workstation, test it in our main target browsers, and deploy it using a visual folder comparison tool. It might have been Beyond Compare; it might have been something else. I honestly can’t remember and the whole thing is lost in the mists of time. But that was basically the process – you’d have the production codebase on one half of your screen and your localhost codebase on the other half, and you’d cherry-pick the bits that needed to be copied across.

Of course, when something went wrong in production, I’d end up reversing the process – edit code directly on live (via UNC share), normally with the phone wedged against my shoulder and a user on the other end; fix the bug, verify the user was happy, and then do a file sync in the other direction to get everything from production back onto localhost. Talk about a tight feedback loop – sometimes I’d do half-a-dozen “deployments” in one phone call. It was a simpler time, dear reader. Rollback plan was to hammer Ctrl-Z until it’s working again; disaster recovery was tape backups of the complete source tree and database every night, and the occasional copy’n’paste backup of wwwroot before doing something ambitious.

Incidentally, I still use Beyond Compare almost daily – I have it configured as my merge tool for fixing Git merge conflicts. It’s fantastic.

2005: Subversion

Once we hired a second developer (hey Dan!) the Beyond Compare approach didn’t really work so well any more, so we set up a Subversion server. You’d get stuff running on localhost, test it, maybe share an http://www.spotlight.com.dylan-pc/ link (hooray for local wildcard DNS) so other people could see it, and when they were happy, you’d do an svn commit, log into the production web server (yep, the production web server – just the one!) and do an svn update. That would pull down the latest code, update everything in-place. There was still the occasional urgent production bugfix. One of my worst habits was that I’d fix something on production and then forget to svn commit the changes, so the next time someone did a deployment (hey Dan!) they’d inadvertently reintroduce whatever bug had just been fixed and we’d get upset people phoning up asking why it was broken AGAIN.

2006: FinalBuilder

This is where we start doing things with ASP.NET in a big way. I still dream about OnItemDataBound sometimes… and wake up screaming, covered in sweat. Fun times. The code has all long since been deleted but I fear the memories will haunt me to my grave.

Anyway. By this point we already had the Subversion server, so we had a look around for something that would check out and compile .NET code, and went with FinalBuilder. It had a GUI for authoring build pipelines and processes, some very neat features, and could deploy .NET applications to IIS servers. This was pretty sophisticated for 2006. 

2008: test server and msdeploy

After one too many botched FinalBuilder deployments, we decided that a dedicated test environment and a better deployment process might be a good idea. Microsoft had just released a preview of a new deployment tool called MSDeploy, and it was awesome. We set up a ‘staging environment’ – it was a spare Dell PowerEdge server that lived under my desk, and I’d know when somebody accidentally wrote an infinite loop because I’d hear the fans spin up. We’d commit changes to Subversion, FinalBuilder would build and deploy them onto the test server, we’d give everything a bit of a kicking in IE8 and Firefox (no Google Chrome until September 2008, remember!) and then – and this was magic back in 2008 – you’d use msdeploy.exe to replicate the entire test server into production! Compared to the tedious and error-prone checking of IIS settings, application pools and so on, this was brilliant. Plus we’d use msdeploy to replicate the live server onto new developers’ workstations, which was a really fast, easy way to get them a local snapshot of a working live system. For the bits that still ran interpreted code, anyway.

2011: TeamCity All The Things!

By now we had separate dev, staging and production environments, and msdeploy just wasn’t cutting it any more. We needed something that can actually build different deployments for each environments – connection strings, credentials, and so on. And there’s now support in Visual Studio for doing XML configuration transforms, so you create a different config file for every environment, check those into revision control, and get different builds for each environment. I can’t remember exactly why we abandoned FinalBuilder for TeamCity, but it was definitely a good idea – TeamCity has been the backbone of our build process ever since, and it’s a fantastically powerful piece of kit.

2012: Subversion to GitHub

At this point, we’d grown from me, on my own doing webmaster stuff, to a team of about six developers. Even Subversion is starting to creak a bit, especially when you’re trying to merge long-lived branches and getting dozens of merge conflicts, so we start moving stuff across to GitHub. It takes a while – I’m talking months – for the whole team to stop thinking of Git as ‘unnecessarily complicated Subversion’ and really grok the workflow, but we got there in the end.

Our deployment process at this point was to commit to the Git master branch, and wait for TeamCity to build the development version of the package. This would get built and deployed. Once it was tested, you’d use TeamCity to build and deploy the staging version – and if that went OK, you’d build and deploy production. Like very step on this journey, it was better than anything we’d had before, but had some obvious drawbacks. Like the fact we had several hundred separate TeamCity jobs and no consistent way of managing them all.

2013: Octopus Deploy and Klondike

When we started migrating from TeamCity 6 to TeamCity 7, it became rapidly apparent that our “build everything several times” process… well, it sucked. It was high-maintenance, used loads of storage space and unnecessary CPU cycles, and we needed a better system.

Enter Octopus Deploy, whose killer feature for us was the ability to compile a .NET web application or project into a deployment NuGet package (an “octopack”), and then apply configuration settings during deployment. We could build a single package, and then use Octopus to deploy and configure it to dev, staging and live. This was an absolute game-changer for us. We set up TeamCity to do continuous integration, so that every commit to a master branch would trigger a package build… and before long, our biggest problem was that we had so many packages in TeamCity that the built-in NuGet server started creaking.

This started life as an experimental build of themotleyfool/NuGet.Lucene – which we actually deployed onto a server we called “Klondike” (because klondike > gold rush > get nuggets fast!) – and it worked rather nicely. Incidentally, that NuGet.Lucene library is now the engine behind themotleyfool/Klondike, a full-spec NuGet hosting application – and I believe our internal hostname was actually the inspiration for their project name. That was a lot of fun for the 18 months or so that Klondike existed but we were still running the old NuGet.Lucene codebase on a server called ‘klondike’. It’s OK, we’ve now upgraded it and everything’s lovely.

It was also in 2013 that we started exploring the idea of automatic semantic versioning – I wrote a post in October 2013 explaining how we hacked together an early version of this. Here’s another post from January 2017 explaining how it’s evolved. We’re still working on it. Versioning is hard.

And now?

So right now, our build process works something like this.

  1. Grab the ticket you’re working on – we use Pivotal Tracker to manage our backlogs
  2. Create a new GitHub branch, with a name like 12345678_fix_the_microfleems – where 12345678 is the ticket ID number
  3. Fix the microfleems.
  4. Push your changes to your branch, and open a pull request. TeamCity will have picked up the pull request, checked out the merge head and built a deployable pre-release package (on some projects, versioning for this is completely automated)
  5. Use Octopus Deploy to deploy the prerelease package onto the dev environment. This is where you get to tweak and troubleshoot your deployment steps.
  6. Once you’re happy, mark the ticket as ‘finished’. This means it’s ready for code review. One of the other developers will pick it up, read the code, make sure it runs locally and deploys to the dev environment, and then mark it as ‘delivered’.
  7. Once it’s delivered, one of our testers will pick it up, test it on the dev environment, run it past any business stakeholders or users, and make sure we’ve done the right thing and done it right.
  8. Finally, the ticket is accepted. The pull request is merged, the branch is deleted. TeamCity builds a release package. We use Octopus to deploy that to staging, check everything looks good, and then promote it to production.

And what’s on our wishlist?

  • Better production-grade smoke testing. Zero-footprint tests we can run that will validate common user journeys and scenarios as part of every deployment – and which potentially also run as part of routine monitoring, and can even be used as the basis for load testing.
  • Automated release notes. Close the loop, link the Octopus deployments back to the Pivotal tickets, so that when we do a production deployment, we can create release notes based on the ticket titles, we can email the person who requested the ticket saying that it’s now gone live, that kind of thing.
  • Deployments on the dashboards. We want to see every deployment as an event on the same dashboards that monitor network, CPU, memory, user sessions – so if you deploy a change that radically affects system resources, it’s immediately obvious there might be a correlation.
  • Full-on continuous deployment. Merge the PR and let the machines do the rest.

So there you go – fourteen years worth of continuous deployments. Of course, alongside all this, we’ve moved from unpacking Dell PowerEdge servers and installing Windows 2003 on them to running Chef scripts that spin up virtual machines in AWS and shut them down again when nobody’s using them – but hey, that’s another story.

Deployment Through the Ages

Quick enzyme JS TDD

With the current wave of change in front-end web development, it’s easy to get lost in a sea of options when it comes to choosing a stack to develop with. The same goes for setting up a test environment depending on which frameworks you’re using, you might be inclined to use different libraries and test runners to better suit the application workflow and logic.

So, we’ll go through all the necessary steps to implement a simple way to get you started with writing tests for React components, using ES2015 syntax and the well documented Airbnb’s testing utility, enzyme. While we’re at it, we’ll set up a constant test runner for TDD

Lets GO!

Quick enzyme JS TDD

Naming Things is Hard: Spotlight Edition

Like most specialist industries, software is rife with mainstream English words that we’ve taken and misappropriated to mean something completely different. Show business is no different. The software team here at Spotlight sits smack-bang in the intersection between these two specialist fields, and so when we’re talking to our customers and product owners about the systems we build, it’s very important to understand the difference between typecasting and type casting, and exactly what sort of actor model we’re talking about. We therefore present this delightful “double glossary” of everyday terms that you’ll hear here at Spotlight Towers. Because as we all know, there’s only two hard problems in software: cache invalidation, naming things, and off-by-one errors.

Actor

Software: A mathematical model of concurrent computation that treats “actors” as the universal primitives of concurrent computation.

Showbiz: A person whose profession is acting on the stage, in films, or on television.

Agent

Software: A software agent is a computer program that acts for a user or other program in a relationship of agency

Showbiz: A person who finds jobs for actors, authors, film directors, musicians, models, professional athletes, writers, screenwriters, broadcast journalists, and other people in various entertainment or broadcast businesses.

Callback

Software: Any executable code that is passed as an argument to other code, which is expected to call back (execute) the argument at a given time.

Showbiz: A follow-up interview or audition

Casting

Software: Explicitly converting a variable from one type to another

Showbiz: Employing actors to play parts in a film, play or other production. Also the act of doing same.

Client

Software: The opposite of a server

Showbiz: An actor, specifically in the context of the actor’s relationship with their agent or manager Internally at Spotlight we have both internal and external clients/customers

Client Profile

Software: A subset of the .NET framework intended to run on mobile and low-powered devices

Showbiz: An actor’s professional CV, as it appears on their agent’s’ website or in various kinds of casting software and directories

Double

Software: Primitive data type representing a floating-point number

Showbiz: A performer who appears in place of another performer, i.e., as in a stunt.

Mirror

Software: A copy of a system that updates from the original in near to real time, often a database or file storage system

Showbiz: An optical device that helps a performer check they’ve applied their makeup correctly

Principal

Software: Used in database mirroring to refer to the primary instance of the database

Showbiz: A performer with lines.

Production

Software: The live infrastructure and code environment

Showbiz: A film, TV or stage show, such as a professional actor might list on their acting CV.

REST

Software: Representational State Transfer – an architectural style used when building hypermedia APIs

Showbiz: What actors do between jobs.

Script

Software: A computer program written in a scripting language

Showbiz: The written dialogue and directions for a play, film or show

“Sequel”

Software: Standard pronunciation of SQL, referring to either the database query language. Also commonly refers to Microsoft’s SQL Server database product.

Showbiz: A published, broadcast, or recorded work that continues the story or develops the theme of an earlier one.

Server

Software: The opposite of a client

Showbiz: Someone working as waiting staff in a restaurant. Who is quite possibly an actor moonlighting as a server to pay the bills between acting jobs.

Spotlight

Software: The native MacOS search application

Showbiz: Our company – www.spotlight.com,  “The Home of Casting” – and the directories and services we have created since 1927. Not to be confused with the 20-odd different ‘Spotlight’ companies around the world, many of whose customers blindly email support@spotlight.com whenever they have a problem. 🙂

Staging

Software: A replica of a production hosting environment used to test new features and deployments.

Showbiz: The method of presenting a play or dramatic performance; also used to refer to the stage structure itself in theatre and live performance.

Naming Things is Hard: Spotlight Edition

Spotlight at Monki Gras 2017

Monki Gras is RedMonk’s annual UK conference about Software, Craft and Experience. I was very happy that my application for their Diversity Scholarships Programme got accepted and I gladly attended the event on the 26th and 27th of January.

Besides providing a full 2-day ticket they also had in place a Mentorship Programme for the scholars. Tracy Miranda – open source evangelist and veteran of the Eclipse community, Bryan Boreham – director of engineering at Weakework and Rachel Stephens – associate analyst at RedMonk made sure the scholars felt welcome, answered questions and facilitated networking.

monkigras

While some industry events have a very corporate feel about them these days, Monki Gras was like a breath of fresh air: less pitching and more content!

The subject of 2017 was “Packaging: convenience is the killer app for great developer and user experiences” and among the speakers were Arianna Aondio from Varnish Software Group, Gordon Haff from Red Hat, Alvaro Videla who works as a distributed systems engineer and was previously a Core Developer for RabbitMq and Abby Kearns from Cloud Foundry Foundation.

panel

My favourite talk was “Metaphors we compute by” by Alvaro Videla, who made a very interesting connection between the Linguistics and Philosophic worlds and the Software Industry. Communication is key even when you only ‘speak’ code!

The conference was held in quite an informal venue, The Bike Shed Motorcycle Club in Shoreditch and the techies were kept focused with artisan coffee, tasty food and locally brewed beer.

Looking forward to their event next year!

Spotlight at Monki Gras 2017

Semantic Versioning with Powershell, TeamCity and GitHub

Here at Spotlight Towers, we’ve been using TeamCity as our main build server since version 6; it’s a fantastic tool and we love it dearly. It got even better a few years back when we paired it with the marvellous Octopus Deploy; TeamCity builds the code and creates a set of deployable packages known as Octopacks; Octopus deploys the packages, and everything works quite nicely. Well, almost everything. One of the few problems that TeamCity + Octopus doesn’t magically solve for us is versioning. In this post, we’re going to look at how we use Git and TeamCity to manage versioning for our individual packages.

If this sounds like your sort of thing, why not come and work for us? That’s right – Spotlight is hiring! We’re looking for developers, testers and a new UX/Web designer – check out jobs.spotlight.com and get in touch if you’re interested.

First, let’s establish some principles

  • We are going to respect the semantic versioning convention of MAJOR.MINOR.PATCH, as described at semver.org.
  • Major and minor versions will be incremented manually. We trust developers to know whether their latest commit should be a new major or minor release according to semantic versioning principles.
  • Building the same codebase from the same branch twice should produce the same semantic version number.
  • Packages created from the master branch are release packages.
  • Packages created from a merge head of an open pull request are pre-release packages.
  • Pre-release packages will use the version number that would be assigned if that branch was accepted for release at build time.

Now, here’s the part where we’re going to deviate from the semantic versioning specification, because our packages actually use a four-part version number. We want to include a build number in our package versions, but the official semver extension for doing this – MAJOR.MINOR.PATCH+BUILD – won’t work with NuGet, so we’re going to use a four-part version number MAJOR.MINOR.PATCH.BUILD. Pre-release packages will be appended with a suffix describing which branch they were built from – MAJOR.MINOR.PATCH.BUILD-BRANCH.

OK, here’s an illustrated example that demonstrates what we’re trying to achieve. Master branch is green. Two developers are working on feature branches – blue and red in this example. To create our pre-release builds, we’re using a little-documented but incredibly useful feature of GitHub known as ‘merge heads’. The idea is that if you have an open pull request, the merge head will give you a snapshot of the codebase that would be created by merging the open pull request into master – so you’re not just testing your new feature in isolation, you’re actually building and testing your new feature plus the current state of the master branch. There is one caveat to this, which I’ll explain below.

So, we’ve got TeamCity set up to build and publish packages every time there’s a commit to master or to the merge head of an open pull request, and we’re also occasionally triggering manual builds just to make sure everything’s hanging together properly. Here’s what happens: 

semantic merging 500px

That line there that’s highlighted in yellow is a gotcha. At this point in our workflow, we’ve merged PR1 into our master branch, but because we haven’t pushed anything to the blue branch since this happened, the blue merge head is out of date. PR2 does NOT reflect the latest changes to master, and if we trigger a build manually, we’ll end up with a package that doesn’t actually reflect the latest state of the codebase. The workaround is pretty simple; if you’re creating pre-release builds from merge heads, never run these builds manually; make sure you always trigger the build by pushing a change to the branch.

Now let’s look at how can we get TeamCity to automatically calculate those semantic version numbers whenever a build is triggered. We’ll start with the major and minor version. We’re going to track these by creating a version.txt file in the root of the project codebase, which just contains the major and minor version numbers. If a developer decides that their feature branch represents a new major or minor version, it’s their responsibility to edit version.txt as part of implementing the feature. This also means that prerelease packages built from that branch will reflect the new version number whilst master branches will continue to use the old version until the branch gets merged, which I think is rather elegant.

For the patch version, we’re going to assume that every commit or merge to the master branch represents a new patch version, according to the following algorithm

  • If the current version.txt represents a NEW major/minor version, the patch number is zero
  • Otherwise, the patch number is the patch number of the latest release, incremented by the number of commits to the master branch since that release.

So – how do we know how many commits there have been since the last release? First, each time we build a release branch, we’re going to use Git tags to tag the repository with the version number we’ve just built. TeamCity will do this for you automatically using a build feature called “VCS labeling”:

image

Assuming every release has a corresponding tag, now we need to find the most recent release number, which we can do from the Git command line.

  git fetch –tags
  git tag –sort=v:refname

Git tags aren’t retrieved by default, so we need to explicitly fetch them before listing them. Then we list all the tags, specifying sort=v:refname which causes tag names to be treated as semantic versions when sorting. (Remember that semver sorting isn’t alphanumeric – in alphanumerics, v9 is higher than v12). Once we’ve got the latest tag, we need to count the number of revisions since that tag was created, which we can do using this syntax:

  git rev-list v1.2.3..HEAD –count

To use this in our TeamCity build, we’ll need to output the various different formats of that version so that TeamCity can use them. We want to do three things here:

  • Label the VCS root with the three-part semantic version number v1.2.3
  • Update the AssemblyInfo.cs files with the four-part version number 1.2.3.456 – note that we can’t put any prerelease suffix in the AssemblyInfo version.
  • Pass the full version – 1.2.3.456-pr789 – to Octopack when creating our deployable packages with Octopus.

I’ve wrapped the whole thing up in a Powershell script which runs as part of the TeamCity build process, which is on GitHub at https://gist.github.com/dylanbeattie/3a2fe5abca14600efee1c88009afc0f8

To use it in your project, add versions.ps1 to the root of your project repo; create a text file called version.txt which contains your major.minor version, and then add a TeamCity build step at the beginning of your build process that looks like this:

image

Finally, it’s worth mentioning that to use command-line git from Powershell, I had to set up TeamCity to use an SSH VCS root rather than HTTP, and install the appropriate SSH keys on the TeamCity build agent. I don’t know whether this is a genuine requirement or a quirk of our configuration; your mileage may vary. And I still find Powershell infuriatingly idiosyncratic, but hey – you probably knew that already. 🙂

Happy versioning! And like I said, if this sort of thing sounds like something you’d like to work on, awesome – we’re hiring! Check out jobs.spotlight.com for more details and get in touch if you’re interested.

Semantic Versioning with Powershell, TeamCity and GitHub

Semantic Versioning with Powershell, TeamCity and GitHub

Here at Spotlight Towers, we’ve been using TeamCity as our main build server since version 6; it’s a fantastic tool and we love it dearly. It got even better a few years back when we paired it with the marvellous Octopus Deploy; TeamCity builds the code and creates a set of deployable packages known as Octopacks; Octopus deploys the packages, and everything works quite nicely. Well, almost everything. One of the few problems that TeamCity + Octopus doesn’t magically solve for us is versioning. In this post, we’re going to look at how we use Git and TeamCity to manage versioning for our individual packages.

If this sounds like your sort of thing, why not come and work for us? That’s right – Spotlight is hiring! We’re looking for developers, testers and a new UX/Web designer – check out jobs.spotlight.com and get in touch if you’re interested.

First, let’s establish some principles

  • We are going to respect the semantic versioning convention of MAJOR.MINOR.PATCH, as described at semver.org.
  • Major and minor versions will be incremented manually. We trust developers to know whether their latest commit should be a new major or minor release according to semantic versioning principles.
  • Building the same codebase from the same branch twice should produce the same semantic version number.
  • Packages created from the master branch are release packages.
  • Packages created from a merge head of an open pull request are pre-release packages.
  • Pre-release packages will use the version number that would be assigned if that branch was accepted for release at build time.

Now, here’s the part where we’re going to deviate from the semantic versioning specification, because our packages actually use a four-part version number. We want to include a build number in our package versions, but the official semver extension for doing this – MAJOR.MINOR.PATCH+BUILD – won’t work with NuGet, so we’re going to use a four-part version number MAJOR.MINOR.PATCH.BUILD. Pre-release packages will be appended with a suffix describing which branch they were built from – MAJOR.MINOR.PATCH.BUILD-BRANCH.

OK, here’s an illustrated example that demonstrates what we’re trying to achieve. Master branch is green. Two developers are working on feature branches – blue and red in this example. To create our pre-release builds, we’re using a little-documented but incredibly useful feature of GitHub known as ‘merge heads’. The idea is that if you have an open pull request, the merge head will give you a snapshot of the codebase that would be created by merging the open pull request into master – so you’re not just testing your new feature in isolation, you’re actually building and testing your new feature plus the current state of the master branch. There is one caveat to this, which I’ll explain below.

So, we’ve got TeamCity set up to build and publish packages every time there’s a commit to master or to the merge head of an open pull request, and we’re also occasionally triggering manual builds just to make sure everything’s hanging together properly. Here’s what happens: 

semantic merging 500px

That line there that’s highlighted in yellow is a gotcha. At this point in our workflow, we’ve merged PR1 into our master branch, but because we haven’t pushed anything to the blue branch since this happened, the blue merge head is out of date. PR2 does NOT reflect the latest changes to master, and if we trigger a build manually, we’ll end up with a package that doesn’t actually reflect the latest state of the codebase. The workaround is pretty simple; if you’re creating pre-release builds from merge heads, never run these builds manually; make sure you always trigger the build by pushing a change to the branch.

Now let’s look at how can we get TeamCity to automatically calculate those semantic version numbers whenever a build is triggered. We’ll start with the major and minor version. We’re going to track these by creating a version.txt file in the root of the project codebase, which just contains the major and minor version numbers. If a developer decides that their feature branch represents a new major or minor version, it’s their responsibility to edit version.txt as part of implementing the feature. This also means that prerelease packages built from that branch will reflect the new version number whilst master branches will continue to use the old version until the branch gets merged, which I think is rather elegant.

For the patch version, we’re going to assume that every commit or merge to the master branch represents a new patch version, according to the following algorithm

  • If the current version.txt represents a NEW major/minor version, the patch number is zero
  • Otherwise, the patch number is the patch number of the latest release, incremented by the number of commits to the master branch since that release.

So – how do we know how many commits there have been since the last release? First, each time we build a release branch, we’re going to use Git tags to tag the repository with the version number we’ve just built. TeamCity will do this for you automatically using a build feature called “VCS labeling”:

image

Assuming every release has a corresponding tag, now we need to find the most recent release number, which we can do from the Git command line.

git fetch –tags
git tag –sort=v:refname

Git tags aren’t retrieved by default, so we need to explicitly fetch them before listing them. Then we list all the tags, specifying sort=v:refname which causes tag names to be treated as semantic versions when sorting. (Remember that semver sorting isn’t alphanumeric – in alphanumerics, v9 is higher than v12). Once we’ve got the latest tag, we need to count the number of revisions since that tag was created, which we can do using this syntax:

git rev-list v1.2.3..HEAD –count

To use this in our TeamCity build, we’ll need to output the various different formats of that version so that TeamCity can use them. We want to do three things here:

  • Label the VCS root with the three-part semantic version number v1.2.3
  • Update the AssemblyInfo.cs files with the four-part version number 1.2.3.456 – note that we can’t put any prerelease suffix in the AssemblyInfo version.
  • Pass the full version – 1.2.3.456-pr789 – to Octopack when creating our deployable packages with Octopus.

I’ve wrapped the whole thing up in a Powershell script which runs as part of the TeamCity build process, which is on GitHub:

https://gist.github.com/dylanbeattie/3a2fe5abca14600efee1c88009afc0f8.js

To use it in your project, add versions.ps1 to the root of your project repo; create a text file called version.txt which contains your major.minor version, and then add a TeamCity build step at the beginning of your build process that looks like this:

image

Finally, it’s worth mentioning that to use command-line git from Powershell, I had to set up TeamCity to use an SSH VCS root rather than HTTP, and install the appropriate SSH keys on the TeamCity build agent. I don’t know whether this is a genuine requirement or a quirk of our configuration; your mileage may vary. And I still find Powershell infuriatingly idiosyncratic, but hey – you probably knew that already. 🙂

Happy versioning! And like I said, if this sort of thing sounds like something you’d like to work on, awesome – we’re hiring! Check out jobs.spotlight.com for more details and get in touch if you’re interested.

Semantic Versioning with Powershell, TeamCity and GitHub