Implementing and Debugging Custom MSBuild Tasks

TL;DR: MSBuild is at the heart of every .NET project. It’s the system that ultimately invokes tools like NuGet or the C# compiler. Additional tools can be integrated via custom MSBuild tasks, however, working with these can be very daunting at first. Knowing a few tricks and tools, the process of implementing, packaging, and debugging a custom task can be greatly simplified. A sample project is available on GitHub.

Much of the tooling around .NET projects ends up having to integrate with MSBuild, the low-level build system in the .NET ecosystem. A few examples of these tools are:

• Refit – REST APIs are generated based on interface declarations before compilation
• Fody – IL code gets rewritten after compilation to add null-checks, merge assemblies to a single file, notify on property changes and much more
• NSwag – Swagger specifications, C#/TypeScript clients and proxies are generated from C# ASP.NET controllers as part of the build
• GitVersion – Semantic versions are calculated based on our commit history and propagated into MSBuild properties before compilation to make them part of the assembly metadata

Some of the scenarios that involve code generation could eventually move to Source Generators, which are already available in the .NET 5 previews. Source generators remove a lot of the burden of writing MSBuild tasks – and sharing workspaces – but still aren’t easy to debug:

Clearly that sets the mood for what a pleasure it is to write an MSBuild task! 😅

MSBuild Integration Options

When we want hook into the execution of MSBuild, we have several options:

• Inline Tasks – We can write code fragments directly into .targets files. They will be compiled into tasks by the RoslynCodeTaskFactory and executed by MSBuild when run. This is great for drafting ideas, but falls short in maintainability and debugging.
• Exec Tasks – Any executable can be invoked in a similar way to Process.Start. We can capture output, validate exit codes, or define regular expressions for custom error/warning messages. However, we will miss some integration points, and if we need to get complex data out of the process, we’ll have to encode it in a single line and decode it in the target.
• Custom Tasks – We can operate with MSBuild infrastructure directly, including ITask, IBuildEngine, and ITaskItem objects. This allows us to log error/warning/info events, inspect item groups with their metadata, create more object-like results, and even to support incremental builds.

Since custom tasks are the most scalable solution, we will put focus on them for the rest of this article.

Implementing the Task

From my own experience, I’d recommend to keep the task assembly small and move complex logic into their own projects. This is also the approach most of the projects mentioned above have taken. Some of the most important types for custom MSBuild tasks are ITask, IBuildEngine, and ITaskItem. In order to gain access, we need to add references for them:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
<ItemGroup>
  <PackageReference Include="Microsoft.Build.Utilities.Core" Version="16.3.0" CopyLocal="false" Publish="false" ExcludeAssets="runtime"/>
  <PackageReference Include="Microsoft.Build.Framework" Version="16.3.0" CopyLocal="false" Publish="false" ExcludeAssets="runtime"/>
  <PackageReference Include="System.Collections.Immutable" Version="1.6.0" CopyLocal="false" Publish="false"/>
  <PackageReference Include="System.Runtime.InteropServices.RuntimeInformation" Version="4.3.0" CopyLocal="false" Publish="false"/>
</ItemGroup>

<ItemGroup Condition="'$(TargetFrameworkIdentifier)' == '.NETFramework'">
  <PackageReference Include="Microsoft.VisualStudio.Setup.Configuration.Interop" Version="1.16.30" CopyLocal="false" Publish="false"/>
</ItemGroup>

<ItemGroup Condition="'$(TargetFrameworkIdentifier)' == '.NETCoreApp'">
  <PackageReference Include="System.Text.Encoding.CodePages" Version="4.6.0" CopyLocal="false" Publish="false"/>
</ItemGroup>

Note that the MSBuild references are part of every MSBuild installation and therefore should not be deployed with our final NuGet package. In order to achieve that, we set the CopyLocal and Publish attribute for each reference to false. We will also need to remove those references from the ReferenceCopyLocalPaths item group:

1
2
3
4
5
6
7
8
<Target Name="RemoveMicrosoftBuildDllsFromOutput" AfterTargets="ResolveReferences">
  <PropertyGroup>
    <NonCopyLocalPackageReferences Condition="'%(PackageReference.CopyLocal)' == 'false'">;@(PackageReference);</NonCopyLocalPackageReferences>
  </PropertyGroup>
  <ItemGroup>
    <ReferenceCopyLocalPaths Remove="@(ReferenceCopyLocalPaths)" Condition="$(NonCopyLocalPackageReferences.Contains(';%(ReferenceCopyLocalPaths.NuGetPackageId);'))"/>
  </ItemGroup>
</Target>

As already hinted, our task assembly will likely have dependencies to other projects or NuGet packages. A while back, this would have taken a huge effort:

Meanwhile we’re at MSBuild 16, and some of the problems that Nate described in his blog have already been addressed. I am by no means an expert in properly resolving dependencies, but Andrew Arnott came up with the ContextAwareTask – originally used in Nerdbank.GitVersion – which is working out great for many folks:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
using System;
using Microsoft.Build.Framework;
using Microsoft.Build.Utilities;

namespace CustomTasks
{
    public class CustomTask : ContextAwareTask
    {
        [Required] public string StringParameter { get; set; }

        public ITaskItem[] FilesParameter { get; set; }

        protected override bool ExecuteInner()
        {
            return true;
        }
    }
}

As for the actual implementation, we won’t go into too much detail but focus on the most important bits. MSBuild calls the Execute method which will later delegate to ExecuteInner and its return value signals whether the task succeeded or failed. The inherited BuildEngine property allows us to log information, warning, and error messages. Users of the task can opt-in to treat warnings of the task as errors by setting the TreatWarningsAsErrors property. The StringParameter property is a required input value. The FilesParameter item group is optional and can contain a list of files. In many situations, a ITaskItem can also be a ordinary string value, like for PackageReference.

Wiring the Task

In this next step we’ll wire up the task implementation in a .targets file, which will be included in our NuGet package and automatically loaded from a referencing project. In this file – here CustomTasks.targets we’ll load the task assembly, create a new XML task element, define a couple default values, and create a new target that calls the task:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

  <PropertyGroup>
    <CustomTasksAssembly>$(MSBuildThisFileDirectory)\$(MSBuildThisFileName).dll</CustomTasksAssembly>
  </PropertyGroup>

  <UsingTask TaskName="$(MSBuildThisFileName).CustomTask" AssemblyFile="$(CustomTasksAssembly)"/>

  <!-- Default Properties -->
  <PropertyGroup>
    <CustomTaskContinueOnError Condition="'$(CustomTaskContinueOnError)' == ''">False</CustomTaskContinueOnError>
    <CustomTaskStringParameter Condition="'$(CustomTaskStringParameter)' == ''">Default Value</CustomTaskStringParameter>
  </PropertyGroup>

  <Target Name="RunCustomTask" BeforeTargets="CoreCompile">
    <CustomTask
            ContinueOnError="$(CustomTaskContinueOnError)"
            StringParameter="$(CustomTaskStringParameter)"
            FilesParameter="@(CustomTaskFilesParameter)"/>
  </Target>

</Project>

Defining the CustomTasksAssembly (Line 5) does not only help us to not repeat ourselves when we reference multiple tasks from the same assembly (Line 8), but is also great for debugging, as we’ll see later. Also note that we’re using a couple of well-known MSBuild properties like MSBuildThisFileDirectory and MSBuildThisFileName to avoid magic strings being scattered around our file. Following best practices makes renaming or relocating the task more effortless! The task invocation also uses the ContinueOnError property (Line 18) – one of the common properties available to all task elements.

Creating the NuGet Package

As already mentioned, we won’t pack the MSBuild tasks project directly, but have another project MainLibrary include the task infrastructure in its package. In order to load the CustomTasks.targets file from the previous section, we create another MainLibrary.props and MainLibrary.targets file in our MainLibrary project:

1
2
3
4
5
6
7
8
9
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

  <PropertyGroup>
    <CustomTasksDirectory Condition="'$(MSBuildRuntimeType)' == 'Core'">$(MSBuildThisFileDirectory)\netcore</CustomTasksDirectory>
    <CustomTasksDirectory Condition="'$(MSBuildRuntimeType)' != 'Core'">$(MSBuildThisFileDirectory)\netfx</CustomTasksDirectory>
  </PropertyGroup>

</Project>

In the .props file, we’re defining a property CustomTasksDirectory that points to the directory containing our task. Note that we need to take the MSBuild runtime into account by checking MSBuildRuntimeType (Line 5-6). A project targeting netcoreapp2.1 would still use .NET Framework for running MSBuild inside Visual Studio, while the same project would use MSBuild for .NET Core when compiling via dotnet build from the command-line. I.e., it must not be subject to framework version folder structure. In the .targets file, we will import the CustomTasks.targets file:

1
2
3
4
5
6
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

  <Import Project="$(CustomTasksDirectory)\CustomTasks.targets" Condition="'$(CustomTasksEnabled)' != 'False'" />

</Project>

We also add a condition to check if CustomTasksEnabled is enabled (Line 4). This little trick allows us to easily opt-out from attaching the task. Faulty MSBuild tasks have a high chance to completely break a referencing project and to cause confusion about where the error originates from:

As one of the last steps, we define the package structure in our MainLibrary.csproj project file:

1
2
3
4
5
6
7
8
<ItemGroup Condition="'$(TargetFramework)' == ''">
  <None Include="$(MSBuildProjectName).props" PackagePath="build" Pack="true"/>
  <None Include="$(MSBuildProjectName).targets" PackagePath="build" Pack="true"/>
  <None Include="..\CustomTasks\CustomTasks.targets" PackagePath="build\netcore" Pack="true"/>
  <None Include="..\CustomTasks\CustomTasks.targets" PackagePath="build\netfx" Pack="true"/>
  <None Include="..\CustomTasks\bin\$(Configuration)\netcoreapp2.1\publish\**\*.*" PackagePath="build\netcore" Pack="true"/>
  <None Include="..\CustomTasks\bin\$(Configuration)\net472\publish\**\*.*" PackagePath="build\netfx" Pack="true"/>
</ItemGroup>

It is important to remember, that to properly create the package, we first need to call dotnet publish for the supported target frameworks. The complete list of invocations should be as follows:

1
2
3
dotnet publish --framework netcoreapp2.1
dotnet publish --framework net472
dotnet pack

Debugging a Task

Let’s get to the most interesting part of how we can effectively debug MSBuild integration in a test project:

Indeed, with the console application approach, things are rather easy. On Windows we can call System.Diagnostics.Debugger.Launch(), which fires up the Just-In-Time Debugger so that we can attach to the process. By default, this will use the Visual Studio JIT Debugger, but we can also configure JetBrains Rider as the Just-In-Time Debugger. As of now, this strategy is not supported on Linux/macOS. The best workaround I’ve found is to call SpinWait.SpinUntil(() => Debugger.IsAttached), which will wait until the debugger is actually attached.

Taking the custom MSBuild task approach, we have a bit more footwork to do. Referencing a package via PackageReference, their main .props and .targets files get automatically included. This is not the case when using ProjectReference on the same project! We could create an actual package, but that has the unpleasant side-effect of getting persisted in our global NuGet cache:

Deleting from the cache, or incrementing versions – all those are rather poor workarounds compared to a possible better development and testing experience for NuGet packages. A better alternative is to manually wire up our test project:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<Project Sdk="Microsoft.NET.Sdk">

  <Import Project="..\MainLibrary\MainLibrary.props"/>

  <PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.1</TargetFramework>
    <CustomTasksEnabled Condition="'$(CustomTasksEnabled)' == ''">False</CustomTasksEnabled>
    <CustomTasksDirectory>$(MSBuildThisFileDirectory)\..\CustomTasks\bin\Debug\netcoreapp2.1\publish</CustomTasksDirectory>
  </PropertyGroup>

  <ItemGroup>
    <ProjectReference Include="..\MainLibrary\MainLibrary.csproj"/>
  </ItemGroup>

  <Import Project="..\MainLibrary\MainLibrary.targets"/>

</Project>

The .props file should be imported at the very beginning of the project file (Line 3), allowing to set default property values. Meanwhile, the .targets file should be important at the very end of the project file (Line 16), to ensure the task is run with the relevant input values. Here, we will also utilize our trick from the previous section, and set a default for CustomTasksEnabled (Line 8), which can either be overridden from the outside (hence the condition) or manually changed to test behavior in the IDE. We should also not forget about adjusting the CustomTasksDirectory property to point to the local version of our task.

With JetBrains Rider we can use run configurations to make this process more convenient. In the first configuration Publish CustomTasks, we will publish the MSBuild task project:

In a second configuration Run CustomTasks we will depend on the first one (see Before launch), and call MSBuild.dll /t:Clean;Restore;Pack /p:CustomTasksEnabled=True to invoke MSBuild on the test project:

Note that this is the place when we need to override the CustomTasksEnabled property to execute our task. Also we should make an educated choice of which targets should be invoked. If our task integrates with the Restore target, then we really need to execute Clean before, because otherwise it may be skipped for consecutive builds.

Now that we’re all set up, we can use the Run CustomTasks configuration to finally debug our task implementation:

Troubleshooting MSBuild

Sooner or later we will run into issues with our MSBuild integration, especially regarding the .props and .targets files. We might reference a wrong property, forget about escaping, or just have a typo in our identifiers. The Project Properties dialog is a good place to start investigations and to see evaluated properties and imports for a project file:

Using the common keyboard shortcut, we can also easily copy values from grid cells. If we need even more insight, then the MSBuild Structured Log Viewer created by Kirill Osenkov can be of great help:

The Structured Log Viewer operates on binary log files, which can be created by passing /binaryLogger:output.binlog to the MSBuild invocation. Binary log files provide the highest level of completeness and verbosity, even compared to most-diagnostic level for text logs. Imports of files and execution of targets are hierarchically visualized using a tree view. Particularly when trying to find a proper target build order for our integration, we can easily check on the flattened temporal order view:

Another benefit is that when testing our task on large projects that imply a time-consuming compilation, we can replay a single task without executing all its dependencies:

For developers on Windows there is a special surprise in JetBrains Rider! We can right-click the test project, choose Advanced Build Actions and execute Rebuild Selected Projects with Diagnostics:

Given that Structured Log Viewer can already run on Avalonia UI, maybe we can see the same feature also for macOS and Linux soon.

Acknowledgements

I want to add that much of my adventures with MSBuild are only of good nature and happy endings because my friend Martin Ullrich is such an MSBuild wizard. If you don’t follow him yet, you really should. Sorry Martin for sending more MSBuild enthusiasts your way! 🤗