There is no Cake in this issue, however we'll set the scene for things to come.
This is the first issue in a series around designing a build system for .NET projects
- The build submodule & Versioning dependencies
- Build, test and pack .NET projects by convention using Cake
- Build and pack .NET applications ready for deployment
- Introducing build configuration options and smart defaults
- Versioning strategy using GitVersion and auto-tagging from your CI build
- Templating new .NET solutions from our build submodule
Whilst GIT submodules can often be tricky (the amount of times I've ended up in a disconnected HEAD situation...), but they are also a really powerful concept. The idea is that you can embedded a pointer to another GIT repository, in another.
So, to get started, we need two repositories - the
build submodule, and the parent solution (that will contain our code).
- The parent repo - https://github.com/Antaris/example-repo-with-build-system
So, the first steps, we need to initialise something into the
build submodule, so let's provide a simple README:
After we commit this we need to add this as a submodule for our parent repo:
git submodule add firstname.lastname@example.org:Antaris/build.git build git submodule update --init
We can see that the submodule materialises as a local folder under our repo. The benefits of this, is we can update our
build submodule entirely independently of the parent repository. When we need to 'update' the submodule, its simply a
git pull within the
build submodule itself.
What is committed to the parent repository, is a pointer to a specific commit in the submodule.
An important thing to consider, is the
buildsubmodule is a GIT repo in itself, which means you can branch, push, pull, rebase, all of the standard operations you can perform in GIT. This also means you need to make sure you commit your changes (and push) in your submodule before commiting changes to your parent repo - otherwise the commit for the build submodule will not change.
Next, let's figure out how we're going to manage our dependencies.
For the purpose of the blog series going forward, I will make all of my changes to the
buildGIT repo, from the context of the parent repo where I have added the submodule.
.NET has long now had a solution for dependencies - NuGet, and this has evolved in the last couple of years to become more streamlined. What started out as an external solution (using
packages.config and a manual NuGet restore), it has become integrated into the MSBuild toolchain. Gone is the need to maintain
packages.config files - now we can utilise the new
<PackageReference /> project element to express our dependencies. The important thing to remember at this point, is that dependencies are expressed at the project level.
NuGet does not currently have a solution (ahem) for expressing dependencies at the solution-level. Sure, Visual Studio provides UI for managing versions (and the Consolidate tab is very useful), but it still comes down to individual
<PackageReference /> elements for each project, each with their own (potentially different) versions. i.e., I could quite easily get into a situation where I depend on
<PackageReference Include="Newtonsoft.Json" Version="10.0.0" /> and
<PackageReference Include="Newtonsoft.Json" Version="12.0.3" />.
One of the design goals of Paket was to provide solution-level tooling for managing dependency versions. We can utilise this same concept to centralise our dependency versions across our repo.
build submodule approach, we're going to first define a new MSBuild file,
dependencies.targets where we will define our dependencies.
You can name this
dependencies.targets, however the convention is that
propsare imported first, and
targetsare imported last - our dependencies need to be imported after they are defined as part of an MSBuild project
I prefer to group my dependencies based on provider, namely
Microsoft packages I keep separate from other third party packages.
You'll hopefully notice something different here, the
<PackageReference /> contains an
Update and not an
Include attribute. This tells MSBuild to update the attributes of a given node, in this case any package references where we
Include="Newtonsoft.Json". If your project does not contain a reference to this package, this operation is a noop.
We need to create one more file before we create a new .NET project. This file is called
Directory.Build.targets, and it will sit at the root of your parent repository.
Directory.Build.(props|targets) are special files to MSBuild - it will detect and automatically import the closest instance to where your .NET project is defined. Having this file at the root means it will be automatically imported into your project.
If you are nesting
Directory.Build.(props|targets), you will need to import a parent instance of the file, as MSBuild only imports the closest file.
Within this file, we need to import our
Now we have the basics of the versioning elements complete, we can add a .NET project and test that our references are resolved.
mkdir src cd src dotnet new classlib --name ExampleLib; cd ../ dotnet new sln --name Example; dotnet sln add ../src/ExampleLib/ExampleLib.csproj
With the above shell commands, we're creating a project structure as follows:
So, let's go ahead an edit the project file to include a reference to
We can now run a
dotnet restore on either the project or the solution and it should happily restore the specific version of
12.0.3. If you load this solution up in Visual Studio, the IDE should confirm this:
If the restore action gives you the following error, or similar to it, it means that the
<PackageReference Update="Newtonsoft.Json" Version="12.0.3" /> node was not being picked up by MSBuild.
Project dependency Newtonsoft.Json does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results. Package 'Newtonsoft.Json 3.5.8' was restored using '.NETFramework,Version=v4.6.1, .NETFramework,Version=v4.6.2, .NETFramework,Version=v4.7, .NETFramework,Version=v4.7.1, .NETFramework,Version=v4.7.2, .NETFramework,Version=v4.8' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.
You'll need to check a few things:
- Package name is spelled correctly in
build.dependenciesand your project file
- You are correctly referencing
build\dependencies.targetsfrom the root
- Your root
Directory.Build.targetsfile is being imported into your MSBuild project correctly.
The MSBuild Log Viewer is a great tool for debugging issues with MSBuild projects.
Committing our changes
Lastly, we need to commit our changes to both the
build submodule, and also our parent repository. Let's run a
git status in our parent repo and see what has changed:
And within the
Let's handle the
build submodule changes first:
cd build git add .\dependencies.targets git commit -m 'Added dependencies.targets' git push
Now we can commit our changes to our parent repo, but first, we probably should create a
.gitignore in our parent repo so we can filter out some noise. Head over to gitignore.io or perhaps use this preconfigured one.
Let's stage our changes and commit:
cd ../ git add -A git commit -m 'Added Example project and solution' git push
Takeaways and next steps
There was a fair amount to go through in this issue. Let's recap:
- We created two repositories, one which will be our parent, and another which will be our submodule.
- We used GIT's
git submodule addcommand to include the
buildGIT repository within our parent repository.
- We added a file,
build\dependencies.targetswhich is an MSBuild file containing a list of
<PackageReference />nodes which
- We added a file,
Directory.Build.targetsat the root of our parent repository which will be automatically imported by MSBuild projects
- We created a new .NET Core class library and a solution file. Within that project file, we added a
<PackageReference Include="Newtonsoft.Json" />with no
- We performed a
dotnet restoreand it successfully restored our packages with the correct version.
Next up in the series, we'll add the Cake build framework to our
build submodule, and have it building, testing and packing our code by convention.