DevSecOps for Delphi: Tests, CI Builds, and Dependency Hygiene

This is the last article in the series I started three years ago, and the one closest to my day job. Delphi and Lazarus projects have a reputation for being built on one developer's machine, by hand, with the "it works here" guarantee. The tools to do better have existed for years. What is usually missing is the habit, so here is the minimum I now set up on day one of any Pascal project.

Unit tests

DUnitX ships with Delphi. A test fixture is a class with attributes, and the runner finds it through RTTI:

unit Temperature.Tests;

interface

uses
  DUnitX.TestFramework;

type
  [TestFixture]
  TTemperatureTests = class
  public
    [Test]
    procedure FreezingPointConverts;
    [Test]
    [TestCase('Body heat', '37,98.6')]
    [TestCase('Boiling',   '100,212')]
    procedure Converts(Celsius, Expected: Double);
  end;

implementation

uses
  Temperature;

procedure TTemperatureTests.FreezingPointConverts;
begin
  Assert.AreEqual(32.0, ToFahrenheit(0), 0.001);
end;

procedure TTemperatureTests.Converts(Celsius, Expected: Double);
begin
  Assert.AreEqual(Expected, ToFahrenheit(Celsius), 0.001);
end;

initialization
  TDUnitX.RegisterTestFixture(TTemperatureTests);

end.

FPCUnit ships with Lazarus and takes the older approach: a class descending from TTestCase whose published methods are the tests.

unit TemperatureTests;

{$mode objfpc}{$H+}

interface

uses
  Classes, SysUtils, fpcunit, testregistry, Temperature;

type
  TTemperatureTests = class(TTestCase)
  published
    procedure TestFreezingPoint;
    procedure TestBoilingPoint;
  end;

implementation

procedure TTemperatureTests.TestFreezingPoint;
begin
  AssertEquals(32.0, ToFahrenheit(0), 0.001);
end;

procedure TTemperatureTests.TestBoilingPoint;
begin
  AssertEquals(212.0, ToFahrenheit(100), 0.001);
end;

initialization
  RegisterTest(TTemperatureTests);

end.

Both have console runners that return a non-zero exit code on failure and can write XML reports, which is all a pipeline needs. Put the tests in a separate project that shares the units under test, and make it the first thing the pipeline runs. Turn on leak detection in the test project as described in the memory management article; a leak should fail the build.

What should you test? Everything that does not touch a form. If a piece of logic is hard to test because it lives in an event handler, that is the signal to move it into a unit of its own, which also makes it reusable between the Delphi and Lazarus versions of a tool.

Building from the command line

A build that only works from the IDE is not a build, it is a ritual.

Delphi projects are MSBuild files. The IDE installs a batch file that sets the environment, after which msbuild compiles a project or project group:

call "C:\Program Files (x86)\Embarcadero\Studio\23.0\bin\rsvars.bat"
msbuild ServiceTool.dproj /t:Build /p:Config=Release /p:Platform=Win64
msbuild ServiceTool.Tests.dproj /t:Build /p:Config=Debug /p:Platform=Win64
Win64\Debug\ServiceTool.Tests.exe

Lazarus has lazbuild, which compiles a project file with the same settings the IDE would use:

lazbuild --build-mode=Release ServiceTool.lpi
lazbuild --build-mode=Debug ServiceTool.Tests.lpi
./ServiceTool.Tests --all --format=plain

Commit the build modes to version control, and make sure the Release mode strips debug information and turns off assertions. Treat compiler warnings as errors where you can: Free Pascal has -Sew for exactly that, and in Delphi you can promote the warnings that matter with {$WARN ... ERROR} directives in a shared include file.

A pipeline in Azure DevOps

Once the build runs from the command line, a pipeline is a short YAML file next to the code. My projects live in Azure DevOps, so the example is Azure Pipelines: a Microsoft-hosted Linux agent builds the Lazarus version, and a private pool builds the Delphi version.

trigger:
  branches:
    include: [main]
pr:
  branches:
    include: [main]

jobs:
- job: Lazarus
  pool:
    vmImage: ubuntu-latest
  steps:
  - script: sudo apt-get update && sudo apt-get install -y lazarus
    displayName: Install Lazarus
  - script: |
      lazbuild --build-mode=Debug ServiceTool.Tests.lpi
      ./ServiceTool.Tests --all --format=plain
    displayName: Unit tests
  - script: lazbuild --build-mode=Release ServiceTool.lpi
    displayName: Release build
  - publish: bin/ServiceTool
    artifact: ServiceTool-linux

- job: Delphi
  pool:
    name: Delphi   # self-hosted Windows agents with a licensed Delphi
  steps:
  - script: |
      call "C:\Program Files (x86)\Embarcadero\Studio\23.0\bin\rsvars.bat"
      msbuild ServiceTool.Tests.dproj /t:Build /p:Config=Debug /p:Platform=Win64
      Win64\Debug\ServiceTool.Tests.exe
      msbuild ServiceTool.dproj /t:Build /p:Config=Release /p:Platform=Win64
    displayName: Build and test
  - task: PublishTestResults@2
    condition: succeededOrFailed()
    inputs:
      testResultsFormat: NUnit
      testResultsFiles: '**/dunitx-results.xml'
  - publish: Win64/Release/ServiceTool.exe
    artifact: ServiceTool-win64

The pr trigger together with a build validation branch policy on main means nothing merges without a green build, which is the single most valuable rule in the whole setup. DUnitX can write NUnit-style XML if you give the test project an XML output file, and the publish task turns that into the Tests tab on every run, with history and failure trends for free.

Delphi needs a licensed installation, so hosted agents are out. The practical setup is a Windows virtual machine in Azure with Delphi installed once, the Azure Pipelines agent registered in a private pool, and nobody logging in to it for anything else. That last part is the point: a build machine with nothing installed by hand is the only honest list of what your application depends on. Lazarus has no such restriction, and the hosted Ubuntu agent installs it fresh on every run.

Dependency hygiene

This is where Pascal projects most often fall behind. The ecosystem has package managers now, GetIt and Boss for Delphi and the Online Package Manager for Lazarus, but a lot of projects still rely on a Components folder that somebody copied from a previous machine.

My rules:

• Every third-party unit is in the repository or pinned to a commit. Vendor the source into the repository, or add it as a submodule at a specific revision. The build must not depend on whatever version a package manager serves today.
• Keep pinned snapshots in Azure Artifacts. If vendoring source into the repository feels heavy, Azure Artifacts universal packages hold a versioned, immutable copy of each component library, and the pipeline restores exactly the version the release was built with.
• Write down what you ship. A text file listing each library, its version, its license and where it came from. Call it a software bill of materials if that helps get it approved. It takes ten minutes and answers every "are we affected by this vulnerability" question for the lifetime of the product.
• Watch the native libraries. OpenSSL and SQLite DLLs sitting next to the executable are dependencies too, and OpenSSL in particular needs updating. Free Pascal programs using opensslsockets depend on the system library instead, which shifts the problem to the operating system but does not remove it.
• Review what you vendor. A component you downloaded from a forum post in 2009 has had nobody looking at it since. Read it once before it ships to customers.

Security basics that come almost for free

• Sign the executable. An unsigned Windows binary triggers SmartScreen warnings, and your service engineers will learn to click past warnings, which is the worst habit you can teach them. A code signing certificate and a signtool step in the pipeline fix this. If you would rather not keep a certificate on the build agent at all, Azure Trusted Signing provides a managed certificate and a signtool integration, so the private key never leaves Azure.
• No secrets in the binary. API keys and connection strings belong in configuration the user controls, never in a constant. Strings in an executable are one tool away from being read. In the pipeline, keep secrets in a variable group linked to Azure Key Vault, so they are injected at build time and never stored in the YAML or the repository.
• Validate at the boundary. Every string from a file, a socket, a serial port or a database is input. Parse it with a function that returns a result and a success flag, like TryStrToFloat, and decide explicitly what happens on failure.
• Keep debug information out of releases. It is not a security control by itself, but shipping a map file to customers is a gift to anyone reverse engineering your protocol.
• Verify certificates. See the REST article. If your tool talks over TLS and does not check the certificate, it is not talking over TLS in any meaningful sense.

None of this is specific to Pascal. What is specific is that the Pascal community grew up before these habits were common, and a lot of good software still misses them.

Closing the series

Ten articles ago I installed Delphi and Lazarus to see whether Pascal still belonged on my workbench. It does. The language is readable, the tools compile in seconds, the binaries run anywhere with nothing to install, and with a few habits from modern practice the result is software I would happily hand to a service engineer and not hear about again for years. Which, if you build tools for a living, is the highest compliment there is.

Questions for you

• Does your Pascal project build from the command line today? If not, what is the first thing that would break?
• How do you track third-party components and their versions? I would like to hear from teams with more than a handful of dependencies.
• What should the next series be? I have had requests for FireMonkey on mobile, for pas2js, and for porting a legacy VCL application to Lazarus. Vote by email.

Write to me at bjorn.nostdahl@nostdahl.com. Thank you to everyone who answered questions along the way; several of your replies ended up shaping these articles.