Container Image Scanning and Supply Chain Security in FinTech

The Wake-Up Call

About eighteen months ago, we ran a routine audit on our payment gateway containers and found a critical CVE in a transitive dependency buried three layers deep in our Go binary's Alpine base image. The vulnerability had been sitting there for eleven weeks. Nobody noticed because our CI pipeline only checked application dependencies — not the OS-level packages baked into the container.

That incident changed how I think about container security in financial services. It's not enough to scan your go.mod or Gemfile.lock. You need to treat the entire container image — every layer, every binary, every shared library — as your attack surface.

Building a Scanning Pipeline That Works

After that incident, we rebuilt our image scanning from scratch. The goal was simple: no container reaches production without being scanned at every layer, and no critical or high CVE gets through without an explicit, time-boxed exception.

We settled on Trivy as our primary scanner. It's fast, it handles OS packages and language-specific dependencies in a single pass, and it integrates cleanly into GitHub Actions. We also run Grype as a secondary scanner — different scanners have different vulnerability databases, and the overlap isn't 100%.

Here's the core of our CI scanning step:

# .github/workflows/image-scan.yml
- name: Build image
  run: docker build -t payment-gateway:${{ github.sha }} .

- name: Trivy vulnerability scan
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: payment-gateway:${{ github.sha }}
    format: 'sarif'
    output: 'trivy-results.sarif'
    severity: 'CRITICAL,HIGH'
    exit-code: '1'

- name: Grype secondary scan
  run: |
    grype payment-gateway:${{ github.sha }} \
      --fail-on high \
      --output table

The exit-code: '1' is the key part — it fails the build if any critical or high severity vulnerability is found. No exceptions in the pipeline itself. If a team needs to ship with a known vulnerability (it happens — sometimes a fix doesn't exist yet), they file an exception in our security tracker with an expiry date, and we add it to Trivy's .trivyignore file.

Base Image Hygiene

The single biggest improvement we made wasn't adding more scanners — it was shrinking our base images. We moved our Go services from alpine:3.18 to gcr.io/distroless/static-debian12. The difference was dramatic.

Distroless images contain only your application and its runtime dependencies — no shell, no package manager, no curl, no wget. That's exactly what you want for a payment service. If an attacker somehow gets code execution inside the container, there's almost nothing to work with. No shell to spawn, no tools to download additional payloads.

For our Ruby services (we still have a few), we use a multi-stage build that compiles everything in a full Ruby image and then copies the app into a slim runtime image. The Dockerfile looks roughly like this:

FROM ruby:3.3-slim AS builder
WORKDIR /app
COPY Gemfile Gemfile.lock ./
RUN bundle config set --local deployment true && \
    bundle install --jobs 4
COPY . .

FROM ruby:3.3-slim AS runtime
RUN groupadd -r appuser && useradd -r -g appuser appuser
WORKDIR /app
COPY --from=builder /app /app
USER appuser
CMD ["bundle", "exec", "puma", "-C", "config/puma.rb"]

SBOMs — The Boring Part That Auditors Love

PCI DSS 4.0 doesn't explicitly mandate SBOMs (Software Bill of Materials), but every auditor I've worked with in the last year has asked about them. Having a machine-readable inventory of every component in your container images makes vulnerability response dramatically faster.

We generate SBOMs in CycloneDX format using Trivy's built-in SBOM generation:

trivy image --format cyclonedx \
  --output sbom.cdx.json \
  payment-gateway:${{ github.sha }}

The SBOM gets stored alongside the image in our OCI registry as an attached artifact. When a new CVE drops, we can query all our SBOMs in seconds to find which services are affected, instead of rebuilding and scanning every image from scratch.

Image Signing with Cosign

Scanning tells you what's in the image. Signing tells you the image hasn't been tampered with since it was scanned. We use Cosign from the Sigstore project to sign every image that passes our scanning pipeline.

The signing happens in CI after the scan passes:

- name: Sign image with Cosign
  run: |
    cosign sign --yes \
      --key env://COSIGN_PRIVATE_KEY \
      $REGISTRY/payment-gateway:${{ github.sha }}

On the deployment side, our Kubernetes admission controller (Kyverno) verifies the signature before allowing any image to run. If an image isn't signed by our CI pipeline's key, it gets rejected. This closes the loop — even if someone pushes a malicious image directly to our registry, it won't have a valid signature and Kubernetes won't run it.

Continuous Monitoring — Not Just CI

Here's the thing most teams miss: scanning at build time is necessary but not sufficient. New CVEs are published daily. An image that was clean when you built it last Tuesday might have three critical vulnerabilities by Friday.

We run a nightly scan of every image currently deployed in production. A CronJob in our cluster pulls the list of running images and scans each one against the latest vulnerability database. If anything new shows up as critical, it pages the on-call engineer and creates a ticket automatically.

The nightly scan has caught issues that build-time scanning missed at least four times in the past year. Each time, it was a newly published CVE affecting a package that was clean at build time.

What We Track

• Mean time to remediate — from CVE publication to patched image in production. Our target is 72 hours for critical, 2 weeks for high.
• Exception count — how many vulnerabilities are currently in our .trivyignore with active exceptions. We review these weekly.
• Base image age — we alert if any production image is running a base image older than 30 days. Stale base images are the number one source of accumulated vulnerabilities.
• SBOM coverage — percentage of production services with a current SBOM attached. Target is 100%.

Lessons Learned

After running this pipeline for over a year across about forty payment-related microservices, here's what I'd tell someone starting from scratch:

1. Start with base image selection. Switching to distroless or scratch images eliminates more vulnerabilities than any scanner will catch. Fewer packages means fewer things that can go wrong.
2. Run two scanners. Trivy and Grype use different vulnerability databases. We've seen cases where one catches something the other misses. The overhead is minimal — an extra 30 seconds in CI.
3. Don't just scan at build time. Nightly production scans are essential. CVEs don't wait for your next deployment.
4. Make exceptions explicit and time-boxed. Every ignored vulnerability should have an owner, a justification, and an expiry date. Review them weekly.
5. Sign everything. Image signing with Cosign plus admission control in Kubernetes is the closest thing to a guarantee that only scanned, approved images run in production.

References

• Trivy Documentation — Aqua Security
• Grype — Anchore Open Source Vulnerability Scanner
• Cosign — Sigstore Container Signing
• Distroless Container Images — Google
• CycloneDX SBOM Specification
• PCI DSS v4.0 — PCI Security Standards Council

Disclaimer: This article reflects the author's personal experience and opinions. Product names, logos, and brands are property of their respective owners. Pricing and features mentioned are subject to change — always verify with official documentation.