CI/CD · SRE · RELEASE ENGINEERING

CI/CD delivery
and SRE operations

I run build, deployment and test environments for R&D from within the QA team, plus monitoring, reverse proxies and internal systems. Since mid-2025 I develop with AI agents and control their output with automated tests and hands-on verification. Every case states my scope, how it was verified, and where each number comes from.

DELIVERY PIPELINE · MY SCOPEBuilt / run by me

Delivery

  1. 01 SOURCER&D mergeGitLab
  2. 02 BUILDBuild imagesGitLab CI
  3. 03 REGISTRYVersioned imagesHarbor

Run

  1. 04 GITOPSAuto syncArgoCD → RKE2
  2. 05 OBSERVEMonitoringGrafana · Loki
  3. 06 SECUREEvent collectionWazuh

INFRASTRUCTUREProxmox VE · TrueNAS · OPNsense · Linux ×100+

~10 yrs
Systems engineering and operations since 2016
100+
Linux hosts under management
9,136 h
Project hours logged in timesheets since Apr 2021

ROLE & SCOPE

What I own

The delivery and operations engineer inside QA: R&D writes the code; I own the pipeline and infrastructure from merge to test, release and monitoring.

Hours are grouped by keywords in timesheet descriptions and may overlap. Product names are anonymised.
Product / systemMy responsibilityPeriodRelated hoursEvidence
Company cloud productR&D test environmentGitOps deployment via GitLab CI → Harbor → ArgoCD; RKE2 cluster, Envoy Gateway routing and TLS2024-10 – now362 h (me ≈175 h · AI ≈187 h)Deploy repo: 419 commits (117 mine) · Read case
R&D build environment6 build lines incl. Web, Go, Java, WindowsGitLab Runners and build hosts; moved Jenkins into containers, later migrated its pipelines to GitLab CI/CD2021-04 – nowGitLab CI 472 h (me ≈427 h · AI ≈45 h) · Jenkins 681 h6 CI/CD handover manuals (Sep 2026)
Public-facing reverse proxyPublic company websitesRebuilt the proxy and WAF, config under GitOps, scheduled upkeep with Ansible2022-10 – now939 h (me ≈843 h · AI ≈96 h)1,425 of my commits to the config repo · Read case
Monitoring platformCompany hosts and the K8s clusterGrafana, Prometheus, Loki, Alertmanager; per-team scoped views2022-06 – now496 h (me ≈422 h · AI ≈74 h)58 alerting rules, 22 recording rules, 18 scrape targets · Read case
Internal systemsProject management, image registry, collaboration toolsUpgrades, data migration, incident investigation and reports2022 – nowProject management 126 h (me ≈76 h · AI ≈50 h)Incident reports and operations log
Identity verification serviceCloud gatewayPlatform side: namespaces and RBAC, gateway routes, restricted deploy identity; the service team owns Helm values2025-12 – now—Self-service guide · handover manual

How to read the numbers

Every result states its source. Where nothing was recorded, no number is given.

System record
Timesheets, Git history, monitoring, test reports
Own estimate
Retrospective range with its basis
Self-assessed
Performance review self-rating or manager feedback

CASES

Featured cases

Each covers context, my scope, verification, results and limits

CI/CDIn use by R&D

GitOps delivery for a cloud product

The product runs as containers on an RKE2 cluster used as R&D's test environment. After a merge, GitLab CI builds images and updates a deploy repo; ArgoCD syncs the cluster. Nobody logs in to a host.

  • GitLab CI
  • Harbor
  • ArgoCD
  • RKE2
  • Envoy Gateway
median ~33 per month
Automated deploys
System record
Partly AI-assistedRead case
SREIn use

Rebuilding the public reverse proxy and WAF

Rebuilt the public reverse proxy, hardened it with ModSecurity (OWASP CRS) and Fail2Ban, put every config change under Git, and surfaced block events in Grafana.

  • Nginx
  • ModSecurity
  • Fail2Ban
  • Ansible
  • AWX
1,425
My commits to the config repo
System record
Partly AI-assistedRead case
SREIn use

Monitoring platform and alert governance

Built the company's monitoring from scratch: Grafana, OpenSearch and Fluent Bit first, then Prometheus, Loki and Alertmanager, with views scoped per service team and alerts by email.

  • Grafana
  • Prometheus
  • Loki
  • Alertmanager
  • Fluent Bit
58 / 22 / 18
Alerting rules / recording rules / scrape targets
System record
Partly AI-assistedRead case
Test automationHanded over to QA

AI-assisted full regression automation for a cloud product

Turned existing manual test cases into Gherkin, Robot Framework and Playwright scripts. Every step keeps before/during/after screenshots and a recording; the product QA reviewed each item before taking over maintenance.

  • Robot Framework
  • Playwright
  • Gherkin
  • Claude Code
541 cases in 2 h 23 min
Cloud product full regression (2026-07-02)
System record
Developed with AI agentsRead case
Self-proposedPOC passed

A K8s cluster build-and-recovery platform: an internal tool I proposed and built with AI agents

Building Kubernetes clusters from scratch and repeated lab work exposed their complexity and learning curve. To help IT staff provision and configure clusters, I began developing a Go management platform with AI agents in August 2025. A VM-based on-premises cluster on Proxmox takes an estimated 20–30 minutes, depending on storage I/O; backup and recovery capabilities are verified separately.

  • Go
  • React
  • OpenTofu
  • Proxmox
  • RKE2
20–30 min (conservative)
Build an on-prem VM-based cluster
Own estimate
Developed with AI agentsRead case

More cases

AI-ASSISTED ENGINEERING

Developing with AI agents

I first paid US$20 a month for AI tools myself; when Claude Code arrived I moved to US$100 a month from 2025-07-09 and began developing with AI agents. Development work, commits and new projects all rose sharply afterwards. My own hands-on share of the time fell from about 80% to about 20% (own estimate).

My hands-on time : AI executionOwn estimate
  1. 2025-02Started working with AI8:2
  2. 2025-07Moved to Claude Code6:4
  3. 2025-09Claude Sonnet 4.55:5
  4. 2025-11Claude Opus 4.54:6
  5. 2026-02Claude Opus 4.63:7
  6. 2026-06Claude 5 generation2:8

MeAI Agent

Early on, steady output from agents took a lot of hand-written scaffolding: CLAUDE.md/AGENTS.md, rules under .claude, SKILL.md files and project indexes, plus my own loop and harness engineering. With each new Claude generation more of that became built in. My time moved to decisions on data control and flow, system architecture and maintainability, while agents handle implementation and execution.

Ratios are my estimate, stepped at Claude model releases. Each case’s logged hours are split by the period’s ratio (at 2:8, 100 h is 20 h mine and 80 h AI); hours before 2025-02-22 count as mine. From 2026 models also run overnight, outside the timesheet, so only company-logged hours are counted and actual AI time is higher.

Share of hours logged as programming
19% → 44%
H1 2025 → H1 2026 (H2 2025: 33%)
System record
My Git commits
326 → 1,296
21 months before → 16 months after
System record
New repositories
3 → 13
Before → after the turning point
System record
Windows test environment setup
1–2 weeks → 1 day
Research included; repeated across Windows versions
Own estimate

Division of work

I define requirements, split tasks, make the data-control and architecture decisions, review and accept; agents draft code, tests and docs.

Keeping the agent honest

AI output is controlled by tests, screenshots, recordings and hands-on checks; all code and config goes through version control and review.

Full details

TIMELINE

Work since 2021

The last entry each year groups short projects, ad-hoc testing and POCs not listed separately; hours come from timesheets and exclude leave, admin and meetings.

  1. 2021
    1. Moved Jenkins into Linux containers with HTTPSLater rewrote its pipelines for GitLab CI/CD and retired Jenkins because of plugin and security upkeep.
    2. Proved a disk I/O bottleneck and moved to ProxmoxTest VMs on VMware ESXi with spinning disks lagged badly; I bought an SSD test machine myself and showed the bottleneck was disk I/O. The next year hardware was funded and CI and test hosts moved to open-source Proxmox.
    3. Second-line product verificationFor 1–2 years while field engineering was short-staffed, verified complex scenarios such as multi-domain Microsoft AD integration and documented them.
    4. PostgreSQL high availability automation (failed project)Built a three-node HA PostgreSQL cluster with pg_auto_failover, Terraform, Ansible and GitLab CI (about 10 minutes to build), verified failover and moved internal system data onto it; also supported HA testing for product releases (2021–2025, ≈386 h logged). It failed mainly on early communication: I owned it as a QA engineer but was outside IT’s decisions and meetings; nobody owned the integration; and the colleagues taking it over had no PostgreSQL operations experience yet.
    5. Other: short projects, ad-hoc testing and POCs (≈594 h)Product release testing and upgrade checks; container and K8s research (Docker Swarm, Rancher, a private registry); Windows AD/ADFS and LDAP integration environments; jump-host and Pushgateway POCs.
  2. 2022
    1. IaC pilotTerraform, Ansible and GitLab CI built test environments automatically (POC).
    2. SonarQube improvements and trainingAdded open-source reports and LDAP, ran internal training; another team owns the platform.
    3. Rebuilt the public reverse proxyWAF, GitOps and Ansible scheduling followed.
    4. Monitoring platform beginsFirst version with Grafana, OpenSearch and Fluent Bit.
    5. Other: short projects, ad-hoc testing and POCs (≈782 h)Product builds on new OS releases (Rocky Linux, Ubuntu) and HA destructive testing; Ansible install scripts; cross-domain Windows, multi-domain Exchange and ADFS automation environments; short trials such as Nexus3 and FTP integration; CI/CD integration across projects.
  3. 2023
    1. Egress gateway for the test networkSquid controls downloads from test machines; later folded into the OPNsense firewall.
    2. Collaboration system build and migrationBuilt, migrated and maintained Mattermost.
    3. Making scan licences go furtherProposed routing scan targets through an Nginx upstream and wrote the scripts; adopted, with scans run by colleagues.
    4. Ansible knowledge transferTrained IT staff on Ansible.
    5. Security event collectionWazuh with OpenCTI.
    6. Other: short projects, ad-hoc testing and POCs (≈513 h)LDAP pass-through and OTP integration tests; Windows AD automation research (including PyAutoGUI); Ansible database restore drills; a mobile-testing CI environment (Appium); Nginx, Let’s Encrypt and self-signed certificate issues; database migration and troubleshooting.
  4. 2024
    1. Two-factor login pilot for internal serversImplemented and tested two-factor login on CI/CD servers and the image registry.
    2. K8s test environment and GitOps begin
    3. Other: short projects, ad-hoc testing and POCs (≈578 h)K8s networking and node troubleshooting (MetalLB, ingress-nginx, DNS forwarding); OPNsense firewall hardware tests and fixes; Exchange test environments and load balancing; Ansible deployment and host-task scripts; GitLab CI/CD project migrations; ModSecurity log parsing and a GeoIP module build; Java Spring Boot research.
  5. 2025
    1. Build-server tool updaterAutomatic tool updates on build servers.
    2. Started developing with AI agents (Jul 9)
    3. Building the K8s cluster platform
    4. Other: short projects, ad-hoc testing and POCs (≈529 h)Rancher/K8s automation scripts and HA cluster tests; a MikroTik and OPNsense lab network; Exchange DAG clusters and multi-domain AD environments; TrueNAS/NAS builds and backup migration; database GitOps and recovery tests; a 24-hour unattended AI-agent development trial; firewall syslog cleanup.
  6. 2026
    1. AI-assisted full regression
    2. K8s platform incidents and upgradesStepped K8s from 1.29 to 1.34 and Longhorn from 1.8 to 1.10; replaced the archived Ingress Nginx with Gateway API; restored Proxmox, NAS and cluster services after a power outage; upgraded Harbor and moved its storage.
    3. Vulnerability risk platform (self-proposed, not deployed)Wazuh vulnerability sync, NVD/EPSS/KEV scoring, Playwright E2E; concluded that network segmentation and firewalling were the practical mitigation.
    4. Client-certificate gateway (mTLS)
    5. Internal project-management system upgradeInventory, compatibility, migration checks and cut-over; investigated and fixed an attachment upload issue after the update. This round logged 62 h: about 12 h mine and about 50 h of AI execution (2:8, own estimate).
    6. CI/CD handover manualsWrote build-host and pipeline handover manuals for six R&D teams.
    7. Operations records platform
    8. Other: short projects, ad-hoc testing and POCs (≈164 h)SAML/OAuth and Exchange MFA architecture POCs; a rack NAS rollout and acceptance; a Renovate-based dependency update tool; blue-green deployment runbooks; MikroTik test-network trials; an e-learning (Moodle) upgrade; lab hardware and AI-agent subscription proposals. (Through October.)

SCOPE

Tools and depth

  • Daily operations
  • Project use
  • Via AI agents
LayerToolsWhat I did
Delivery
  • GitLab CI
  • ArgoCD
  • Harbor
  • Jenkins
Runners and build hosts, GitOps deploys, Harbor upgrades, Jenkins migration
Containers
  • RKE2
  • Rancher
  • Envoy Gateway
  • Longhorn
RKE2 cluster build; K8s 1.29 → 1.34 upgrades; Ingress Nginx to Gateway API
Observability
  • Grafana
  • Prometheus
  • Loki
  • Fluent Bit
Alert rules and notices, per-team dashboards, central logs
Virtualisation and network
  • Proxmox VE
  • TrueNAS
  • OPNsense
  • Wazuh
Disk I/O and storage incidents, recovery after outages, security events
Automation and testing
  • Ansible
  • Robot Framework
  • Playwright
  • Go (Via AI agents)
Ansible operations automation; product test suites; Go projects written by AI agents, reviewed and verified by me

EXPERIENCE

Experience

  1. 2021.04 — present

    Test Automation Engineer・ChangingTec · QA, IT Services

    R&D build and deploy pipelines, the K8s test environment, monitoring, the public reverse proxy and internal systems; product test automation, release testing and second-line verification.

  2. 2019.01 — 2021.04

    Operations Engineer・Blue Planet Inc.

    Ran customer systems and websites; introduced vulnerability scanning and patched against advisories; automated operations with Bash and PowerShell.

  3. 2017.09 — 2018.08

    Systems Engineer・NEWEBINFO

    IBM Power server and storage replacement, AIX maintenance and upgrades, restore drills with customers.

  4. 2016.09 — 2017.08

    Systems Engineer・SYSTEX

    Deployed and maintained security products, wrote product documentation, supported customer POCs.

Education B.A., National Quemoy University Certifications RHCSA (2016)・MCSA: Windows Server 2012・IBM Certified System Administrator – AIX