Bob is available for hire
Hire BobBob Clarke
Verified Expert in Engineering
DevOps Engineer and Developer
Location
London, United Kingdom
Toptal Member Since
March 15, 2021
Bob is a highly experienced DevOps lead with a track record of designing and building large-scale, highly available systems within the financial sector, both cloud-based and on-premise. Bob has done container orchestration (Kubernetes), infrastructure as code, CI/CD, service mesh (Istio), TCP/IP networking, performance engineering, and system telemetry. He has worked with large programs and small greenfield projects alike. Bob is passionate about building robust solutions to deliver value.
Portfolio
Dentsu International
Kubernetes, Terraform, Helm, Azure, Apache Kafka, CI/CD Pipelines, TCP/IP, Go...
Lloyds Banking Group
Ansible, Splunk, Kubernetes, Linux, Terraform, Helm, TCP/IP, CI/CD Pipelines...
HSBC
Ansible, Docker, TCP/IP, PKI, Linux, CI/CD Pipelines...
Experience
Availability
Part-time
Preferred Environment
Linux, Kubernetes, Azure, Helm, Docker, Bash, Terraform, Istio, Prometheus, Distributed Tracing
The most amazing...
...thing I've achieved is the design and implementation of a large-scale production Kubernetes infrastructure to host my clients' global microservices platform.
Work Experience
DevOps Lead
2018 - PRESENT
Dentsu International
- Designed and built a large-scale production Kubernetes platform in Azure Kubernetes Service (AKS) to host my client's global microservices platform. Currently in the process of implementing Istio service mesh on this platform.
- Established and led a Platform Engineering and DevOps team of 16 engineers responsible for infrastructure design, build automation, operations, and DevOps tooling/release management.
- Built a continuous delivery and release management system to support over 25 development teams running over 120 pipelines. The tech stack used is Brigade.
- Established standard infrastructure as code (IaC) patterns and developed parameterized Terraform modules for all components used on the platform.
- Implemented monitoring and centralized logging for my client's Kubernetes estate using Prometheus, Grafana, and Loki. Currently in the process of adding distributed tracing by leveraging the native support for this that is built into Istio.
- Built and automated a large-scale Kafka platform on Kubernetes to support my client's workflow platform. Developed a new pattern for exposing Kafka broker endpoints via Kubernetes Ingress.
- Defined architecture patterns and engineering standards for the migration to the Istio service mesh.
Technologies: Kubernetes, Terraform, Helm, Azure, Apache Kafka, CI/CD Pipelines, TCP/IP, Go, Linux, DevOps, Grafana, Prometheus, Infrastructure as Code (IaC), Docker, Loki, GitHub, Transport Layer Security (TLS), PKI, Distributed Tracing, Env0, Containers, Container Orchestration
Senior DevOps Engineer
2017 - 2018
Lloyds Banking Group
- Developed automated build scripts for WebSphere Liberty.
- Developed Splunk dashboards for platform telemetry.
- Provided support and training for the IBM UrbanCode platform.
- Built Jenkins pipeline libraries to provide a common set of functions for development teams.
Technologies: Ansible, Splunk, Kubernetes, Linux, Terraform, Helm, TCP/IP, CI/CD Pipelines, Docker, DevOps, Infrastructure as Code (IaC), GitHub, Transport Layer Security (TLS), PKI, Container Orchestration, Containers
DevOps Lead
2016 - 2017
HSBC
- Managed the infrastructure design, DevOps tooling, automated environment builds, and configuration management for HSBC's DTC program.
- Developed a fully automated, on-demand runtime to provide dedicated CI/CD machinery, including dedicated pipeline tooling and a set of VMs for test environments, for each development team.
- Developed a library of Terraform modules to facilitate the automation of each system component, such as Postgres, Redis, Nginx, and more.
Technologies: Ansible, Docker, TCP/IP, PKI, Linux, CI/CD Pipelines, Transport Layer Security (TLS), Shell Scripting, DevOps, Infrastructure as Code (IaC), GitHub
Engineering Lead
2014 - 2016
Lloyds Banking Group
- Acted as engineering lead for LBG's UrbanCode automation program. The program objective was to deliver an automation platform to provide deployment and configuration management for my client's retail online banking platform.
- Developed automated builds for WebSphere Application Server, DataPower, and F5 BigIP.
- Defined engineering standards for the production of the WebSphere infrastructure. This included automation, network security, high availability, monitoring, and capacity management.
Technologies: Linux, TCP/IP, CI/CD Pipelines, Transport Layer Security (TLS), Shell Scripting, PKI, DevOps, Infrastructure as Code (IaC), GitHub, Containers
Engineering Consultant
2012 - 2013
Deutsche Bank
- Designed and built Deutsche's strategic business process management (BPM) platform for the DB Palace project.
- Developed automated builds for BPM platform and rolled out across route to live and production.
- Created automation to test and release BPM workflows.
- Built a custom web-based monitoring and inventory system.
- Worked closely with development teams post-implementation to iron out operational issues.
- Produced system documentation and run-book, carried out handovers and system sign-offs, and provided technical training for Deutsche Bank support teams.
Technologies: PKI, Linux, TCP/IP, Transport Layer Security (TLS), Shell Scripting
Lead Technical Architect
2010 - 2012
PwC UK
- Designed and implemented a business process management infrastructure based on IBM's business process management (BPM) system.
- Developed automated build scripts and rolled out test and production environments.
- Provided training for in-house teams and produced system run-books for support teams.
Technologies: TCP/IP
Senior Technical Consultant
2004 - 2010
Fidelity International
- Led the migration for the Fidelity FundsNetwork platform from WebSphere version 4.0 to 6.1.
- Designed and developed a toolset for WebSphere's automated delivery and configuration management.
- Designed and developed a browser-based inventory tool that provided a dynamic view of server utilization, configuration, and software inventory across the server estate.
- Carried out the platform's day-to-day systems administration, support, and monitoring.
Technologies: Linux, TCP/IP, Shell Scripting
Market Data Specialist
2004 - 2004
Tullett Prebon
- Designed and built a new Reuters Market Data System (RMDS) platform and migrated 400 trading floor positions from the legacy system.
- Developed automated build and configuration management scripts for the platform.
- Created a browser-based tool for monitoring key systems metrics and usage data.
Technologies: TCP/IP, Shell Scripting
Market Data Specialist
1998 - 2004
Dresdner Kleinwort Wasserstein
- Provided operational support of the production RMDS infrastructure.
- Provided system design and engineering services for in-house development teams.
- Developed a browser-based market data usage and cost reporting tool.
Technologies: Shell Scripting
Market Data Specialist
1996 - 1998
NatWest Markets
- Migrated existing RMDS infrastructure from a standalone network to my client's corporate LAN.
- Co-designed and developed a browser-based monitoring and management tool using Perl/CGI.
- Provided day-to-day support for trading floor users.
Technologies: Data Analytics (Marketing), Solaris, Infrastructure
Experience
Large-scale Production Kubernetes Platform
I was the technical lead for a project to build a large-scale Kubernetes platform (approx 30 clusters) on Azure (AKS).
As part of this project, I designed and automated the build of a new internet landing zone based on Azure Application Gateway and migrated to this from my client's legacy F5-based system.
I automated TLS certificate creation and renewals using Let's Encrypt and implemented system-wide monitoring and logging using Prometheus, Grafana, and Loki.
I also set up a CI/CD toolchain to support a global team of approximately 200 developers, and I automated all builds using Terraform. During the project, I established and documented a set of IaC standards for adoption by other engineering teams.
As part of this project, I designed and automated the build of a new internet landing zone based on Azure Application Gateway and migrated to this from my client's legacy F5-based system.
I automated TLS certificate creation and renewals using Let's Encrypt and implemented system-wide monitoring and logging using Prometheus, Grafana, and Loki.
I also set up a CI/CD toolchain to support a global team of approximately 200 developers, and I automated all builds using Terraform. During the project, I established and documented a set of IaC standards for adoption by other engineering teams.
Production Kafka Platform
The requirement was to implement an automated, scalable, and resilient Kafka platform to be hosted on Kubernetes (AKS). Another requirement was that the Kafka broker endpoints needed to be web-facing.
Automating Kafka and its related components was fairly straightforward. We chose Confluent Kafka and this came with a set of Helm charts. I needed to modify these Helm charts as they did not ship with Ingress resources (which I needed in order to comply with our existing Kubernetes Ingress standards). This being the case, I forked the chart code and hosted the packaged charts on our internal Chart Museum service.
The biggest and most interesting challenge was exposure to the internet via Kubernetes Ingress. Although the patterns were available in the community for exposing Kafka endpoints to the internet via Kubernetes services, there did not appear to be any existing patterns for doing so via Kubernetes Ingress. With this in mind, I set about forming a new pattern that leveraged Nginx TCP services. This worked very well, so I formalized the pattern with our architecture team and automated the build with Terraform.
Automating Kafka and its related components was fairly straightforward. We chose Confluent Kafka and this came with a set of Helm charts. I needed to modify these Helm charts as they did not ship with Ingress resources (which I needed in order to comply with our existing Kubernetes Ingress standards). This being the case, I forked the chart code and hosted the packaged charts on our internal Chart Museum service.
The biggest and most interesting challenge was exposure to the internet via Kubernetes Ingress. Although the patterns were available in the community for exposing Kafka endpoints to the internet via Kubernetes services, there did not appear to be any existing patterns for doing so via Kubernetes Ingress. With this in mind, I set about forming a new pattern that leveraged Nginx TCP services. This worked very well, so I formalized the pattern with our architecture team and automated the build with Terraform.
Istio Rollout to Production
The implementation of Istio provided my client with the following new capabilities:
• Granular, policy-based network controls (east-west and egress)
• JWT validation at Ingress Gateway
• Transport encryption between pods
• Telemetry
I was the technical lead for the project; the key challenges I had to overcome were as follows:
• Migration of Ingress resources to Virtual Services in 200+ existing Helm charts
• Cutover from Ingress controllers to Istio Ingress Gateway
• Auditing services that would be MESH_EXTERNAL and configuring necessary Istio ServiceEntry's
• Developing a Terraform module for Istio deployment
• Updating Helm charts for Kubernetes Jobs to ensure Istio sidecar exit
• Integration of Kiali with existing Prometheus estate
• Granular, policy-based network controls (east-west and egress)
• JWT validation at Ingress Gateway
• Transport encryption between pods
• Telemetry
I was the technical lead for the project; the key challenges I had to overcome were as follows:
• Migration of Ingress resources to Virtual Services in 200+ existing Helm charts
• Cutover from Ingress controllers to Istio Ingress Gateway
• Auditing services that would be MESH_EXTERNAL and configuring necessary Istio ServiceEntry's
• Developing a Terraform module for Istio deployment
• Updating Helm charts for Kubernetes Jobs to ensure Istio sidecar exit
• Integration of Kiali with existing Prometheus estate
Skills
Tools
Terraform, Helm, Istio, GitHub, Ansible, Grafana, Splunk, Azure Application Gateway
Paradigms
DevOps
Platforms
Kubernetes, Docker, Linux, Unix, Azure, Apache Kafka
Other
Infrastructure as Code (IaC), TCP/IP, CI/CD Pipelines, Shell Scripting, Container Orchestration, Prometheus, HashiCorp Vault, Loki, IT Infrastructure, Transport Layer Security (TLS), PKI, Content Delivery Networks (CDN), Distributed Tracing, Env0, Containers
Languages
Go