Senior DevOps Engineer
2021 - 2022Graphiant- Orchestrated a greenfield around 25 services local development stack using Kubernetes and Tilt, allowing developers to run a full stack of internally developed services, multiple databases, and open source components on top of Linux and macOS.
- Architected and deployed a multi-environment Azure infrastructure topology consisting of Azure Kubernetes Service (AKS) clusters and managed PostgreSQL, multiple VNets with VNet peering, and OpenVPN for staff VNet access using Terraform.
- Built a management network in Azure with site-to-site VPN connectivity to on-premises equipment across multiple environments and regions using Terraform CDK for Python and GitLab CI.
- Deployed and managed internally hosted GitLab Runners on Azure AKS and reduced average total CI pipeline runtimes from around 12 mins to approximately four mins via aggressive pipeline tuning and caching.
Technologies: Go, Linux, Kubernetes, Azure Kubernetes Service (AKS), Azure, Terraform, Helm, Python, Networking, Git, GitLab, GitLab CI/CD, DevOps, CI/CD Pipelines, DevOps Engineer, System Administration, Grafana, VPNSenior DevOps Engineer
2015 - 2021TED Conferences, LLC- Defined and architected a Docker strategy using Amazon ECS with Packer AMIs for underlying nodes, Consul for service discovery, a custom Go CLI tool to wrap Docker CLI, and Amazon ECS APIs to simplify multi-repo deploys.
- Built an Elasticsearch operational deployment strategy with nodes split between EC2 and ECS.
- Created a semi-immutable infrastructure using Packer to build AMIs and a test suite in Terratest and Goss. Primary logging clusters handle more than 450 GB or 368 million docs daily.
- Maintained the co-architected centralized logging infrastructure using Filebeat at the edge and custom Go CDN log parser for scale.
- Ingested all the above into Elasticsearch, RSpec tests for end-to-end Logstash pipeline, and Go and Docker tests for end-to-end CDN log ingest pipeline.
Technologies: Go, Ruby, Ruby on Rails (RoR), Node.js, Terraform, Bash, Amazon Elastic Container Service (Amazon ECS), Docker, Docker Compose, Docker Hub, Networks, Cloud Storage, Storage, Linux, MacOS, GitHub, Git, Amazon Web Services (AWS), DevOps, CI/CD Pipelines, DevOps Engineer, System Administration, Grafana, Amazon Virtual Private Cloud (VPC), AWS Cloud Architecture, Amazon EC2, Amazon RDS, Cost Analysis, Cost ControlDevOps Engineer
2013 - 2015Kixeye- Oversaw load testing and pre-launch tasks in the first three months, quick ramp-up from onboarding.
- Implemented a Vagrant development environment to allow developers to run the whole game stack locally, including MySQL, MongoDB, RabbitMQ, and Nginx/PHP, tie-ing in with Perforce.
- Architected an AWS migration strategy, including creating ephemeral load test environments using EC2, Packer, and a custom Ruby script.
Technologies: Linux, MySQL, MongoDB, RabbitMQ, NGINX, Perforce, Rackspace, Amazon Web Services (AWS), DevOps, DevOps Engineer, System Administration, Amazon Virtual Private Cloud (VPC), AWS Cloud Architecture, Amazon EC2
Nathan Sullivan
DevOps Engineer and Developer in Brisbane, Queensland, Australia
Member since November 21, 2022
Nathan is a senior DevOps engineer with over a decade of experience building cloud and on-premises server infrastructure and running various Ops, DevOps, and SRE-related coding projects. Before that, he worked in network administration systems holding administration, NOC, and software development roles. With a comprehensive skillset, Nathan is a dedicated professional willing to leverage his experience in exciting new projects.
Nathan is now available for hire
Portfolio
- GraphiantGo, Linux, Kubernetes, Azure Kubernetes Service (AKS), Azure, Terraform, Helm...
- TED Conferences, LLCGo, Ruby, Ruby on Rails (RoR), Node.js, Terraform, Bash...
- KixeyeLinux, MySQL, MongoDB, RabbitMQ, NGINX, Perforce, Rackspace...
Availability
Part-time
Preferred Environment
Linux, MacOS, Windows, Vim Text Editor, Slack, Firefox, Chrome, Amazon EKS
The most amazing...
...team I've been part of allowed me to build and maintain the server infrastructure behind TED Talks.
Employment
Experience
- CDN Log Parser for TED Talkshttps://www.ted.com
Replaced the CDN log ingest tooling for TED Talks, used to analyze and troubleshoot any end-user playback issues. Originally written in Ruby, as views increased, it became clear that the previous tooling couldn't keep up with the ingestion rate and needed something more performant.
I designed and implemented a replacement in Go, with three phases using three pools of goroutines with channels between respectively. The first phase consisted of downloading the following log file from Amazon S3, extracting and pushing lines onto a channel. The next step consisted of parsing and validating the log lines, sanitizing fields, applying GeoIP data, and then pushing this to another channel. Finally, the lines were batched and pushed to the Elasticsearch API. This data was then available via Kibana for analysis.
I applied back pressure testing at each stage automatically based on the channel max lengths and adjusted performance by tuning the number of goroutines per phase.
To my knowledge, this is still in production use today, parsing and ingesting 100+ million log lines daily.
Skills
Languages
Ruby, Go, Bash, PHP, Python, LuaFrameworks
Chrome, Ruby on Rails (RoR)Tools
Terraform, Amazon Elastic Container Service (Amazon ECS), Docker Compose, NGINX, GitLab CI/CD, GitHub, Vim Text Editor, Slack, Azure Kubernetes Service (AKS), Helm, Docker Hub, Git, GitLab, Grafana, Amazon Virtual Private Cloud (VPC), VPN, RabbitMQ, Perforce, Amazon EKSPlatforms
Firefox, Docker, Linux, MacOS, Windows, Kubernetes, Azure, Amazon Web Services (AWS), Amazon EC2, RackspaceOther
Cloud Storage, Storage, Load Balancers, Elastic Load Balancers, Infrastructure, Immutable Infrastructure, Infrastructure as Code (IaC), System Administration, Networking, Networks, HAProxy, CI/CD Pipelines, DevOps Engineer, AWS Cloud Architecture, Amazon RDS, Cost Analysis, Cost Control, Content Delivery Networks (CDN)Paradigms
Continuous Integration (CI), Continuous Delivery (CD), DevOpsStorage
MySQL, MongoDB, ElasticsearchLibraries/APIs
Node.js
