Software Developer in Fredericton, NB, Canada

The product was comprised of distributed appliances with each appliance having multiple services. I was assigned an issue that required me to launch a task on particular services on all appliances, triggered by the master node. At the time, no existing solution existed that would work in our environment. I went ahead and designed a solution that could trigger a task for a particular service on a particular appliance, as well as track its status and cancel tasks. Before I finalized the implementation, several other teams were already using it successfully for their own needs.

Technical tools used: Java, PostgreSQL, Java Message Service (JMS), iptables.

I joined this company when they were still in the POC stage and were collecting security-related data via a custom collector. They were storing this in Redshift and had a custom data pipeline for processing and storage. Due to this, and storage inefficiencies, this cost over $15,000/month with sub-par performance. Our custom processing was at least four hours behind and was consistently dropping or not enhancing the data. During my time there, I switched the pipeline to Apache Spark on EMR, and utilized AWS Athena with Redis for aggregated data. This allowed us to process the data live as it came in, and we never again fell behind. It also made the pipeline much more maintainable and easy to work with. The cost was brought down to about $3,000/month, resulting in significant savings.

I set up a microservice architecture from development to production. I was responsible for setting up the standards and processes used by developers to deliver features from start to finish. Tasks included:
* Project structure
* Git structure
* Automated CI/CD
* E2E testing standards
* Deployment routing and authorization