Daman Singh

The system consisted of several microservices written in Go and deployed using Amazon Elastic Container Service (Amazon ECS) with Fargate as compute resource. The microservices were responsible for tasks such as data ingestion, processing, and visualization.

AWS Kinesis was used as a distributed streaming platform to handle real-time data streams and ensure fault tolerance and scalability. The data was ingested into the platform using Kinesis producers and consumed by microservices for processing and analysis.

The platform incorporated real-time concurrency handling techniques like channels and mutexes to ensure thread-safe access to shared resources and prevent race conditions.

The platform also had a web-based dashboard that displayed real-time analytics, such as charts and graphs, based on the data ingested.

The platform allows users to add any property owned within the UK. It displays multiple trends (historical and forecast), including price, rentals, etc., and comparisons with district and city-based benchmarks.

As part of my role, I:
• Developed the entire back end using Go and multiple AWS services like RDS, DynamoDB, SQS, SageMaker, Lambda, EKS, etc.
• Used Terraform to manage and version control infrastructure and GitHub actions for CI/CD.
• Created and fine-tuned the low latency address autocomplete feature for a dataset of millions of addresses using a combination of Postgres indexes.
• Built a reliable async workflow to run ML predictions using SQS and SageMaker.