Priyanshu Hasija

The Roambee Corporation is an IoT supply chain and enterprise asset visibility company. It offers real-time visibility of assets and goods outside the four walls of a global enterprise with patented hardware and software technology combined with an array of sensor data, proprietary analytics, predictive reporting, and open APIs.

Roambee bees (devices) continuously send heartbeats that involve many useful components like coordinates, temperature, battery life, and pictures. We gather this information on AWS S3, and then real-time tracking of the goods is shown on the UI. The front end was built on Node.js and the back end with Spark real-time streaming.

The project is based on an ETL model where data is collected from various sources, namely MDM and Salesforce, in an S3 data lake and fed to the PySpark pipelines for data quality checks and processing. Data flows through various layers from raw to curated to conform. The final data is moved to Redshift for analytics and reporting using Tableau.

This project deals with ETL pipelines hosted on AWS. Incoming data from various sources like SFDC and Mulesoft. Real-time streams are ingested into a data lake on S3. As data arrives on S3, Lambda placed on S3 buckets triggers the data pipeline jobs written on PySpark and hosted on AWS EMR/AWS Glue. Data then moves from raw to transformed and then to the conformed layer, which is loaded to Redshift for analytics and reporting.

Project deals with processing experimental images and extracting required information. I designed the Python interface, which creates a connection with PostgreSQL using SQLAlchemy. The interface takes raw experimental images as input and based on client requirements. The required information is extracted from the images which are then inserted into the database. The interface has the functionality to process, download, and delete images from the database as per the requirements.