Ivan Lorencin

I led the development of a system for automatic speech transcription from video CVs in any language to concise English summaries using Whisper and the OpenAI API. This system enhances accessibility and understanding globally.

As the lead developer, I designed the system architecture and oversaw its implementation. I integrated Whisper for accurate speech-to-text conversion across languages and dialects. Leveraging the OpenAI API, I ensured that translated text was transformed into standardized English summaries, focusing on education, work experience, skills, and achievements.

Key technologies included Whisper for speech recognition and the OpenAI API for translation. Automated algorithms were developed for summarization and optimizing content for readability and relevance.

Implemented efficiencies reduced video CV review times by at least 20%, benefiting multinational organizations and HR teams globally. Enhanced accessibility of candidate profiles and improved decision-making through standardized summaries.

I developed a retrieval-augmented generation (RAG) solution for a prominent travel agency, one of Croatia's largest, where every 8th client uses their services. As the leading Destination Management Company (DMC) in Croatia and Southeast Europe, our solution provides personalized ski recommendations.

Our system begins by aggregating data from diverse sources to analyze customer preferences, utilizing RAG, the OpenAI API, FAISS, and Streamlit for efficient processing. This enables us to generate ski recommendations tailored to clients' unique preferences and needs. This allows us to generate tailored ski recommendations that align with each client's unique needs and preferences.

Moreover, our platform streamlines the interaction and booking processes, optimizing the overall customer experience. This advancement is expected to significantly reduce agents' time manually searching for destinations, averaging over half an hour per query. Agents' roles will shift to verifying and approving the generated proposals, leading to increased efficiency and productivity.

The link showcases the prototype of our solution. The actual system is in advanced development stages.

This Streamlit application is a prototype for developing an email prioritization system using the OpenAI API. It integrates advanced AI to evaluate email priority and spam status based on predefined criteria. My role included designing and implementing the system architecture with Streamlit for the UI, OpenAI for AI functionalities, and operating system for file management. The system aims to enhance user efficiency and reduce manual email sorting time, optimizing operational workflows within the travel agency. It targets a 10% reduction in query analysis time, benefiting a leading DMC in Southeast Europe. Currently, in the mockup stage, the system will integrate into the DMC's email infrastructure to boost efficiency and productivity in managing client communications.

This prototype application illustrates how XCLIP, a language vision model, can expedite and simplify video classification solutions without needing model training. By leveraging pre-trained capabilities from the "microsoft/xclip-base-patch16-zero-shot" model through the Transformers library, the system efficiently processes textual descriptions and video frames for classification tasks. This approach accelerates implementation and enhances accuracy by utilizing a robust pre-trained model. Thus, it represents a significant advancement in deploying video classification solutions rapidly and effectively in various applications. The plan is to integrate this prototype into multiple processes to leverage its capabilities for fast video classification. Currently, efforts are underway to implement this solution in online exam monitoring to prevent cheating. Additionally, there are ongoing initiatives to apply this technology in the employment industry to sort received video materials automatically. These expansions aim to enhance integrity in online assessments and streamline the evaluation process of candidate submissions in hiring processes.

The OWL-ViT zero-shot detection solution leverages advanced vision transformers for detecting unauthorized construction on the island of Šolta, Croatia, where this innovative system aims to automate illegal construction and waste disposal detection processes.

The solution utilizes models from the Hugging Face repository, specifically OWL-ViT, tailored for zero-shot learning tasks. Given the terrain and architectural specifics of Šolta, conventional pre-trained YOLO models are impractical. Therefore, OWL-ViT represents the most effective option for accurately detecting and monitoring illegal activities.

Empowering municipal inspectors responsible for enforcing building regulations significantly reduces oversight errors and lightens the workload of regulatory bodies.