Vu Quang Hoa
Verified Expert in Engineering
Software Developer
Hanoi, Vietnam
Toptal member since August 9, 2016
Hoa, nicknamed Joe, is a brilliant engineer capable of grasping new concepts very quickly. His most striking quality is the commitment he shows in whatever he does. He specializes in full-stack, highly scalable Python-Django applications. He has over 10 years of experience developing applications on the Django framework at many companies worldwide, one of them being the StoryTree company—one of the top 500 startups in the US.
Portfolio
Experience
Availability
Preferred Environment
Python 3, Django, Celery, ChatGPT, Redis, Docker, Vue, Node.js, Microsoft AI, Amazon
The most amazing...
...thing I've built was the query page app which turns SQL commands into an analytics page with graphs and a variety of input types (date, time, number, & string).
Work Experience
Senior Web Scraping Developer
Lotto Edge
- Maintained the Lambda functions to collect lottery information.
- Created reports based on the collected data, and output Excel/CSV files according to the required format.
- Speeded up the current scraping system and optimize the scraper code (do not use the infinite scheduler; use a cronjob with a smaller amount of data).
- Designed and created database for better lookup and querying.
Full-stack Developer
Toptal Client
- Scraped product details from these websites: winsupplyinc.com, ferguson.com, supplyhouse.com, and supplyhouse.com.
- Designed an electrical and water equipment database and inserted scraped data into the Django models.
- Integrated Scrapy, Scrapyd, and Django to stream Scrapy logs and automatically call Scrapyd from the Django admin app.
- Researched and implemented a scraping system to automatically collect data on demand to serve dynamic search on the website.
- Experienced OpenAI to automatically scrape data in a structured format so we can optimize the extracting process faster.
Full-stack Developer (via Toptal)
Thomas Robb
- Designed, implemented, and maintained a complicated scraping application with Google News.
- Integrated Scrapy, Scrapyd, and Django to run scrapers and stream logs easily.
- Designed and implemented the data flow using Scrapy and Django PostgreSQL.
- Enhanced the Django admin to adapt with many filters/actions and admin dashboards.
Full-stack Web Developer
Schoolmaster.io
- Brainstormed the ideas, scraped data using ChatGPT, and discussed with the owner to consult him on how to build the app with minimum features.
- Designed database and system architecture for the whole system. Planned the project timeline and milestones.
- Worked closely with the designer and owner to develop a reasonable design and user flow.
- Built a fancy UI/UX, different pages (dashboard, authentication, admission) with complicated logic regarding the application process using Angular.
- Integrated the front end with the back end RESTful APIs.
- Integrated with Stripe for the subscription feature and direct payment process.
- Integrated with SendGrid to send email, Sentry for error reporting, and NewRelict for performance tracking.
- Designed and implemented RESTful APIs using Django, DRF, Celery, and Redis.
- Maintained high-quality code using pre-commit, isort, Flake8, Black, and other libraries.
Full-stack Engineer
Quantlink, Inc.
- Configured auto-deployment on GitHub CI/CD using Amazon EC2, Nginx, Supervisord, and uWSGI (React and Django).
- Implemented Docker Compose and Docker files for many services in the same container.
- Customized the Django admin with multiple filters and added some custom admin pages.
- Analyzed and optimized Django query sets in admin and views using SQLInspect and Django debug toolbar.
- Refactored React views, components, and customized authentication components.
Scraping/CRM Developer
Miguel Pinto
- Created an automation bot to scrape corporation information from the Sunbiz website.
- Scraped LexisNexis to collect customer phone and email.
- Integrated with ChatGPT to extract human names from PDF files.
- Deployed and managed AWS Lambda functions and Amazon EC2 servers.
- Integrated with Salesforce API to update document information and create leads.
- Scraped different counties to collect properties and PDF documentation.
Scraping/CRM Expert (via Toptal)
Jason Abitbol PA
- Designed and implemented a CRM flow system to allow more sales to work with customers and properties.
- Built a scraper tool to collect corporation information from Sunbiz.
- Implemented LexisNexis scraper to collect customers' phone numbers and emails.
- Used Django-import-export to import/extract Excel files that contain different formats and data from another CRM application.
- Utilized django CMS for the content management system with blogs, posts, images, videos, and pages.
- Adjusted the Django admin to provide more filters and custom pages with report charts using Chart.js.
Senior Python Developer
Maxeon - Main
- Designed and built the Django web application to manage email content, classification, sentences, and labels.
- Implemented RESTful APIs for email classification, labeling, and prediction.
- Designed the email processing flow, encompassing email extraction, email embedding value calculation, comparison, and return. Built the entire flow using Django, Celery, and Redis.
- Brainstormed and discussed directly with the manager about the project deadline, tasks, and plan. Contributed actively to the project development.
- Optimized the email classification and extraction speed.
- Utilized OpenAI ChatGPT 4 for classification, embedding calculation, and chat completion.
Web Scraper
FTI Consulting
- Designed and implemented the scraper using Scrapy and PlayWright to scrape music/art events on different websites.
- Parsed and extracted HTML, XML, JSON, and CSV content using pandas and openpyxl.
- Set up an auto-scaling mechanism for auto scraping, daily jobs, and heavy background tasks.
Web Scraping Expert
Benjamin Kolber
- Scraped news websites to collect author, title, created-date, publication, and main content using Scrapy, Newspaper3k, Playwright, Selenium, and Beautiful Soup.
- Designed a scraping system with error notification, accuracy check, auto-retry, and auto-scaling on demand using Celery, Celery beat, Redis, and Django.
- Implemented RESTful APIs for mobile apps to consume. Some APIs are asynchronous using Django async support.
- Designed the database schema to adapt to new changes from the application and newly requested features.
- Integrated with OpenAI- ChatGPT to get text summary.
- Built a Chrome/Firefox extension to collect URL content and extract and send extracted data to the back-end API.
- Integrated with Slack to send notification messages regarding the bot's performance.
Web Scraping Developer
Perfect Cube, LLC
- Scraped thousands of LinkedIn profiles and latest posts using Python, Playwright, and IP proxy.
- Integrated with Coda to get and update data using the Python/Coda package.
- Set up, deployed, and automatically scaled the scrapers based on API requests.
- Set up a Flask application with Celery tasks to perform scraper and RESTFUL APIs to receive requests from Coda.
- Designed the database schema and optimized queries using New Relic service and query-inspector packages.
Web Scraping Expert
Pixelcut Inc.
- Designed a scraper system to scale up and down using Celery and Redis as task queue and task management.
- Deployed the scraper application to a server and configured all processes with supervisorctl.
- Scraped many eCommerce sites (Zara, Zalando, Farfetch, Nordstrom, H&M, Net-a-Porter Madewell, PrettyLittleThing, Unplash) to collect data for AI model processing. The data includes product color, price, and images by Playwright, Scrapy, and Requests.
- Scraped and uploaded images to an S3 bucket and matched uploaded images with image links for each product.
- Optimized the scrapers, e.g., memory leak due to huge load from some Playwright pages, IP blocking, and JavaScript rendering required.
- Refactored and cleaned up code and configured pre-commit.
Senior Django Developer
Endeverus Inc.
- Designed and implemented the post feature where students and SME companies can post messages on a project and user dashboard.
- Designed and implemented the comment feature where students and SME companies can comment on a project and user post.
- Built RESTFUL APIs for users using JWT tokens as authentication methods with a custom payload.
- Worked directly with the CEO and CTO to discuss project features' requirements and deadlines, estimate their cost, and plan the milestones and deadlines.
- Integrated back-end APIs with a front-end UI/UX using JavaScript and jQuery.
- Upgraded the front end from pure JavaScript to Vue and refactored the codebase.
Web Scraping Developer
Research Team in Hanoi National University
- Scraped job portal sites, such as jobs.statushp.com, careers.mercy.net, sorbs.net, ema.us, jobs.mayoclinic.org, and others to collect job information and save it to the PostgreSQL database.
- Built a background tasks system with Celery + Redis to run scraper processes.
- Designed and implemented notification systems with Slack to inform the team about the scraper performance, including the number of scraped jobs in an hour, 404 pages.
- Planned daily/weekly tasks for myself and two other developers so we could move forward easier.
- Optimized the scraper to make it more accurate and faster without harming the site performance or violating scraper ethics.
- Customized Django Admin to make it more UI/UX friendly, easy to use, and extract data.
Back-end Developer
Keyturn Inc
- Designed and developed the subscriptions feature for the application that allows customers to subscribe and make requests.
- Integrated with Stripe subscription and built a webhook to listen to Stripe events, transactions, and activities.
- Architected the database schema for new features and optimized queries on PostgreSQL.
- Analyzed and enhanced the application performance using Sentry and New Relic and used cache to speed up static queries.
- Built RESTful APIs for subscriptions, memberships, and credit management.
Full-stack Django and Python Developer
Test4me LLC
- Designed and implemented a complete web application using Django for the back and front end to manage the education test system.
- Maintained the old web app and upgraded its features and performance with the help of New Relic for performance and Sentry for error notifications.
- Implemented many Celery tasks to perform background tasks to calculate student test results, analyze student performance, and notify teachers.
- Designed and implemented the student/teacher dashboards using AdminLTE.
- Developed and optimized database queries to improve the performance of the application.
- Integrated with OpenAI to recommend better resources for students with bad test results.
Senior Python Back-end Engineer
Rapyuta Robotics Co., Ltd.
- Designed the database for warehouse management, robot moving, and shipment tracking information for a robotics firm using PostGIS.
- Created the microsystem with many microservices and how they communicate with a common authentication service. Different services use different tech stacks to cope with the requirements.
- Implemented hundreds of RESTful APIs with high performance to allow microservices to communicate quickly and easily, with or without authentication.
- Documented APIs using Python/Sphinx and Swagger that were maintained and updated daily.
- Wrote performance tests and optimized the APIs and functionalities. The more complicated the service is, the more stress/performance tests we need.
- Wrote unit and integration tests with greater than or equal to 90% code test coverage using Pytest and Django unit tests.
- Built and implemented a simple authentication service using FastAPI and Asyncio.
- Planned sprints and discussed and estimated effort and time for each task. Cleaned the backlog every week.
- Monitored application performance, suggested improvements, and optimized slow queries and transactions.
Full-stack Django Developer
Learning Seeds
- Acted as a Django developer to debug and enhance educator workflow tools; investigated the slow queries and bottlenecks of the application, then optimized and improved them. As a result, the system is 100 times faster.
- Integrated with 3rd-party services to collect data, including Amazon S3, Google API, and Loom.
- Designed and applied Django best practices and libraries to improve the source code quality and application performance.
- Implemented RESTful APIs for different resources with multiple levels of permission and JWT authentication.
- Integrated with Slack API to push notifications when the web application models/resources change and reports are generated.
- Created unit tests and integration tests with pytest, sped up the test time, and then went back to optimize the source code again.
- Researched and evaluated new service integrations to add more features to the current application.
Django Back-end Developer
ELMLAB Sdn. Bhd.
- Designed and implemented dynamic form generation using the Django framework, helping organization managers to create different forms with input and output and a workflow for other members to fill and report.
- Implemented, documented, and optimized RESTful APIs for front-end consumption.
- Created and implemented a report notification feature via Slack API to report errors and employee performances.
- Conducted integration with the email service SendGrid to send emails with proper templates when there is a workflow completion.
- Wrote unit tests to cover all functions and views, wrote stress tests to analyze the application performance, and refactored the code to optimize.
- Followed the best practices recommended by the Django community, books, and experienced developers.
Django and Python Developer
DSX Pty Ltd
- Maintained a sophisticated back-end system with thousands of background tasks using Django, Celery, and RabbitMQ.
- Designed and implemented fast and reliable RESTful APIs for candle charts in coin trading and documented the APIs using Sphinx.
- Enhanced the Django admin pages with additional user interface elements, animations, and data visualization tools to make the user experience more intuitive and engaging.
- Created comprehensive unit and integration tests to ensure the highest levels of code test coverage, exceeding 90%. Ensured all existing source code was covered by tests and verified that all code behaved as expected.
- Improved the logic for hedge trade daily between multiple coin trading systems.
- Stayed up to date with best practices and library updates.
- Designed and optimized the Candlestick Chart APIs using many mechanisms, including temporary report tables, concurrent queries, and smart data selection.
Python and Django Back-end Developer
EQTBLE
- Built the project from scratch, designed and applied Django best practices and libraries, including DRF, Celery, and Django extensions, unit tests, integration tests, coverage, clean code, fat models, thin views, and utility functions.
- Implemented RESTful APIs for different resources and models with complicated logic and validations.
- Extended Django OAuth to cope with new project requirements.
- Integrated multiple 3rd-party APIs to process recruitment statistics data.
- Created comprehensive unit and integration tests to ensure the highest levels of code test coverage, exceeding 90%. Ensured that all existing source code was covered by tests and verified that all code behaved as expected.
- Designed and implemented reliable and secure data storage solutions for various applications utilizing PostgreSQL on the AWS Relational Database Service (RDS).
- Added pre-commit with a security check, style check, and unit-test run.
Senior Full-stack Developer
DsTax
- Created and executed an efficient application deployment process on DigitalOcean utilizing the Gunicorn web server, the supervisorctl process control system, and the Git version control system.
- Designed and built RESTful APIs for tax clients, whitelisted and blacklisted IPS, and throttle requests.
- Debugged issues with performance, conducted stress tests, analyzed performance issues, and devised a plan and solution for coding and design.
- Integrated third-party APIs and services, including ONESOURCE, QuickBooks, Shopify, Fishbowl, Acumatica, Vertex, Stripe, and Sovos.
- Built an alert system for customer service, including Slack notifications, Sentry notifications for the development team, and performance notifications for the development and business teams.
- Customized the UI/UX pages for clients and company admins using jQuery, Bootstrap, HTML, and CSS.
- Integrated with QuickBooks APIs, ONESOURCE APIs, Vertex APIs, Stripe, and Chargebee APIs.
Technical Leader
STLPro
- Designed and implemented a scraper system that collects product information from suppliers and lists them in the Amazon store; used Scrapy, Selenium, and Playwright technologies.
- Built thousands of background tasks using Celery and Redis to sync the Django application with the Amazon store for orders and products, using Amazon MWS API and Selling Partner API (SP API).
- Customized and enhanced the UI/UX of the staff dashboard to manage the store using jQuery, Bootstrap, HTML, and CSS.
- Designed the database for store management, product history, and Amazon API requests and responses.
- Optimized the APIs and functionalities by writing unit, performance, stress, and integration tests using Django unit tests and pytest.
- Implemented RESTful APIs for different resources with multiple levels of permission using the Django REST framework and Django filter.
- Integrated with Slack API for notification messages.
Python Engineer
HealthLab
- Built RESTful APIs for the local development team to get workers' information.
- Designed the database schema for new features, making it easy to upgrade and maintain.
- Optimized the system performance due to a massive load of queries using paging, analyzing performance metrics, and cleaning the code.
- Refactored the code, cleaned up messy code and libraries, and upgraded the app engine machines.
- Updated the UI/UX for the web portal using HTML, CSS, JavaScript, and jQuery.
Python Expert Developer
Sikoia
- Designed and built a web application using Python and Django, Celery, and RabbitMQ from scratch.
- Created unit and integration tests for the current library functions and classes, mainly in data analysis.
- Analyzed data based on the current algorithm and added more filters and complex logic to clean output.
- Worked closely with the owner, who was also the data analysis researcher, to finalize the expected output and identify patterns.
- Deployed the web application in Azure architecture using DevOps and Visual Studio deployment.
Python and Django Engineer
ARTA
- Designed and implemented RESTful APIs on top of Django, DRF, Celery, and RabbitMQ.
- Designed microservices and their APIs interface and maintained multiple libraries across all services.
- Customized the Django admin features, integrated with Google Geolocation APIs.
- Created thousands of unit and integration tests using Django unit tests to cover at least 90% of the source code.
- Improved and refactored the codebase to make it more efficient using Query Inspect and New Relic.
- Designed the database schema, created indexes, and optimized database settings.
- Refactored the code following the best practices from the Django community, blog, and the book "Django Two Scoops."
Back-end Developer
Rasoul Holdings, LLC
- Implemented RESTful APIs using Django and DRF to be consumed by mobile applications that use JWT for authentication.
- Involved in designing and deploying the application structure in AWS using Lambda for deployment.
- Integrated Sentry, New Relic, Thumbnail API, AWS, and Slack with third-party services.
- Implemented a web page for Tudoonow.com using HTML, JavaScript, and CSS with fancy UI/UX and animation.
- Customized a Django admin dashboard using Django Grappelli and added many filters, actions, and charts.
- Composed unit and integration tests to cover 90% of the codebase.
- Wrote performance stress tests to detect the bottleneck of the application.
- Refactored code daily to improve performance based on best practices recommended by the Django community and the book "Django Two Scoops."
Django Developer
Atomic One CC
- Built an exquisite UI/UX website and RESTful API based on Python, Django, DRF, Celery, and RabbitMQ.
- Deployed a Docker container to host a web application using Nginx as a web server, RabbitMQ for message brokering, and Gunicorn to serve the application.
- Created an administrative dashboard with Material Design styling and the powerful Django admin framework.
- Implemented soft deletion and history of changes for all objects using Django.
- Integrated AWS, Sentry, Rollbar, engageSpark, SendGrid and NewRelic.
- Created and executed hundreds of automated unit and integration tests to ensure the quality and accuracy of software functionality.
- Refactored code daily to improve performance based on the best practices recommended by the Django community and the book "Django Two Scoops."
Freelance Django Developer
Atomic One CC
- Developed a management portal for administrators using the Django admin framework and Material Design for a modern, streamlined look. This portal provides administrators with the tools and resources to manage their organization's data effectively.
- Maintained a SYNC API built entirely in the Python and Django framework while the internet network was down.
- Integrated user-facing elements developed by the front-end developers with server-side logic using jQuery, Ajax, HTML, and CSS.
- Created and maintained unit tests to cover 90% of the codebase, then refactored code following the "Clean Code" book.
- Followed the best practices recommended by the Django community and Django books.
Freelance Full-stack Developer
DealzTap
- Designed and optimized a server to meet high-demand requests on AWS EC2 (Dealztap.com).
- Maintained RESTful APIs built entirely with Python, Django, Django REST Framework, and RabbitMQ/Celery.
- Integrated many third-party services, including ONTRAPORT and Google OAuth 2.0.
- Researched and collaborated with the business team to develop the best solution for technical and commercial issues.
- Designed and implemented the database on AWS RDS to cope with new features.
- Built a push notification management system using Firebase.
- Refactored the code daily to improve performance based on the best practices recommended by the Django community and the book "Django Two Scoops."
- Involved in the analysis and design of the application features.
Full-stack Developer
The Lori System
- Created and maintained a website for transportation quotes in Kenya for trucks only for Lorisystems.com using the Django framework, HTML, CSS, and jQuery.
- Assisted in the successful deployment of an application to both Heroku and the Google App Engine, ensuring a smooth transition from development to production.
- Built RESTful APIs entirely with the Django REST framework powered by Celery and RabbitMQ.
- Created an automated tracking system utilizing the Google Geolocation API to provide real-time location data.
- Researched and built a seamless integration of third-party services, including a secure payment service (PesaPal) and tracking with Google Geolocation API, making it easier to manage payments and track shipments in real time.
- Created a testing automation service inside the project to test, including Style Guide PEP8 (Flake8), API (unit test, Factory Boy), code coverage (coverage.py), and property-based tests (hypothesis).
- Followed the best practices from the Django community, blog, and the book "Django Two Scoops" to refactor the code.
- Created a comprehensive database structure and application infrastructure to ensure efficient data storage and processing.
Back-end Developer
StoryTree
- Developed and maintained a RESTful API built entirely in Python with Django, Django Rest Framework, Celery, and RabbitMQ.
- Deployed and maintained an application on an Amazon EC2 server using version control with Git and automation with Fabric.
- Created and deployed PostgreSQL databases on AWS Relational Database Service (AWS RDS) to store and manage data.
- Optimized the application performance to meet users' demands using Nginx for the EC2 server and Heroku apps for free workers.
- Integrated various third-party APIs such as tracking for USPS, FedEx, UPS, logging (Sentry), server performance (New Relic), push notification (Parse), advertising analytics (Fiksu, Tune, Facebook, AdWords), Payment (Stripe), and email (SendGrid).
- Integrated user-facing elements developed by the front-end developers with server-side logic using jQuery, Ajax, HTML, and CSS.
- Built an alert system for customer service using Slack notifications and integration and for project managers using SendGrid email.
- Constructed an AB testing system to test new features based on metrics like the chi-square test and the two-sample test.
- Maintained side projects related to the SimplePrints app using the Node.js framework, Ember.js, and Express.js.
- Customized the Django admin interface by adding extra filters, actions, UI/UX, jQuery DataTable, and Chart.js.
Experience
Python Developer
Python and Django Developer to Enhance Education System
https://www.learningseeds.com/Recapp
Education Application for Students
https://adaptivex.io/Education
Master's Degree in Computer Science
University of New South Wales - Sydney, Australia
Bachelor's Degree in Computer Science
University of Engineering and Technology, Hanoi National University - Hanoi, Vietnam
Skills
Libraries/APIs
Facebook API, Google API, TensorFlow, REST API, Requests, Node.js, jQuery, Stripe Development, Gmail API, Google Maps Development, Selenium WebDriver, Playwright, Twilio Development, Pandas, NumPy, Flask-RESTful, React.js, Django, Django-taggit, Django-allauth, SQL, QuerySet API, QuickBooks Development, Stripe Development, Amazon API, Beautiful Soup, Slack API, Salesforce API, Oracle Development, Office 365 API, Vue.js, Puppeteer, Chart.js, D3.js, AWS, LinkedIn API
Tools
Celery, RabbitMQ, Git, GitLab CI/CD, Zapier Development, Sublime Text 3, AWS, GIS, Pytest, Sentry, Docker Compose, Virtualenv, Pyenv, Slack Development, Acumatica, ChatGPT, Microsoft AI, GitHub, AWS, Coda, Amazon Textract
Languages
SQL, JavaScript, Python, Python, CSS, HTML, PHP, Java, XML, TypeScript
Frameworks
Django, Django, Scrapy, Selenium, Ember.js, Express.js, Django, AngularJS, Flask, OAuth, Django, JSON Web Tokens (JWT), Bootstrap, Angular, Angular 6, Redux, Laravel
Paradigms
Unit Testing, RESTful Development, Agile Development, Functional Programming, Clean Code, Design Patterns, REST, Automated Testing, Agile Development, DevOps, Azure DevOps, Test Automation, Microservices Development, Web Architecture, Automation
Platforms
AWS, Amazon EC2, Heroku, Linux, Docker, Twilio Development, OS X, Google App Engine, AWS Lambda, Shopify, CMS, Amazon, DigitalOcean, New Relic, Web Development, Sovos, Magento, WordPress Development, MacOS, Azure, Salesforce Design, Cloud Engineering
Storage
MySQL, PostgreSQL, Database, JSON, Redis, Azure, Amazon S3, Google Cloud Development, GIS, Redis Cache, Cloud Deployment, XML, Database
Other
Freelance Design, Data Science, Data Analysis, Full-stack, Software Development, Front-end Development, Web Scraping, CSV, Software Architecture, Integration Testing, Unix Shell Scripting, Web Services, Scraping, Financials, Business Technology, System Security, Chrome Extension Development, Firefox Extensions, Jupiter, Intuit QuickBooks, Coin, Physics, Social Impact, APIs, Server-side PDF Generation, Django, FastAPI, Workers, Background Jobs, Web UI, APM, Recommendation Systems, Symphony, Algorithms, Back-end Developers, Data Scraping, Web Development, Architecture, Integration, QuickBooks Development, CSV File Processing, GPT-4, OpenAI GPT-3 API, AWS RDS, Amazon Marketplace, E-commerce marketing, Data Processing, Data Processing Automation, API Integration, System Architecture, Web Servers, Vertex, OneSource, PDF Scraping, Newspapers, HTML Parsing, Document Parsing, Multitenancy, Containerization, Slackbot, Parsers, Text Classification, Amazon FBA, Sunbiz, LexisNexis, Full-stack, CI/CD Pipelines, OpenAI, ChatGPT API, Vite, Data Extraction, Scrapyd, Web Development, Newsfeeds, News, MERN Stack, Data Science, OCR, Lottery
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring