#Python Posts

The Toptal Engineering Blog is a hub for in-depth development tutorials and new technology announcements created by professional software engineers in the Toptal network.
Guillaume Ferry
Python Machine Learning Prediction with a Flask REST API

Employing Python to make machine learning predictions can be a daunting task, especially if your goal is to create a real-time solution. However, Tensorflow and Scikit-Learn can significantly speed up implementation.

In this article, Toptal Python Developer Guillaume Ferry outlines a simple architecture that should help you progress from a basic proof of concept to a minimal viable product without much hassle.

Continue reading →
Radu Balaban
Intro to Python Image Processing in Computational Photography

Computational photography is about enhancing the photographic process with computation. While we normally tend to think that this applies only to post-processing the end result (similar to photo editing), the possibilities are much richer since computation can be enabled at every step of the photographic process—starting with scene illumination.

In this article, Toptal OpenCV Expert Radu Balaban walks us through two examples of computational photography: low light and high dynamic range.

Continue reading →
Son Nguyen Kim
Python Logging: An In-Depth Tutorial

As applications become more complex, having good logs can be very useful, not only when debugging but also to provide insight in application issue/performance.

The Python standard library comes with a logging module that provides most of the basic logging features and is very handy but contains some quirks that can cause hours of headaches

Continue reading →
Vlad Miller
Exploring Supervised Machine Learning Algorithms

While machine learning sounds highly technical, an introduction to the statistical methods involved quickly brings it within reach. In this article, Toptal Freelance Software Engineer Vladyslav Millier explores basic supervised machine learning algorithms and scikit-learn, using them to predict survival rates for Titanic passengers.

Continue reading →
Oliver Holloway
From Solving Equations to Deep Learning: A TensorFlow Python Tutorial

TensorFlow makes implementing deep learning on a production scale a breeze. However, understanding its core mechanisms and how dataflow graphs work is an essential step in leveraging the tool’s power.

In this article, Toptal Freelance Software Engineer Oliver Holloway demonstrates how TensorFlow works by first solving a general numerical problem and then a deep learning problem.

Continue reading →
Iulian Gulea
A Guide to Performance Testing and Optimization With Python and Django

Donald Knuth said that “premature optimization is the root of all evil.” But there comes a time, usually in mature projects with high loads, when the need to optimize presents itself. In this article, Toptal Freelance Software Engineer Iulian Gulea talks about five common methods to optimize a web project’s code using principles that can be used in Django as well as other frameworks and languages. Using these principles, he demonstrates how to reduce the response time of a query from 77 to 3.7 seconds.

Continue reading →
Rustem Kamun
Orchestrating a Background Job Workflow in Celery for Python

In this article, I will try to give you a good understanding of which scenarios could be covered by Celery. Not only will you see interesting examples, but will also learn how to apply Celery with real world tasks such as background mailing, report generation, logging and error reporting. I will share my own way of testing tasks beyond emulation and explain a few tricks that go beyond the official documentation and took me hours of research to discover myself.

Continue reading →
Hanee' Medhat
Apache Spark Streaming Tutorial: Identifying Trending Twitter Hashtags

Social networks are among the biggest sources of data today, and this means they are an extremely valuable asset for marketers, big data specialists, and even individual users like journalists and other professionals. Harnessing the potential of real-time Twitter data is also useful in many time-sensitive business processes.

In this article, Toptal Freelance Software Engineer Hanee’ Medhat explains how you can build a simple Python application to leverage the power of Apache Spark, and then use it to read and process tweets to identify trending hashtags.

Continue reading →
Peter Goodspeed-Niklaus
How to Integrate OAuth 2 Into Your Django/DRF Back-end Without Going Insane

So you’ve implemented user authentication. Now, you want to allow your users to log in with Twitter, Facebook or Google. No problem. You’re only a few lines of code away from doing so.

But while there are hundreds of OAuth 2 packages that pip knows, only a few actually do what they’re supposed to do.

In this article, Toptal Software Engineer Peter Goodspeed-Niklaus explains how to integrate OAuth 2 into your Django or Django Rest Framework using Python Social Auth.

Continue reading →
Zhuyi Xue
A Comprehensive Introduction To Your Genome With the SciPy Stack

Genome data is one of the most widely analyzed datasets in the realm of Bioinformatics. The SciPy stack offers a suite of popular Python packages designed for numerical computing, data transformation, analysis and visualization, which is ideal for many bioinformatic analysis needs.

In this tutorial, Toptal Software Engineer Zhuyi Xue walks us through some of the capabilities of the SciPy stack. He also answers some interesting questions about the human genome, including: How much of the genome is incomplete? How long is a typical gene?

Continue reading →
Andrea Nalon
The Rise Of Automated Trading: Machines Trading the S&P 500

More than 60 percent of trading activities with different assets rely on automated trading and machine learning instead of human traders. Today, specialized programs based on particular algorithms and learned patterns automatically buy and sell assets in various markets, with a goal to achieve a positive return in the long run.

In this article, Toptal Freelance Data Scientist Andrea Nalon explains how to predict, using machine learning and Python, which trade should be made next on the S&P 500 to get a positive gain.

Continue reading →
Jongwook Kim
How to Create a Simple Python WebSocket Server Using Tornado

The threaded model and global interpreter lock has always been in the way of Python handling thousands of concurrent long-lived connections. Modern web frameworks, such as Tornado, use non-blocking network I/O to make Python feasible for implementing WebSocket servers.

In this article, Toptal engineer Jongwook Kim walks us through the process of implementing a simple WebSocket-based web application in Python using the Tornado web framework.

Continue reading →
Dario Bertini
To Python 3 and Back Again: Is It Worth the Switch?

Since its debut in 2008, Python 3 has come a long way. Gone are the days when it lacked support for almost all useful libraries and tools. Python 3 offers many improvements and amazing new features that make writing robust code in Python easier than ever.

In this article, Toptal engineer Dario Bertini discusses some of the improvements and features that Python 3 has to offer, and explains whether switching to Python 3 is a smart choice right now.

Continue reading →
Adnan Ademovic
An Introduction to Robot Operating System: The Ultimate Robot Application Framework

Robot Operating System, a framework for building robot applications, allows developers to assemble a complex system by connecting existing solutions for small problems.

In this article, Toptal engineer Adnan Ademovic gives us a step-by-step tutorial to building software for an onboard computer that allows us to remotely control and monitor a robot and running it in a simulated world using Robot OS.

Continue reading →
Andrew Crosio
Service Oriented Architecture with AWS Lambda: A Step-by-Step Tutorial

AWS Lambda offers a relatively thin service with a rich set of ancillary configuration options, making it possible to implement easily scalable and maintainable applications leveraging these services.

In this article, Toptal engineer Andrew Crosio gives us a step-by-step tutorial for building an image uploading and resizing service and demonstrates how AWS Lambda can be used as a platform to easily build service oriented architecture applications.

Continue reading →
Andrei Boyanov
Python Design Patterns: For Sleek And Fashionable Code

Python is a powerful, object-based, high-level programming language with dynamic typing and binding. Due to its flexibility and power, developers often employ certain rules, or Python design patterns. What makes them so important and what do does this mean for the average Python developer?

In this post, Toptal Senior Software Engineer Andrei Boyanov explains why Python is great for design patterns, and how they can be used to unlock even more potential, or to streamline development and make code more maintainable.

Continue reading →
Leandro Lima
WSGI: The Server-Application Interface for Python

Nowadays, almost all Python frameworks use WSGI as a means, if not the only means, to communicate with their web servers. This is how Django, Flask and many other popular frameworks do it.

This article intends to provide the reader with a glimpse into how WSGI works, and allow the reader to build a simple WSGI application or server.

Continue reading →
Elder Santos
Data Mining for Predictive Social Network Analysis

Analysts have come to recognize social network data as a virtual treasure trove of information for sensing public opinion trends and groundswells of support. In this article, Toptal Engineer Elder Santos describes the techniques he employed for a proof-of-concept that effectively analyzed Twitter Trend Topics to predict, as a sample test case, regional voting patterns in the 2014 Brazilian presidential election.

Continue reading →
Ivan Voras
Installing Django on IIS: A Step-by-Step Tutorial

Although the most wide-spread and supported way of running Django is on a Linux system (e.g., with uwsgi and nginx), it actually doesn’t take much work to get it to run on IIS. In this article, Toptal Engineer Ivan Voras walks you through a step-by-step tutorial, clearly explaining how to install Django on IIS.

Continue reading →
Nikolay Derkach
A Tutorial for Reverse Engineering Your Software's Private API: Hacking Your Couch

Reverse engineering and hacking are usually related to malicious activities that result in sleepless nights of engineers responsible for system maintenance.

Reverse engineering is a tool that we can utilize to find the flaws and improve our software in many aspects. This article shows us how to use these techniques to learn more about different implementations of web API.

Continue reading →
Martin Chikilian
Buggy Python Code: The 10 Most Common Mistakes That Python Developers Make

Python’s simple, easy-to-learn syntax can mislead Python developers – especially those who are newer to the language – into missing some of its subtleties and underestimating the power of the language.

In this article, Toptal’s Martin Chikilian presents a “top 10” list of somewhat subtle, harder-to-catch mistakes that can trip up even the most advanced Python developer.

Continue reading →
Denis Kyorov
Django, Flask, and Redis Tutorial: Web Application Session Management Between Python Frameworks

I love and use Django in lots of my personal and client projects, mostly for those involving relational databases and more classical web applications. However, by design, Django is very tightly coupled with its ORM, Template Engine System, and Settings object. Plus, it’s not a new project: it carries a lot of baggage from the past to remain backwards compatible.

In a few of my client projects, we’ve chosen to give up on Django and use a micro framework like Flask, typically when the client wants to do some interesting stuff with the framework. At the same time, we often need user registration, login, and more, all of which is easily handled with Django.

The question emerged: is Django an all-or-nothing deal? Should we drop it completely from the project, or is there a way to combine some it with the flexibility of other frameworks?

Continue reading →
Charles Marsh
Python Class Attributes: An Overly Thorough Guide

In a recent phone screen, I decided to use a class attribute in my implementation of a certain Python API. My interviewer challenged me, questioning whether my code was syntactically valid, when it was executed, etc. In fact, I wasn’t sure of the answers myself. So I did some digging.

Python class attributes. No one really knows when (or how) to use ‘em. In this guide, I walk through common pitfalls and conclude with a list of valid use-cases that could save you time, energy, and lines of code.

Continue reading →
Charles Marsh
Computational Geometry in Python: From Theory to Application

When people think computational geometry, in my experience, they typically think one of two things:

  1. Wow, that sounds complicated.
  2. Oh yeah, convex hull.

In this post, I’d like to shed some light on computational geometry, starting with a brief overview of the subject before moving into some practical advice based on my own experiences in computational geometric programming with Python.

Continue reading →
Naftuli Kay
An Introduction to Mocking in Python

More often than not, the software we write directly interacts with what we would label as “dirty” services. In layman’s terms: services that are crucial to our application, but whose interactions have intended but undesired side-effects—that is, undesired in the context of an autonomous test run.

For example: perhaps we’re writing a social app and want to test out our new ‘Post to Facebook feature’, but don’t want to actually post to Facebook every time we run our test suite.

The Python unittest library includes a subpackage named unittest.mock—or if you declare it as a dependency, simply mock—which provides extremely powerful and useful means by which to mock and stub out these undesired side-effects.

Continue reading →
Charles Marsh
Why Are There So Many Pythons? A Python Implementation Comparison

Python is amazing.

Surprisingly, that’s a fairly ambiguous statement. What do I mean by ‘Python’? Do I mean Python the abstract interface? Do I mean CPython, the common Python implementation? Or do I mean something else entirely? Maybe I’m obliquely referring to Jython, or IronPython, or PyPy. Or maybe I’ve really gone off the deep end and I’m talking about RPython or RubyPython (which are very, very different things).

While the technologies mentioned above are commonly-named and commonly-referenced, some of them serve completely different purposes (or at least operate in completely different ways). In this post, I’ll start from scratch and move through the various Python implementations, concluding with a thorough introduction to PyPy, which I believe is the future of the language.

Continue reading →
Gergely Kalman
How I Made Porn 20x More Efficient with Python Video Streaming

Porn is a big industry. There aren’t many sites on the Internet that can rival the traffic of its biggest players.

And juggling this immense traffic is tough. To make things even harder, much of the content served from porn sites is made up of low latency live streams rather than simple static video content. But for all of the challenges involved, rarely have I read about the developers who take them on. So I decided to write about my own experience on the job.

Continue reading →