Python Posts

The Toptal Engineering Blog is a hub for in-depth development tutorials and new technology announcements created by professional freelance software engineers in the Toptal network.
Subscribe for daily updates today and check out the wealth of technical resources published by the Toptal network.
Dino Causevic
Getting Started with TensorFlow: A Machine Learning Tutorial

TensorFlow is more than just a machine intelligence framework. It is packed with features and tools that make developing and debugging machine learning systems easier than ever.

In this article, Toptal Freelance Software Engineer Dino Causevic gives us an overview of TensorFlow and some auxiliary libraries to debug, visualize, and tweak the models created with it.

Continue reading →
Oliver Holloway
From Solving Equations to Deep Learning: A TensorFlow Python Tutorial

TensorFlow makes implementing deep learning on a production scale a breeze. However, understanding its core mechanisms and how dataflow graphs work is an essential step in leveraging the tool’s power.

In this article, Toptal Freelance Software Engineer Oliver Holloway demonstrates how TensorFlow works by first solving a general numerical problem and then a deep learning problem.

Continue reading →
Cody Nash
Create Data from Random Noise with Generative Adversarial Networks

Generative adversarial networks, among the most important machine learning breakthroughs of recent times, allow you to generate useful data from random noise. Instead of training one neural network with millions of data points, you let two neural networks contest with each other to figure things out.

In this article, Toptal Freelance Software Engineer Cody Nash gives us an overview of how GANs work and how this class of machine learning algorithms can be used to generate data in data-limited situations.

Continue reading →
Iulian Gulea
A Guide to Performance Testing and Optimization With Python and Django

Donald Knuth said that “premature optimization is the root of all evil.” But there comes a time, usually in mature projects with high loads, when the need to optimize presents itself. In this article, Toptal Freelance Software Engineer Iulian Gulea talks about five common methods to optimize a web project’s code using principles that can be used in Django as well as other frameworks and languages. Using these principles, he demonstrates how to reduce the response time of a query from 77 to 3.7 seconds.

Continue reading →
Rustem Kamun
Orchestrating a Background Job Workflow in Celery for Python

In this article, I will try to give you a good understanding of which scenarios could be covered by Celery. Not only will you see interesting examples, but will also learn how to apply Celery with real world tasks such as background mailing, report generation, logging and error reporting. I will share my own way of testing tasks beyond emulation and explain a few tricks that go beyond the official documentation and took me hours of research to discover myself.

Continue reading →
Hanee' Medhat
Apache Spark Streaming Tutorial: Identifying Trending Twitter Hashtags

Social networks are among the biggest sources of data today, and this means they are an extremely valuable asset for marketers, big data specialists, and even individual users like journalists and other professionals. Harnessing the potential of real-time Twitter data is also useful in many time-sensitive business processes.

In this article, Toptal Freelance Software Engineer Hanee’ Medhat explains how you can build a simple Python application to leverage the power of Apache Spark, and then use it to read and process tweets to identify trending hashtags.

Continue reading →
Peter Goodspeed-Niklaus
How to Integrate OAuth 2 Into Your Django/DRF Back-end Without Going Insane

So you’ve implemented user authentication. Now, you want to allow your users to log in with Twitter, Facebook or Google. No problem. You’re only a few lines of code away from doing so.

But while there are hundreds of OAuth 2 packages that pip knows, only a few actually do what they’re supposed to do.

In this article, Toptal Software Engineer Peter Goodspeed-Niklaus explains how to integrate OAuth 2 into your Django or Django Rest Framework using Python Social Auth.

Continue reading →
Zhuyi Xue
A Comprehensive Introduction To Your Genome With the SciPy Stack

Genome data is one of the most widely analyzed datasets in the realm of Bioinformatics. The SciPy stack offers a suite of popular Python packages designed for numerical computing, data transformation, analysis and visualization, which is ideal for many bioinformatic analysis needs.

In this tutorial, Toptal Software Engineer Zhuyi Xue walks us through some of the capabilities of the SciPy stack. He also answers some interesting questions about the human genome, including: How much of the genome is incomplete? How long is a typical gene?

Continue reading →
Andrea Nalon
The Rise Of Automated Trading: Machines Trading the S&P 500

More than 60 percent of trading activities with different assets rely on automated trading and machine learning instead of human traders. Today, specialized programs based on particular algorithms and learned patterns automatically buy and sell assets in various markets, with a goal to achieve a positive return in the long run.

In this article, Toptal Freelance Data Scientist Andrea Nalon explains how to predict, using machine learning and Python, which trade should be made next on the S&P 500 to get a positive gain.

Continue reading →
Lovro Iliassich
Clustering Algorithms: From Start To State Of The Art

Clustering algorithms are very important to unsupervised learning and are key elements of machine learning in general. These algorithms give meaning to data that are not labelled and help find structure in chaos. But not all clustering algorithms are created equal; each has its own pros and cons.

In this article, Toptal Freelance Software Engineer Lovro Iliassich explores a heap of clustering algorithms, from the well known K-Means algorithm to the elegant, state-of-the-art Affinity Propagation technique.

Continue reading →
Jongwook Kim
How to Create a Simple Python WebSocket Server Using Tornado

The threaded model and global interpreter lock has always been in the way of Python handling thousands of concurrent long-lived connections. Modern web frameworks, such as Tornado, use non-blocking network I/O to make Python feasible for implementing WebSocket servers.

In this article, Toptal engineer Jongwook Kim walks us through the process of implementing a simple WebSocket-based web application in Python using the Tornado web framework.

Continue reading →
Dario Bertini
To Python 3 and Back Again: Is It Worth the Switch?

Since its debut in 2008, Python 3 has come a long way. Gone are the days when it lacked support for almost all useful libraries and tools. Python 3 offers many improvements and amazing new features that make writing robust code in Python easier than ever.

In this article, Toptal engineer Dario Bertini discusses some of the improvements and features that Python 3 has to offer, and explains whether switching to Python 3 is a smart choice right now.

Continue reading →
Adnan Ademovic
An Introduction to Robot Operating System: The Ultimate Robot Application Framework

Robot Operating System, a framework for building robot applications, allows developers to assemble a complex system by connecting existing solutions for small problems.

In this article, Toptal engineer Adnan Ademovic gives us a step-by-step tutorial to building software for an onboard computer that allows us to remotely control and monitor a robot and running it in a simulated world using Robot OS.

Continue reading →
Andrew Crosio
Service Oriented Architecture with AWS Lambda: A Step-by-Step Tutorial

AWS Lambda offers a relatively thin service with a rich set of ancillary configuration options, making it possible to implement easily scalable and maintainable applications leveraging these services.

In this article, Toptal engineer Andrew Crosio gives us a step-by-step tutorial for building an image uploading and resizing service and demonstrates how AWS Lambda can be used as a platform to easily build service oriented architecture applications.

Continue reading →
Andrei Boyanov
Python Design Patterns: For Sleek And Fashionable Code

Python is a powerful, object-based, high-level programming language with dynamic typing and binding. Due to its flexibility and power, developers often employ certain rules, or Python design patterns. What makes them so important and what do does this mean for the average Python developer?

In this post, Toptal Senior Software Engineer Andrei Boyanov explains why Python is great for design patterns, and how they can be used to unlock even more potential, or to streamline development and make code more maintainable.

Continue reading →
Necati Demir
Ensemble Methods: Elegant Techniques to Produce Improved Machine Learning Results

Machine Learning, in computing, is where art meets science. Perfecting a machine learning tool is a lot about understanding data and choosing the right algorithm. But why choose one algorithm when you can choose many and make them all work to achieve one thing: improved results.

In this article, Toptal Engineer Necati Demir walks us through some elegant techniques of ensemble methods where a combination of data splits and multiple algorithms is used to produce machine learning results with higher accuracy.

Continue reading →
Leandro Lima
WSGI: The Server-Application Interface for Python

Nowadays, almost all Python frameworks use WSGI as a means, if not the only means, to communicate with their web servers. This is how Django, Flask and many other popular frameworks do it.

This article intends to provide the reader with a glimpse into how WSGI works, and allow the reader to build a simple WSGI application or server.

Continue reading →
Leandro Lima
Building a Rest API with the Bottle Framework

REST APIs have become a common way to establish an interface between web back-ends and front-ends, and between different web services. The simplicity of this kind of interface, and the ubiquitous support of the HTTP and HTTPS protocols across different networks and frameworks, makes it an easy choice when considering interoperability issues.

Bottle is a minimalist Python web framework. It is lightweight, fast, and easy to use, and is well-suited to building RESTful services. In this article, I’ll provide a walkthrough of how to build a RESTful API service using Bottle.

Continue reading →
Nick McCrea
Learn to Code: Wisdom and Tools for the Journey

It’s no surprise that more and more people, from all kinds of backgrounds, are deciding to learn to code. But, each person who tackles the task is soon faced with an unpleasant reality: Learning to program is hard. Contrary to expectations, the feeling of “I don’t get it,” may persist unabated long into the journey, making once bright-eyed beginners feel hopeless, lost, and ready to give up.

The moral of the story is this: Be prepared. The path to programmer paradise is a long one, and without the right mindset at the beginning, it can quickly lose its appeal. In this article, I’ll attempt to give you some guidance on what to expect on your journey, how best to go about it, and what tools and resources you may find helpful along the way.

Continue reading →
Elder Santos
Data Mining for Predictive Social Network Analysis

Analysts have come to recognize social network data as a virtual treasure trove of information for sensing public opinion trends and groundswells of support. In this article, Toptal Engineer Elder Santos describes the techniques he employed for a proof-of-concept that effectively analyzed Twitter Trend Topics to predict, as a sample test case, regional voting patterns in the 2014 Brazilian presidential election.

Continue reading →
Ivan Voras
Installing Django on IIS: A Step-by-Step Tutorial

Although the most wide-spread and supported way of running Django is on a Linux system (e.g., with uwsgi and nginx), it actually doesn’t take much work to get it to run on IIS. In this article, Toptal Engineer Ivan Voras walks you through a step-by-step tutorial, clearly explaining how to install Django on IIS.

Continue reading →
Nikolay Derkach
A Tutorial for Reverse Engineering Your Software's Private API: Hacking Your Couch

Reverse engineering and hacking are usually related to malicious activities that result in sleepless nights of engineers responsible for system maintenance.

Reverse engineering is a tool that we can utilize to find the flaws and improve our software in many aspects. This article shows us how to use these techniques to learn more about different implementations of web API.

Continue reading →
Martin Chikilian
Buggy Python Code: The 10 Most Common Mistakes That Python Developers Make

Python’s simple, easy-to-learn syntax can mislead Python developers – especially those who are newer to the language – into missing some of its subtleties and underestimating the power of the language.

In this article, Toptal’s Martin Chikilian presents a “top 10” list of somewhat subtle, harder-to-catch mistakes that can trip up even the most advanced Python developer.

Continue reading →
Denis Kyorov
Django, Flask, and Redis Tutorial: Web Application Session Management Between Python Frameworks

I love and use Django in lots of my personal and client projects, mostly for those involving relational databases and more classical web applications. However, by design, Django is very tightly coupled with its ORM, Template Engine System, and Settings object. Plus, it’s not a new project: it carries a lot of baggage from the past to remain backwards compatible.

In a few of my client projects, we’ve chosen to give up on Django and use a micro framework like Flask, typically when the client wants to do some interesting stuff with the framework. At the same time, we often need user registration, login, and more, all of which is easily handled with Django.

The question emerged: is Django an all-or-nothing deal? Should we drop it completely from the project, or is there a way to combine some it with the flexibility of other frameworks?

Continue reading →
Charles Marsh
Python Class Attributes: An Overly Thorough Guide

In a recent phone screen, I decided to use a class attribute in my implementation of a certain Python API. My interviewer challenged me, questioning whether my code was syntactically valid, when it was executed, etc. In fact, I wasn’t sure of the answers myself. So I did some digging.

Python class attributes. No one really knows when (or how) to use ‘em. In this guide, I walk through common pitfalls and conclude with a list of valid use-cases that could save you time, energy, and lines of code.

Continue reading →
Charles Marsh
Computational Geometry in Python: From Theory to Application

When people think computational geometry, in my experience, they typically think one of two things:

  1. Wow, that sounds complicated.
  2. Oh yeah, convex hull.

In this post, I’d like to shed some light on computational geometry, starting with a brief overview of the subject before moving into some practical advice based on my own experiences in computational geometric programming with Python.

Continue reading →
Naftuli Kay
An Introduction to Mocking in Python

More often than not, the software we write directly interacts with what we would label as “dirty” services. In layman’s terms: services that are crucial to our application, but whose interactions have intended but undesired side-effects—that is, undesired in the context of an autonomous test run.

For example: perhaps we’re writing a social app and want to test out our new ‘Post to Facebook feature’, but don’t want to actually post to Facebook every time we run our test suite.

The Python unittest library includes a subpackage named unittest.mock—or if you declare it as a dependency, simply mock—which provides extremely powerful and useful means by which to mock and stub out these undesired side-effects.

Continue reading →
Charles Marsh
Why Are There So Many Pythons? A Python Implementation Comparison

Python is amazing.

Surprisingly, that’s a fairly ambiguous statement. What do I mean by ‘Python’? Do I mean Python the abstract interface? Do I mean CPython, the common Python implementation? Or do I mean something else entirely? Maybe I’m obliquely referring to Jython, or IronPython, or PyPy. Or maybe I’ve really gone off the deep end and I’m talking about RPython or RubyPython (which are very, very different things).

While the technologies mentioned above are commonly-named and commonly-referenced, some of them serve completely different purposes (or at least operate in completely different ways). In this post, I’ll start from scratch and move through the various Python implementations, concluding with a thorough introduction to PyPy, which I believe is the future of the language.

Continue reading →
Alejandro Rigatuso
Growing Growth: Perform Your Own Cohort Analysis with This Open Source Code

But this isn’t just another article about cohort analysis. If you already know the importance of the topic and want to skip the introduction, you can jump to the simulator, where you can either simulate startup growth based on retention, churn, and a number of other factors, or analyze your own PayPal logs with the code I’ve open sourced.

If, however, you don’t realize that these are some of the most important metrics around–continue reading.

Continue reading →
Michael Truog
CloudI: Bringing Erlang's Fault-Tolerance to Polyglot Development

Clouds must be efficient to provide useful fault-tolerance and scalability, but they also must be easy to use.

CloudI (pronounced “cloud-e” /klaʊdi/) is an open source cloud computing platform that is most closely related to the Platform as a Service (PaaS) clouds. CloudI differs in a few key ways, most importantly: software developers are not forced to use specific frameworks, slow hardware virtualization, or a particular operating system. By allowing cloud deployment to occur without virtualization, CloudI leaves development process and runtime performance unimpeded, while quality of service can be controlled with clear accountability.

Continue reading →
Gergely Kalman
How I Made Porn 20x More Efficient with Python Video Streaming

Porn is a big industry. There aren’t many sites on the Internet that can rival the traffic of its biggest players.

And juggling this immense traffic is tough. To make things even harder, much of the content served from porn sites is made up of low latency live streams rather than simple static video content. But for all of the challenges involved, rarely have I read about the developers who take them on. So I decided to write about my own experience on the job.

Continue reading →