Database Posts

The Toptal Engineering Blog is a hub for in-depth development tutorials and new technology announcements created by professional software engineers in the Toptal network.
Joaquin Cid
Angular 6 Tutorial: New Features with New Power

Angular 6 is out! The most outstanding changes are in its CLI and how services get injected. In this tutorial, Toptal Freelance Angular Developer Joaquin Cid goes over the basic steps of initial setup and then creates a small diary app using Material Design for the front end and Firebase for the back end.

Continue reading →
Viktor Bojović
Developing a Bioinformatics Database for Disulfide Bonds Research

The Protein Data Bank (PDB) bioinformatics database is the world’s largest repository of experimentally-determined structures of proteins, nucleic acids, and complex assemblies. All data is gathered using experimental methods such as X-ray, spectroscopy, crystallography, NMR, etc. This article explains how to extract, filter, and clean data from the PDB to make it suitable for further analysis.

Continue reading →
Ivan Pavlov
Database Migrations: Turning Caterpillars into Butterflies

For a large codebase, managing database schema can become tedious, especially if you maintain multiple testing environments or customers that update the product at different paces. Sometimes, documenting the latest schema or database changes isn’t enough.

In this article, Toptal Database Engineer Ivan Pavlov introduces us to concepts that help manage database states.

Continue reading →
Mohammad Altarade
The Definitive Guide to NoSQL Databases

Limited SQL scalability has prompted the industry to develop and deploy a number of NoSQL database management systems, with a focus on performance, reliability, and consistency. The trend was driven by proprietary NoSQL databases developed by Google and Amazon. Eventually, open-source systems like MongoDB, Cassandra, and Hypertable brought NoSQL within reach of everyone.

In this post, Senior Software Engineer Mohamad Altarade dives into some of them and explains why NoSQL will probably be with us for years to come.

Continue reading →
Ken Hu
A Data Engineer's Guide To Non-Traditional Data Storages

With the rise of big data and data science, storage and retrieval have become a critical pipeline component for data use and analysis. Recently, new data storage technologies have emerged. But the question is: Which one should you choose? Which one is best suited for data engineering?

In this article, Toptal Data Scientist Ken Hu compares three prominent storage technologies within the context of data engineering.

Continue reading →
Dallas H. Snider
An HDFS Tutorial for Data Analysts Stuck With Relational Databases

The Hadoop Distributed File System (HDFS) is a scalable, open source solution for storing and processing large volumes of data. With its built-in replication and resilience to disk failures, HDFS is an ideal system for storing and processing data for analytics.

In this step-by-step tutorial, Toptal Database Developer Dallas H. Snider details how to migrate existing data from a PostgreSQL database into the more efficient HDFS.

Continue reading →
Jeffrey Shumaker
Guide To Budget-friendly Data Mining

Although database programming does not evolve at nearly the same pace as traditional application programming, recent advancements in several fields are bringing new techniques and technologies within the reach of small and independent developers.

In this guide, Toptal Freelance Software Engineer Jeffrey Shumaker explains how developers can quickly and easily tap these methods to identify database issues they may not even be aware of, and how they can build excellent data mining tools without spending a lot on expensive software licenses.

Continue reading →
Ivan Bojovic
MySQL Master-Slave Replication on the Same Machine

Developers often work on only one machine, and have their whole development environment on that machine. Testing database replication before deploying changes in this kind of a development environment can be a challenging task.

In this article, Toptal engineer Ivan Bojovic guides us through a step-by-step tutorial on how to implement MySQL master-slave replication on one machine.

Continue reading →
Marcin Kmiec
Simplifying RESTful API Use and Data Persistence on iOS with Mantle and Realm

A large number of modern mobile applications interact with web services in one way or another, and iOS applications are no different. Mantle (a model framework) and Realm (a mobile database) come with the promise of simplifying some of the hurdles in consuming web services through RESTful APIs and persisting data locally. In this article, Toptal engineer Marcin Kmiec shows how to build a simple iOS application using Mantle and Realm and demonstrates how this approach helps to eliminate a large amount of boilerplate code.

Continue reading →
Sripal Reddy Vindyala
How to Tune Microsoft SQL Server for Performance

To retain its users, any application or website must run fast. For mission critical environments, a couple of milliseconds delay in getting information might create big problems. As database sizes grow day by day, we need to fetch data as fast as possible, and write the data back into the database as fast as possible. To make sure all operations are executing smoothly, we have to tune Microsoft SQL Server for performance.

Continue reading →
Avinash Kaza
Business Intelligence Platform: Tutorial Using MongoDB Aggregation Pipeline

In today’s data driven world, researches are busy answering interesting questions by churning through huge volumes of data. Some obvious challenges they face are due the sheer size of dataset that they have to deal with. In this article, we take a peek at a simple business intelligence platform implemented on top of the MongoDB Aggregation Pipeline.

Continue reading →
Doug Sparling
Full Text Search of Dialogues with Apache Lucene: A Tutorial

Apache Lucene is a powerful Java library used for implementing full-text search on a corpus of text. With its wide array of configuration options and customizability, it is possible to tune Apache Lucene specifically to the corpus at hand - improving both search quality and query capability.

This article gives us a glimpse of the simplicity and ease of customization of the Apache Lucene analysis pipeline.

Continue reading →
Rodrigo Koch
SQL Database Performance Tuning for Developers

Database tuning can be an incredibly difficult task, particularly when working with large-scale data where even the most minor change can have a dramatic (positive or negative) impact on performance.

In mid-sized and large companies, most database tuning will be handled by a Database Administrator (DBA). But there are plenty of developers who have to perform DBA-like tasks; meanwhile, DBAs often struggle to work well with developers.

In this article, Toptal Freelance Software Engineer Rodrigo Koch provides developers with database tuning tips and explains how developers and DBAs can work together effectively.

Continue reading →