Dmitri Ivanovich Arkhipov

A Maven/Spring and Spring Boot Integration service that executes dynamically determined processing stage paths.

The Spring Integration is a framework for developing message-passing systems. One component not part of the framework is a dynamic router, next hop, and a current function—this project implements that component.

I built this command-line interface (CLI) to access data vendor data. The data vendor gives us a daily file that tells us information about a given stock ticker, how many people were tweeting about that ticker, and how many of those tweets were positive.

I made sure that the data was conveniently accessible for evaluation. The data is cleaned, ingested into SQLite using Pandas, and is made available via a CLI interface.

It pulls from the Kinesis Stream every N seconds and then pushes to a new stream. Amazon Kinesis Data Streams collect and process large streams of data records in real time.

Kineses are used to create data-processing applications, known as Kinesis Data Streams applications. A typical Kinesis Data Streams application reads data from a data stream as data records.

Online advertisers are concerned with the Internet Advertising Bureau (IAB) categorizations. These categorizations represent themes or topics and can apply broadly to websites advertisements or other web data objects.

I developed and deployed a RESTFul microservice using Flask to train and update classifiers learning from web data and classify any document into a normalized vector representing the theme breakdown of the document into IAB categories. This service used Flask as the REST service layer, the multiprocessing module for process control, MALLET software (Mallet.cs.umass.edu/) for classification, beautiful soup for data preprocessing, and Nutch/SOLR for crawling and indexing web data.

Determining the next ad producer to solicit for a particular ad impression is not trivial. A good algorithm will consider many factors; the two most immediately noticeable factors are revenue for the publisher (website hosting the advertisement) and the time to fill the ad slot.

I've developed a suite of algorithms for solving this problem under different assumptions in my work. A version of this work deployed in production use data aggregation over terabytes of log data to build distributions representing how much time a solicitation will take. The deployment used Flask as a RESTful microservice layer, Python, and SciPy for the main algorithms. More advanced research solutions make use of real-time dynamic programming to achieve solutions with bounded suboptimality.

As part of SDN research work, I developed basic software-defined networking (SDN) solution based on Python, Berkley Sockets, and Python remote objects (Pythonhosted.org/Pyro4). The SDN is built over TCP/IP and allows dynamically reconfigurable networking paths. In fact, in the paper on this topic, the reconfiguration is done based on period flow estimates to ensure that devices are given uncongested networking paths.

To our knowledge, we are the first group to build an SDN over the remote object paradigm. With this approach, the client requests a connection rather than an address from the gateway. The link returned is a remote object now shared by both the client and the network used for reads and writes by the client and reconfigured by the network administration as network conditions change.