Staff Software Engineer
2017 - PRESENTStripe- Made the ML feature pipeline faster and more reliable which saved $3 million annually.
- Optimized a data pipeline's performance by 20x so a product can be launched on time.
- Rebuilt an online feature store based on a Redis cluster and Lua script that cuts latency from 100 milliseconds to 5 milliseconds (ms).
- Led a team to design/implement a multivariate experimentation service that can handle 3,000 QPS (queries per second) per node and a client-side p50 of 5 ms.
Technologies: Redis, Apache Kafka, Hadoop, Scalding, Spark, Python, Java, ScalaSenior Software Engineer
2014 - 2017LinkedIn- Implemented a serialization system which reduced the p50 latency by 30% and CPU usage by 15%, and achieved $2.5M annual saving.
- Mentored an intern to design and implement a prototype for facet search, and boosted exploratory search CTR by 15%.
- Migrated 80% of Linkedin's search traffic from a legacy search system to the new APIs.
Technologies: Python, Apache Pig, Apache Avro, Apache ZooKeeper, Apache Lucene, Apache Kafka, JavaMember of Technical Staff
2013 - 2014Oracle- Developed an OpenStack Neutron plugin for Oracle Virtual Network Controller.
- Built a UI to visualize and automate virtual network setup.
- Developed an API back end to interact with OpenStack and virtual machines.
Technologies: OpenStack, Apache ZooKeeper, Java, C++
Tianshi Zhu
Software Developer in San Francisco, CA, United States
Member since April 28, 2020
Tianshi loves optimizing distributed systems, big data pipelines, and real-time streaming applications. It thrills him when an out-of-memory issue is fixed or a Spark job becomes 50% faster, or a streaming application's throughput is increased by 10x. Tianshi looks forward to building the next reliable, scalable, and highly available distributed system.
Tianshi is now available for hire
Experience
- Python 10 years
- Java 10 years
- Big Data Architecture 6 years
- Distributed Systems 6 years
- Apache Kafka 6 years
- Spark 3 years
- Scala 3 years
- Scalding 3 years
Availability
Part-time
Preferred Environment
G Suite, Zoom, Slack, Windows
The most amazing...
...optimization I've done is a 100-line change that makes Redis transactions faster by 20x.
Employment
Experience
- An ML Feature System That Supports Batch and Real-time Processing
An ML feature system that allows users to define the business logic in one place and compiled it to multiple back ends, including Spark, Scalding, and real-time streaming processing. The machine learning (ML) features are guaranteed to not have time-traveling issues.
Skills
Frameworks
Spark, Hadoop, DjangoLibraries/APIs
Scalding, Apache LucenePlatforms
Apache Kafka, Windows, Apache Pig, OpenStack, Amazon Web Services (AWS), KubernetesOther
Big Data Architecture, Distributed Systems, AWSLanguages
Scala, Java, Python, C++Paradigms
MapReduce, Functional ProgrammingStorage
Redis, PostgreSQLTools
Slack, Zoom, G Suite, Apache ZooKeeper, Apache Avro
Education
- Master's Degree in Telecommunications2011 - 2013University of Pennsylvania - Philadephia, PA, USA
- Bachelor's Degree in Telecommunications2007 - 2011Fudan University - Shanghai, China
