Publication
A Deep Dive Into Reinforcement Learning
https://www.toptal.com/developers/machine-learning/deep-dive-into-reinforcement-learning
Adam is currently unavailable
Adam Stelmaszczyk
Verified Expert in Engineering
Software Development Developer
Warsaw, Poland
Toptal member since September 14, 2017
Bio
As a curiosity-driven constant learner, Adam is interested in mathematics, computer science, and artificial intelligence. Having come across programming at a very young age, he grew up solving problems and especially enjoys it when the solutions involve creativity. A good communicator and people person, Adam has a good eye for detail. He has worked in teams of all sizes and actively participates in open source and AI competitions.
Portfolio
Datagran
Google Ads, Facebook Ads, Google Cloud, BigQuery, MongoDB, Git, Python...
GridCell (via Toptal)
Python, Google Cloud, TensorFlow, Deep Reinforcement Learning
Smarkets, Hanson Applied Sciences
Amazon Web Services (AWS), BigQuery, PostgreSQL, GitLab, Jenkins, Docker, Flask...
Experience
- Software Development - 7 years
- Algorithms - 6 years
- Artificial Intelligence (AI) - 5 years
- Python - 5 years
- Optimization - 5 years
- Machine Learning - 5 years
- C++ - 3 years
- Neural Networks - 3 years
Preferred Environment
PyCharm, Vim Text Editor, Git, Linux, GNU
The most amazing...
...thing I've done is teach a human model how to walk using deep reinforcement learning and 80 CPUs.
Work Experience
Machine Learning Engineer
2018 - 2018
Datagran
- Optimized Facebook and Google ads.
- Devised and implemented five ads optimization algorithms, with one showing a significant improvement.
- Introduced and automatized split testing to compare the algorithms fairly.
- Contributed to the jobs architecture, code repository structure, reviewed code, and helped others.
- Designed, conducted, and analyzed experiments, prepared technical reports and a strategy for next steps.
- Devised a way to find similar campaigns and to have even more data to leverage.
- Conducted 1-hour AI learning session for 15 people.
Technologies: Google Ads, Facebook Ads, Google Cloud, BigQuery, MongoDB, Git, Python, Monte Carlo, Testing, Mathematics, Back-end
Machine Learning Specialist
2017 - 2018
GridCell (via Toptal)
- Implemented custom reinforcement learning environments in Python compatible with OpenAI gym.
- Tested various deep reinforcement learning algorithms such as DQN and PPO.
- Deployed, trained, and hyperparameter tuned models on Google Cloud Machine Learning Engine.
- Provided technical advice.
- Documented every step and environment detail needed to reproduce the experiments.
- Learned about Differentiable Neural Computers and implemented new training task in the DeepMind's code.
Technologies: Python, Google Cloud, TensorFlow, Deep Reinforcement Learning
Software Engineer
2015 - 2017
Smarkets, Hanson Applied Sciences
- Collaborated on research, design, and implementation of state-of-the-art market pricing algorithms.
- Partnered in research, design, and implementation of anti-fraud micro-service for auto flagging suspicious activity, using Random Forests.
- Increased the trading performance N times by parallelizing computations onto N shards.
- Discovered, wrote tests, and fixed multiple severe bugs. Fixed more than a 100 smaller ones.
- Proposed an idea and wrote a hybrid mobile app that replaced the old native one completely, thanks to that we had only one codebase and team instead of three (web, Android, iOS).
- Conducted 57 technical in-person interviews, introduced online coding tests which more than 1000 candidates took, devised a method for auto grading, introduced employee referral scheme that brought more than 5 excellent new employees to the company.
- Worked in a self-management setting in a period of a rapid development (from 28 to over 90 employees), contributed to Smarkets growth and culture.
Technologies: Amazon Web Services (AWS), BigQuery, PostgreSQL, GitLab, Jenkins, Docker, Flask, Bash, Linux, GNU, Haskell, Scikit-learn, Pandas, NumPy, IPython, SciPy, Python, Machine Learning
Cohort Member
2015 - 2015
Entrepreneur First
- Accepted to the EF4 cohort consisting of 36 top tech people throughout Europe. There were 1200 applicants.
- Collaborated with other members to conceptualize ideas.
- Created SportsTracking proof of concept, an automatic extraction of statistics from sport videos using image recognition (Python, OpenCV).
- Built prototype of BitVisual, a visualization of Bitcoin transactions (Neo4j graph database).
- Developed first version of Fluently.io, a marketplace for translators (Scala, Play Framework).
Technologies: Scala, Bitcoin, Neo4j, OpenCV, Python
Software Engineer
2014 - 2015
Hewlett-Packard
- Implemented core functionalities of Store Intelligence Android application in a team of 4.
- Developed features for identity system for Flemish Government using Wicket (Java) in a team of 5.
- Fixed several reported issues in the back-end of large Java EE system for Polish Ministry of Health.
- Built a mobile hybrid application written in AngularJS and packed with Cordova for Android. Introduced Jenkins for automatic builds, which saved us about 5% of time everyday and resulted in less context-switching.
- Conducted video-recorded, 1-hour Brown Bag Seminar titled "Android: basics, tips & tricks" to an audience of 30 people.
- Co-organized HP Developer Day, brand-new, one-day programming competition for Polish developers. 2014 event gathered 40 selected participants and met with a very positive feedback.
Technologies: Jenkins, Android, Java
Android Freelance Developer
2012 - 2012
BlackDigits
- Gathered requirements, met and communicated with a client.
- Implemented Android financial application which provided live share and corporate bond prices, plus a list of company announcements, dedicated to the Maltese listed market.
- Advised and deployed finished product to the Google Play market.
Technologies: Java, Android
Software Engineer Intern
2011 - 2011
Samsung
- Implemented part of S Pen Library and developed demo app.
- Wrote a tutorial + an exemplary game showing how to use the camera in Samsung Linux Platform (now Tizen).
Technologies: Java, Android, C++, C
Junior Programmer
2009 - 2009
DirektPoint
- Joined a startup (2 people at that time) to learn while developing websites such as puds.pl and sportowa.warszawa.pl.
Technologies: Drupal, PHP, JavaScript, CSS, HTML
Experience
Game Tree Search Algorithms
https://github.com/AdamStelmaszczyk/gtsaC++ library for AI bot programming, useful in competitions.
Shield for Clash of Clans
Winner of Hack & Tell #2 London: https://twitter.com/hackandtellLDN/status/596430235895205888
Utility/hack for Clash of Clans, one of the most popular mobile games ever. It received 1000 installs after just 12 days and another 1000 installs after only 6 days through word of mouth between players. So far it has been installed on 200,000 devices in the world and counting.
Utility/hack for Clash of Clans, one of the most popular mobile games ever. It received 1000 installs after just 12 days and another 1000 installs after only 6 days through word of mouth between players. So far it has been installed on 200,000 devices in the world and counting.
Swipe Master 2: Space
Android game developed over the course of a year as a personal project. Swipe Master has been installed on more than 30,000 devices in the world. It wasn't our primary goal, fun and learning were, but we even earned some money on ads.
++C Programming Language
Dynamically typed C. I also made string manipulation more convenient by making the string a built-in type. Also removed the need for declarations being before definitions, which is annoying in original C.
For proof of concept, I wrote ++C to C++ translator (in C++, using Flex and Bison).
For proof of concept, I wrote ++C to C++ translator (in C++, using Flex and Bison).
10th (out of 4014) in Halite 3
During 3 months of competition, I created an AI bot in C++ that reached top 10.
4th (out of 880) in Battle of Bots #3
During 1 week of competition, I created an AI bot in C++ that reached 4th place.
7th (out of 2700) in IndiaHacks: Bot Challenge
During 1 week of competition, I created an AI bot in C++ that reached 7th place.
Distinction on Engineer's Thesis from Warsaw University of Technology
Thesis title: "Tool for detecting similar program code fragments."
Top 3% Overall on Stack Overflow
Sorted by reputation points I'm in top 9000 users.
Our NIPS 2017: Learning to Run Approach
For three months, from July to November 13, a friend and I participated in the NIPS 2017: Learning to Run competition. In this publication, we describe our experience.
Skills: Deep reinforcement learning, TensorFlow, physics-based simulation
Skills: Deep reinforcement learning, TensorFlow, physics-based simulation
Education
2017 - 2018
Progress Toward a Ph.D. in Artificial Intelligence, Deep Reinforcement Learning
University of Warsaw - Warsaw, Poland
2013 - 2014
Master's Degree in Artificial Intelligence
Warsaw University of Technology - Warsaw, Poland
2009 - 2013
Engineer's Degree in Computer Science
Warsaw University of Technology - Warsaw, Poland
2011 - 2012
Engineer's Degree in Computer Science
University of Malta - Malta
Certifications
DECEMBER 2014 - PRESENT
Oracle Certified Professional, Java SE 6 Programmer
Oracle
Skills
Libraries/APIs
Keras, TensorFlow, NumPy, Pandas, Standard Template Library (STL), Scikit-learn, Matplotlib, SciPy, OpenCV
Tools
GitLab, Shell, Git, OpenAI Gym, Jenkins, PDB, GNU Debugger (GDB), Vim Text Editor, PyCharm, IPython, BigQuery
Languages
C++, Python, Java, SQL, R, C, HTML, CSS, JavaScript, PHP, Octave, Assembler x86, Bash, Scala, Haskell
Frameworks
JUnit, Swagger, Bootstrap, Django, Boost, Flask, Play Framework, Caffe
Paradigms
Object-oriented Programming (OOP), Continuous Integration (CI), Test-driven Development (TDD), Unit Testing, REST, Testing, Functional Programming
Platforms
Linux, Drupal, Google Ads, Google Cloud Platform (GCP), Heroku, Amazon Web Services (AWS), Docker, Android
Storage
MySQL, PostgreSQL, Neo4j, Google Cloud, MongoDB
Other
Game Theory, Optimization, Data Science, Reinforcement Learning, Evolutionary Algorithms, Neural Networks, Software Development, Machine Learning, Deep Learning, Data Structures, Artificial Intelligence (AI), Algorithms, Genetic Algorithms, Research, Software Engineering, Back-end, Bitcoin, Statistics, Mathematics, GNU, Deep Reinforcement Learning, Monte Carlo, Facebook Ads, Compilers, Security
Collaboration That Works
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
1
Share your needs
Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2
Choose your talent
Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3
Start your risk-free talent trial
Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.
Top talent is in high demand.
Start hiring