Cover image
6 minute read

The Need for Speed: A Toptal JavaScript Coding Challenge Retrospective

Toptal’s JavaScript Speed Coding Challenge invited creative solutions from the beginning. As the week played out, competitors got closer and closer to the maximum theoretical score. Then something unexpected happened…

Toptal started the JavaScript coding challenge web app as a way to attract people to our conference booths. Seeing how successful it was, we decided to make a pilot on the web, open for everyone in our community and their networks.

A screenshot of Toptal's JavaScript Speed Coding Challenge, showing one of the first questions. A function named "double" is supplied, and its body consists of a comment, "return x doubled."

At launch time, we had encouraged motivated developers to come up with creative ways to score high on the overall JavaScript coding challenge. Some of the key success factors of great independent freelancers are the ability to think outside the box and find creative ways to work within a set of constraints.

JavaScript Coding Challenge Questions

The gauntlet consisted of a number of JavaScript questions—akin to those that might be used as interview questions—ranging from really basic JavaScript challenge questions:

box.double = function double (x) {
    //return x doubled

…to more intermediate ones:

box.dateRank = function dateRank (x) {
    // x is a date in 2019 as string (example: "06/30/2019")
    // return the rank of the day in 2019 (i.e., "09/01/2019" translates to 244)

We wanted both beginners and advanced developers to have fun and invite their friends and colleagues to compete with them for the highest scores. The questions were sorted by points, so that even junior developers could score some points. The order for questions having the same amount of points was random, so the experience was slightly different every time, while the full set of questions stayed the same throughout the week.

The goal was to complete as many tasks as possible within the time limit of three minutes. In case someone found a way to complete all the tasks in the JavaScript coding challenge, 10 points would be awarded for each second that was left. We allowed multiple attempts to complete the challenge.

The first thing that we anticipated would happen is that people would take some time to solve the questions at a leisurely pace, and then copy and paste the answers into the web application.

After one hour and 40 minutes of launching the challenge, the first person followed this approach and got the 1445 maximum points that the application allowed, plus some extra points that we awarded for every second left.

A Turning Point in the JavaScript Coding Challenge

With the copy-and-paste approach, top contestants had nothing more to gain from focusing on coding the answers themselves. Instead, they turned their attention to bringing the speed to their automation skills.

The easiest approach at this point was to write some JavaScript that would solve each task while waiting on a loop until the buttons were ready, and copy-paste it into the browser console:

const solutions = {
  'double': 'return x*2',
  'numberToString': '...',
  'square': '...',
  'floatToInt': '...',
  'isEven': '...',
  'squareroot': '...',
  'removeFirstFive': '...',
  // ...
  'dateRank': '...',
  // ...
const get_submit_button = () => document.querySelector('.task-buttons > .col > .btn');
const solve = () => {
  const ace_editor = ace.edit(document.querySelector('.ace_editor'))
  const submission = ace_editor.getValue()
  for (const key in solutions) {
    if (submission.includes('box.' + key + ' ')) {
      setTimeout(() => {
        setTimeout(() => {
        }, 400)
      }, 900)

This part could also be automated using regular automation tools like Selenium. But a faster way would be to automate use of the API, sending the solutions to the tasks:

const request = require('request');
const runTask = (data, entryId, callback) => {
  const tests = data.nextTask.tests_json;
  const results = Object.fromEntries(
    Object.entries(tests).map(([key, value]) => [key, value.result])
  );`${entryId}/attemptTask`, {
    form: {
      attempt_id: data.attemptId,
      tests_json: JSON.stringify(results),
  }, (error, res, body) => {
    if (error) throw error;
    const next = JSON.parse(body).data
    if (next.isChallengeEntryFinished) {
    runTask(next, entryId, callback)
const runEntry = (callback) => {'', {
    form: {
      challengeSlug: 'toptal-speedcoding',
      email: ...,
      leaderboardName: ...,
      isConfirmedToBeContacted: ...,
      dateStop: ...
  }, (error, res, body) => {
    if (error) throw error;
    const {
    } = JSON.parse(body);
    const entryId =
    runTask(data, entryId, callback)

One thing to note is that, in this version of the speedcoding challenge, the code was tested only on the client side. Because of this, it was possible to simply send out the answers to the test cases instead of the code. This allowed for an optimization and to cut some milliseconds on the client side.

After three days, the competition really began to heat up, with the number of attempts per three-hour bucket going suddenly from under 1,000 to nearly 100,000.

For three days, the scores remained the same. Some people were writing micro optimizations for their code, and several people were submitting their solutions in a loop, hoping the server would become less crowded so they could get a few points ahead. We were due for a big surprise.

Breaking the JavaScript Coding Challenge’s Maximum Score

First, let’s do some quick math: We had a total of 1445 points awarded for the completion of all tasks, and a time allowance of 180 seconds. If we award 10 points per second left in the application, the maximum theoretical achievable score would be 3245—in the case of submitting all answers instantly.

One of our users achieved a score of over 6000, which kept steadily growing over time.

How could someone get such a high score?

After a brief review, we found what had been happening. Our top contestant, a professional full-stack developer and Toptaler with more than 15 years of competitive programming experience, found a loophole in the setup of the coding challenge. He spawned multiple bots, which slowed down the server; meanwhile, he could complete the same task (the one that awarded the most points) as many times as possible and assign their scores to a single entry, continuously adding to that entry’s score.

This wasn’t against the rules, as we allowed for creative solutions; however, the particular method that he was using caused the server to be considerably busier, making network requests slower for everyone else. The first thing that we did was increase our server power, which only made him go from 56,000 to 70,000 points and stay in the first place.

While we didn’t want to intervene in the way people were interacting with the challenge, these attempts slowed down the server to the extent that the challenge was hard to use for other users, so we decided to fix the loophole.

The fix made it impossible for other people to achieve the same score during the last day of the JavaScript coding challenge. Because of this, we decided to extend the number of awards given to the top contestants. Originally, the top prize—a pair of AirPods—was supposed to go to the one top contestant only. In the end, AirPods were given to those holding the first six places.

Humble Beginnings and Ferocious Ends: Some JavaScript Coding Challenge Stats

Even our highest scorers had some difficulty starting out. In fact, of all the people who made five attempts or more, the top score for anyone’s first attempt was 645, and the median score for first attempts in that group was just 25 points.

By the end of the contest, one could guess the strategy being attempted from the total number of attempts. While some were more efficient than others, the top contestant far and away had the highest attempt count.

A combined, logarithmic bar graph showing the top 20 contestants' highest scores and total attempt counts. While the top attempt count went to the top scorer, the second-, third-, and fourth-highest attempt counts went to the 13th-, ninth-, and second-place contestants, respectively. Exactly half of the top 20, including the fourth- and sixth-place contestants, had under 1,000 attempts. Two contestants had under 100 attempts, and another one even had under 10.

Moving Forward

What does the future hold?

This was only the first JS coding challenge iteration. We want to do many more JavaScript programming challenges in the future, learn lessons from the previous runs, and make it even more exciting. The first thing we want to do is to implement attempt throttling to limit the number of submissions. We also want to expand beyond coding challenges in JavaScript, making them available in a wide range of programming languages.

Lastly, while we are taking measures to make the JavaScript coding challenge more secure, we plan to keep allowing the use of bots and other creative approaches for future challenges.

Special thanks to Pavel Vydra, Anton Andriievskyi, Tiago Chilanti, and Matei Copot for their contributions to the challenge and this article, and to @Zirak for the open-source project that formed the basis of the contest app. Likewise, thanks to everyone who participated in—and ran—the contest.

Understanding the basics

To become an expert at solving coding challenges, you should constantly improve by selecting increasingly harder problems. Toptal's JavaScript challenge contains a wide array of task difficulty levels. Don't skip problems that look tedious: The only way to improve is to focus on solving these too.

A JavaScript challenge is designed to test your programming knowledge in a timed environment. To complete one successfully, you should be able to intelligently select data structures and design algorithms quickly.

Exercism, Codewars, Codility, and Toptal's speed coding challenges (when available) are great places to practice JavaScript coding—competitive or otherwise.


Ruben Dario Carvajal Herrera
este sitio es genial
Tiago Chilanti
It is so amazing how people get creative when you challenge them. I´m really proud to have taken part on the development of this challenge.
Let me get it straight. Some guys cheated and hacked your app but you are saying it's ok and they got the top places. This kind of creativity when interacting with some real world apps can get you into legal trouble.
Eva Bojorges
I wouldn't call this cheating. We announced from the beginning that we allowed for creativity in the solutions, so any way of solving it was allowed.
Michael Hays
Personally, I think it's great to say that anything goes. It gives us the opportunity to be truly creative, and beating the system is fun :) I do think it would be fun to have "true" speed-programming challenges too, though. Seems like the only way to prevent people from knowing the questions ahead of time is to have a small window (~1 minute) where everyone has to begin their single attempt. Otherwise, people will always be able to collaborate.
How did you fix the loophole?
The data visualization graph above shows that only a couple of guys shared that belief that "anything goes". I would have said nothing if everyone was playing by the same rules but clearly other participants considered exploiting bugs, DDOSing and botnets to be dubious practices and not the "creative approach".
Michael Hays
You're taking this silly competition a bit too seriously. Relax, it's all in good fun -- the top prize was a pair of AirPods! Besides, other top scorers were using scripts to submit answers; I'm sure they'd have exploited the bug if they'd known how.
comments powered by Disqus