Promises Made, Promises Actually Kept - via JavaScript

I started learning JavaScript in March 2017, primarily via freeCodeCamp (fCC) via their Front End Developer certification path and lightly supplemented by free courses offered by Codecademy. By the beginning of May, I’d completed most of fCC’s Front End path, had signed up for the “Chingu Cohorts” and had taken on the Chingu’s “Build to Learn” (B2L) project signup, an open-ended project wherein you can choose what you want to work on. For my project, the decision was to use the Trello API to build “something” — not really sure what (and it’s still not fully realized what the ultimate goal is), but the desire was to learn Trello’s API by way of creating HTML views based on the Trello data.

My starting approach was to consider that, regardless of whatever other views may result from the project, building a view that shows all of the user’s boards, lists and cards on a single page would give me a decent understanding of not only Trello’s API but also (perhaps more importantly) the data structure behind it.

At that point, my experiences with API use had been mostly limited to work I’d done on four projects for the fCC certification path — a Random Quote Generator, a Local Weather app, a Wikipedia search tool, and a Twitch user status listing. The Twitch project required me to use multiple API calls in order to create the initial HTML view for that page, so I figured a similar approach would be my starting point for obtaining the Trello data and subsequently generating/painting the HTML content.

If you’re not at all familiar with Trello, it is essentially an organizational tool; you create “boards” which can represent large-scope “big picture” concepts, within the boards you create “lists” which can represent aspects of a concept, within the lists you create “cards” which can represent more detailed/nuanced items. There’s more to it than that, of course, but as a basic overview this should be enough to get an idea of what Trello is and what it can be used for.

So, following a model similar to what was used in the Twitch project, I wrote a chain of functions to gather the Trello data and paint HTML content representing that data. Here is a pseudo-code representation of what that looked like (note: going forward, “async1”, “a1”, &c, represent the Trello boards; “async2”, “a2”, &c, represent the lists; “async3”, “a3”, &c., represent the cards):

function getData() {
  //get Trello boards, make container, label for each
  async1(params, function(data1) {
    for (var i = 0; i < data1.length; i++) {
      a1PaintHtml;

//get lists from each board, make container, label in board for each
      async2(param + i.key + param, function(data2) {
        for (var j = 0; j < data2.length; j++) {
          a2PaintHtml;

//get cards from each list, make container, label in list for each
          async3(param + j,key + param, function(data3) {
            a3PaintHtml;
          });
        }
      });
    }
  });
}

Fairly simple and straightforward, that. But there were two things I didn’t like about it. First, the appearance of the page load was such that it was visibly showing the boards, lists and cards being constructed; from a UX perspective it makes the page look, for lack of a better term, clunky — I wanted the page to load in a way that made all the data appear to show up at the same time. Second, I’d done enough research and console testing with Trello’s API to know that, regarding the cards, I was making more async calls than were needed — even though the cards are functionally nested within the lists, they can be obtained by referencing the board itself, just like how the lists are obtained.

It became immediately obvious that what I needed to do was obtain all the data first and then paint the HTML content — since the HTML painting happens almost instantaneously, I’d get the UX perspective I was looking for. So, I began looking into how to delay the HTML painting until all the data was gathered. Using a setTimeout() did initially come to mind but I tossed that thought out quickly, knowing that any sort of delay I’d use would be almost entirely arbitrary and not guaranteed across platforms.

I then took to Google and began searching how to accomplish a non-arbitrary delay. Frequently, I came across references to “callback hell” and the “pyramid of doom”, which outlined essentially the same approach I had initially taken with my code. This made me feel pretty good because, I figured, if it’s a commonly discussed issue, there must be an answer. Sure enough, I began seeing the terms “promise” and “deferred” coming up in the articles and posts on Stack Overflow that I was reading. A light bulb went off in my head — during my many perusals of jQuery’s API documentation I’d seen .promise and .deferred listed as available methods, so I went back there for further research. In particular I was interested in reading about .deferred; because that’s what I was trying to do, right? “Defer” painting HTML content until after I have all the data. Made sense to me.

After reading more about jQuery’s .deferred — actually, it’s jQuery.Deferred() — things seemed to be lining up. I could create deferred objects that are tied to a listener ( jQuery.when() ) which then triggers the painting of the HTML content once the deferred objects are in a “resolved” state. Man, that’s awesome! And really easy! I was excited and wrote new code, which looked like this:

//establish master arrays for each async method result
var a1array = [], a2array = [], a3array = [];

//establish Deferreds for each async master array, html
var a1arrDfd = $.Deferred(), a1htmlDfd = $.Deferred();
var a2arrDfd = $.Deferred(), a2htmlDfd = $.Deferred();
var a3arrDfd = $.Deferred(), a3htmlDfd = $.Deferred();

//get a1 data
async1(params).done(function(a1data) {

  //iterate over a1 return and push to a1 master array
  var i, ilen = a1data.length, idone = 0;
  for (i = 0; i < ilen; i++) {
    a1array.push(a1data[i]);
    idone++;

    //if at last iteration of a1data array,
    //a1's master array is complete
    if (idone === ilen) { a1arrDfd.resolve(); }
  }
}).done(function(a1data) {
  var j, jlen = a1data.length, jdone = 0;

  //loop thru a1data array to get a2 data and
  //push a2data to a2 master array
  for (j = 0; j < jlen; j++) {
    async2(param + a1data[j].key + param).done(function(a2data) {
      var k, klen = a2data.length, kdone = 0;
      jdone++;

      //loop thru a2data array to push to a2 master array
      for (k = 0; k < klen; k++) {
        a2array.push(a2data[k]);
        kdone++;

        //if at last a1data iteration and a2data iterant,
        //a2's master array is complete
        if (jdone === jlen && kdone === klen) { a2arrDfd.resolve(); }
      }
    });
  }
}).done(function(a1data) {
  var l, llen = a1data.length, ldone = 0;

  //loop thru a1data array to get a3 data and
  //push a3data to a3 master array
  for (l = 0; l < llen; l++) {
    async3(param + a1data[l].key + param).done(function(a3data) {
      var m, mlen = a3data.length, mdone = 0;
      ldone++;

      //loop thru a3data array to push to a3 master array
      for (m = 0; m < mlen; m++) {
        a3array.push(a3data[m]);
        mdone++;

        //if at last a1data iteration and a3data iterant
        //a3's master array is complete
        if (ldone === llen && mdone === mlen) { a3arrDfd.resolve(); }
      }
    });
  }
});

//listen for completion of async1array, then
//paint html containers for each object in async1 array
$.when(a1arrDfd).done(function() {
  var n, nlen = a1array.length, ndone = 0;
  for (n = 0; n < nlen; n++) {
    a1paintHtml;
    ndone++;

    //if at last a1array iteration,
    //a1's html painting is complete
    if (ndone === nlen) { a1htmlDfd.resolve(); }
  }
});

//listen for completion of async1 html containers and
//async2 array, then paint containers for
//each object in async2 array
$.when(a1htmlDfd, a2arrDfd).done(function() {
  var o, olen = a2array.length, odone = 0;
  for (o = 0; o < olen; o++) {
    a2paintHtml;
    odone++;

    //if at last a2array iteration,
    //a2's html painting is complete
    if (odone === olen) { a2htmlDfd.resolve(); }
  }
});

//listen for completion of async2 html containers and
//async3 array, then paint containers for
//each object in async3 array
$.when(a2htmlDfd, a3arrDfd).done(function() {
  var p, plen = a3array.length, pdone = 0;
  for (p = 0; p < plen; p++) {
    a3paintHtml;
    pdone++;

    //if at last a3array iteration,
    //a3's html painting is complete
    if (pdone === plen) { a3htmlDfd.resolve(); }
  }
});

Ok, this code was a bit longer than my original attempt; however, the resulting UX was pretty much exactly what I was looking for — all the content appeared to show up at the same time, after a short delay during which the data was being gathered. I was fairly pleased with this, but had a nagging suspicion that it was too easy, that someone who’d been learning JavaScript for two months (me) must be missing something — it works, sure, but is it optimal?

I decided that I wasn’t going to delay working on my project but I did want to, in parallel, look into this curiosity further. At first I thought to create a question on Stack Overflow, but I’d read enough comments on posts at that site to recognize an online community that, while often helpful, can at times be somewhat viciously discerning about the supposed worthiness of questions posted there, especially by people who hadn’t previously posted much if at all. Instead, I took my question to the freeCodeCamp user forums and posted a link to that forum post on the Chingu cohort’s Slack channel dedicated to “help and feedback”.

After a little more than a day, I hadn’t received much in the way of a response, except for one fCC user suggesting that I should, in fact, take the question to Stack Overflow. Well, I figured, that’s the next step — I’d tried what I knew to be more friendly resources, and frankly I’m not emotionally involved enough to worry about what the Stack Overflow people think of my question; it’s fine if they want to be jerks about it, so long as I get an answer or at least a hint that leads me to an answer.

So I posted my question and waited. Later that day, I did get a response, albeit not in the form of a direct answer: “Avoid the deferred antipattern!”.

Ok… though not an answer per se, it was a hint and if nothing else it did tell me that I was right to think I was not writing optimal code.

I was glad to not be in a state of knowledge limbo, however I was in a bit of a quandary as to my next steps: after a few cursory reads regarding the “deferred anti-pattern” it became clear to me that I’d need to spend more time learning Promises and how to use them more effectively. Though I knew that one way or another I would need to spend this time, I wasn’t confident that was the right moment for me to start that process — the cohort was nearing its end so I wanted to get the first phase of the project completed (knowing that I’d be returning to it multiple times in the near future), added to which even though I’d thoroughly enjoyed the cohort and was looking forward to the next one, my progress on the fCC Front End certification path had been effectively halted. My learning plan, in that state of its evolution, was to finish that certification path and then take a step back to more thoroughly study/research JavaScript and round out my overall front end development proficiencies (including frameworks, e.g., Vue, React, Angular). The first goal post-certification was to read/study the book series You Don’t Know JS (YDKJS) and in that series is a book titled “Async &Performance”, in which there is a chapter dedicated to Promises. At that point, I figured, I’d learn enough to revisit my project and apply a more optimal approach than what I had.

That said, I did realize, rather embarrassingly, that even with an approach using deferred objects, it was completely unnecessary for me to tie such objects to the HTML painting in and of itself. So, I went back to the code and eliminated the aspects relying on those unnecessary objects:

var a1array = [], a2array = [], a3array = [];

var a1arrDfd = $.Deferred(), a2arrDfd = $.Deferred(), a3arrDfd = $.Deferred();

async1(params).done(function(a1data) {
  var i, ilen = a1data.length, idone = 0;
  for (i = 0; i < ilen; i++) {
    a1array.push(a1data[i]);
    idone++;
    if (idone === ilen) { a1arrDfd.resolve(); }
  }
}).done(function(a1data) {
  var j, jlen = a1data.length, jdone = 0;
  for (j = 0; j < jlen; j++) {
    async2(param + a1data[j].key + param).done(function(a2data) {
      var k, klen = a2data.length, kdone = 0;
      jdone++;
      for (k = 0; k < klen; k++) {
        a2array.push(a2data[k]);
        kdone++;
        if (jdone === jlen && kdone === klen) { a2arrDfd.resolve(); }
      }
    });
  }
}).done(function(a1data) {
  var l, llen = a1data.length, ldone = 0;
  for (l = 0; l < llen; l++) {
    async3(param + a1data[l].key + param).done(function(a3data) {
      var m, mlen = a3data.length, mdone = 0;
      ldone++;
      for (m = 0; m < mlen; m++) {
        a3array.push(a3data[m]);
        mdone++;
        if (ldone === llen && mdone === mlen) { a3arrDfd.resolve(); }
      }
    });
  }
});

$.when(a1arrDfd, a2arrDfd, a3arrDfd).done(function() {
  a1PaintHtml();
  a2PaintHtml();
  a3PaintHtml();
});

I was happy enough to leave things in that state until I’d continued on with my learning plan long enough to read the aforementioned YDKJS chapter on Promises.

Two weeks ago, I read that chapter and, sure enough, light bulbs again started going off in my head. I began inspecting my existing code with fresh eyes, and also decided that since I was in a new Chingu cohort it couldn’t hurt to reposit my situation in the new cohort’s “help and feedback” channel. I was making some headway on my own, but a reply from cohort member @pkh1162 got me pointed in the right direction and resulted in the following:

const a1array = [], a2array = [], a3array = [];

const a1get = async1(params).then(function(data) {
  const len = data.length;
  for (let i = 0; i < len; i++) {
    a1array.push(data[i]);
  }
  return a1array;
});

const a2get = a1get.then(function(a1data) {
  const promiseArray = a1data.map(function(a1) {
    return async2(param + a1.key + param)
      .then(function(data) {
        const len = data.length;
        for (let i = 0; i < len; i++) {
          a2array.push(data[i]);
        }
      });
  });
  return Promise.all(promiseArray);
});

const a3get = a1get.then(function(a1data) {
  const promiseArray = a1data.map(function(a1) {
    return async3(param + a1.key + param)
      .then(function(data) {
        const len = data.length;
        for (let i = 0; i < len; i++) {
          a3array.push(data[i]);
        }
      });
  });
  return Promise.all(promiseArray);
});

Promise.all([a1get, a2get, a3get]).then(function() {
  a1paintHtml();
  a2paintHtml();
  a3paintHtml();
});

This felt good, I was finally rid of the “deferred anti-pattern”. But it didn’t feel concise enough, particularly considering that in my time away from the project I’d picked up some more knowledge regarding ES6 syntax. After combing through my code yet again (and again), I finally landed on something I was truly happy with — a concise, effective, reusable pattern:

const a1array = [], a2array = [], a3array = [];

const a1get = async1(params).then(data => {
  a1array.push(...data);
  return a1array;
})

const a2get = a1get.then(a1data =>
  Promise.all(a1data.map(a1 =>
    async2(param + a1.key + param)
      .then(data => a2array.push(...data)))));

const a3get = a1get.then(a1data =>
  Promise.all(a1data.map(a1 =>
    async3(param + a1.key + param)
      .then(data => a3array.push(...data)))));

Promise.all([a1get, a2get, a3get]).then(() => {
  a1paintHtml();
  a2paintHtml();
  a3paintHtml();
});

Even as I write this article, I am awash with relaxed feeling when I compare what I started with to the end result. Do I fully know Promises inside and out, backward and forward, up and down? Not yet, however — having not only written a concise code pattern that works, but knowing why and how it works — I now have a solid foundation from which I can comfortably base my future endeavors using Promises in any other coding project that can benefit from their use.

Promises made, Promises actually kept — the JS way.