The data join and dynamic charts

About halfway through the last video, we encountered a bug with our charts. We saw that while our charts can handle the initial render of their data, if we change the data after render, nothing happens in response. In this video we're going to update our bar chart component to be dynamic, so that it re-renders in response to changing data.

Let's start by reverting a change we made in the last video. If we come to our code and open up our posts controller, we can scroll down to the properties that are powering our bar charts. We have our postsByAuthor property here, and right now it groups the model by author. So let's make it group the posts instead, because this posts property is now a dynamic array of posts that changes whenever the user selects different bars.

postsByAuthor: groupBy('posts', 'author')

If we save this and come back to the app, we see our charts render correctly, and if we select a category, we know the posts property has changed since the table is re-rendering – but our Authors chart here doesn't do anything. Now what if we reload the page? Well now the chart changes, and again, that's because we're using the query param to filter down the data, and the bar chart's initial render works. So right now it's just rendering that filtered dataset. The problem is that the chart is not re-rendering.

So, we want to update our code so that our chart can handle re-rendering as well.

Now, our strategy is going to be very similar to when we added the selectable interaction to our chart earlier in this series. If we open our bar chart component, recall that earlier we extracted the rendering logic for updating the opacities of our bars into this separate updateOpacities method, and that let us call it whenever we needed to, which we're doing right here, here and here. We want to take a similar approach for all the other logic in our chart concerned with rendering our bars.

So, we'll come down here, and we'll start by writing a new renderChart method. And, we'll come up here, and we'll just grab all of this logic, and move it to renderChart. And then we'll make sure to come up to didInsertElement and call renderChart where our code used to be.

didInsertElement() {
  this.renderChart();
  this.updateOpacities();
  this.set('didRenderChart', true);
}

Let's call save and make sure this still works. Looks good.

Ok, now that we have our rendering logic in this method, and we call it here in didInsertElement, and we want our chart to re-render on updates. So we'll just grab this and call it again in didUpdateAttrs.

didUpdateAttrs() {
  this.renderChart();
  this.updateOpacities();
}

Let's save this, come back to our app and try it out. Ok, nothing seems to be happening. Let's open our console - we don't see any errors. So what's going on here?

Well, let me show you something really quick.

  let bars = svg.selectAll('rect').data(this.get('data'))
    .enter()
    .append('rect')
+   .merge(svg.selectAll('rect').data(this.get('data')))

Now when we click... our chart updates! And it also re-sorts, even when we change the sorting order of the table!

So, what was this line of code that I just added to make our chart update? To explain this, we need to take a closer look at this line of code, which is our D3 data join.

svg.selectAll('rect').data(this.get('data'))
  .enter()

Now first, when we call the D3 method selectAll, what this really gives us is a representation of an array of elements. And then, once we call .data and bind this representation to our actual array, D3 gives us what's called a "data join" to work with.

Now, the data join is a key concept in D3 and it gives us complete control over how our visualizations respond to changing data. The primary data join can be broken down into three subselections:

• There's the enter subselection, which controls how our visualization responds to new data;
• The update subselection, which controls how our visualization responds to existing data that D3 already knows about, but whose values have changed;
• And the exit subselection, which controls how our visualization responds to data that's no longer present in our data array

So, whether a particular bar is new, being updated, or being removed depends on the data that's coming into our chart and being joined to our D3 selection.

Ok, so up till now we've just been working exclusively with the enter subselection, and this is because we're calling .enter right here:

svg.selectAll('rect').data(this.get('data'))
  .enter()

What this means is that all the transforms that follow — appending a rectangle, setting those rectangle's widths, heights, x and y positions and fill color — these only apply to data that D3 is seeing for the first time. But if we come back to our app, when we select a category, the posts array is being filtered down, so our Authors chart gets a new dataset with new values. And that new data belongs to D3's update subselection, which right now we're not working with anywhere in our chart's code.

Here, let's refactor our code a bit to make this a little bit more clear where each of the subselections lives.

The first one is this main one, which is actually the update subselection. And this is just the default selection that D3 uses whenever you call .data. So down here we're calling .enter to get the enter subselection, but by default the data join returns the update subselection. And that's just part of D3's API, because the update subselection is such a common one to work with.

Now the way we get at the other two subselections is actually off of the update subselection. Now this is what we've been working with so far, which is the enter subselection. And again now we can just delete this and replace it with barsUpdate, it's the same thing, move this up to the same line, and call this barsEnter. And then down here we'll call all of these methods off of barsEnter, and now we're back to what we had before.

let barsUpdate = svg.selectAll('rect').data(this.get('data'));
let barsEnter = barsUpdate.enter().append('rect');

barsEnter
  // .merge(svg.selectAll('rect').data(this.get('data')))
  .attr('width', `${this.get('xScale').bandwidth()}%`)

So again, the default data join that's returned from .data is the update subselection – we don't need to call .update() or anything like that, it's just the default one returned – and then we get the enter subselection by calling .enter() on the update subselection.

And so now that we have update and enter defined, and we know we want all of these transformations applied to both new and existing bars whose values are changing, we can use this merge method and call .merge, and then we can pass in the updating bars.

barsEnter.merge(barsUpdate)

So altogether, these instructions are saying: set up our data join; for new bars, append a rectangle to the DOM; and then grab all of our new bars, merge them with our existing bars, and then set the widths, heights and other attributes accordingly; and then finally we add event handlers, but only on initial render.

Let's save this, and then check out our app.

So now, when we select a category, and the counts in the data array change and flow back into our Authors chart, these bars' attributes now get updated. So, all of the commands in our code apply to both initial render and re-render.

---

Alright, so now our bar chart is re-rendering correctly! We've fixed it so that it handles both the initial render of the data, and it responds to new data dynamically. But if we take a closer look, we'll actually see we have an issue.

If we look at the first bar in our Authors chart for the unfiltered dataset, we'll see that it's for Edison. But if we select Economics and our authors chart rerenders, this first bar is now for Giuseppe. So the bars look like their heights and widths are correct, but their positions are actually changing. And that makes it confusing.

The problem is that our grouped author data is derived from the filtered array, and once we choose a category the first author in that array happens to change. And we can see this by looking at our table. The unfiltered dataset shows Edison first, but the filtered dataset shows Giuseppe. And our bar chart is reflecting this.

We can fix this by coming to our bar chart component and sorting our data. So we'll come up to the top of renderChart, and we'll make a local data property where we get our data and sort by the label property.

let data = this.get('data').sortBy('label');

And now let's just replace all the instances of this.get('data') with our new local property. Let's save this, come back, and now the first bar is Edison, whether or not we have any of these bars selected, or even if we resort the data in the table.

Ok, so we've improved our chart to dynamically re-render when both new bars are entering our chart, and when existing bars are updating. What about exiting bars?

Well, we can see what happens by selecting a single post from the comments chart. We can see by looking at the table that our dataset filters down to a single record. In response, our Authors chart has highlighted this single bar, but the other five have actually stayed in the same place. And that's because we haven't defined any rules for what happens when data exits the chart. So let's do that now.

Just like with the enter subselection, we can define our exit from our upate subselection by calling .exit():

let barsExit = barsUpdate.exit();

And then let's come down here after we've drawn our bar chart and define what happens for exiting. Now D3 has a convenient .remove method that we can call directly from our exit subselection. And this will remove the bars that correspond to the exiting data from the DOM.

barsExit.remove();

Let's save this, and come back and check it out. Now when we select a post, our Authors chart becomes a single bar. We can also see this with the Literature category, there's only five authors who have posts in the Literature category, and we can see that the Authors chart is correctly rendering.

So now we've defined transformations on all three subselections, and our charts stays in sync no matter how that data changes - whether we select a new bar, or we use the back and forward buttons to change the selected properties, or we refresh the page and let our bar charts initially render with a filtered dataset. So this is pretty neat!

---

In the next video, we'll refactor our code a bit and separate out the static parts of our chart from the dynamic parts.