Making error states easy to test

• The Gist of the asyncThrows custom helper

In this video I want to show you how to test your application when it gets into an error state as a result of a failed Ajax request.

We’ll be writing some tests for the Blog posts route. This index page shows us a list of all blog posts in our system, and we can click a post, and then if we click Edit post, we can make some changes and then click Save. So let’s start by writing some acceptance tests for the behavior we already have.

We don’t have any tests yet so we’ll start by coming to the terminal and running

ember g acceptance-test posts

and then we’ll start our test server alongside our development server with ember t -s. If we open up localhost:7357 in our browser, we can see that our test suite is currently passing. And we can see that most of these are ESLint test that are automatically generated for us.

We can find our new Posts module here and just run these. And we see there’s this default test running and passing - so let’s take a look at it in our code.

Right now it just visits /posts and asserts that the URL is /posts. Let’s update it to actually work with some Mirage data.

Back in our development app, we see that the /posts page renders out this list of posts - so that would be a nice high-level test we could write to make sure that this page is working. We could say, given that there are 3 posts in our server, when we visit this page we expect to see three of these rows in this table rendered. And we can see that these are each <tr> elements in a <tbody>.

So let’s write that test. We’ll rename this to I can videw all the posts. And remember our test says that, given we have 3 posts. So we’ll start by creating 3 posts in Mirage’s database. Then we’ll visit /posts. And then let’s throw in a pauseTest here so we can see our app. Alright, looks good - I can see these three posts being rendered. And in the console we can experiment, we’ll want to do something like $(‘tbody tr’).length to make sure we have all 3. And that looks like it works, so let’s come back and in the assertion write assert.equal(find('tbody tr').length, 3). And we’ll get rid of the pauseTest and then try it out.

test(‘I can view all the posts’, function(assert) {
  server.createList('post', 3);

  visit('/posts');

  andThen(function() {
    assert.equal(find('tbody tr').length, 3);
  });
});

Ok great, so now we have our first passing test. Now I’ve been using async/await to write my tests so let’s go ahead switch over to that really quick. We need to install ember-maybe-import-regenerator. And then we can update our test

test('I can view all the posts', async function(assert) {
  server.createList('post', 3);

  await visit('/posts');

  assert.equal(find('tbody tr').length, 3);
});

And it still passes. Very nice!

Alright - now we have a passing test, and we could move on, but what we want to do here is, code an error scenario for this feature. After all, this route is fetching the list of posts from the server by making a GET request to /posts - so what happens if our server doesn’t respond with what we expect?

It’s easy to ignore these error states but if you want a robust application, you’ll not only write them but also write tests for them. So let’s come back our development app and first code some error handling for this route.

Let’s start by simulating an error response to this endpoint in Mirage. We’ll open our mirage config and right below our resource declaration for posts, we’ll force Mirage to error with

this.get('/posts', { errors: [ ] }, 500);

This object is a JSON:API errors object, and the array contains a list of reasons, but for now we’ll just leave it empty.

If we do this and come back to our app, we see our app has white-screened. If we reload on the homepage, and then click Blog posts, we don’t get an feedback. And if we open the console, we can see in these cases Ember Data is throwing an error as a result of the 500 response, and Ember’s router is canceling the transition.

Just to show you what’s going on, I can open the post index route, and we can see we’re returning this findAll from the model hook. So, Ember’s router is aware of this, and Ember has a nice way to handle errors that happen in these transition hooks on the route - and it’s called the error substate.

If we define an error template here under posts/error/template.hbs and let’s just add an h1 with “Uh-oh…” in it, now if we check out our app we can see that when the AJAX request errors, we see this error message. So that’s nice, because now our users know something happened.

Another nice thing about these error templates is that the JSON payload included in our API response is set as the model for this template. So let’s come back to our mirage config and add a reason in our response

this.get('/posts', { errors: [ 'The database is on vacation' ] }, 500)

And now in the template, we can say

<ul class="bullets">
  {{#each model.errors as |error|}}
    <li>{{error}}</li>
  {{/each}}
</ul>

and now when we visit the route, we see the reason being shown in the server. So this is just an example of how we can show server errors in our template, to give our users some feedback and also give our app a chance to recover without the user having to reload the whole thing.

So, let’s come back to our test and write a test for this. Now the first thing you’ll notice is that our first test is now failing. This is because we forced Mirage to error in our mirage/config file, which is global. So let’s come back and comment that out. 
So we’ll write a new test called “If there's a problem loading the posts, I see an error message”, it takes an async function, and now we can start. Let’s grab the code from our first test. It says, GIVEN three posts, when I visit posts, I see three table rows rendered. For this test what we want is, given 3 posts and given that our server will respond to our app’s GET request with an error, WHEN I visit /posts THEN I see an error message. So let’s come grab our line from mirage/config, and we can put this right here, and we’ll just change this to server - they’re the same thing. So this line is forcing Mirage to error.

Now when we visit /posts, we know we want to assert there’s an error, but let’s drop a pauseTest in here and see how our test is doing.

Ok - looks like we can see our error page. So we’ll just write a simple assertion that says we can see the message from our server.

test("If there's a problem loading the posts, I see an error message", async function(assert) {
  server.createList('post', 3);
  server.get('/posts', { errors: [ 'The database is on vacation' ] }, 500); // force Mirage to error

  await visit('/posts');

  assert.ok(find(":contains(The database is on vacation)").length > 1);
});

And with that we have a passing test that validates our error state! Not too shabby.

Now, you’ll notice one thing here in the console. Ember Data is actually logging the 500 error response. This is helpful to us when we’re doing normal development, but if our test suit is behaving the way we expect, we really should have a clean console. Otherwise our CI logs can be distracting and we might ignore warnings or other messages that we really should pay attention to.

So, I’m going to bring in this custom test assertion that I wrote to help me test these server errors. And don’t worry, I’ll link the gist in the summary below so you can copy it into your own app, as well as a blog post we have that goes into more detail about this helper.

Now if we look at the gist, what it’s describing is this assert.asyncThrows helper. And this does a few things, but the way we use it is by wrapping the parts of our app that make a server request and get back an error inside this helper. And in doing so, it’ll make sure our console is clean, and the error we get is the one we actually expect. So let’s install this and use it in our new test.

We can see from this How to use section that we need to install this Custom Assertions library, so let’s do that first. We’ll click on the link and run this ember install command.

This is a nice library from Dockyard that makes it easy to define new kinds of assertions for our tests - and I use it a lot because I find it makes my tests a bit more expressive than only using assert.ok and assert.equal.

If we scroll down we’ll also see there’s a Setup section, we need to import these two helpers and run them before and after our test app sets up. So let’s copy this, go to our module-for-acceptance, paste it in, then we paste assertionInjectior here and we can paste assertionCleanup here.

And now we can define new assertions under tests/assertions. So if we come back to the gist, we see we want to make a new file called async-throws, and then we can copy this over like this.

Now let’s start up our dev and test servers again. Our tests are still running, but we see the console message. So let’s use our new helper.

await assert.asyncThrows(() => {
  return visit('/posts');
});

Now if we use it like this, we see our test will fail. That’s because the helper expects us to pass in a string that will match the actual error message it received from the server. We can see here everything it received, and so we can grab a descriptive part of this like “GET /posts returned a 500”, and come back and pass it in as the second arg.

await assert.asyncThrows(() => {
  return visit('/posts');
}, "GET /posts returned a 500");

And now our tests pass, and our console is clean!

The string description of the error message here is important, because now if Mirage happens to respond with an error to any other request that we didn’t expect, this test would fail and we’d know about it.

Alright, let’s move on to our last feature! In the app, if we navigate to a blog post, we can click Edit, change the title or body, and click save. So let’s write a test for this behavior.

test('I can edit a post', async function(assert) {
  server.create('post', { text: 'Old post body' });

  await visit('/posts/1/edit');
  await fillIn('textarea', 'New post body');
  await click('button:contains(Save)');

  assert.equal(currentURL(), '/posts/1');
  assert.ok(find(':contains(New post body)').length > 0);
});

Alright, now we have our happy path test. It’s time to see what happens when our server errors.

Back in our dev app, I’m going to click Edit. And then let’s switch over to our Mirage config, and just like before we’ll override this route handler with one that errors:

this.patch('/posts/:id', { errors: [ ] }, 500);

If we click save back in our app, now we see that nothing happens. If we open the console, we can see that Ember Data threw an exception, but we can still use the app. So here we can see the difference between server errors when they’re handled by the route or not.

In our first example, our app white-screened, because Ember’s router went into a bad state. But here, we’re just calling an ember concurrency task on our controller. And I can pull that up right here, if I open the post/edit template, we can see the Save button is performing the save task, which is defined right here.

So, if this fails, our App doesn’t white screen, but also there’s no feedback to show the user, and we can’t just define an error template the way we did when it was a route-level promise that rejected. So instead, let’s use Ember concurrency’s derived state to show the user some feedback in the event of a server error response.

If we come back to the template, right here under the button we can add

{{#if save.last.isError}}
  <p class='mt3 red'>
    {{fa-icon 'exclamation-circle'}}
    Whoops - your post was not saved. Please try again.
  </p>
{{/if}}

And now, when we click save, we can see a message. Great! Let’s come write a test for this.

We’ll comment out the global override in our mirage/config, and come write a new test:

test('if editing a post errors, I see a message', async function(assert) {
  server.create('post', { text: 'Old post body' });
  server.patch('/posts/:id', { errors: [ 'A bad thing happened' ] }, 500);

  await visit('/posts/1/edit');
  await fillIn('textarea', 'New post body');

  await assert.asyncThrows(() => {
    return click('button:contains(Save)');
  }, 'PATCH /posts/1 returned a 500');

  assert.ok(find(':contains(Whoops - your post was not saved)').length > 0, 'I see the message');
});

And now we have four tests, two happy path and two errors, and our console is clean as a whistle!

This asyncThrows helper that I wrote may change over time, especially because QUnit recently got a new assert.rejects API that’s promise-aware, but so far it’s served us really well. I always find it worthwhile to take the time to make nice helpers like this, so me and other developers on my team have no excuse for testing important scenarios like error handing in our apps. So if there’s one lesson I’d want you to take away from this video, it’s to make sure your tests are easy to write, and your console is always kept clean of any extraneous information that shouldn't be there!