"global failure" for last test when acceptance test is present

[allthesignals]

global failure:

beforeEach failed on global failure: Cannot read property 'setContext' of null:

Source: | TypeError: Cannot read property 'setContext' of null     at Object.beforeEach (http://localhost:7357/assets/test-support.js:9324:16)     at callHook (http://localhost:7357/assets/test-support.js:4072:25)     at runHook (http://localhost:7357/assets/test-support.js:4095:7)     at Object.advance (http://localhost:7357/assets/test-support.js:3691:26)     at Test.queue (http://localhost:7357/assets/test-support.js:4264:22)     at test (http://localhost:7357/assets/test-support.js:4498:12)     at Object.onError (http://localhost:7357/assets/test-support.js:5212:5)     at window.onerror (http://localhost:7357/assets/test-support.js:6558:18)

This global failure occurs when I have an acceptance test present in my tests directory. NOTE: The tests run fine (including acceptance) in chrome headless browser, but not in the interactive qunit interface. I'm not sure how to approach this...

Here is my package.json:

{
  "name": "labs-nyc-factfinder",
  "version": "0.0.0",
  "description": "Small description for labs-nyc-factfinder goes here",
  "license": "MIT",
  "author": "",
  "directories": {
    "doc": "doc",
    "test": "tests"
  },
  "repository": "",
  "scripts": {
    "build": "ember build",
    "start": "ember server",
    "test": "ember test"
  },
  "devDependencies": {
    "babel-eslint": "^7.2.3",
    "broccoli-asset-rev": "^2.4.5",
    "broccoli-clean-css": "^1.1.0",
    "d3": "^4.11.0",
    "ember-ajax": "^3.0.0",
    "ember-browserify": "^1.2.0",
    "ember-cli": "~2.14.0",
    "ember-cli-app-version": "^3.0.0",
    "ember-cli-babel": "^6.3.0",
    "ember-cli-dependency-checker": "^1.3.0",
    "ember-cli-eslint": "^3.0.0",
    "ember-cli-foundation-6-sass": "0.0.24",
    "ember-cli-htmlbars": "^2.0.1",
    "ember-cli-htmlbars-inline-precompile": "^0.4.3",
    "ember-cli-inject-live-reload": "^1.4.1",
    "ember-cli-numeral": "^1.0.0",
    "ember-cli-qunit": "^4.0.0",
    "ember-cli-sass": "^6.2.0",
    "ember-cli-shims": "^1.1.0",
    "ember-cli-sri": "^2.1.0",
    "ember-cli-uglify": "^1.2.0",
    "ember-d3": "^0.3.4",
    "ember-data": "~2.14.3",
    "ember-decorators": "^1.3.1",
    "ember-export-application-global": "^2.0.0",
    "ember-fetch": "^3.4.3",
    "ember-font-awesome": "^4.0.0-alpha.13",
    "ember-load-initializers": "^1.0.0",
    "ember-lodash": "^4.17.5",
    "ember-math-helpers": "^2.3.0",
    "ember-native-dom-helpers": "^0.5.4",
    "ember-resize": "0.0.17",
    "ember-resolver": "^4.0.0",
    "ember-source": "~2.14.0",
    "ember-test-helpers": "^0.7.0-beta.4",
    "eslint": "^4.3.0",
    "eslint-config-airbnb": "^15.1.0",
    "eslint-plugin-import": "^2.7.0",
    "eslint-plugin-jsx-a11y": "^5.1.1",
    "eslint-plugin-react": "^7.1.0",
    "foundation-sites": "^6.4.3",
    "loader.js": "^4.2.3"
  },
  "engines": {
    "node": "^4.5 || 6.* || >= 7.*"
  },
  "private": true,
  "dependencies": {
    "@turf/bbox": "^5.0.0",
    "babel-eslint": "^8.0.1",
    "ember-jane-maps": "0.0.5",
    "numeral": "^2.0.6"
  }
}

It seems to be caused by the visit('/report/demographic'); helper.

Here's the full test:

import { test } from 'qunit';
import moduleForAcceptance from 'labs-nyc-factfinder/tests/helpers/module-for-acceptance';

moduleForAcceptance('Acceptance | report shows correct statistics');

test('visiting /report', function(assert) {
  visit('/report/demographic');

  andThen(function() {
    assert.equal(currentURL(), '/report/demographic');
  });
});

[rwjblue]

Hmm, this is an odd one. setContext is a method provided by ember-test-helpers (the version you are using is here), and must be present if the tests are actually running and passing like you mentioned.

Are there any customizations to tests/helpers/module-for-acceptance.js?

Also, this statement confuses me a bit:

but not in the interactive qunit interface

Can you describe how you are running the tests (what commands) when the do work vs when they do not work?

[allthesignals]

Hi @rwjblue - thank you for the response. I don't have customizations in my module-for-acceptance which can be seen below (I think it is just the standard).

"qunit interface" is just the web-based interface you see at http://localhost:7357/ as opposed to the console-based interface (which is also interactive so I can understand the confusion):

The first tab is headless Chrome, the second is web-based.

Tests work when:

• I run them in headless Chrome
• They're run through CI
• Reload based on file changes

Tests do not work when:

• I have an acceptance test present, and I use the web-based tester (the entire test suite runs fine, then a split second later, "global failure"). I used ember g acceptance-test test-name to generate.

After doing a git bisect, I pinned the 'presence-of-acceptance-test-breaking-web-based-tester' behavior to one commit that adds ember-browserify to package.json... that seems strange to me, possibly unrelated. However, after more digging, I found the use of a single addon component (which we built) seems to trigger the global failure. When it's commented out, no global failure, acceptance test passes. I can even re-add ember-browserify.

There are some weird behaviors deep inside that addon we use that I believe are causing this, specifically an error about setting properties on destroyed objects. I'm going to try to solve that issue, but it is confusing how the behavior differs between headless and web-based, and how an addon issue seems to be getting swallowed up.

I'll keep digging, but I think this is something project-specific rather than an issue with ember-cli-qunit. The branch I'm using is here.

module-for-acceptance.js

import { module } from 'qunit';
import Ember from 'ember';
import startApp from '../helpers/start-app';
import destroyApp from '../helpers/destroy-app';

const { RSVP: { resolve } } = Ember;

export default function(name, options = {}) {
  module(name, {
    beforeEach() {
      this.application = startApp();

      if (options.beforeEach) {
        return options.beforeEach.apply(this, arguments);
      }
    },

    afterEach() {
      let afterEach = options.afterEach && options.afterEach.apply(this, arguments);
      return resolve(afterEach).then(() => destroyApp(this.application));
    }
  });
}

[praya]

Any progress on this issue?