Here are some cool links to other Stuff. Enjoy, or Don't. I'm not your boss.

Weekly Update: Tackling the Testing problem for Browsers

Recently I've been working on a series of projects. These projects are really all scoped under one massive goal that I've had for awhile though. One of these projects (like most things today), includes building a website. Building a websites frontend is always something I've had much trouble with. Not the backend, and even to an extent not the frontend code. Specifically I've ran into two problems with the frontend: Making something that looks objectively pretty, and performing cross browser testing.

Now if you're like me, you love testing. Testing is great when done right. The key word there is right. A test should never be a chore to right, or constantly require change, or be slow. They should be fast, reproducible, and should be easy to write. This is why I generally hate code coverage rules unless they're very lax. Take for example a 90% code coverage rating on say lines, and functions. In theory this sounds like an amazing goal "we test 90% of all of our code". Until you realize that probably around 50% the code we write, testing is essentially meaningless when paired with code review. Take for example the following block of javascript (note I cut out some of the imports):

import {formatLockObject} from '../LockItemFormat'

export default class LockBanner extends Component {
  static propTypes = {
    isLocked: PropTypes.bool.isRequired,
    itemLocks: propTypes.itemLocks,
  }

  static defaultProps = {
    itemLocks: {
      content: false,
      points: false,
      settings: false,
      due_dates: false,
      availability_dates: false,
    }
  }

  static setupRootNode () {
    const bannerNode = document.createElement('div')
    bannerNode.id = 'blueprint-lock-banner'
    bannerNode.setAttribute('style', 'margin-bottom: 2em')
    const contentNode = document.querySelector('#content')
    return contentNode.insertBefore(bannerNode, contentNode.firstChild)
  }

  componentDidUpdate (prevProps) {
    if (this.props.isLocked && !prevProps.isLocked) {
      $.screenReaderFlashMessage(I18n.t('%{attributes} locked', { attributes: formatLockObject(this.props.itemLocks) }))
    }
  }

  render () {
    if (this.props.isLocked) {
      return (
        <Alert>
          <Typography weight="bold" size="small">{I18n.t('Locked:')}&nbsp;</Typography>
          <Typography size="small">{formatLockObject(this.props.itemLocks)}</Typography>
        </Alert>
      )
    }

    return null
  }
}

This seems like a pretty normal component. Easy to test, and does one thing. Now lets take a look at a test for this component:

test('displays locked description text appropriately when one attribute is locked', () => {
  const props = defaultProps()
  const tree = enzyme.mount(<LockBanner {...props} />)
  const text = tree.find('Typography').at(1).text()
  equal(text, 'Content')
})

This seems like a pretty reasonable test, until you realize a couple things:

  1. This test is depending on the raw output of text. If that text ever changes for any reason (Product decides it wants to be changed, we start internationalizing text to a different language or want to test a different language, we find a type, etc.). This enforces us to go back and have to change it multiple places.
  2. This test depends on the raw location of the Typography block. If we want to change the markup at all to say contain a secondary typography block there's a high change this test will break. Again there's something hardcoded in this test that when I change the component I have to go change this thing.
  3. The text actually being tested for here  "Content". If you notice is being rendered through `formatLockObject`. `formatLockObject` has a test that tests for the exact same output! So I'm actually duplicating the test.
  4. The test really isn't testing any of my code (the test in `formatLockObject` is ensuring it chooses the right one, but again this is duplicating that for no reason other than code coverage). It's testing that the text I put in which I've already validated through my `formatLockObject` tests is being outputted by the framework. I'm not testing my code, I'm testing the framework.

So why do we have this test? Because there are code coverage numbers, and we want to ensure that 90% of our lines, and functions are tested. So we write a test like this. Look through your favorite frontend code base you'll find tests like these pretty frequently when there's code coverage requirements. For a framework this test would make sense, your testing your framework renders the data passed in, but inside this application it makes no sense here. (Test actual behaviors not test everything).

Anyway enough about poorly written tests in order to hit "code coverage". Let's actually talk about testing in the frontend. When building a frontend app there's a couple things I want to test:

  1. When an action happens, the state gets updated in the way I'm expecting. E.g. user clicks a button so I increment a counter.
  2. Cross browser differences (say IE doing something weird).
  3. That the components look like they're supposed to cross browser (e.g. IE not supporting some CSS Rule).

Personally this app was built by using ReasonML, and ReactReason. So the first step is easy, because React has a pretty dedicated community around one test runner. Jest. Even if there's parts of jest I don't fully agree with (like snapshot testing). It's a widely supported solution, and I'm only using it for testing state changes (and maybe timers), so it's a clear win.

The problem comes in with the last two points. I wish I could just run my own super big selenium grid, but unfortunately I have neither the money nor operational patience for setting something like that up. So no problem, I just need something that works Cross Browser (Chrome/Firefox/Safari/And hopefully IE at the least), is Cheap-"ish" for a single person private repo, and allows me to write automated tests in a fairly simple way in the same way for all browsers. No problem right?

Turns out that this is either a really really hard problem, or a really really expensive (for me problem). So if you excuse me it's back to digging into super expensive solutions in order to hopefully find a diamond in the rough.

Weekly Roundup

Beginning the New Year