E2E testing with Japa and Playwright

[HipsterBrown]

End-to-end (E2E) testing provides the most realistic and broadest coverage of full stack AdonisJS applications. Playwright is handy open-source project for cross-browser automation that can also be used to test web applications.

This guide will show how to integrate Playwright into the built-in Japa testing framework to make E2E testing as seamless as possible.

Some customization may be required for each application use case, i.e. auth.

TL;DR View a RealWorld example app with end-to-end testing coverage

Japa Integration

The core piece of this integration is a custom Japa plugin (unpublished at the time of writing):

// assuming this is placed in a new file called `tests/playwright-client.ts`
import type { Config, PluginFn } from '@japa/runner'
import playwright from 'playwright'
import type { Browser, Page } from 'playwright'
import { getDocument, getQueriesForElement, queries } from 'playwright-testing-library'

declare module '@japa/runner' {
  interface TestContext {
    page: Page
    login(email: string, password: string): Promise<void>
    getScreen(): Promise<ReturnType<typeof getQueriesForElement>>
  }
}

type BrowserName = 'firefox' | 'webkit' | 'chromium'
type PlaywrightClientArgs = {
  /** defaults to firefox */
  browser?: BrowserName
  suiteName?: string
}

export function playwrightClient({
  browser: browserName = 'firefox',
  suiteName = 'e2e',
}: PlaywrightClientArgs = {}) {
  return async function (_config, runner) {
    runner.onSuite((suite) => {
      if (suite.name === suiteName) {
        let browser: Browser

        suite.setup(async () => {
          // create a new browser instance, headless by default
          browser = await playwright[browserName].launch({ headless: process.env.HEADLESS !== '0' })
          return () => browser.close()
        })

        suite.onGroup((group) => {
          group.each.setup(async (test) => {
            // for each test create a new page so session info is not shared across tests
            test.context.page = await browser.newPage({
              baseURL: `http://${process.env.HOST}:${process.env.PORT}`,
            })
            // this may need to be customized depending on the app setup
            test.context.login = async (email, password) => {
              await test.context.page.goto(test.context.route('login')) // go to login page
              const loginDoc = await getDocument(test.context.page)
              const emailInput = await queries.getByLabelText(loginDoc, 'email') // get the email input
              const passwordInput = await queries.getByLabelText(loginDoc, 'password') // get the password input

              await emailInput.fill(email) // enter email arg
              await passwordInput.fill(password) // enter password arg

              await (await queries.findByRole(loginDoc, 'button', { name: 'Sign in' })).click() // submit the form
            }
            test.context.getScreen = async () => {
              return getQueriesForElement(await getDocument(test.context.page))
            }

            return async () => {
              await test.context.page.close()
            }
          })
        })
      }
    })
  } as PluginFn
}

You may notice the use of playwright-testing-library for the getScreen helper to provide more accessible selection of elements in the browser. Read more about this testing philosophy here: https://testing-library.com/docs/guiding-principles

The plugin can be imported and used in the tests/bootstrap.ts file:

import { playwrightClient } from './playwright-client'

export const plugins: Config['plugins'] = [
  assert(),
  runFailedTests(),
  apiClient(),
  playwrightClient(),
]

Also, make sure the HTTP server is started for that suite in the configureSuite hook in that bootstrap file (this assumes the suite is called e2e:

export const configureSuite: Config['configureSuite'] = (suite) => {
  if (suite.name === 'functional') {
    suite.setup(() => TestUtils.httpServer().start())
  }
  if (suite.name === 'e2e') {
    suite.setup(() => TestUtils.httpServer().start())
  }
}

My .adonisrc.json is configured for the e2e suite of tests, with a longer timeout than the functional tests due to potentially increased latency when interacting with a browser environment:

  "tests": {
    "suites": [
      {
        "name": "functional",
        "files": [
          "tests/functional/**/*.spec(.ts|.js)"
        ],
        "timeout": 60000
      },
      {
        "name": "e2e",
        "files": [
          "tests/e2e/**/*.spec.ts"
        ],
        "timeout": 100000
      }
    ]
  },

Writing tests

With all that in place, let's take a look at an example test case:

import Database from '@ioc:Adonis/Lucid/Database'
import { test } from '@japa/runner'
import { ArticleFactory } from 'Database/factories/ArticleFactory'
import { ProfileFactory } from 'Database/factories/ProfileFactory'

test.group('articles/show', (group) => {
  group.each.setup(async () => {
    await Database.beginGlobalTransaction()
    return async () => {
      await Database.rollbackGlobalTransaction()
    }
  })

  test('displays article content', async ({ assert, page, route, getScreen }) => {
    const profile = await ProfileFactory.with('user').create()
    const article = await ArticleFactory.merge({ userId: profile.userId }).create()

    await page.goto(route('articles.show', article))
    let screen = await getScreen()

    assert.exists(await screen.findByRole('heading', { level: 1, name: article.title }))
    assert.lengthOf(await screen.findAllByRole('link', { name: profile.name }), 2)
    assert.lengthOf(await screen.findAllByRole('button', { name: /Follow / }), 2)
    assert.lengthOf(await screen.findAllByRole('button', { name: /Favorite post/ }), 2)
    assert.exists(await screen.findByText(/to add comments on this article/))
  })
})

Things like factories and database interactions can still be used in these tests, and the text context now includes the page object and getScreen helper. Once we have a screen instance from getScreen, we get access to all the Testing Library queries for asserting on the markup of the page.

If we want to test an authenticated user experience, we can use the login helper from the test context as demonstrated below:

  test('creates new articles for logged in user', async ({ assert, page, login, getScreen }) => {
    const profile = await ProfileFactory.with('user', 1, (user) =>
      user.merge({ email: '[email protected]', password: 'SuperSecret123' })
    ).create()

    await login('[email protected]', 'SuperSecret123')

    await page.goto('/')
    let screen = await getScreen()

    await screen.findByRole('link', { name: 'New Article' }).then((el) => el.click())

    screen = await getScreen()

    const title = await screen.findByLabelText('article title')
    const description = await screen.findByLabelText("What's this article about?")
    const content = await screen.findByLabelText('article content')
    const tags = await screen.findByLabelText('enter tags (comma separated)')
    const publishButton = await screen.findByRole('button', { name: 'Publish Article' })

    await title.fill('My new article')
    await description.fill('This is a brand new article')
    await content.fill(
      `## New Article\n\nRendering markdown is no problem [here](http://example.com)`
    )
    await publishButton.click()

    screen = await getScreen()
    assert.exists(await screen.findByRole('heading', { level: 1, name: 'My new article' }))
    assert.exists(await screen.findByRole('heading', { level: 2, name: 'New Article' }))
    assert.exists(await screen.findByRole('link', { name: 'here' }))
    assert.lengthOf(await screen.findAllByRole('link', { name: profile.name }), 3)
  })

These tests can be run like any other suite with node ace test. If you want to view the live browser (rather than running headless), set the HEADLESS environment variable to 0: HEADLESS=0 node ace test

Running Tests on CI

For continuous feedback in pull requests or before deployment, the tests can be run in a continuous integration environment, like GitHub Actions:

name: E2E Tests
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
jobs:
  test:
    timeout-minutes: 20
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Setup node
        uses: actions/setup-node@v2
        with:
          node-version: "16.x"

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright Browsers
        run: npx playwright install --with-deps

      - name: Run tests
        run: node ace test

Read the Playwright docs for more CI configuration examples; be sure to trade out references to npx playwright test with node ace test instead.

Conclusion

This recipe provides the most basic browser testing features from Playwright. The plugin could eventually become a separate package with official auth support (rather than manually logging in through the UI) and be configured to provide automatic screenshots after failures or for visual regression testing. Iterating on that, there could be video capture for full user flows and automatically generating test interactions.

Until that time, I hope you've found this information helpful!