End to end tests

End-to-end tests (e2e) are focused on testing entire flows of our application to ensure that all components and interactions work together correctly. These tests guarantee that our application as a whole fulfills its requirements rather than simply testing individual elements in isolation.

Already know the basics of Views? If not, read the introduction.

Stack

We use Cypress for our e2e tests for its testing facilities, developer experience, automatic assertions, smart asynchronous DOM elements management, low flakiness, and easy maintainability.

Running end-to-end tests

To execute all e2e tests in one go, use the following command from the universe/views directory:

views-e2e

You can also execute any of the single specs available with:

views-e2e <some_spec>

For example views-e2e specs/trial/test_trial.

For an interactive experience, particularly useful during setup and result analysis, you can open the Cypress Electron application. To do this, run:

cviews-e2e open

This will launch the Cypress Test Runner, allowing you to choose a browser for testing and running tests from a user-friendly interface.

Note

For the Cypress Test Runner, it is recommended to select the “Electron” browser, as it is reliable and operates without issues.

Adding custom commands

The Commands Cypress API is useful for adding new custom commands or overriding existing ones. For further details, please check the Commands API documentation.

To add new commands and be compliant with typing code quality, first extend the Chainable interface with the definition of your commands in the views/e2e/cypress/support/commands_def.ts file:

declare namespace Cypress {
  interface Chainable<Subject> {    
    yourCustomCommand(arg: string): Chainable<Subject>;
    anotherCustomCommand(arg1: string, arg2: number): Chainable<Subject>;
    ...
  }
}

Then, add the implementation with the Commands.add method in the views/e2e/support/commands.ts file:

Cypress.Commands.add("yourCustomCommand", (arg: string) => {  
  cy.log("Running your custom command");  
  // your command code goes here  
  ...
});

Cypress.Commands.add("anotherCustomCommand", (arg1: string, arg2: number) => {  
  cy.log("Running another custom command");  
  // more custom command execution   
  ...
});

Write tests

Write your tests in a spec file in the views/e2e/specs directory with file extension test_{your-test-name}.cy.ts.

// test_{your-test-name}.cy.ts 
describe("Test findings", () => {

  // Type here all tests which will start visisting /this/url  
});

describe("Test findings", () => {

  // Type here all tests which will visiting /this/one/other/url
});

Once in your spec file, group your tests by the target URL to be visited for the test inside a description block as follows:

CI, users, and roles

Cypress runs in CI with a different user (or users) per node instance. A single CI job can run tests with multiple users, but no tests should be executed with the same user in multiple CI nodes; such a level of redundancy is not necessary. This is a parallelization strategy that does not depend on Cypress.io's proprietary tools but on a different job split strategy based on the user(s) mapped to execute them. A mapping between users and CI nodes can be found in views/e2e/support/user.ts and contains a mapping logic which mimics the following behavior:

User1, User2 and User3    (Should run in) => CI Node 1
User 4                    (Should run in) => CI Node 2
User 5 and User6          (Should run in) => CI Node 3
...

When writing a test, it is required to choose which users will log in under the hood during Cypress execution, and will concurrently run in the pipeline. For selecting the users who will run your test, use the runForUsers() function:

describe("Test an awesome aspect", () => {

    runForUsers(
     [
       // users for running the following tests
       IntegratesUsers.integratesmanager,
       IntegratesUsers.continuoushack2,
       IntegratesUsers.integratesadmin,
     ],
     (user) => {
       // Now you can place your tests and will only run for the previous users!
       it("Test feature 1", ()=>{...})
       it("Test feature 2", ()=>{...})
       it("Test feature 3", ()=>{...})
     }
    );
});

In this approach, we distribute tests across multiple CI instances, introducing redundancy in testing for various roles. This helps prevent unanticipated feature breakage due to changes impacting specific roles.

Code quality

Currently, the Cypress linting process is done through eslint-plugin-cypress, and the TypeScript compiler is targeted for type check only.

Once your e2e is ready, run its linting with:

views-lint

Best practices

No interdependencies: Each test should be independent of the others. This means that tests should not rely on the state of the application after another test has run.
Make tests retriable: Tests should be able to be retried without any manual intervention. This is important to ensure that the tests are reliable and can be run in a CI/CD pipeline.
Test a single unit: Each test should test a single unit of the application. This could be a single page, a single component, or a single feature. For example, a single test should not create a root, and then edit a user, and then delete a user. Instead, there should be three separate tests for each of these actions.
Avoid too long tests: Tests should be short and focused. This makes them easier to read, understand, and maintain.
Decouple data: Tests should not rely on specific data. This is important to ensure that the tests are reliable and can be run in different environments.
Follow happy paths: Tests should follow the happy path, i.e., the most common path that a user would take through the application. It is not the responsibility of the e2e tests to test all possible edge cases; this should be covered in unit tests.

Tip

Have an idea to simplify our architecture or noticed docs that could use some love? Don't hesitate to open an issue or submit improvements.

End to End Tests | Cypress Testing Framework | Fluid Attacks Help