End to end tests

End to end tests (e2e) are focused in testing entire flows of our application to ensure that all components and interactions work together correctly. These tests guarantee that our application as a whole fulfills its requirements rather than simply testing individual elements in isolation.

Stack

We use Cypress for our e2e tests for its testing facilities, developer experience, automatic assertions, smart asynchronous DOM elements management, low flakiness and easy maintainability.

Running end to end tests

To execute all e2e tests in one go, use the following command from the universe/views directory:

views-e2e

You can also execute any of the single specs available with:

views-e2e <some_spec>

For example views-e2e specs/trial/test_trial.

For an interactive experience, particularly useful during setup and result analysis, you can open the Cypress Electron application. To do this, run:

cviews-e2e open

This will launch the Cypress Test Runner, allowing you to choose a browser for testing and running tests from a user-friendly interface.

Note

For the Cypress Test Runner, it is recommended to select the “Electron” browser, as it is reliable and operates without issues.

Adding custom commands

The Commands Cypress API is useful for adding new custom commands or overriding existing ones. For further detais, please check the Commands API documentation.

For adding new commands and being compliant with typing code quality, first extend the Chainable interface with the definition of your commands in the views/e2e/cypress/support/commands_def.ts file:

declare namespace Cypress {
  interface Chainable<Subject> {    
    yourCustomCommand(arg: string): Chainable<Subject>;
    anotherCustomCommand(arg1: string, arg2: number): Chainable<Subject>;
    ...
  }
}

Then, add the implementation with the Commands.add method in the views/e2e/support/commands.ts file:

Cypress.Commands.add("yourCustomCommand", (arg: string) => {  
  cy.log("Running your custom command");  
  // your command code goes here  
  ...
});

Cypress.Commands.add("anotherCustomCommand", (arg1: string, arg2: number) => {  
  cy.log("Running another custom command");  
  // more custom command execution   
  ...
});

Write tests

Write your tests in a spec file in the views/e2e/specs directory with file extension test_{your-test-name}.cy.ts.

// test_{your-test-name}.cy.ts 
describe("Test findings", () => {

  // Type here all tests which will start visisting /this/url  
});

describe("Test findings", () => {

  // Type here all tests which will visiting /this/one/other/url
});

Once in your spec file, group your tests by the target URL to be visited for the test inside a description block as follows:

CI, users and roles

Cypress runs in CI with a different user (or users) per node instance. A single CI job can run tests with multiple users, but no tests should be executed with a same user in multiple CI nodes, such level of redundancy is not necessary. This is a parallelization strategy which does not depend on cypress.io proprietary tools but from a different job split strategy based on the user(s) mapped to execute them. A mapping between users an CI nodes can be found in views/e2e/support/user.ts and contains a mapping logic which mimics the following behavior:

User1, User2 and User3    (Should run in) => CI Node 1
User 4                    (Should run in) => CI Node 2
User 5 and User6          (Should run in) => CI Node 3
...

When writing a test, it is required to choose which users will log in under the hood during cypress execution, and will concurrently run in the pipeline. For selecting the users which will run your test, user the runForUsers() function:

describe("Test an awesome aspect", () => {

    runForUsers(
     [
       // users for running the following tests
       IntegratesUsers.integratesmanager,
       IntegratesUsers.continuoushack2,
       IntegratesUsers.integratesadmin,
     ],
     (user) => {
       // Now you can place your tests and will only run for the previous users!
       it("Test feature 1", ()=>{...})
       it("Test feature 2", ()=>{...})
       it("Test feature 3", ()=>{...})
     }
    );
});

In this approach, we distribute tests across multiple CI instances, introducing redundancy in testing for various roles. This helps prevent unanticipated feature breakage due to changes impacting specific roles.

Code quality

Currently, cypress linting process is done through eslint-plugin-cypress and the typescript compiler targeted for type check only.

Once your e2e is ready, run its linting with:

views-lint

Best practices

No interdependencies: Each test should be independent of the others. This means that tests should not rely on the state of the application after another test has run.
Make tests retriable: Tests should be able to be retried without any manual intervention. This is important to ensure that the tests are reliable and can be run in a CI/CD pipeline.
Test a single unit: Each test should test a single unit of the application. This could be a single page, a single component, or a single feature. For example, a single test should not create a root, and then edit a user, and Then delete a user. Instead, there should be three separate tests for each of these actions.
Avoid too long tests: Tests should be short and focused. This makes them easier to read, understand, and maintain.
Decouple data: Tests should not rely on specific data. This is important to ensure that the tests are reliable and can be run in different environments.
Follow happy paths: Tests should follow the happy path, i.e., the most common path that a user would take through the application. It is not responsability of the e2e tests to test all possible edge cases, this should be covered in unit tests.

Tip

Have an idea to simplify our architecture or noticed docs that could use some love? Don't hesitate to open an issue or submit improvements.

End to End Tests | Cypress Testing Framework | Fluid Attacks Help