Skims functional tests

To develop a functional test, follow the following steps:

Generate a configuration file: Generate a configuration file that emulates the desired execution scenario. Store this file in skims/test/data/config.

Implement test logic: In skims/test/functional, write the basic test logic using the skims utility function. This function executes the CLI with the provided configuration file, analyzing specified modules and test files. It facilitates leveraging pytest features like mocking and patching. The utility function runs the equivalent of nix run .#skims scan config.yaml within a test environment.

code, stdout, stderr = skims("scam", path)
assert code == 0, stdout
assert "[INFO] Startup work dir is:" in stdout
assert "[INFO] An output file has been written:" in stdout
assert not stderr, stderr
check_that_csv_results_match(finding)

Assign test group: Assign the test to a group and add it to skims/test_groups.json. After updating this file, run the test locally using the command nix run .#skims-test test_group_name.

Update CI/CD pipeline: Execute nix run .#skims-pipeline to update the CI/CD pipeline YAML configuration on GitLab, ensuring its execution on the platform.

For adding tests specifically for new methods, integrate mock data into the appropriate folder within skims/test/data/lib_ , and verify the target finding is defined in skims/test/functional/test_findings.py. These tests utilize the common config.yaml template to automatically generate and execute functional tests on specific paths.

Tip

Have an idea to simplify our architecture or noticed docs that could use some love? Don't hesitate to open an issue or submit improvements.

Functional Tests | Skims CLI Testing Framework | Fluid Attacks Help

Skims functional tests