`opengrep test`

See also: opengrep scan, --config, opengrep validate, --autofix, --matching-diagnosis, --test, fix

opengrep test [FLAGS] [PATHS...]
opengrep test --config RULES [FLAGS] TARGETS...

Runs rules on test files and compares the findings with the annotations in those files. The default PATH is the current directory.

Pairing rules with targets

Without --config, each rule file (.yaml or .yml) found under PATHS is tested against the files in the same directory whose names are the same except for the extensions. For example, find-eval.yaml is tested against find-eval.py and find-eval.test.yaml. A directory with the same name as the rule file, such as find-eval/, provides all the files under it as targets.

With --config, the given rules run on the given targets, regardless of their names.

Annotations

An annotation is a comment on the line before the code it is about, or on the same line. The comment is written in the target language, for example #, //, /* */,  or (* *).

Annotation	Meaning
`ruleid: <id>`	The rule must report this line.
`ok: <id>`	The rule must not report this line.
`todoruleid: <id>`	The rule should report this line but does not yet. The test passes either way.
`todook: <id>`	The rule reports this line but should not. The test passes either way.

Several rule ids can be given, separated by commas: # ruleid: rule-a, rule-b. The test fails when a rule misses a ruleid: line ("missed lines") or reports a line that no ruleid: annotation names ("incorrect lines"). A todoruleid: or todook: line never fails a test, regardless of whether the rule reports it. The deepok:, deepruleid:, prook: and proruleid: prefixes are accepted for compatibility and are ignored.

Autofix tests

When a target x.py has a file x.fixed.py next to it, opengrep test also applies the fixes of the rules to x.py and checks that the result equals x.fixed.py. The line No tests for fixes found. means that there is no such file.

Exit status

0 when all tests pass, 1 when a test fails, and 2 on a fatal error.

Flags

Flag	Summary
`--config`	Load rules from a file, a directory, a URL, a git repository or the registry.
`--debug`	Log the messages of --verbose and the diagnostics of the engine.
`--develop`	Listed in Internal and debugging interfaces.
`--disable-intrafile`	Analyze every taint rule one function at a time, including rules that set taint_intrafile or taint_interfile.
`--enable-interfile`	Run cross-file taint analysis for the rules that set the taint_interfile option.
`--force-color`	Style the output even when it does not go to a terminal.
`--interfile-depth`	The longest call chain the cross-file taint analysis follows.
`--json`	Print the findings as a JSON document.
`--matching-diagnosis`	Explain why a failing rule test did not match.
`--max-memory`	The memory that the analysis of one file may use before it is abandoned.
`--opengrep-ignore-pattern`	An additional comment prefix that suppresses the findings on a line.
`--profile`	Listed in Internal and debugging interfaces.
`--quiet`	Print only the findings, and nothing on standard error.
`--strict`	Fail the run when a file cannot be parsed or another warning-level error occurs.
`--taint-intrafile`	Follow taint through calls to functions in the same file, for every taint rule.
`--timeout`	The time limit of a rule on a file, in seconds; 0 sets no limit.
`--timeout-threshold`	The number of rules that may time out on a file before the file is skipped.
`--verbose`	Log the progress of the run, at the info level.

Examples

A rule next to its test file

find-eval.yaml

rules:  - id: find-eval    pattern: eval(...)    message: found eval    languages: [python]    severity: WARNING

find-eval.py

# ruleid: find-evaleval(user_input)# ok: find-evalevaluate(user_input)

Command and result

$ opengrep test .
1/1: ✓ All tests passed
No tests for fixes found.

A failing test

The second eval call is incorrectly annotated ok:.

find-eval.yaml

rules:  - id: find-eval    pattern: eval(...)    message: found eval    languages: [python]    severity: WARNING

find-eval.py

# ruleid: find-evaleval(a)# ok: find-evaleval(b)

Command and result

$ opengrep test .; echo "exit status: $?"
0/1: 1 unit tests did not pass:
--------------------------------------------------------------------------------
	✖ find-eval
	missed lines: [], incorrect lines: [4]
	test file path: <tmp>/find-eval.py


No tests for fixes found.
exit status: 1

opengrep test

## Pairing rules with targets

## Annotations

## Autofix tests

## Exit status

## Flags

## Examples

### A rule next to its test file

### A failing test