Skip to content

Conversation

@ST-DDT
Copy link
Contributor

@ST-DDT ST-DDT commented Oct 25, 2025

Prerequisites checklist

What is the purpose of this pull request? (put an "X" next to an item)

[ ] Documentation update
[ ] Bug fix (template)
[ ] New rule (template)
[ ] Changes an existing rule (template)
[ ] Add autofix to a rule
[ ] Add a CLI option
[x] Add something to the core
[ ] Other, please explain:

Implements #19921, eslint/rfcs/pull/137

What changes did you make? (Give an overview)

Adds new assertionOptions to RuleTester.run to allow for a consistent base level of test assertions.

  • requireMessage: Ensures the message or messageId is validated
  • requireLocation: Ensures that the error location is validated

Is there anything you'd like reviewers to focus on?

@github-project-automation github-project-automation bot moved this to Needs Triage in Triage Oct 25, 2025
@netlify
Copy link

netlify bot commented Oct 25, 2025

Deploy Preview for docs-eslint canceled.

Name Link
🔨 Latest commit 0535874
🔍 Latest deploy log https://app.netlify.com/projects/docs-eslint/deploys/6934070f5c20c300088e9cb0

@eslint-github-bot eslint-github-bot bot added the feature This change adds a new feature to ESLint label Oct 25, 2025
@github-actions github-actions bot added the core Relates to ESLint's core APIs and features label Oct 25, 2025
@ST-DDT ST-DDT marked this pull request as ready for review October 25, 2025 14:15
@ST-DDT ST-DDT requested a review from a team as a code owner October 25, 2025 14:15
@aladdin-add aladdin-add moved this from Needs Triage to Implementing in Triage Oct 25, 2025
@aladdin-add aladdin-add added the accepted There is consensus among the team that this change meets the criteria for inclusion label Oct 25, 2025
@aladdin-add
Copy link
Member

It may conflict with #20135

Copy link
Member

@mdjermanovic mdjermanovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also update the RuleTester documentation?

https://eslint.org/docs/latest/integrate/nodejs-api#ruletester

@mdjermanovic
Copy link
Member

And types:

eslint/lib/types/index.d.ts

Lines 1927 to 1991 in 2584187

// #region RuleTester
export class RuleTester {
static describe: ((...args: any) => any) | null;
static it: ((...args: any) => any) | null;
static itOnly: ((...args: any) => any) | null;
constructor(config?: Linter.Config);
run(
name: string,
rule: RuleDefinition,
tests: {
valid: Array<string | RuleTester.ValidTestCase>;
invalid: RuleTester.InvalidTestCase[];
},
): void;
static only(
item: string | RuleTester.ValidTestCase | RuleTester.InvalidTestCase,
): RuleTester.ValidTestCase | RuleTester.InvalidTestCase;
}
export namespace RuleTester {
interface ValidTestCase {
name?: string;
code: string;
options?: any;
filename?: string | undefined;
only?: boolean;
languageOptions?: Linter.LanguageOptions | undefined;
settings?: { [name: string]: any } | undefined;
before?: () => void;
after?: () => void;
}
interface SuggestionOutput {
messageId?: string;
desc?: string;
data?: Record<string, unknown> | undefined;
output: string;
}
interface InvalidTestCase extends ValidTestCase {
errors: number | Array<TestCaseError | string>;
output?: string | null | undefined;
}
interface TestCaseError {
message?: string | RegExp;
messageId?: string;
/**
* @deprecated `type` is deprecated and will be removed in the next major version.
*/
type?: string | undefined;
data?: any;
line?: number | undefined;
column?: number | undefined;
endLine?: number | undefined;
endColumn?: number | undefined;
suggestions?: SuggestionOutput[] | undefined;
}
}
// #endregion

@lumirlumir lumirlumir linked an issue Oct 27, 2025 that may be closed by this pull request
1 task
@mdjermanovic
Copy link
Member

I think 5ec4b83 should be reverted because requireMessage: true has an effect of disallowing errors: number, so it isn't redundant.

Copy link
Member

@mdjermanovic mdjermanovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall. Just a few small suggestion, then LGTM.

@ST-DDT
Copy link
Contributor Author

ST-DDT commented Dec 5, 2025

After this gets merged, we should probably re-sort/re-group the rule-tester unit tests to align them with the methods in the source file.

e.g.

  • run
    • valid
      • assertValidTestCase
      • testValidTemplate
    • invalid
      • assertInvalidTestCase
        • assertErrorsProperty
      • testInvalidTemplate

Because I found it pretty hard to find the correct location for the new tests in that 6200 lines of code file.

Do you accept those kind of PRs?

Comment on lines +1386 to +1395
if (requireLocation) {
const missingKeys = locationProperties.filter(
key =>
!hasOwnProperty(error, key) &&
hasOwnProperty(message, key),
);
assert.ok(
missingKeys.length === 0,
`Error is missing expected location properties: ${missingKeys.join(", ")}`,
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we move this check to assertErrorsProperty for early feedback?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it requires the test execution to determine the actually required fields.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This wouldn't be possible as properties that don't exist on the actual error message are not mandatory, so we need to run the rule first.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be possible to check only line and column, as those should always be present on the actual error message, but I think splitting this into two checks would be an unnecessary complication, so it's fine to leave this as already implemented in this PR (check all location properties together here).

});
});

describe("messageId", () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test case where we have placeholders in error message when requireMessage : "messageId"?

hasOwnProperty(error, "message"),
`errors[${number}] should specify 'message' (and not 'messageId') when 'assertionOptions.requireMessage' is 'message'.`,
);
} else if (requireMessage === "messageId") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ideally if the value is messageId we should ensure that all the placeholder values are also passed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In eslint/rfcs#137 this was discussed, but it was excluded from the first iteration.
Search for requireData.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When data property is passed, RuleTester already checks if all needed placeholder values are passed in it. Requiring data property to be passed is excluded from the first iteration, as @ST-DDT noted.

Co-authored-by: Nitin Kumar <snitin315@gmail.com>
Copy link
Member

@mdjermanovic mdjermanovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks! Leaving open for a second review.

Copy link
Member

@lumirlumir lumirlumir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Would like one more check from the team before merging.

@lumirlumir lumirlumir moved this from Implementing to Second Review Needed in Triage Dec 7, 2025
@mdjermanovic
Copy link
Member

Merging since there are already two approvals.

@mdjermanovic mdjermanovic merged commit f148a5e into eslint:main Dec 8, 2025
30 checks passed
@github-project-automation github-project-automation bot moved this from Second Review Needed to Complete in Triage Dec 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

accepted There is consensus among the team that this change meets the criteria for inclusion contributor pool core Relates to ESLint's core APIs and features feature This change adds a new feature to ESLint

Projects

Status: Complete

Development

Successfully merging this pull request may close these issues.

Change Request: Add options to rule-tester requiring certain assertions

6 participants