Skip to content

Consider supporting cargo-nextest #4092

@heaths

Description

@heaths

As we scale the number of crates in the repo, we'll start to hit some scalability issues. Among them is test run time. Anything that runs a bunch of tests - like cargo test --workspace --no-fail-fast - is going to be slow. cargo-nextest can help, but we'd need to solve a couple problems:

  1. It's configuration is centralized. While we can use test patterns to apply defaults for a profile (like a "ci" profile), there's no easy way to build a hierarchy of configuration files like, say, .gitignore).
  2. This runs more tests in parallel, which we could also accomplish with Run tests in parallel #4091; however, in local testing I found it not only breaks a bunch of tests, but has some persistent effect:
    cargo test --no-fail-fast # works
    cargo nextest run --no-fail-fast # a bunch of tests fail - mostly recorded but, IIRC, not all recorded
    cargo test --no-fail-fast # fails, and from a small sample, it's all the same tests as the previous step

We also have to make sure cargo test keeps working because that's idiomatic and rustup nor cargo have any keep way to preinstall third-party dependencies like some languages' package managers do (there are some, but they have to be installed first: chicken/egg problem). We could recommend it in CONTRIBUTING.md, but then use it in our PRs and CIs with profiles as needed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    EngSysThis issue is impacting the engineering system.test-enhancement

    Projects

    Status

    Untriaged

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions