feat: add bench-ai CLI for natural language bench command help#1710
Open
younis-ali wants to merge 3 commits intofrappe:developfrom
Open
feat: add bench-ai CLI for natural language bench command help#1710younis-ali wants to merge 3 commits intofrappe:developfrom
younis-ali wants to merge 3 commits intofrappe:developfrom
Conversation
Add bench-ai console script (OpenAI-compatible API), local context from bench --help and optional Frappe help, suggestion validation, and tests. Signed-off-by: younis-ali <lone.younis1993@gmail.com>
Split build_context and bench_ai into helpers; BENCH_SHELL_PREFIX/BENCH_BINARY; mock get_api_key in test_missing_api_key so CliRunner env does not leak keys. Signed-off-by: younis-ali <lone.younis1993@gmail.com>
Use split-based extraction and a max reply length instead of DOTALL (.*?). Signed-off-by: younis-ali <lone.younis1993@gmail.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Title
feat: add
bench-aiCLI for natural-language bench command helpDescription
Summary
Adds an optional
bench-aiconsole entry point that turns plain-English questions into structured guidance and a suggestedbench commands. It uses the existingrequestsdependency to call an OpenAI-compatible Chat Completions HTTP API. No new runtime dependencies.Context is built from this machine’s
benchClick CLI (full native subcommand paths +bench --helptext) and, when run inside a Frappe bench, the same framework command appendix asbench --help(viaget_frappe_help/get_env_frappe_commands). When not in a bench, a short static framework crib plus validation fallback still allows common commands (e.g.doctor) to be suggested conservatively.Suggested lines are checked against the local CLI where possible;
--runrefuses invalid or placeholder commands. Default behavior is print-only.What changed
bench/commands/ai.pythe CLI, context builder, API client, parsing, validation.pyproject.toml:bench-ai = "bench.commands.ai:main".bench/tests/test_ai_command.py(mocked HTTP, parsing, validation, Click path collection).Configuration
Env varaibles
OPENAI_API_KEYorBENCH_AI_API_KEY: OPEN API keyBENCH_AI_BASE_URL: Defaulthttps://api.openai.com/v1BENCH_AI_MODEL: Defaultgpt-4o-miniHow to test
Dependencies
None added, uses existing
requests(~2.32.x perpyproject.toml).Documentation
README updated with install/setup, flags, and security note (review before
--run).Notes for reviewers
_()sincebenchCLI generally doesn’t i18n user messages the same way as Frappe apps—happy to adjust if project policy requires it.Checklist
unittestmodule (no pytest required)