Code Review Agent 代码评审智能体

• 1: 启用和配置
• 1.1:
• 2: 指令参考
• 2.1:
• 2.2:
• 2.3:
• 2.4:
• 2.5:
• 2.6:

1 - 启用和配置

服务端配置

要使用 Code Review Agent，管理员首先需要在服务端添加可用的大模型API访问地址，并配置模型访问参数。当前支持的模型包括

• GPT系列模型：支持 gpt-3.5-turbo-16k、gpt-4、gpt-4o，并支持使用 OpenAI 或者 Azure OpenAI 服务。
• DeepSeek系列模型：支持 deepseek api服务、 deepseek v2 16b，并支持使用自建的模型API服务，包括使用Nvidia GPU或者华为晟腾910B等硬件平台。

有关模型部署和推理服务的更多信息，请联系部署实施人员。

配置大模型

默认模型为 gpt-3.5-turbo-16k，常用的模型还包括：gpt-4o 和 deepseek/deepseek-chat ,如果需要使用其他模型，请联系部署实施人员来修改。

# pr_agent\settings\configuration.toml 
[config]
model="gpt-3.5-turbo-16k"
model_turbo="gpt-3.5-turbo-16k"
fallback_models=["gpt-3.5-turbo-16k"]

配置模型访问参数

以下是 在 .secrets.toml 中 Azure OpenAI 的配置示例：

[openai]
key = "<OpenAI API key>"
api_type = "azure" 
api_version = '2023-03-15-preview'  
api_base = "<endpoint url>"  
deployment_id = "<your deployment_id>"  

以下是 在 .secrets.toml 中 deepseek 的配置示例：

[openai]
key = "<model key>" 
api_base = "<model api endpoint>" 

DevOps 平台支持

Code Review Agent 代码评审智能体 支持多种 DevOps 平台，包括 Azure DevOps、GitLab 等。当前我们已经提供对 Azure DevOps 平台的的支持，对于GitLab 平台的支持正在适配中，末未正式支持后我们会补充这部分的文档。

Azure DevOps 配置

目前 Code Review Agent 代码评审智能体 已支持 Azure DevOps Servies和Azure DevOps Server的集成。 在使用前需要先配置好webhook, 目前使用到了两个webhook,一个是创建PR事件和提交PR Comment事件。

另外还需配置启用的 Azure DevOps 机构。

获取访问token

需要配置用于Code Review Agent 调用 Azure DevOps API 的 访问 token（pat）。

Azure DevOps 支持2种第三方服务认证方式，分别为：PAT 令牌 或 DefaultAzureCredential 认证（仅适用于Azure DevOps 在线服务）。

• PAT 创建速度更快，并有内置的到期日期，，PAT需要附属于某个真实用户，需要用户对PAT的过期时间进行小心的维护，否则额很容易出现因为PAT过期而影响使用的情况。
• 使用 DefaultAzureCredential，可以使用托管身份或服务主体，安全性更好，并且会为代理创建单独的 ADO 用户身份（通过 AAD）；但创建过程复杂，而且仅适用于 Azure DevOps 在线服务）。

如果选择了 PAT，您可以在 .secrets.toml 中分配该值。如果选择了 DefaultAzureCredential，您可以直接分配 AZURE_CLIENT_SECRET 等额外的环境变量。

以下是 .secrets.toml 的配置示例：

[azure_devops]
# For Azure devops personal access token
org = "<azure devops org or collection>" 
pat = "<pat>" 

设置服务访问用户和密码

要在 Azure DevOps 中通过 Service Hook 调用 Code Review Agent，首先我们需要配置或者获取 .secrets.toml 中的访问用户名和密码。

以下是 .secrets.toml 文件中的配置示例，您剋打开这个我文件，修复并记录其中的 webhook_username 和 webhook_password，在后续的配置中需要用到。

## in # pr_agent/settings/.secrets.toml 中
[azure_devops_server]
webhook_username = "<basic auth user>" #default value is aisedevpr
webhook_password = "<basic auth password>" #default password is <TO BE Fill>

确保 webhook 端点仅通过 HTTPS 访问，以降低使用基本身份验证时凭据被拦截的风险。

Code Review Agent提供和api支持基本认证和匿名访问（不需要认证）两种模式，前者适用于外网环境，后者适用于内网环境。如果想采用匿名方式（无认证），只里面要把

在 .secrets.toml中，对整个节点注释(或者删除)即可.

# [azure_devops_server]
# webhook_username = "<basic auth user>" 
# webhook_password = "<basic auth password>" 

另外azure devops 的webhook配置那里帐号和密码留空即可，修改完后需要重启 Code Review Agent 服务才能生效。

配置Service Hook

使用Code Review Agent之前需要在Azure DevOps对应的团队项目(Team Project)中配置对应的Web Hook，所需要配置的Web Hook主要有两个。

• Pull Request created: 当PR创建的时候自动触发
• Pull request commented on：当PR上被用户添加了新的评论的时候触发

Web Hooks 配置界面如图：

Web Hook 的具体配置参数如下

• URL：需要只想当前的 /workspace-api/pragent/
• 认证方式中的用户名和密码可以在我们提供的物料中获取

完成以上配置后，Code Review Agent 就可以正常工作了。

1.1 -

Local configuration file

By uploading a local .pr_agent.toml file to the root of the repo’s main branch, you can edit and customize any configuration parameter. Note that you need to upload .pr_agent.toml prior to creating a PR, in order for the configuration to take effect.

For example, if you set in .pr_agent.toml:

[pr_reviewer]
extra_instructions="""\
- instruction a
- instruction b
...
"""

Then you can give a list of extra instructions to the review tool.

Extra instructions

All Code Review Agent tools have a parameter called extra_instructions, that enables to add free-text extra instructions. Example usage:

/update_changelog --pr_update_changelog.extra_instructions="Make sure to update also the version ..."

Working with large PRs

TODO

Changing a model

TODO

Patch Extra Lines

By default, around any change in your PR, git patch provides three lines of context above and below the change.

@@ -12,5 +12,5 @@ def func1():
 code line that already existed in the file...
 code line that already existed in the file...
 code line that already existed in the file....
-code line that was removed in the PR
+new code line added in the PR
 code line that already existed in the file...
 code line that already existed in the file...
 code line that already existed in the file...

For the review, describe, ask and add_docs tools, if the token budget allows, Code Review Agent tries to increase the number of lines of context, via the parameter:

[config]
patch_extra_lines=3

Increasing this number provides more context to the model, but will also increase the token budget. Code Review Agent automatically sets this number to 0, using the original git patch.

Editing the prompts

The prompts for the various Code Review Agent tools are defined in the pr_agent/settings folder. In practice, the prompts are loaded and stored as a standard setting object. Hence, editing them is similar to editing any other configuration value - just place the relevant key in .pr_agent.tomlfile, and override the default value.

For example, if you want to edit the prompts of the describe tool, you can add the following to your .pr_agent.toml file:

[pr_description_prompt]
system="""
...
"""
user="""
...
"""

Note that the new prompt will need to generate an output compatible with the relevant post-process function.

2 - 指令参考

指令参考

当前 Code Review Agent 所支持的指令内容如下，我们持续改进现有指令的生成效果并根据需要持续添加新的指令内容：

• /summary: 生成PR描述，创建PR时会自动生成，也可以手工执行。
• /review：触发代码审查并提供反馈。
• /ask [问题]：提出具体问题并获取回答。
• /update_changelog：自动生成并更新变更日志。
• /generate_labels：自动生成标签，并添加到当前PR中

2.1 -

Overview

The summary(same with describe) tool scans the PR code changes, and generates a description for the PR - title, type, summary, walkthrough and labels.

The tool can be triggered automatically every time a new PR be created, or it can be invoked manually by commenting on any PR:

/describe

Example usage

Manual triggering

Invoke the tool manually by commenting /describe on any PR:

{width=512}

After ~30 seconds, the tool will generate a description for the PR:

{width=512}

If you want to edit configurations, add the relevant ones to the command:

/describe --pr_description.some_config1=... --pr_description.some_config2=...

Automatic triggering

To run the describe automatically when a PR is opened, define in a configuration file:

[azure_devops_server]
pr_commands = [
    "/describe",
    #"/review --pr_reviewer.num_code_suggestions=0",
    #"/update_changelog"
    #"/improve", #dont add this ,mabey has loop exec bug
]


[gitlab]
url = "https://gitlab.com" # URL to the gitlab service
pr_commands = [
    "/describe --pr_description.final_update_message=false",
    "/review --pr_reviewer.num_code_suggestions=0",
    "/improve",
]
handle_push_trigger = false
push_commands = [
    "/describe",
    "/review --pr_reviewer.num_code_suggestions=0",
]

[pr_description]
publish_labels = ...
...

• The pr_commands lists commands that will be executed automatically when a PR is opened.
• The [pr_description] section contains the configurations for the describe tool you want to edit (if any).

Configuration options

!!! example “Possible configurations”

Markers template

To enable markers, set pr_description.use_description_markers=true. Markers enable to easily integrate user’s content and auto-generated content, with a template-like mechanism.

For example, if the PR original description was:

User content...

## PR Type:
pr_agent:type

## PR Description:
pr_agent:summary

## PR Walkthrough:
pr_agent:walkthrough

The marker pr_agent:type will be replaced with the PR type, pr_agent:summary will be replaced with the PR summary, and pr_agent:walkthrough will be replaced with the PR walkthrough.

{width=512}

→

{width=512}

Configuration params:

• use_description_markers: if set to true, the tool will use markers template. It replaces every marker of the form pr_agent:marker_name with the relevant content. Default is false.
• include_generated_by_header: if set to true, the tool will add a dedicated header: ‘Generated by Code Review Agent at …’ to any automatic content. Default is true.

Usage Tips

!!! tip “Automation” - When you first install , the default mode for the describe tool is: pr_commands = ["/describe", ...] meaning the describe tool will run automatically when any PR be created, with the default configurations.

• Markers are an alternative way to control the generated description, to give maximal control to the user. If you set:

pr_commands = ["/describe --pr_description.use_description_markers=true", ...]

the tool will replace every marker of the form pr_agent:marker_name in the PR description with the relevant content, where marker_name is one of the following: * type: the PR type. * summary: the PR summary. * walkthrough: the PR walkthrough.

• Note that when markers are enabled, if the original PR description does not contain any markers, the tool will not alter the description at all.

2.2 -

Overview

The review tool scans the PR code changes, and automatically generates a PR review. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR:

/review

Example usage

Manual triggering

Invoke the tool manually by commenting /review on any PR:

{width=512}

After ~30 seconds, the tool will generate a review for the PR:

{width=512}

If you want to edit configurations, add the relevant ones to the command:

/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

Automatic triggering

To run the review automatically when a PR is opened, define in a configuration file:

[github_app]
pr_commands = [
    "/review",
    ...
]

[pr_reviewer]
num_code_suggestions = ...
...

• The pr_commands lists commands that will be executed automatically when a PR is opened.
• The [pr_reviewer] section contains the configurations for the review tool you want to edit (if any).

Incremental Mode

Incremental review only considers changes since the last Code Review Agent review. This can be useful when working on the PR in an iterative manner, and you want to focus on the changes since the last review instead of reviewing the entire PR again. For invoking the incremental mode, the following command can be used:

/review -i

Note that the incremental mode is only available for GitHub.

{width=512}

PR Reflection

By invoking:

/reflect_and_review

The tool will first ask the author questions about the PR, and will guide the review based on their answers.

Configuration options

!!! example “General options”

!!! example “Enable\disable specific sub-sections”

!!! example “SOC2 ticket compliance 💎”

This sub-tool checks if the PR description properly contains a ticket to a project management system (e.g., Jira, Asana, Trello, etc.), as required by SOC2 compliance. If not, it will add a label to the PR: “Missing SOC2 ticket”.

!!! example “Adding PR labels”

You can enable\disable the review tool to add specific labels to the PR:

!!! example “Auto-approval”

If enabled, the review tool can approve a PR when a specific comment, /review auto_approve, is invoked.

Usage Tips

!!! tip “General guidelines”

The `review` tool provides a collection of possible feedbacks about a PR.
It is recommended to review the [Configuration options](#configuration-options) section, and choose the relevant options for your use case.

Some of the features that are disabled by default are quite useful, and should be considered for enabling. For example: 
`require_score_review`, `require_soc2_ticket`, and more.

On the other hand, if you find one of the enabled features to be irrelevant for your use case, disable it. No default configuration can fit all use cases.

!!! tip “Automation” When you first install Code Review Agent app, the default mode for the review tool is: pr_commands = ["/review --pr_reviewer.num_code_suggestions=0", ...] Meaning the review tool will run automatically on every PR, without providing code suggestions. Edit this field to enable/disable the tool, or to change the used configurations.

!!! tip “Code suggestions”

If you set `num_code_suggestions`>0 , the `review` tool will also provide code suggestions.

Notice If you are interested **only** in the code suggestions, it is recommended to use the [`improve`](./improve.md) feature instead, since it is a dedicated only to code suggestions, and usually gives better results.
Use the `review` tool if you want to get more comprehensive feedback, which includes code suggestions as well.

!!! tip “Possible labels from the review tool”

The `review` tool can auto-generate two specific types of labels for a PR:

- a `possible security issue` label that detects if a possible [security issue](settings/pr_reviewer_prompts.toml#L136) exists in the PR code (`enable_review_labels_security` flag)
- a `Review effort [1-5]: x` label, where x is the estimated effort to review the PR (`enable_review_labels_effort` flag)

Both modes are useful, and we recommended to enable them.

!!! tip “Extra instructions”

Extra instructions are important.
The `review` tool can be configured with extra instructions, which can be used to guide the model to a feedback tailored to the needs of your project.

Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify the relevant sub-tool, and the relevant aspects of the PR that you want to emphasize.

Examples for extra instructions:
```
[pr_reviewer]
extra_instructions="""\
In the code feedback section, emphasize the following:
- Does the code logic cover relevant edge cases?
- Is the code logic clear and easy to understand?
- Is the code logic efficient?
...
"""
```
Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

!!! tip “Auto-approval”

Code Review Agent can approve a PR when a specific comment is invoked.

To ensure safety, the auto-approval feature is disabled by default. To enable auto-approval, you need to actively set in a pre-defined configuration file the following:
```
[pr_reviewer]
enable_auto_approval = true
```
(this specific flag cannot be set with a command line argument, only in the configuration file, committed to the repository)


After enabling, by commenting on a PR:
```
/review auto_approve
```
Code Review Agent will automatically approve the PR, and add a comment with the approval.


You can also enable auto-approval only if the PR meets certain requirements, such as that the `estimated_review_effort` label is equal or below a certain threshold, by adjusting the flag:
```
[pr_reviewer]
maximal_review_effort = 5
```

2.3 -

Overview

The ask tool answers questions about the PR, based on the PR code changes. Make sure to be specific and clear in your questions. It can be invoked manually by commenting on any PR:

/ask "..."

Example usage

{width=512}

{width=512}

Note that the tool does not have “memory” of previous questions, and answers each question independently.

2.4 -

Overview

The update_changelog tool automatically updates the CHANGELOG.md file with the PR changes. It can be invoked manually by commenting on any PR:

/update_changelog

Example usage

{width=768}

{width=768}

Configuration options

Under the section pr_update_changelog, the configuration file contains options to customize the ‘update changelog’ tool:

• push_changelog_changes: whether to push the changes to CHANGELOG.md, or just print them. Default is false (print only).
• extra_instructions: Optional extra instructions to the tool. For example: “focus on the changes in the file X. Ignore change in …

2.5 -

Overview

The improve tool scans the PR code changes, and automatically generates suggestions for improving the PR code. The tool can be triggered automatically every time a new PR is opened, or it can be invoked manually by commenting on any PR:

/improve

Example usage

Manual triggering

Invoke the tool manually by commenting /improve on any PR. The code suggestions by default are presented as a single comment:

![code suggestions as comment]code_suggestions_as_comment.png){width=512}

To edit configurations related to the improve tool, use the following template:

/improve --pr_code_suggestions.some_config1=... --pr_code_suggestions.some_config2=...

For example, you can choose to present the suggestions as commitable code comments, by running the following command:

/improve --pr_code_suggestions.commitable_code_suggestions=true

{width=512}

Note that a single comment has a significantly smaller PR footprint. We recommend this mode for most cases. Also note that collapsible are not supported in Bitbucket. Hence, the suggestions are presented there as code comments.

Automatic triggering

To run the improve automatically when a PR is opened, define in a configuration file:

[github_app]
pr_commands = [
    "/improve",
    ...
]

[pr_code_suggestions]
num_code_suggestions_per_chunk = ...
...

• The pr_commands lists commands that will be executed automatically when a PR is opened.
• The [pr_code_suggestions] section contains the configurations for the improve tool you want to edit (if any)

Extended mode

An extended mode, which does not involve PR Compression and provides more comprehensive suggestions, can be invoked by commenting on any PR by setting:

[pr_code_suggestions]
auto_extended_mode=true

(This mode is true by default).

Note that the extended mode divides the PR code changes into chunks, up to the token limits, where each chunk is handled separately (might use multiple calls to GPT-4 for large PRs). Hence, the total number of suggestions is proportional to the number of chunks, i.e., the size of the PR.

Configuration options

!!! example “General options”

!!! example “params for ’extended’ mode”

Usage Tips

!!! tip “Extra instructions”

Extra instructions are very important for the `imrpove` tool, since they enable you to guide the model to suggestions that are more relevant to the specific needs of the project.

Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify relevant aspects that you want the model to focus on.

Examples for extra instructions:
```
[pr_code_suggestions] # /improve #
extra_instructions="""\
Emphasize the following aspects:
- Does the code logic cover relevant edge cases?
- Is the code logic clear and easy to understand?
- Is the code logic efficient?
...
"""
```
Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

!!! tip “Review vs. Improve tools comparison”

- The [review](./review.md) tool includes a section called 'Possible issues', that also provide feedback on the PR Code.
In this section, the model is instructed to focus **only** on [major bugs and issues](settings/pr_reviewer_prompts.toml#L71).
- The `improve` tool, on the other hand, has a broader mandate, and in addition to bugs and issues, it can also give suggestions for improving code quality and making the code more efficient, readable, and maintainable (see [here](pr_code_suggestions_prompts.toml#L34)).
- Hence, if you are interested only in feedback about clear bugs, the `review` tool might suffice. If you want a more detailed feedback, including broader suggestions for improving the PR code, also enable the `improve` tool to run on each PR.

2.6 -

Overview

The help tool provides a list of all the available tools and their descriptions. For Code Review Agent Pro users, it also enables to trigger each tool by checking the relevant box.

It can be invoked manually by commenting on any PR:

/help

Example usage

An example result:

{width=750}

→

{width=750}