Code Review Agent Code Review Intelligent Agent

• 1:
• 2:
• 3:
• 4:
• 5:
• 6:

1 -

Overview

The ask tool answers questions about the PR, based on the PR code changes. Make sure to be specific and clear in your questions. It can be invoked manually by commenting on any PR:

/ask "..."

Example usage

{width=512}

{width=512}

Note that the tool does not have “memory” of previous questions, and answers each question independently.

2 -

Overview

The describe tool scans the PR code changes, and generates a description for the PR - title, type, summary, walkthrough and labels.

The tool can be triggered automatically every time a new PR be created, or it can be invoked manually by commenting on any PR:

/describe

Example usage

Manual triggering

Invoke the tool manually by commenting /describe on any PR:

{width=512}

After ~30 seconds, the tool will generate a description for the PR:

{width=512}

If you want to edit configurations, add the relevant ones to the command:

/describe --pr_description.some_config1=... --pr_description.some_config2=...

Automatic triggering

To run the describe automatically when a PR is opened, define in a configuration file:

[azure_devops_server]
pr_commands = [
    "/describe",
    #"/review --pr_reviewer.num_code_suggestions=0",
    #"/update_changelog"
    #"/improve", #dont add this ,mabey has loop exec bug
]


[gitlab]
url = "https://gitlab.com" # URL to the gitlab service
pr_commands = [
    "/describe --pr_description.final_update_message=false",
    "/review --pr_reviewer.num_code_suggestions=0",
    "/improve",
]
handle_push_trigger = false
push_commands = [
    "/describe",
    "/review --pr_reviewer.num_code_suggestions=0",
]

[pr_description]
publish_labels = ...
...

• The pr_commands lists commands that will be executed automatically when a PR is opened.
• The [pr_description] section contains the configurations for the describe tool you want to edit (if any).

Configuration options

!!! example “Possible configurations”

Markers template

To enable markers, set pr_description.use_description_markers=true. Markers enable to easily integrate user’s content and auto-generated content, with a template-like mechanism.

For example, if the PR original description was:

User content...

## PR Type:
pr_agent:type

## PR Description:
pr_agent:summary

## PR Walkthrough:
pr_agent:walkthrough

The marker pr_agent:type will be replaced with the PR type, pr_agent:summary will be replaced with the PR summary, and pr_agent:walkthrough will be replaced with the PR walkthrough.

{width=512}

→

{width=512}

Configuration params:

• use_description_markers: if set to true, the tool will use markers template. It replaces every marker of the form pr_agent:marker_name with the relevant content. Default is false.
• include_generated_by_header: if set to true, the tool will add a dedicated header: ‘Generated by PR Agent at …’ to any automatic content. Default is true.

Usage Tips

!!! tip “Automation” - When you first install , the default mode for the describe tool is: pr_commands = ["/describe", ...] meaning the describe tool will run automatically when any PR be created, with the default configurations.

• Markers are an alternative way to control the generated description, to give maximal control to the user. If you set:

pr_commands = ["/describe --pr_description.use_description_markers=true", ...]

the tool will replace every marker of the form pr_agent:marker_name in the PR description with the relevant content, where marker_name is one of the following: * type: the PR type. * summary: the PR summary. * walkthrough: the PR walkthrough.

• Note that when markers are enabled, if the original PR description does not contain any markers, the tool will not alter the description at all.

3 -

Overview

The help tool provides a list of all the available tools and their descriptions. For PR-Agent Pro users, it also enables to trigger each tool by checking the relevant box.

It can be invoked manually by commenting on any PR:

/help

Example usage

An example result:

{width=750}

→

{width=750}

4 -

Overview

The improve tool scans the PR code changes, and automatically generates suggestions for improving the PR code. The tool can be triggered automatically every time a new PR is opened, or it can be invoked manually by commenting on any PR:

/improve

Example usage

Manual triggering

Invoke the tool manually by commenting /improve on any PR. The code suggestions by default are presented as a single comment:

![code suggestions as comment]code_suggestions_as_comment.png){width=512}

To edit configurations related to the improve tool, use the following template:

/improve --pr_code_suggestions.some_config1=... --pr_code_suggestions.some_config2=...

For example, you can choose to present the suggestions as commitable code comments, by running the following command:

/improve --pr_code_suggestions.commitable_code_suggestions=true

{width=512}

Note that a single comment has a significantly smaller PR footprint. We recommend this mode for most cases. Also note that collapsible are not supported in Bitbucket. Hence, the suggestions are presented there as code comments.

Automatic triggering

To run the improve automatically when a PR is opened, define in a configuration file:

[github_app]
pr_commands = [
    "/improve",
    ...
]

[pr_code_suggestions]
num_code_suggestions_per_chunk = ...
...

• The pr_commands lists commands that will be executed automatically when a PR is opened.
• The [pr_code_suggestions] section contains the configurations for the improve tool you want to edit (if any)

Extended mode

An extended mode, which does not involve PR Compression and provides more comprehensive suggestions, can be invoked by commenting on any PR by setting:

[pr_code_suggestions]
auto_extended_mode=true

(This mode is true by default).

Note that the extended mode divides the PR code changes into chunks, up to the token limits, where each chunk is handled separately (might use multiple calls to GPT-4 for large PRs). Hence, the total number of suggestions is proportional to the number of chunks, i.e., the size of the PR.

Configuration options

!!! example “General options”

!!! example “params for ’extended’ mode”

Usage Tips

!!! tip “Extra instructions”

Extra instructions are very important for the `imrpove` tool, since they enable you to guide the model to suggestions that are more relevant to the specific needs of the project.

Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify relevant aspects that you want the model to focus on.

Examples for extra instructions:
```
[pr_code_suggestions] # /improve #
extra_instructions="""\
Emphasize the following aspects:
- Does the code logic cover relevant edge cases?
- Is the code logic clear and easy to understand?
- Is the code logic efficient?
...
"""
```
Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

!!! tip “Review vs. Improve tools comparison”

- The [review](./review.md) tool includes a section called 'Possible issues', that also provide feedback on the PR Code.
In this section, the model is instructed to focus **only** on [major bugs and issues](settings/pr_reviewer_prompts.toml#L71).
- The `improve` tool, on the other hand, has a broader mandate, and in addition to bugs and issues, it can also give suggestions for improving code quality and making the code more efficient, readable, and maintainable (see [here](pr_code_suggestions_prompts.toml#L34)).
- Hence, if you are interested only in feedback about clear bugs, the `review` tool might suffice. If you want a more detailed feedback, including broader suggestions for improving the PR code, also enable the `improve` tool to run on each PR.

5 -

Overview

The review tool scans the PR code changes, and automatically generates a PR review. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR:

/review

Example usage

Manual triggering

Invoke the tool manually by commenting /review on any PR:

{width=512}

After ~30 seconds, the tool will generate a review for the PR:

{width=512}

If you want to edit configurations, add the relevant ones to the command:

/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

Automatic triggering

To run the review automatically when a PR is opened, define in a configuration file:

[github_app]
pr_commands = [
    "/review",
    ...
]

[pr_reviewer]
num_code_suggestions = ...
...

• The pr_commands lists commands that will be executed automatically when a PR is opened.
• The [pr_reviewer] section contains the configurations for the review tool you want to edit (if any).

Incremental Mode

Incremental review only considers changes since the last PR-Agent review. This can be useful when working on the PR in an iterative manner, and you want to focus on the changes since the last review instead of reviewing the entire PR again. For invoking the incremental mode, the following command can be used:

/review -i

Note that the incremental mode is only available for GitHub.

{width=512}

PR Reflection

By invoking:

/reflect_and_review

The tool will first ask the author questions about the PR, and will guide the review based on their answers.

Configuration options

!!! example “General options”

!!! example “Enable\disable specific sub-sections”

!!! example “SOC2 ticket compliance 💎”

This sub-tool checks if the PR description properly contains a ticket to a project management system (e.g., Jira, Asana, Trello, etc.), as required by SOC2 compliance. If not, it will add a label to the PR: “Missing SOC2 ticket”.

!!! example “Adding PR labels”

You can enable\disable the review tool to add specific labels to the PR:

!!! example “Auto-approval”

If enabled, the review tool can approve a PR when a specific comment, /review auto_approve, is invoked.

Usage Tips

!!! tip “General guidelines”

The `review` tool provides a collection of possible feedbacks about a PR.
It is recommended to review the [Configuration options](#configuration-options) section, and choose the relevant options for your use case.

Some of the features that are disabled by default are quite useful, and should be considered for enabling. For example: 
`require_score_review`, `require_soc2_ticket`, and more.

On the other hand, if you find one of the enabled features to be irrelevant for your use case, disable it. No default configuration can fit all use cases.

!!! tip “Automation” When you first install PR-Agent app, the default mode for the review tool is: pr_commands = ["/review --pr_reviewer.num_code_suggestions=0", ...] Meaning the review tool will run automatically on every PR, without providing code suggestions. Edit this field to enable/disable the tool, or to change the used configurations.

!!! tip “Code suggestions”

If you set `num_code_suggestions`>0 , the `review` tool will also provide code suggestions.

Notice If you are interested **only** in the code suggestions, it is recommended to use the [`improve`](./improve.md) feature instead, since it is a dedicated only to code suggestions, and usually gives better results.
Use the `review` tool if you want to get more comprehensive feedback, which includes code suggestions as well.

!!! tip “Possible labels from the review tool”

The `review` tool can auto-generate two specific types of labels for a PR:

- a `possible security issue` label that detects if a possible [security issue](settings/pr_reviewer_prompts.toml#L136) exists in the PR code (`enable_review_labels_security` flag)
- a `Review effort [1-5]: x` label, where x is the estimated effort to review the PR (`enable_review_labels_effort` flag)

Both modes are useful, and we recommended to enable them.

!!! tip “Extra instructions”

Extra instructions are important.
The `review` tool can be configured with extra instructions, which can be used to guide the model to a feedback tailored to the needs of your project.

Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify the relevant sub-tool, and the relevant aspects of the PR that you want to emphasize.

Examples for extra instructions:
```
[pr_reviewer]
extra_instructions="""\
In the code feedback section, emphasize the following:
- Does the code logic cover relevant edge cases?
- Is the code logic clear and easy to understand?
- Is the code logic efficient?
...
"""
```
Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

!!! tip “Auto-approval”

PR-Agent can approve a PR when a specific comment is invoked.

To ensure safety, the auto-approval feature is disabled by default. To enable auto-approval, you need to actively set in a pre-defined configuration file the following:
```
[pr_reviewer]
enable_auto_approval = true
```
(this specific flag cannot be set with a command line argument, only in the configuration file, committed to the repository)


After enabling, by commenting on a PR:
```
/review auto_approve
```
PR-Agent will automatically approve the PR, and add a comment with the approval.


You can also enable auto-approval only if the PR meets certain requirements, such as that the `estimated_review_effort` label is equal or below a certain threshold, by adjusting the flag:
```
[pr_reviewer]
maximal_review_effort = 5
```

6 -

Overview

The update_changelog tool automatically updates the CHANGELOG.md file with the PR changes. It can be invoked manually by commenting on any PR:

/update_changelog

Example usage

{width=768}

{width=768}

Configuration options

Under the section pr_update_changelog, the configuration file contains options to customize the ‘update changelog’ tool:

• push_changelog_changes: whether to push the changes to CHANGELOG.md, or just print them. Default is false (print only).
• extra_instructions: Optional extra instructions to the tool. For example: “focus on the changes in the file X. Ignore change in …