Skip to content

Commit

Permalink
Improving Readme and non-English Language support
Browse files Browse the repository at this point in the history
  • Loading branch information
mageroni committed Jul 27, 2024
1 parent d8bdd82 commit 4c11cec
Show file tree
Hide file tree
Showing 13 changed files with 40 additions and 30 deletions.
6 changes: 4 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
# The ID of your GitHub App
# Required. Details about your GitHub App
APP_ID=
PRIVATE_KEY=
WEBHOOK_SECRET=development
# Optional. Language to be used for the bot. If empty, it will default to English
BOT_LANGUAGE=en

# Use `trace` to get verbose logging or `info` to show less
# Optional. Use `trace` to get verbose logging or `info` to show less
LOG_LEVEL=debug
30 changes: 19 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ As more companies adopt GitHub Copilot, it becomes increasingly important to mea

Quantitative feedback from the Developer at the time of creating a PR provides valuable insights on the time savings experienced by the Developer. Time savings is needed first before other downstream impacts like velocity increases, or other improvements can happen. The level of granularity provides multiple feedback opportunities for Developers and can capture a variety of PRs so we can understand adoption challenges and improvement opportunities. If helpful, the Survey results may also be combined with Key Performance Indicators (KPIs) that the product provides to further contextualize the survey responses.

The survey responses are stored in a file called results.csv in a new branch "copilot-survey-engine-results" to provide insights into how developers are using the tool, the value they report, and the challenges they encounter.
The survey responses are stored in a file called **results.csv** in the **copilot-survey-engine-results** branch to provide insights into how developers are using the tool, the value they report, and the challenges they encounter.

We hope that this project provides value to your organization, and we encourage you to contribute and build upon it. Your contributions can help further enhance the survey capabilities and provide even greater insights into the developer experience with Copilot.

Expand All @@ -18,14 +18,14 @@ The application actively monitors three key events: the closure of a pull reques

### How a survey gets created

When a pull request is closed, the app automatically creates an issue that prompts the user with relevant survey questions. Our application is equipped to handle multiple languages, including English, Spanish, Portuguese, and French. For this, the engine performs a language analysis on the pull request description, matching it with the appropriate language when generating the corresponding issue.
When a pull request is closed, the app automatically creates an issue that prompts the user with relevant survey questions. Our application is equipped to handle multiple languages, including English, Spanish, Portuguese, and French. Selection will be made according to the `BOT_LANGUAGE` value in the .env file.

Note: *If the env file does not contain a Language API Key or Endpoint, the analysis will be skipped and the default language will always be English.*
> **Note**: *If the env file does not contain a value for `BOT_LANGUAGE` the default language will always be English.*
### Sample screenshot of a survey
### Copilot Usage Survey

1. ***Did you use Copilot in developing this PR? (If you select No, just answer question 5***
1. ***Did you use Copilot in developing this PR? (If you select No, just answer question 5)***
- [ ] No
- [ ] Yes

Expand All @@ -36,8 +36,8 @@ Note: *If the env file does not contain a Language API Key or Endpoint, the anal
(Example: The PR would normally take 5 days, but only took 4 days with Copilot then the answer is 20%)
- [ ] 0%
- [ ] > 0% but < 10%
- [ ] > 11% but < 20%
- [ ] > 21% but < 30%
- [ ] 11% but < 20%
- [ ] 21% but < 30%
- [ ] ≥ 31% but < 40%
- [ ] ≥ 41%

Expand All @@ -62,9 +62,9 @@ Note: *If the env file does not contain a Language API Key or Endpoint, the anal
- [ ] Work on other items in the backlog
- [ ] Other. Please explain in the comment

### Where does the app store surveys?
### Where does the app store survey results?

As we receive edits on the issue, the App will validate the responses received (options selected) and once all questions have been answered, the issue will be closed automatically and the responses will be saved into a results.csv file in the same repo in which the issue was created.
As we receive edits on the issue, the App will validate the responses received (options selected) and once all questions have been answered, the issue will be closed automatically and the responses will be saved into a results.csv file in the same repo in which the issue was created on a new branch called *copilot-survey-engine-results*.

## Setup. Deploy on your own environment

Expand All @@ -75,7 +75,7 @@ The web server is the minimum requirement for this app to work and can be hosted

### Step 1. Execute locally and configure secrets

To run the application on you local machine you'll need to have installed NodeJS. Once you have it, you can access to the folder where you've cloned this project and run the following:
To run the application on you local machine you'll need to have installed [NodeJS](https://nodejs.org/). Once you have it, you can access to the folder where you've cloned this project and run the following:

```sh
# Install dependencies
Expand All @@ -85,11 +85,19 @@ npm install
npm start
```

As a first time execution probot will prompt you for creating a new GitHub App or connect it with an existing App. As you complete the requested information, a .env file will get created in your local source code and all the private information regarding your GitHub App will be automatically written there. If you need guidance on how to configure your first GitHub App, please review this guide https://probot.github.io/docs/development/#configuring-a-github-app.
Once the service is running, you will see a listening endpoint in your terminal. Please follow the link, and you will find a screen like the following image.

![Alt text](image-1.png)

As a first time execution probot is prompting you for creating a new GitHub App or connect it with an existing GitHub App. As you complete the requested information, a .env file will get created in your local source code and all the private information regarding your GitHub App will be automatically written there. If you need guidance on how to configure your first GitHub App, please review this guide https://probot.github.io/docs/development/#configuring-a-github-app.

> **Note**: Make sure to delete the `WEBHOOK_PROXY_URL` value from the env file and confirm that all details regarding the GitHub App are correct.
### Step 3. Deploy your App!

Once you have your source code deployed in the web server or hosting platform of your choosing, finally go to your GitHub App and update your webhook URL to reflect your Web Server URL.
Once you have your source code deployed in the web server or hosting platform of your choosing, finally go to your GitHub App General settings and update your webhook URL to reflect your Web Server URL.

![Alt text](image-2.png)

### Step 4. Test your App!

Expand Down
Binary file added image-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added image-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion index.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ module.exports = (app) => {
let pr_number = context.payload.pull_request.number;
let pr_author = context.payload.pull_request.user.login;
let organization_name = context.payload.repository.owner.login;
let detectedLanguage = "en";
let detectedLanguage = process.env.BOT_LANGUAGE ? process.env.BOT_LANGUAGE : "en";

// read file that aligns with detected language
const issue_body = fs.readFileSync(
Expand Down
4 changes: 2 additions & 2 deletions issue_template/copilot-usage-en.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ For Pull Request XXX:
(Example: The PR would normally take 5 days, but only took 4 days with Copilot then the answer is 20%)
- [ ] 0%
- [ ] > 0% but < 10%
- [ ] > 11% but < 20%
- [ ] > 21% but < 30%
- [ ] 11% but < 20%
- [ ] 21% but < 30%
- [ ] ≥ 31% but < 40%
- [ ] ≥ 41%

Expand Down
8 changes: 4 additions & 4 deletions issue_template/copilot-usage-es.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Para el Pull Request XXX:

1. *** ¿Utilizó Copilot para desarrollar este PR? (Si selecciona No, simplemente responda la pregunta 5)***
1. ***¿Utilizó Copilot para desarrollar este PR? (Si selecciona No, simplemente responda la pregunta 5)***
- [ ] No
- [ ] Si

Expand All @@ -11,8 +11,8 @@ Para el Pull Request XXX:
(Ejemplo: el PR normalmente tomaría 5 días, pero solo tomó 4 días con Copilot, entonces la respuesta es 20%)
- [ ] 0%
- [ ] > 0% pero < 10%
- [ ] > 11% pero < 20%
- [ ] > 21% pero < 30%
- [ ] 11% pero < 20%
- [ ] 21% pero < 30%
- [ ] ≥ 31% pero < 40%
- [ ] ≥ 41%

Expand All @@ -22,7 +22,7 @@ Para el Pull Request XXX:

4. ***¿Con qué frecuencia usaste Copilot en este PR?***
- [ ] Todo o la mayor parte del tiempo
- [] Aproximadamente la mitad del tiempo
- [ ] Aproximadamente la mitad del tiempo
- [ ] Algo de tiempo
- [ ] No mucho

Expand Down
4 changes: 2 additions & 2 deletions issue_template/copilot-usage-fr.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ Pour le Pull Request XXX :
(Exemple : le PR prendrait normalement 5 jours, mais n'a pris que 4 jours avec Copilot alors la réponse est 20 %)
- [ ] 0%
- [ ] > 0 % mais < 10 %
- [ ] > 11 % mais < 20 %
- [ ] > 21 % mais < 30 %
- [ ] 11 % mais < 20 %
- [ ] 21 % mais < 30 %
- [ ] ≥ 31 % mais < 40 %
- [ ] ≥ 41%

Expand Down
4 changes: 2 additions & 2 deletions issue_template/copilot-usage-pt.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ Para o Pull Request XXX:
(Exemplo: o PR normalmente levaria 5 dias, mas demorou apenas 4 dias com o Copilot, então a resposta é 20%)
- [ ] 0%
- [ ] > 0% mas < 10%
- [ ] > 11% mas < 20%
- [ ] > 21% mas < 30%
- [ ] 11% mas < 20%
- [ ] 21% mas < 30%
- [ ] ≥ 31% mas < 40%
- [ ] ≥ 41%

Expand Down
4 changes: 2 additions & 2 deletions test/fixtures/issue_body.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ For Pull Request #44:
(Example: The PR would normally take 5 days, but only took 4 days with Copilot then the answer is 20%)
- [ ] 0%
- [ ] > 0% but < 10%
- [ ] > 11% but < 20%
- [ ] > 21% but < 30%
- [ ] 11% but < 20%
- [ ] 21% but < 30%
- [ ] ≥ 31% but < 40%
- [ ] ≥ 41%

Expand Down
2 changes: 1 addition & 1 deletion test/fixtures/issue_comment.created.json
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@
"assignee": null,
"assignees": [],
"author_association": "NONE",
"body": "### Copilot Usage Survey\n\nFor Pull Request #44:\n\n1. ***Did you use Copilot in developing this PR? (If you select No, just answer question 5)***\n- [ ] No\n- [x] Yes\n\n2. Compared to your previous experience coding WITHOUT using Copilot (This number represents 100%) ***How much less time did the coding take during this PR with Copilot?***\n\n(Example: The PR would normally take 5 days, but only took 4 days with Copilot then the answer is 20%)\n- [ ] 0%\n- [ ] > 0% but < 10%\n- [x] > 11% but < 20%\n- [ ] > 21% but < 30%\n- [ ] ≥ 31% but < 40%\n- [ ] ≥ 41%\n\n3. ***Describe your thought process for calculating (or estimating) the time saved in Question 2***\n\n- (Please tell us in a comment)\n\n4. ***How often did you use Copilot in this PR?***\n- [x] All or most of the time\n- [ ] About Half of the time\n- [ ] Some of the time\n- [ ] Not very much\n\n5. ***What other information can you share about Copilot's ability to save you time coding?*** \n\n- (Please tell us in a comment)\n\n6. ***Where did you invest your Copilot Time Savings?***\n- [x] Resolve vulnerabilites\n- [ ] Experiment + Learn and Wellness\n- [ ] Technical debt and refactorization\n- [ ] Work on other items in the backlog\n- [ ] Other. Please explain in the comment",
"body": "### Copilot Usage Survey\n\nFor Pull Request #44:\n\n1. ***Did you use Copilot in developing this PR? (If you select No, just answer question 5)***\n- [ ] No\n- [x] Yes\n\n2. Compared to your previous experience coding WITHOUT using Copilot (This number represents 100%) ***How much less time did the coding take during this PR with Copilot?***\n\n(Example: The PR would normally take 5 days, but only took 4 days with Copilot then the answer is 20%)\n- [ ] 0%\n- [ ] > 0% but < 10%\n- [x] ≥ 11% but < 20%\n- [ ] ≥ 21% but < 30%\n- [ ] ≥ 31% but < 40%\n- [ ] ≥ 41%\n\n3. ***Describe your thought process for calculating (or estimating) the time saved in Question 2***\n\n- (Please tell us in a comment)\n\n4. ***How often did you use Copilot in this PR?***\n- [x] All or most of the time\n- [ ] About Half of the time\n- [ ] Some of the time\n- [ ] Not very much\n\n5. ***What other information can you share about Copilot's ability to save you time coding?*** \n\n- (Please tell us in a comment)\n\n6. ***Where did you invest your Copilot Time Savings?***\n- [x] Resolve vulnerabilites\n- [ ] Experiment + Learn and Wellness\n- [ ] Technical debt and refactorization\n- [ ] Work on other items in the backlog\n- [ ] Other. Please explain in the comment",
"closed_at": null,
"comments": 1,
"comments_url": "https://api.github.com/repos/mageroni/TestRepo/issues/60/comments",
Expand Down
2 changes: 1 addition & 1 deletion test/fixtures/issues.edited.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"assignee": null,
"assignees": [],
"author_association": "NONE",
"body": "### Copilot Usage Survey\n\nFor Pull Request #44:\n\n1. ***Did you use Copilot in developing this PR? (If you select No, just answer question 5)***\n- [ ] No\n- [x] Yes\n\n2. Compared to your previous experience coding WITHOUT using Copilot (This number represents 100%) ***How much less time did the coding take during this PR with Copilot?***\n\n(Example: The PR would normally take 5 days, but only took 4 days with Copilot then the answer is 20%)\n- [ ] 0%\n- [ ] > 0% but < 10%\n- [x] > 11% but < 20%\n- [ ] > 21% but < 30%\n- [ ] ≥ 31% but < 40%\n- [ ] ≥ 41%\n\n3. ***Describe your thought process for calculating (or estimating) the time saved in Question 2***\n\n- (Please tell us in a comment)\n\n4. ***How often did you use Copilot in this PR?***\n- [x] All or most of the time\n- [ ] About Half of the time\n- [ ] Some of the time\n- [x] Not very much\n\n5. ***What other information can you share about Copilot's ability to save you time coding?*** \n\n- (Please tell us in a comment)\n\n6. ***Where did you invest your Copilot Time Savings?***\n- [x] Resolve vulnerabilites\n- [ ] Experiment + Learn and Wellness\n- [ ] Technical debt and refactorization\n- [ ] Work on other items in the backlog\n- [ ] Other. Please explain in the comment",
"body": "### Copilot Usage Survey\n\nFor Pull Request #44:\n\n1. ***Did you use Copilot in developing this PR? (If you select No, just answer question 5)***\n- [ ] No\n- [x] Yes\n\n2. Compared to your previous experience coding WITHOUT using Copilot (This number represents 100%) ***How much less time did the coding take during this PR with Copilot?***\n\n(Example: The PR would normally take 5 days, but only took 4 days with Copilot then the answer is 20%)\n- [ ] 0%\n- [ ] > 0% but < 10%\n- [x] ≥ 11% but < 20%\n- [ ] ≥ 21% but < 30%\n- [ ] ≥ 31% but < 40%\n- [ ] ≥ 41%\n\n3. ***Describe your thought process for calculating (or estimating) the time saved in Question 2***\n\n- (Please tell us in a comment)\n\n4. ***How often did you use Copilot in this PR?***\n- [x] All or most of the time\n- [ ] About Half of the time\n- [ ] Some of the time\n- [x] Not very much\n\n5. ***What other information can you share about Copilot's ability to save you time coding?*** \n\n- (Please tell us in a comment)\n\n6. ***Where did you invest your Copilot Time Savings?***\n- [x] Resolve vulnerabilites\n- [ ] Experiment + Learn and Wellness\n- [ ] Technical debt and refactorization\n- [ ] Work on other items in the backlog\n- [ ] Other. Please explain in the comment",
"closed_at": "2023-03-19T01:12:10Z",
"comments": 0,
"comments_url": "https://api.github.com/repos/mageroni/TestRepo/issues/62/comments",
Expand Down
4 changes: 2 additions & 2 deletions test/index.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -157,8 +157,8 @@ describe("My Probot app", () => {
});

test("updates results.csv record if issue id exists", async () => {
const initialContent = "ZW50ZXJwcmlzZV9uYW1lLG9yZ2FuaXphdGlvbl9uYW1lLHJlcG9zaXRvcnlfbmFtZSxpc3N1ZV9pZCxpc3N1ZV9udW1iZXIsUFJfbnVtYmVyLGFzc2lnbmVlX25hbWUsaXNfY29waWxvdF91c2VkLHNhdmluZ19wZXJjZW50YWdlLHVzYWdlX2ZyZXF1ZW5jeSxzYXZpbmdzX2ludmVzdGVkLGNvbW1lbnQsY3JlYXRlZF9hdCxjb21wbGV0ZWRfYXQKLCxUZXN0UmVwbywyMDAwMDk1NjI0LDE3LDUsbWFnZXJvbmksMSwsLCwyMDIzLTExLTE3VDIzOjUzOjUyWiwyMDIzLTExLTE3VDIzOjU0OjQyWgosbWFnZXJvbmksVGVzdFJlcG8sMTYzMDYzMzg3NSw2Miw0NCwsMSw+IDIxJSBidXQgPCAzMCUsQWxsIG9yIG1vc3Qgb2YgdGhlIHRpbWUsRXhwZXJpbWVudCArIExlYXJuIGFuZCBXZWxsbmVzcywsMjAyMy0wMy0xOFQyMjowMDozNFosMjAyMy0wMy0xOVQyMjowMDozNFo=";
const expectedContent = "ZW50ZXJwcmlzZV9uYW1lLG9yZ2FuaXphdGlvbl9uYW1lLHJlcG9zaXRvcnlfbmFtZSxpc3N1ZV9pZCxpc3N1ZV9udW1iZXIsUFJfbnVtYmVyLGFzc2lnbmVlX25hbWUsaXNfY29waWxvdF91c2VkLHNhdmluZ19wZXJjZW50YWdlLHVzYWdlX2ZyZXF1ZW5jeSxzYXZpbmdzX2ludmVzdGVkLGNvbW1lbnQsY3JlYXRlZF9hdAosLFRlc3RSZXBvLDIwMDAwOTU2MjQsMTcsNSxtYWdlcm9uaSwxLCwsLDIwMjMtMTEtMTdUMjM6NTM6NTJaLDIwMjMtMTEtMTdUMjM6NTQ6NDJaCixtYWdlcm9uaSxUZXN0UmVwbywxNjMwNjMzODc1LDYyLDQ0LCwxLD4gMTElIGJ1dCA8IDIwJSxBbGwgb3IgbW9zdCBvZiB0aGUgdGltZSB8fCBOb3QgdmVyeSBtdWNoLFJlc29sdmUgdnVsbmVyYWJpbGl0ZXMsLDIwMjMtMDMtMThUMjI6MDA6MzRaLDIwMjMtMDMtMTlUMDM6Mzk6MDZaCg==";
const initialContent = "ZW50ZXJwcmlzZV9uYW1lLG9yZ2FuaXphdGlvbl9uYW1lLHJlcG9zaXRvcnlfbmFtZSxpc3N1ZV9pZCxpc3N1ZV9udW1iZXIsUFJfbnVtYmVyLGFzc2lnbmVlX25hbWUsaXNfY29waWxvdF91c2VkLHNhdmluZ19wZXJjZW50YWdlLHVzYWdlX2ZyZXF1ZW5jeSxzYXZpbmdzX2ludmVzdGVkLGNvbW1lbnQsY3JlYXRlZF9hdCxjb21wbGV0ZWRfYXQKLCxUZXN0UmVwbywyMDAwMDk1NjI0LDE3LDUsbWFnZXJvbmksMSwsLCwyMDIzLTExLTE3VDIzOjUzOjUyWiwyMDIzLTExLTE3VDIzOjU0OjQyWgosbWFnZXJvbmksVGVzdFJlcG8sMTYzMDYzMzg3NSw2Miw0NCwsMSziiaUgMjElIGJ1dCA8IDMwJSxBbGwgb3IgbW9zdCBvZiB0aGUgdGltZSxFeHBlcmltZW50ICsgTGVhcm4gYW5kIFdlbGxuZXNzLCwyMDIzLTAzLTE4VDIyOjAwOjM0WiwyMDIzLTAzLTE5VDIyOjAwOjM0Wg==";
const expectedContent = "ZW50ZXJwcmlzZV9uYW1lLG9yZ2FuaXphdGlvbl9uYW1lLHJlcG9zaXRvcnlfbmFtZSxpc3N1ZV9pZCxpc3N1ZV9udW1iZXIsUFJfbnVtYmVyLGFzc2lnbmVlX25hbWUsaXNfY29waWxvdF91c2VkLHNhdmluZ19wZXJjZW50YWdlLHVzYWdlX2ZyZXF1ZW5jeSxzYXZpbmdzX2ludmVzdGVkLGNvbW1lbnQsY3JlYXRlZF9hdAosLFRlc3RSZXBvLDIwMDAwOTU2MjQsMTcsNSxtYWdlcm9uaSwxLCwsLDIwMjMtMTEtMTdUMjM6NTM6NTJaLDIwMjMtMTEtMTdUMjM6NTQ6NDJaCixtYWdlcm9uaSxUZXN0UmVwbywxNjMwNjMzODc1LDYyLDQ0LCwxLOKJpSAxMSUgYnV0IDwgMjAlLEFsbCBvciBtb3N0IG9mIHRoZSB0aW1lIHx8IE5vdCB2ZXJ5IG11Y2gsUmVzb2x2ZSB2dWxuZXJhYmlsaXRlcywsMjAyMy0wMy0xOFQyMjowMDozNFosMjAyMy0wMy0xOVQwMzozOTowNloK";
const mock = nock("https://api.github.com")
// Test that we correctly return a test token
.post("/app/installations/35217443/access_tokens")
Expand Down

0 comments on commit 4c11cec

Please sign in to comment.