diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS new file mode 100644 index 00000000..5fd7c90a --- /dev/null +++ b/.github/CODEOWNERS @@ -0,0 +1 @@ +* @ansjcy @jainankitk @deshsidd @dzane17 \ No newline at end of file diff --git a/.github/ISSUE_TEMPLATE/BUG_TEMPLATE.md b/.github/ISSUE_TEMPLATE/BUG_TEMPLATE.md new file mode 100644 index 00000000..1d2ea2d2 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/BUG_TEMPLATE.md @@ -0,0 +1,24 @@ +--- +name: 🐛 Bug report +about: Create a report to help us improve +title: '[BUG]' +labels: 'bug, untriaged' +assignees: '' +--- +### What is the bug? +_A clear and concise description of the bug._ + +### How can one reproduce the bug? +_Steps to reproduce the behavior._ + +### What is the expected behavior? +_A clear and concise description of what you expected to happen._ + +### What is your host/environment? +_Operating system, version._ + +### Do you have any screenshots? +_If applicable, add screenshots to help explain your problem._ + +### Do you have any additional context? +_Add any other context about the problem._ diff --git a/.github/ISSUE_TEMPLATE/FEATURE_REQUEST_TEMPLATE.md b/.github/ISSUE_TEMPLATE/FEATURE_REQUEST_TEMPLATE.md new file mode 100644 index 00000000..55b3ddf3 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/FEATURE_REQUEST_TEMPLATE.md @@ -0,0 +1,18 @@ +--- +name: 🎆 Feature request +about: Request a feature in this project +title: '[FEATURE]' +labels: 'enhancement, untriaged' +assignees: '' +--- +### Is your feature request related to a problem? +_A clear and concise description of what the problem is, e.g. I'm always frustrated when [...]._ + +### What solution would you like? +_A clear and concise description of what you want to happen._ + +### What alternatives have you considered? +_A clear and concise description of any alternative solutions or features you've considered._ + +### Do you have any additional context? +_Add any other context or screenshots about the feature request here._ diff --git a/.github/ISSUE_TEMPLATE/PROPOSAL_TEMPLATE.md b/.github/ISSUE_TEMPLATE/PROPOSAL_TEMPLATE.md new file mode 100644 index 00000000..9276d050 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/PROPOSAL_TEMPLATE.md @@ -0,0 +1,40 @@ +--- +name: 💭 Proposal +about: Suggest an idea for a specific feature you wish to propose to the community for comment +title: '[PROPOSAL]' +labels: proposal +assignees: '' +--- +## What/Why +### What are you proposing? +_In a few sentences, describe the feature and its core capabilities._ + +### What users have asked for this feature? +_Highlight any research, proposals, requests or anecdotes that signal this is the right thing to build. Include links to GitHub Issues, Forums, Stack Overflow, Twitter, Etc_ + +### What problems are you trying to solve? +_Summarize the core use cases and user problems and needs you are trying to solve. Describe the most important user needs, pain points and jobs as expressed by the user asks above. Template: When \ , a \ wants to \, so they can \. (Example: When **searching by postal code**, **a buyer** wants to **be required to enter a valid code** so they **don’t waste time searching for a clearly invalid postal code.**)_ + +### What is the developer experience going to be? +_Does this have a REST API? If so, please describe the API and any impact it may have to existing APIs. In a brief summary (not a spec), highlight what new REST APIs or changes to REST APIs are planned. as well as any other API, CLI or Configuration changes that are planned as part of this feature._ + +#### Are there any security considerations? +_Describe if the feature has any security considerations or impact. What is the security model of the new APIs? Features should be integrated into the OpenSearch security suite and so if they are not, we should highlight the reasons here._ + +#### Are there any breaking changes to the API +_If this feature will require breaking changes to any APIs, ouline what those are and why they are needed. What is the path to minimizing impact? (example, add new API and deprecate the old one)_ + +### What is the user experience going to be? +_Describe the feature requirements and or user stories. You may include low-fidelity sketches, wireframes, APIs stubs, or other examples of how a user would use the feature via CLI, OpenSearch Dashboards, REST API, etc. Using a bulleted list or simple diagrams to outline features is okay. If this is net new functionality, call this out as well._ + +#### Are there breaking changes to the User Experience? +_Will this change the existing user experience? Will this be a breaking change from a user flow or user experience perspective?_ + +### Why should it be built? Any reason not to? +_Describe the value that this feature will bring to the OpenSearch community, as well as what impact it has if it isn't built, or new risks if it is. Highlight opportunities for additional research._ + +### What will it take to execute? +_Describe what it will take to build this feature. Are there any assumptions you may be making that could limit scope or add limitations? Are there performance, cost, or technical constraints that may impact the user experience? Does this feature depend on other feature work? What additional risks are there?_ + +### Any remaining open questions? +_What are known enhancements to this feature? Any enhancements that may be out of scope but that we will want to track long term? List any other open questions that may need to be answered before proceeding with an implementation._ diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 00000000..67f6e2a3 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,7 @@ +contact_links: + - name: OpenSearch Community Support + url: https://discuss.opendistrocommunity.dev/ + about: Please ask and answer questions here. + - name: AWS/Amazon Security + url: https://aws.amazon.com/security/vulnerability-reporting/ + about: Please report security vulnerabilities here. diff --git a/.github/workflows/add-untriaged.yml b/.github/workflows/add-untriaged.yml new file mode 100644 index 00000000..9dcc7020 --- /dev/null +++ b/.github/workflows/add-untriaged.yml @@ -0,0 +1,19 @@ +name: Apply 'untriaged' label during issue lifecycle + +on: + issues: + types: [opened, reopened, transferred] + +jobs: + apply-label: + runs-on: ubuntu-latest + steps: + - uses: actions/github-script@v6 + with: + script: | + github.rest.issues.addLabels({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + labels: ['untriaged'] + }) diff --git a/.github/workflows/bake_time.yml b/.github/workflows/bake_time.yml new file mode 100644 index 00000000..6c0f3379 --- /dev/null +++ b/.github/workflows/bake_time.yml @@ -0,0 +1,20 @@ +name: Bake time + +on: + pull_request: + types: + - opened + - synchronize + workflow_dispatch: + schedule: + - cron: '0 */1 * * *' # Runs every 1 hour + +jobs: + baking_pull_request: + name: "Baking pull request..." + runs-on: ubuntu-latest + steps: + - uses: peternied/bake-time@v3 + with: + check-name: "Baking pull request..." + delay-hours: 48 diff --git a/.gitignore b/.gitignore new file mode 100644 index 00000000..e69de29b diff --git a/ADMINS.md b/ADMINS.md new file mode 100644 index 00000000..dc748e8f --- /dev/null +++ b/ADMINS.md @@ -0,0 +1,59 @@ +- [Overview](#overview) +- [Current Admins](#current-admins) +- [Admin Permissions](#admin-permissions) + - [Prioritize Security](#prioritize-security) + - [Enforce Code of Conduct](#enforce-code-of-conduct) + - [Add/Remove Maintainers](#addremove-maintainers) + - [Adopt Organizational Practices](#adopt-organizational-practices) +- [New Repos](#new-repos) + +## Overview + +This document explains who the admins are (see below), what they do in opensearch-project, and how they should be doing it. These individuals are members of an "admin" GitHub team that is given Admin-level permissions to every repository in opensearch-project organization. These are individuals that worked on creating the OpenSearch fork, and those that currently support the organization-wide infrastructure, such as the public [CI/CD](https://build.ci.opensearch.org/). + +If you're interested in becoming a maintainer, see [MAINTAINERS](MAINTAINERS.md). If you're interested in contributing, see [CONTRIBUTING](CONTRIBUTING.md). + +## Current Admins + +| Admin | GitHub ID | Affiliation | +|--------------------| --------------------------------------------------- | ----------- | +| Daniel Doubrovkine | [dblock](https://github.com/dblock) | Amazon | +| Rishabh Singh | [rishabh6788](https://github.com/rishabh6788) | Amazon | +| Prudhvi Godithi | [prudhvigodithi](https://github.com/prudhvigodithi) | Amazon | +| Sayali Gaikawad | [gaiksaya](https://github.com/gaiksaya) | Amazon | +| Peter Zhu | [peterzhuamazon](https://github.com/peterzhuamazon) | Amazon | +| Zelin Hao | [zelinh](https://github.com/CEHENKLE) | Amazon | + +## Admin Permissions + +Admins have [admin-level permissions on a repository](https://docs.github.com/en/organizations/managing-access-to-your-organizations-repositories/repository-permission-levels-for-an-organization). Use those privileges to serve the community and protect the repository as follows. + +### Prioritize Security + +Security is your number one priority. Manage security keys and safeguard access to the repository. + +Note that this repository is monitored and supported 24/7 by Amazon Security, see [Reporting a Vulnerability](SECURITY.md) for details. + +### Enforce Code of Conduct + +Act on [CODE_OF_CONDUCT](CODE_OF_CONDUCT.md) violations by revoking access, and blocking malicious actors. + +### Add/Remove Maintainers + +Perform administrative tasks, such as [adding](RESPONSIBILITIES.md#adding-a-new-maintainer) and [removing maintainers](RESPONSIBILITIES.md#removing-a-maintainer). + +Please note that maintainers typically do not have admin-level permissions in their repos in this organization. Admin-level permissions allow for sensitive and destructive actions, such as managing security, or deleting a repository. Therefore, admin access in opensearch-project was designed to be deliberately centralized in ways that requires that two people to make any one sensitive change. If you need to perform an admin function, such as adding or removing a maintainer, please [follow the maintainer nomination process](RESPONSIBILITIES.md#becoming-a-maintainer), then ask to effect permissions by tagging `@admin` in your pull request, or the [#maintainers channel on the public Slack](https://opensearch.slack.com/archives/C05L60S4UBT). One of the above-mentioned admins will make this change for you. + +### Adopt Organizational Practices + +Adopt organizational practices documented in this repo, work in the open, and collaborate with other admins by opening issues before making process changes. Prefer consistency, and avoid diverging from practices in the opensearch-project rganization. + +## New Repos + +There are currently two ways new repositories may appear in the opensearch-project organization: creating a new repo and adopting, or moving a repo from outside of the organization into it. The process is the same. + +The AWS Open Source Program Office (OSPO) currently owns and manages the opensearch-project organization, and has permissions to create a new repo. While the admins above have admin-level permissions, they do not have permissions to create new repositories or move repositories into the organization. + +All new repositories inside opensearch-project follow the [security response process](SECURITY.md), and therefore require an Amazon team to be engaged when necessary. If you wish to create a repository in this organization, or move a repository into opensearch-project, please contact one of the above-mentioned admins via the [#maintainers channel on the public Slack](https://opensearch.slack.com/archives/C05L60S4UBT). + +While the opensearch-project organization has adopted [opensearch-plugin-template-java](https://github.com/opensearch-project/opensearch-plugin-template-java), or [opensearch-project/opensearch-learning-to-rank-base](https://github.com/opensearch-project/opensearch-learning-to-rank-base) in the past, we generally encourage you to start and run with your open-source project outside of the organization, and only consider making it part of it when you wish to include your already very popular component or tool in the "official" distribution. diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index 5b627cfa..cb235477 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -1,4 +1,24 @@ -## Code of Conduct -This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). -For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact -opensource-codeofconduct@amazon.com with any additional questions or comments. + +This code of conduct applies to all spaces provided by the OpenSource project including in code, documentation, issue trackers, mailing lists, chat channels, wikis, blogs, social media, events, conferences, meetings, and any other communication channels used by the project. + +**Our open source communities endeavor to:** + +* Be Inclusive: We are committed to being a community where everyone can join and contribute. This means using inclusive and welcoming language. +* Be Welcoming: We are committed to maintaining a safe space for everyone to be able to contribute. +* Be Respectful: We are committed to encouraging differing viewpoints, accepting constructive criticism and work collaboratively towards decisions that help the project grow. Disrespectful and unacceptable behavior will not be tolerated. +* Be Collaborative: We are committed to supporting what is best for our community and users. When we build anything for the benefit of the project, we should document the work we do and communicate to others on how this affects their work. + +**Our Responsibility. As contributors, members, or bystanders we each individually have the responsibility to behave professionally and respectfully at all times. Disrespectful and unacceptable behaviors include, but are not limited to:** + +* The use of violent threats, abusive, discriminatory, or derogatory language; +* Offensive comments related to gender, gender identity and expression, sexual orientation, disability, mental illness, race, political or religious affiliation; +* Posting of sexually explicit or violent content; +* The use of sexualized language and unwelcome sexual attention or advances; +* Public or private harassment of any kind; +* Publishing private information, such as physical or electronic address, without permission; +* Other conduct which could reasonably be considered inappropriate in a professional setting; +* Advocating for or encouraging any of the above behaviors. + +**Enforcement and Reporting Code of Conduct Issues:** + +Instances of abusive, harassing, or otherwise unacceptable behavior may be reported. [Contact us](mailto:opensource-codeofconduct@amazon.com). All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. diff --git a/COMMUNICATIONS.md b/COMMUNICATIONS.md new file mode 100644 index 00000000..0617056e --- /dev/null +++ b/COMMUNICATIONS.md @@ -0,0 +1,54 @@ +# OpenSearch Project Communication + +- [Overview](#overview) +- [Slack](#slack) + - [Getting Started](#getting-started) + - [Workspace Channels](#workspace-channels) + - [Tips](#tips) + +## Overview + +The purpose of this document is to provide information regarding the communication channels for the OpenSearch Project. All communication is subject to the [OpenSearch Code of Conduct](CODE_OF_CONDUCT.md). Please see [CONTRIBUTING](CONTRIBUTING.md) if you're interested in contributing to the project. + +## Slack + +OpenSearch has a workspace on [Slack](https://opensearch.slack.com) to provide open communication channels for anyone interested in OpenSearch. + +The following guidelines include a Getting Started for steps to register and setup the workspace along with tips for using the workspace. Please read through these guidelines carefully and thoroughly. + +### Getting Started + +_Join Workspace_ + + * Visit the [slack landing page](https://opensearch.org/slack.html) on OpenSearch.org for information on registering your account. You will be required to enroll in two-factor authentication to ensure the best possible security of your account. + +_Update Profile_ + + * You may add your interests or keywords in the “What I do” section of your profile along with your Title and Company/Organization. This helps participants network and connect with each other by common interests. + * Feel free to update your profile photo to something unique to you. All profile photos are subject to the [OpenSearch Code of Conduct](CODE_OF_CONDUCT.md) policies. + * Optionally share your pronouns by adding them to the end of your Full Name in your profile. This way other participants will be able to see them and be respectful to your communication preferences. Please also be respectful of other participants’ pronouns. + +### Workspace Channels + +After you join OpenSearch workspace you will be automatically added to the following channels: + + * **#announcements** - Read only channel for overall project announcements. This may include critical security issues, releases, milestones or general information related to the OpenSearch Project community. + * **#general** - a channel for asking general questions. Based on the question or discussion users will mostly likely be directed to more specific channels based on the topics of interest. + * **#random** - consider this the water cooler in a face to face workspace. Feel free to introduce yourself to the community, share what you’re working on, or what you’re excited about learning with OpenSearch. + +There are dedicated channels for the following (note that participants might be directed to more specific channels based on the nature of the discussion): + + * **#admin-requests** - slack related feature requests and questions for the administration team. + * **#dev** - primary channel for development questions. This is a good place to start for participants interested in OpenSearch development but are not sure which channels to join. + * **#core** - primary channel for core OpenSearch. General questions and discussions about the core can be asked here. + * **#dashboards** - primary channel for Dashboards. General questions and discussions about dashboards can be asked here. + * **#infra** - primary channel for infra. General discussions, questions, or infra requests can be posted here. + +### Tips + + * Try to keep communications in the open - not only does this facilitate better sharing of information but back channeled discussions are not retained for search among other community members. + * Slack conversations are not searchable outside the workspace. For this reason we encourage using the [OpenSearch Discussion Forum](https://forum.opensearch.org) for technical support discussions or, at least, open a summary discussion on the forum so useful findings can be shared with the rest of the community. + * Keep discussions organized - use threads to respond to comments inline instead of posting to the entire channel. + * Expect asynchronous response time - do not assume immediate response time, or possibly any response at all. This is a community driven communication tool with participants in different time zones. The workspace should not be treated as a formal support or training tool. + * Protecting IP and legally protected information is your responsibility, this is a public forum. Don’t presume anything said here will remain private. + * **DO NOT** post CVEs or Security issues publicly. See [SECURITY](SECURITY.md) for guidelines on handling security issues. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c4b6a1c5..6f1a55ff 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,59 +1,174 @@ -# Contributing Guidelines +- [Contributing to OpenSearch](#contributing-to-opensearch) +- [First Things First](#first-things-first) +- [Ways to Contribute](#ways-to-contribute) + - [Bug Reports](#bug-reports) + - [Feature Requests \& Proposals](#feature-requests--proposals) + - [Documentation Changes](#documentation-changes) + - [Contributing Code](#contributing-code) +- [Developer Certificate of Origin](#developer-certificate-of-origin) + - [Troubleshooting Failed DCO Checks](#troubleshooting-failed-dco-checks) +- [License Headers](#license-headers) + - [Java](#java) + - [Python, Ruby, Shell](#python-ruby-shell) +- [Review Process](#review-process) +- [Using Feature Branches](#using-feature-branches) +- [Experimental Features](#experimental-features) -Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional -documentation, we greatly value feedback and contributions from our community. +## Contributing to OpenSearch -Please read through this document before submitting any issues or pull requests to ensure we have all the necessary -information to effectively respond to your bug report or contribution. +OpenSearch is a community project that is built and maintained by people just like **you**. We're glad you're interested in helping out. There are several different ways you can do it, but before we talk about that, let's talk about how to get started. +## First Things First -## Reporting Bugs/Feature Requests +1. **When in doubt, open an issue** - For almost any type of contribution, the first step is opening an issue. Even if you think you already know what the solution is, writing down a description of the problem you're trying to solve will help everyone get context when they review your pull request. If it's truly a trivial change (e.g. spelling error), you can skip this step -- but as the subject says, when it doubt, [open an issue](../../issues). -We welcome you to use the GitHub issue tracker to report bugs or suggest features. +2. **Only submit your own work** (or work you have sufficient rights to submit) - Please make sure that any code or documentation you submit is your work or you have the rights to submit. We respect the intellectual property rights of others, and as part of contributing, we'll ask you to sign your contribution with a "Developer Certificate of Origin" (DCO) that states you have the rights to submit this work and you understand we'll use your contribution. There's more information about this topic in the [DCO section](#developer-certificate-of-origin). -When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already -reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: +3. When you're ready to start, we have [a step-by-step onboarding guide](ONBOARDING.md) to help you get oriented. -* A reproducible test case or series of steps -* The version of our code being used -* Any modifications you've made relevant to the bug -* Anything unusual about your environment or deployment +## Ways to Contribute +### Bug Reports -## Contributing via Pull Requests -Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: +Ugh! Bugs! -1. You are working against the latest source on the *main* branch. -2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. -3. You open an issue to discuss any significant work - we would hate for your time to be wasted. +A bug is when software behaves in a way that you didn't expect and the developer didn't intend. To help us understand what's going on, we first want to make sure you're working from the latest version. -To send us a pull request, please: +Once you've confirmed that the bug still exists in the latest version, you'll want to check to make sure it's not something we already know about on the [open issues GitHub page](../../issues). -1. Fork the repository. -2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. -3. Ensure local tests pass. -4. Commit to your fork using clear commit messages. -5. Send us a pull request, answering any default questions in the pull request interface. -6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. +If you've upgraded to the latest version and you can't find it in our open issues list, then you'll need to tell us how to reproduce it Provide as much information as you can. You may think that the problem lies with your query, when actually it depends on how your data is indexed. The easier it is for us to recreate your problem, the faster it is likely to be fixed. -GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and -[creating a pull request](https://help.github.com/articles/creating-a-pull-request/). +### Feature Requests & Proposals +If you've thought of a way that OpenSearch could be better, we want to hear about it. We track `feature requests` ([examples](https://github.com/search?q=org%3Aopensearch-project+%22Is+your+feature+request+related+to+a+problem%3F%22&type=Issues)) using GitHub, so please feel free to open an issue which describes the feature you would like to see, why you need it, and how it should work. If you would like contribute code toward building it, you might consider a `feature proposal` ([examples](https://github.com/search?q=org%3Aopensearch-project+%22How+did+you+come+up+with+this+proposal%3F%22&type=Issues)) instead. A feature proposal is the first step to helping the community better understand what you are planning to contribute, why it should be built, and collaborate on ensuring you have all the data points you need for implementation. -## Finding contributions to work on -Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. +### Documentation Changes +There are two types of documentation in OpenSearch: developer documentation, which describes how OpenSearch is designed internally, and user documentation, which describes how to use OpenSearch. -## Code of Conduct -This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). -For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact -opensource-codeofconduct@amazon.com with any additional questions or comments. +Developer documentation is maintained in the repository to which it pertains. The workflow for contributing developer documentation is the same as the one for contributing code. +User documentation is maintained in the [documentation-website](https://github.com/opensearch-project/documentation-website/) repo. You can find the rendered documentation at [opensearch.org/docs](https://opensearch.org/docs). To learn how to contribute user documentation, see the [contribution guidelines](https://github.com/opensearch-project/documentation-website/blob/main/CONTRIBUTING.md). -## Security issue notifications -If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. +### Contributing Code +As with other types of contributions, the first step is to [open an issue on GitHub](../../issues/new/choose). Opening an issue before you make changes makes sure that someone else isn't already working on that particular problem. It also lets us all work together to find the right approach before you spend a bunch of time on a PR. So again, when in doubt, open an issue. -## Licensing +## Developer Certificate of Origin -See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. +OpenSearch is an open source product released under the Apache 2.0 license (see either [the Apache site](https://www.apache.org/licenses/LICENSE-2.0) or the [LICENSE.txt file](LICENSE.txt)). The Apache 2.0 license allows you to freely use, modify, distribute, and sell your own products that include Apache 2.0 licensed software. + +We respect intellectual property rights of others and we want to make sure all incoming contributions are correctly attributed and licensed. A Developer Certificate of Origin (DCO) is a lightweight mechanism to do that. + +The DCO is a declaration attached to every contribution made by every developer. In the commit message of the contribution, the developer simply adds a `Signed-off-by` statement and thereby agrees to the DCO, which you can find below or at [DeveloperCertificate.org](http://developercertificate.org/). + +``` +Developer's Certificate of Origin 1.1 + +By making a contribution to this project, I certify that: + +(a) The contribution was created in whole or in part by me and I + have the right to submit it under the open source license + indicated in the file; or + +(b) The contribution is based upon previous work that, to the + best of my knowledge, is covered under an appropriate open + source license and I have the right under that license to + submit that work with modifications, whether created in whole + or in part by me, under the same open source license (unless + I am permitted to submit under a different license), as + Indicated in the file; or + +(c) The contribution was provided directly to me by some other + person who certified (a), (b) or (c) and I have not modified + it. + +(d) I understand and agree that this project and the contribution + are public and that a record of the contribution (including + all personal information I submit with it, including my + sign-off) is maintained indefinitely and may be redistributed + consistent with this project or the open source license(s) + involved. + ``` + +We require that every contribution to OpenSearch is signed with a Developer Certificate of Origin. DCO checks are enabled via a [DCO workflow app](https://github.com/apps/dco) across the entire opensearch-project organization, and your PR will fail CI without it. + +Additionally, we kindly ask you to use your real name. We do not accept anonymous contributors, nor those utilizing pseudonyms. + +Each commit must include a DCO which looks like this, which includes a real name and a valid email address where you can receive emails: + +``` +Signed-off-by: Jane Smith +``` + +You may type this line on your own when writing your commit messages. However, if your `user.name` and `user.email` are set in your `git config`, you can use `-s` or `--signoff` to add the `Signed-off-by` line to the end of the commit message automatically. + +Forgot to add DCO to a commit? Amend it with `git commit --amend -s`. + +### Troubleshooting Failed DCO Checks + +The DCO workflow app requires signatures on every commit that's part of a PR. + +If you've already signed all your commits and your PR still fails the DCO check, it's likely because the email address you signed the commit with doesn't match the one on your GitHub account. + +To fix: + +1. Go to [your GitHub email settings](https://github.com/settings/emails) and uncheck "Keep my email address private". +2. Following the [GitHub documentation](https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-personal-account-on-github/managing-email-preferences/setting-your-commit-email-address#setting-your-commit-email-address-in-git), set your commit email address to your primary GitHub email address. + +## License Headers + +New files in your code contributions should contain the following license header. If you are modifying existing files with license headers, or including new files that already have license headers, do not remove or modify them without guidance. + +### Java + +``` +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * +*/ +``` + +### Python, Ruby, Shell +``` +# Copyright OpenSearch Contributors +# SPDX-License-Identifier: Apache-2.0 +# +# The OpenSearch Contributors require contributions made to +# this file be licensed under the Apache-2.0 license or a +# compatible open source license. +``` + +## Review Process + +We deeply appreciate everyone who takes the time to make a contribution. We will review all contributions as quickly as possible. As a reminder, [opening an issue](../../issues/new/choose) discussing your change before you make it is the best way to smooth the PR process. This will prevent a rejection because someone else is already working on the problem, or because the solution is incompatible with the architectural direction. + +During the PR process, expect that there will be some back-and-forth. Please try to respond to comments in a timely fashion, and if you don't wish to continue with the PR, let us know. If a PR takes too many iterations for its complexity or size, we may reject it. Additionally, if you stop responding we may close the PR as abandoned. In either case, if you feel this was done in error, please add a comment on the PR. + +If we accept the PR, a [maintainer](MAINTAINERS.md) will merge your change and usually take care of backporting it to appropriate branches ourselves. + +If we reject the PR, we will close the pull request with a comment explaining why. This decision isn't always final: if you feel we have misunderstood your intended change or otherwise think that we should reconsider then please continue the conversation with a comment on the PR and we'll do our best to address any further points you raise. + +## Using Feature Branches + +Our recommended approach for development is doing frequent small PR merges to main. This lets us catch integration issues earlier, makes it easier to review your PRs and makes your development visible to everyone. It's okay if it's not the complete feature, as long as the PR won’t break a build or any existing functionality. + +But sometimes it may be useful to create a feature branch. This allows you work on long-running disruptive features in isolation. The reason we don't recommend it is because it still requires maintainer access to merge changes, and the overhead of rebasing is high. If you do want to use a feature branch: + +1. Treat feature branch PR's the same as PR's to `main`. CI should run on all PR's, no incomplete work should be merged, tests are necessary, etc. + a. This maintains the code quality going into each feature making the integration to main PR's much easier and quicker. + b. More visibility during development since it gives reviewers the necessary time to review each PR. +2. Please use Feature specific labels: This helps identify feature related issues and PR's. + +All the safeguard here are not rules but guidelines and should be adopted by each repository based on their specific requirements. This is to ensure that feature branch development is less likely to have code quality issues and massive merge to main PR's. + +For contributors looking to add a new feature that would require the creation of a feature branch, the process begins by opening an issue in the repo with the feature proposal. Depending on the nature of the feature, the maintainers of the project can decide to create a feature branch and use this model. + +## Experimental Features + +Experimental releases are used to gather early feedback on features. Features should only be marked experimental if there is a high likelihood that API or experience will change. We strongly advise people to avoid using experimental features in production as there is no guarantee the API or experience won't change before release. It is best to avoid experimental feature releases unless it is necessary. Our goal is to have all experimental features into production or removed within 2 minor releases. We generally use [Feature Flags](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#experimental-development) to isolate experimental features in backend code. diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md new file mode 100644 index 00000000..e69de29b diff --git a/MAINTAINERS.md b/MAINTAINERS.md new file mode 100644 index 00000000..6c1b8239 --- /dev/null +++ b/MAINTAINERS.md @@ -0,0 +1,21 @@ +- [Overview](#overview) +- [Current Maintainers](#current-maintainers) +- [Emeritus](#emeritus) + +## Overview + +This document contains a list of maintainers in this repo. See [opensearch-project/.github/RESPONSIBILITIES.md](https://github.com/opensearch-project/.github/blob/main/RESPONSIBILITIES.md#maintainer-responsibilities) that explains what the role of maintainer means, what maintainers do in this and other repos, and how they should be doing it. If you're interested in contributing, and becoming a maintainer, see [CONTRIBUTING](CONTRIBUTING.md). + +## Current Maintainers + +| Maintainer | GitHub ID | Affiliation | +| ------------------ | --------------------------------------------------------- | ----------- | +| Chenyang Ji | [ansjcy](https://github.com/ansjcy) | Amazon | +| Ankit Jain | [jainankitk](https://github.com/jainankitk) | Amazon | +| Siddhant Deshmukh | [deshsidd](https://github.com/deshsidd) | Amazon | +| David Zane | [dzane17](https://github.com/dzane17) | Amazon | + +## Emeritus + +| Maintainer | GitHub ID | Affiliation | +| ------------------ | --------------------------------------------------------- | ----------- | diff --git a/README.md b/README.md index 847260ca..08fe5373 100644 --- a/README.md +++ b/README.md @@ -1,17 +1,42 @@ -## My Project +- [Welcome!](#welcome) +- [Project Resources](#project-resources) +- [Project Style Guidelines](#project-style-guidelines) +- [Code of Conduct](#code-of-conduct) +- [License](#license) +- [Copyright](#copyright) -TODO: Fill this README out! +## Welcome! -Be sure to: +**OpenSearch** is [a community-driven, open source fork](https://aws.amazon.com/blogs/opensource/introducing-opensearch/) of [Elasticsearch](https://en.wikipedia.org/wiki/Elasticsearch) and [Kibana](https://en.wikipedia.org/wiki/Kibana) licensed under the [Apache v2.0 License](LICENSE.txt). For more information, see [opensearch.org](https://opensearch.org/). -* Change the title in this README -* Edit your repository description on GitHub +## Project Resources -## Security +* [Project Website](https://opensearch.org/) +* [Downloads](https://opensearch.org/downloads.html) +* [Documentation](https://opensearch.org/docs/latest/) +* Need help? Try [Forums](https://forum.opensearch.org/) +* [Project Principles](https://opensearch.org/about.html#principles-for-development) +* [Contributing to OpenSearch](CONTRIBUTING.md) +* [Onboarding Guide](ONBOARDING.md) +* [Maintainer Responsibilities](RESPONSIBILITIES.md) +* [Release Management](RELEASING.md) +* [Organization Admins](ADMINS.md) +* [Repo Maintainers](MAINTAINERS.md) +* [Issue Triage](TRIAGING.md) +* [Security](SECURITY.md) -See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information. +## Project Style Guidelines + +The [OpenSearch Project style guidelines](https://github.com/opensearch-project/documentation-website/blob/main/STYLE_GUIDE.md) and [OpenSearch terms](https://github.com/opensearch-project/documentation-website/blob/main/TERMS.md) documents provide style standards and terminology to be observed when creating OpenSearch Project content. + +## Code of Conduct + +This project has adopted the [Amazon Open Source Code of Conduct](CODE_OF_CONDUCT.md). For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq), or contact [opensource-codeofconduct@amazon.com](mailto:opensource-codeofconduct@amazon.com) with any additional questions or comments. ## License -This project is licensed under the Apache-2.0 License. +This project is licensed under the [Apache v2.0 License](LICENSE.txt). + +## Copyright +Copyright OpenSearch Contributors. See [NOTICE](NOTICE.txt) for details. diff --git a/RELEASING.md b/RELEASING.md new file mode 100644 index 00000000..e0e3bb1c --- /dev/null +++ b/RELEASING.md @@ -0,0 +1,161 @@ +- [Overview](#overview) +- [Branching](#branching) + - [OpenSearch Branching](#opensearch-branching) + - [Plugin Branching](#plugin-branching) + - [Feature Branches](#feature-branches) +- [Versioning](#versioning) + - [Version Numbers](#version-numbers) + - [Incrementing Versions](#incrementing-versions) +- [Tagging](#tagging) +- [Release Labels](#release-labels) +- [Releasing](#releasing) + - [Release Process for OpenSearch Bundle Minor Releases and Patch Releases](#Release-Process-for-OpenSearch-Bundle-Minor-Releases-and-Patch-Releases) + - [Entrance Criteria to Start Release Window](#Entrance-Criteria-to-Start-Release-Window) + - [Exit Criteria to Close Release Window](#Exit-Criteria-to-Close-Release-Window) +- [Security Reviews](#security-reviews) +- [Backporting](#backporting) + +## Overview + +This document explains the release strategy for artifacts in this organization. + +## Branching + +Projects create a new branch when they need to start working on 2 separate versions of the product, with the `main` branch being the furthermost release. + +### OpenSearch Branching + +[OpenSearch](https://github.com/opensearch-project/OpenSearch) typically tracks 3 releases in parallel. For example, given the last major release of 2.0, OpenSearch in this organization maintains the following active branches. + +* **main**: The _next major_ release, currently 3.0. This is the branch where all merges take place, and code moves fast. +* * **2.x**: The _next minor_ release. Once a change is merged into `main`, decide whether to backport it to `2.x`. +* **1.3**: The _next patch_ release. Once a change is merged into `main`, decide whether to backport it to `1.3`. + +Label PRs with the next major version label (e.g. `3.0.0`) and merge changes into `main`. Label PRs that you believe need to be backported as `2.x`, `1.x` and `1.3`. Backport PRs by checking out the versioned branch, cherry-pick changes and open a PR against each target backport branch. + +### Plugin Branching + +Plugins are bundled and shiped together along with OpenSearch for every release. Plugin branching follows OpenSearch core branching that will allow working on 3 releases at the same time. + +### Feature Branches + +Do not creating branches in the upstream repo, use your fork, for the exception of long lasting feature branches that require active collaboration from multiple developers. Name feature branches `feature/`. Once the work is merged to `main`, please make sure to delete the feature branch. + +## Versioning + +OpenSearch versioning [follows semver](https://opensearch.org/blog/technical-post/2021/08/what-is-semver/). + +### Version Numbers + +The build number of the engine is 3-digit `major.minor.patch` (e.g. `2.9.0`), while plugins use 4 digits (`2.9.0.45`). See [OpenSearch#1093](https://github.com/opensearch-project/OpenSearch/issues/1093) for a proposal to remove this difference. + +### Incrementing Versions + +Versions are incremented as soon as development starts on a given version to avoid confusion. In the examples above versions are as follows. + +* OpenSearch: `main` = 3.0.0, `2.x` = 2.10.0 `1.x` = 1.4.0, and `1.2` = 1.2.5 +* job-scheduler: `main` = 3.0.0.0, `2.x` = 2.10.0.0 , `1.x` = 1.4.0.0 and `1.2` = 1.2.5.0 + +## Tagging + +Create tags after a release that match the version number, `major.minor.patch`, without a `v` prefix. + +* [OpenSearch tags](https://github.com/opensearch-project/OpenSearch/tags): [1.0.0](https://github.com/opensearch-project/OpenSearch/releases/tag/1.0.0) +* [job-scheduler tags](https://github.com/opensearch-project/job-scheduler/tags): [1.0.0.0](https://github.com/opensearch-project/job-scheduler/releases/tag/1.0.0.0) + +For a discussion on whether to add a prefixing `v` to release tags, see [#35](https://github.com/opensearch-project/.github/issues/35). + +## Release Labels + +Repositories create consistent release labels, such as `v2.9.0`, `v1.0.0`, `v1.1.0` and `v2.0.0`, as well as `patch` and `backport`. Use release labels to target an issue or a PR for a given release. See [MAINTAINERS](MAINTAINERS.md#triage-open-issues) for more information on triaging issues. + +## Releasing + +OpenSearch follows semver, which means we will only release breaking changes in major versions. All minor versions are compatible with every other minor version for that major. For example, 1.2.0 will work with 1.3.2, 1.4.1, etc, but may not work with 2.0. + +OpenSearch uses a “release-train” model. For minor version, that train leaves approximately every six weeks we release a new minor version which includes all the new features and fixes that are ready to go. Having a set release schedule makes sure OpenSearch is releasing in a predictable way and prevents a backlog of unreleased changes. + +In contrast, OpenSearch releases new major versions only when there are a critical mass of breaking changes (e.g. changes that are incompatible with existing APIs). These tend to be tied to Lucene major version releases, and will be announced in the forums at least 4 weeks prior to the release date. + +### Release Process for OpenSearch Bundle Minor Releases and Patch Releases + +At the beginning of every year, the project will publish on [OpenSearch.org](https://opensearch.org/releases.html) the “release windows start” dates for the year. These dates will be spaced out ~6 weeks. + +On release window start date: +1. We generate the first candidate with all plug-ins/components that have met the entrance criteria. If a plug-in/component hasn't met all of the criteria, we'll version bump the last released version and release that. Once the release window opens and the first candidate is generated, no additioanl features can be added, and we will not delay the start of a release window for any plug-in/component. +1. During the release window we will do final quality testing, documentation and performance testing. Bug fixes can be added in during this time, but no new features will be added. +1. We will generate a new candidate every day and post on the release issue about the status of the exit criteria. When all the exit criteria have been met, we'll announce that the candidate is ready and release it 2 days later (unless that falls on Friday, in which case we'll release on Monday) +1. If we cannot clear the exit criteria within 2 weeks from the start of the window (1 week for patch releases), we will cancel that release window and try again in the next window. + +Please note: This process is for regularly scheduled minor and patch releases. For "hot" patches (patches required for security or other urgent issues) we will build, test and release as quickly as possible. + +#### Entrance Criteria to Start Release Window +* Documentation draft PRs are up and in tech review for all component changes. +* Sanity testing is done for all components. +* Code coverage has not decreased (all new code has tests). +* Release notes are ready and available for all components. +* Roadmap is up-to-date (information is available to create release highlights). +* Release ticket is cut, and there's a forum post announcing the start of the window. +* [Any necessary security reviews](##Security-Reviews) are complete. + +#### Exit Criteria to Close Release Window +* Performance tests are run, results are posted to the release ticket and there no unexpected regressions +* Documentation has been fully reviewed and signed off by the documentation community. +* All integration tests are passing +* Release blog is ready + +### Release Process for OpenSearch Major Releases + +OpenSearch only does major releases when there are significant breaking changes that are ready for release. Once we become aware of the need for a major version, we will start a major version release window which will be similar to a minor release, except for two things: Participation is mandatory for all components and the release window will be at least 4 weeks long to accommodate the testing required. + +For the actual steps to build a release, please see [Releasing OpenSearch](https://github.com/opensearch-project/opensearch-build/blob/main/README.md#releasing-opensearch). + + +### Security Reviews + +If you discover a potential security issue in this project we ask that you notify the OpenSearch Security Team by email at security@opensearch.org. Please do not create a public GitHub issue. See [SECURITY.md](SECURITY.md) for more information on the security response process. + +The OpenSearch Project currently performs security reviews before releasing signed artifacts. These are typically conducted for any of the following: + +* Releases from a new GitHub repository, such as a new plugin or extension, client, or tool. +* Major new features added to an existing application, including significant UX or API changes. +* Changes to authentication, authorization, cryptography, or other security-impacting functions. +* New software or infrastructure deployed to support the project, such as CI/CD. +* New 3rd-party providers or vendors being used by the project. + +The review process consists of building a threat model for the proposed change and optionally engaging a specialist to perform additional testing, such as a penetration testing. This process is done in parallel and in private within the project, during development, and usually takes 4-10 weeks. A repository maintainer will assess the scope of the new changes, initiate and manage a security review, provide public updates, and, if needed, communicate privately by email with the contributors. Please add a note in your pull request if you believe a security review is warranted. + +Please see [opensearch-project/.github#81](https://github.com/opensearch-project/.github/issues/81) for a discussion on improving this and other security-related processes.​ + +## Backporting + +This project follows [semantic versioning](https://semver.org/spec/v2.0.0.html). Backwards-incompatible changes always result in a new major version and will __never__ be backported. Small improvements and features will be backported to a new minor version (e.g. `2.9.0`). Security fixes will be backported to a new patch version (e.g. `2.9.1`). + +Here are the commands we typically run to backport changes to release branches: + +1. Checkout the target release branch and pull the latest changes from `upstream`. In the examples below, our target release branch is `2.x`. + +``` +git checkout 2.x +git pull upstream 2.x +``` + +2. Create a local branch for the backport. A convenient naming convention is _backport-\[PR-id\]-\[target-release-branch\]_. + +``` +git checkout -b backport-pr-xyz-1.x +``` + +3. Cherry-pick the commit to backport. Remember to include [DCO signoff](CONTRIBUTING.md#developer-certificate-of-origin). + +``` +git cherry-pick -s +``` + +4. Push the local branch to your fork. + +``` +git push origin backport-pr-xyz-1.x +``` + +5. Create a pull request for the change. diff --git a/RESPONSIBILITIES.md b/RESPONSIBILITIES.md new file mode 100644 index 00000000..e5fc5f9b --- /dev/null +++ b/RESPONSIBILITIES.md @@ -0,0 +1,167 @@ +- [Overview](#overview) +- [Current Maintainers](#current-maintainers) +- [Maintainer Responsibilities](#maintainer-responsibilities) + - [Uphold Code of Conduct](#uphold-code-of-conduct) + - [Prioritize Security](#prioritize-security) + - [Review Pull Requests](#review-pull-requests) + - [Merging a Pull Request](#merging-a-pull-request) + - [Triage Open Issues](#triage-open-issues) + - [Automatically Label Issues](#automatically-label-issues) + - [Be Responsive](#be-responsive) + - [Maintain Overall Health of the Repo](#maintain-overall-health-of-the-repo) + - [Keep Dependencies up to Date](#keep-dependencies-up-to-date) + - [Manage Roadmap](#manage-roadmap) + - [Add Continuous Integration Checks](#add-continuous-integration-checks) + - [Use Semver](#use-semver) + - [Release Frequently](#release-frequently) + - [Promote Other Maintainers](#promote-other-maintainers) + - [Describe the Repo](#describe-the-repo) +- [Becoming a Maintainer](#becoming-a-maintainer) + - [Nomination](#nomination) + - [Interest](#interest) + - [Addition](#addition) +- [Removing a Maintainer](#removing-a-maintainer) + - [Moving On](#moving-on) + - [Inactivity](#inactivity) + - [Negative Impact on the Project](#negative-impact-on-the-project) + +## Overview + +This document explains who maintainers are, what they do in various repos of opensearch-project, and how they should be doing it. If you're interested in contributing, see [CONTRIBUTING](CONTRIBUTING.md). + +## Current Maintainers + +Each repo contains a [MAINTAINERS.md](MAINTAINERS.md) file that lists current maintainers, and points to this document. + +## Maintainer Responsibilities + +Maintainers are active and visible members of the community, and have [maintain-level permissions on a repository](https://docs.github.com/en/organizations/managing-access-to-your-organizations-repositories/repository-permission-levels-for-an-organization). Use those privileges to serve the community and evolve code as follows. + +### Uphold Code of Conduct + +Model the behavior set forward by the [Code of Conduct](CODE_OF_CONDUCT.md) and raise any violations to other maintainers and [members of the admin team](https://github.com/opensearch-project/.github/blob/main/ADMINS.md). + +### Prioritize Security + +Security is your number one priority. Maintainer's Github keys must be password protected securely and any reported security vulnerabilities are addressed before features or bugs. + +Note that this repository is monitored and supported 24/7 by Amazon Security, see [Reporting a Vulnerability](SECURITY.md) for details. + +### Review Pull Requests + +It's our responsibility to ensure the content and code in pull requests are correct and of high quality before they are merged. Here are some best practices: + +- Leverage the issue triaging process to review pull requests and assign them to maintainers for review. +- In cases of uncertainty on how to proceed, search for related issues and reference the pull request to find additional collaborators. +- When providing feedback on pull requests, make sure your feedback is actionable to guide the pull request towards a conclusion. +- If a pull request is valuable but isn't gaining traction, consider reaching out to fulfill the necessary requirements. This way, the pull request can be merged, even if the work is done by several individuals. +- Lastly, strive for progress, not perfection. + +### Merging a Pull Request + +It is important that commit messages are helpful in understanding the reasons for a given commit and maintain good commit hygiene by only keeping the relevant information. + +Most repositories in [opensearch-project](https://github.com/opensearch-project) are configured to require commits to be squashed into a single commit when merging pull requests. If the pull request contains multiple commits then messages from all commits will be appended into a single message, which usually requires editing to produce a high quality commit message. When merging pull requests, edit commit messages by following these steps as much as possible: + +- The commit subject should be concise (ideally within 50 characters) and clearly convey what is being merged. +- The commit body should include the details (if any, and possibly within 72 characters) about the commit, typically inline with the PR description. +- The commit body should include the 'Signed-off-by:*' for all committers involved in the change and thereby indicating that all code contributors agree to the [DCO](https://github.com/opensearch-project/.github/blob/main/CONTRIBUTING.md#developer-certificate-of-origin). +- There need to be a matching 'Signed-Off-By:' line for the `This commit will be authored by *` email address otherwise backport DCO checks will fail. + +### Triage Open Issues + +Manage labels, review issues regularly, and triage by labelling them. + +All repositories in this organization have a standard set of labels, including `bug`, `documentation`, `duplicate`, `enhancement`, `good first issue`, `help wanted`, `blocker`, `invalid`, `question`, `wontfix`, and `untriaged`, along with release labels, such as `v1.0.0`, `v1.1.0`, `v2.0.0`, `patch`, and `backport`. + +Use labels to target an issue or a PR for a given release, add `help wanted` to good issues for new community members, and `blocker` for issues that scare you or need immediate attention. Request for more information from a submitter if an issue is not clear. Create new labels as needed by the project. + +See [TRIAGING](TRIAGING.md) for more information on how to attend triage meetings. + +#### Automatically Label Issues + +There are many tools available in GitHub for controlling labels on issues and pull requests. Use standard issue templates in the [./.github/ISSUE_TEMPLATE](./.github/ISSUE_TEMPLATE) directory to apply appropriate labels such as `bug` and `untriaged`. Repositories can choose to use GitHub actions such as [add-untriaged.yml](./.github/workflows/add-untriaged.yml) to apply labels automatically. + +### Be Responsive + +Respond to enhancement requests, and forum posts. Allocate time to reviewing and commenting on issues and conversations as they come in. + +### Maintain Overall Health of the Repo + +Keep the `main` branch at production quality at all times. Backport features as needed. Cut release branches and tags to enable future patches. + +#### Keep Dependencies up to Date + +Maintaining up-to-date dependencies on third party projects reduces the risk of security vulnerabilities. The Open Source Security Foundation (OpenSSF) [recommends](https://github.com/ossf/scorecard/blob/main/docs/checks.md#dependency-update-tool) either [dependabot](https://docs.github.com/en/code-security/dependabot) or [renovatebot](https://docs.renovatebot.com/). Both of these applications generate Pull Requests for dependency version updates. + - Renovate is integrated as part of the Remediate app in [Mend for Github](https://github.com/apps/mend-for-github-com), which is enabled on all opensearch-project repositories. It can be enabled in the `.whitesource` configuration file as described in the [Mend Remediate and Renovate](https://docs.mend.io/bundle/integrations/page/mend_remediate_and_renovate.html#Integration-with-Mend-Renovate) documentation. The [Merge Confidence](https://docs.renovatebot.com/merge-confidence/) feature can be configured to provide maintainers more information on the age, adoption rate, and percent test passing rate of other repositories. Mend maintains a "Dependency Dashboard" Issue in the repository with centralized information on pending version update PRs. + - Dependabot is integrated with GitHub and can be enabled by adding a [`dependabot.yml`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuring-dependabot-version-updates) file to the repo. Dependabot does not have any centralized management dashboard, so maintainers may use tags or other PR filters to track pending updates. + +### Manage Roadmap + +Ensure the repo highlights features that should be elevated to the project roadmap. Be clear about the feature’s status, priority, target version, and whether or not it should be elevated to the roadmap. Any feature that you want highlighted on the OpenSearch Roadmap should be tagged with "roadmap". The OpenSearch [project-meta maintainers](https://github.com/opensearch-project/project-meta/blob/main/MAINTAINERS.md) will highlight features tagged "roadmap" on the project wide [OpenSearch Roadmap](https://github.com/orgs/opensearch-project/projects/1). + +### Add Continuous Integration Checks + +Add integration checks that validate pull requests and pushes to ease the burden on Pull Request reviewers. + +### Use Semver + +Use and enforce [semantic versioning](https://semver.org/) and do not let breaking changes be made outside of major releases. + +### Release Frequently + +Make frequent project releases to the community. + +### Promote Other Maintainers + +Assist, add, and remove [MAINTAINERS](MAINTAINERS.md). Exercise good judgement, and propose high quality contributors to become co-maintainers. See [Becoming a Maintainer](#becoming-a-maintainer) for more information. + +### Describe the Repo + +Make sure the repo has a well-written, accurate, and complete description. See [opensearch-project/.github#38](https://github.com/opensearch-project/.github/issues/38) for some helpful tips to describe your repo. + +## Becoming a Maintainer + +You can become a maintainer by actively [contributing](CONTRIBUTING.md) to any project, and being nominated by an existing maintainer. + +### Nomination + +Any current maintainer starts a private e-mail thread (until we have a better mechanism, e-mail addresses can usually be found via MAINTAINERS.md + DCO) with all other maintainers on that repository to discuss nomination using the template below. In order to be approved, at least three positive (+1) maintainer votes are necessary, and no vetoes (-1). In rare cases when there are fewer than three maintainers, the positive (+1) votes from all maintainers are required. Any disagreements can be escalated to the [members of the admin team](https://github.com/opensearch-project/.github/blob/main/ADMINS.md). + +The nomination should clearly identify the person with their real name and a link to their GitHub profile, and the rationale for the nomination, with concrete example contributions. + +### Interest + +Upon receiving at least three positive (+1) maintainer votes, and no vetoes (-1), from existing maintainers after a one week period, the nominating maintainer asks the nominee whether they might be interested in becoming a maintainer on the repository via private e-mail message. + +> This is great work! Based on your valuable contribution and ongoing engagement with the project, the current maintainers invite you to become a co-maintainer for this project. Please respond and let us know if you accept the invitation to become maintainer. + +Individuals accept the nomination by replying, or commenting, for example _"Thank you! I would love to."_ + +### Addition + +Upon receiving three positive (+1) maintainer votes, and no vetoes (-1), from other maintainers, and after having privately confirmed interest with the nominee, the maintainer opens a pull request adding the proposed co-maintainer to MAINTAINERS.md. The pull request is approved and merged. + +> _Content from the above nomination._ +> +> The maintainers have voted and agreed to this nomination. + +The [members of the admin team](https://github.com/opensearch-project/.github/blob/main/ADMINS.md) adjusts the new maintainer’s permissions accordingly, and merges the pull request. + +## Removing a Maintainer + +Removing a maintainer is a disruptive action that the community of maintainers should not undertake lightly. There are several reasons a maintainer will be removed from the project, such as violating the [code of conduct](https://github.com/opensearch-project/.github/blob/main/CODE_OF_CONDUCT.md), or taking other actions that negatively impact the project. + +### Moving On + +There are plenty of reasons that might cause someone to want to take a step back or even a hiatus from a project. Existing maintainers can choose to leave the project at any time, with or without reason, by making a pull request to move themselves to the "Emeritus" section of MAINTAINERS.md, and asking an [members of the admin team](https://github.com/opensearch-project/.github/blob/main/ADMINS.md) to remove their permissions. + +### Inactivity + +Maintainer status never expires. If a maintainer becomes inactive for a time (usually several months), or a maintainer can confirm that they are no longer involved with the project for any reason, the [members of the admin team](https://github.com/opensearch-project/.github/blob/main/ADMINS.md) may make a pull request to move them to the "Emeritus" section of the MAINTAINERS.md, remove them from CODEOWNERS, and upon merging the pull request, revoke the maintainer level access. Any past maintainer can be reinstated via another pull request, and have their permissions restored by the [members of the admin team](https://github.com/opensearch-project/.github/blob/main/ADMINS.md) at any time upon request. + +If the repo is left without any maintainers, either by maintainer inactivity or moving on, the repo is considered unmaintained. The [members of the admin team](https://github.com/opensearch-project/.github/blob/main/ADMINS.md) will seek out new maintainers and note the maintenance status in the repo README file. + +### Negative Impact on the Project + +Actions that negatively impact the project will be handled by the [members of the admin team](https://github.com/opensearch-project/.github/blob/main/ADMINS.md), in coordination with other maintainers, in balance with the urgency of the issue. Examples would be [Code of Conduct](CODE_OF_CONDUCT.md) violations, deliberate harmful or malicious actions, and security risks. diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 00000000..e573b576 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,88 @@ +# Security Issue Response Process + +## Introduction + +Security issues happen as part of the normal lifecycle of software development of OpenSearch. This document describes the process for reporting these issues, responding to these issues, and ensuring we treat them not only with the appropriate priority but also with respect for the discoverers, software providers and their users, and the OpenSearch community in general. + +If you discover a potential security issue in this project we ask that you notify the OpenSearch Security Team via email to security@opensearch.org. Please do **not** create a public GitHub issue. + +*Giving credit where credit is due, this policy is heavily influenced by the [Xen Project’s security response process](https://xenproject.org/developers/security-policy/), that was put to the test during the [embargo period for XSA-108 back in 2014](https://xenproject.org/2014/10/22/xen-project-security-policy-improvements-get-involved/) and improved its clarity around managing the pre-disclosure list and the deployment of fixes during embargo. We are standing on the shoulders of these battle-tested giants.* + +## The Security Team + +The OpenSearch Security Team is a subset of the project’s maintainers responsible for looking after the project’s security, including the security issue response process outlined below. Maintainers can be added to the Security Team by submitting a PR updating this document and adding their name to the list below (the process for becoming a maintaner can be found [here](https://github.com/opensearch-project/.github/blob/main/MAINTAINERS.md#becoming-a-maintainer)). + +The OpenSearch Security Team will address reported issues on a best effort basis, prioritizing them based on several factors, including severity. + +### Current Members + +| Security Team member | GitHub Alias | Affiliation | +| ------------------------ | --------------------------------------------- | ----------- | +| Kunal Khatua | [kkhatua](https://github.com/kkhatua) | Amazon | +| Daniel (dB.) Doubrovkine | [dblock](https://github.com/dblock) | Amazon | +| Anirudha (Ani) Jadhav | [anirudha](https://github.com/anirudha) | Amazon | +| Anan Zhuang | [ananzh](https://github.com/ananzh) | Amazon | +| Eli Fisher | [elfisher](https://github.com/elfisher) | Amazon | + +## Process + +Anyone finding an issue that is already publicly disclosed (for example, a CVE in one of the project’s dependencies) should feel free to create an issue and discuss openly on GitHub. The process below is only intended for issues that have not been publicly disclosed yet. + +1. We request that instead of opening a GitHub issue, it is reported via email at security@opensearch.org. Please include a description of the issue, and any other information that could help in the reproduction and creation of a fix (version numbers, configuration values, reproduction steps...) +2. The OpenSearch Security Team will negotiate the conditions for an embargo period and a disclosure timeline with the discoverer (see [Embargo schedule](#embargo-schedule)). +3. After the vulnerability is confirmed, if no CVE number is already reserved, the OpenSearch Security Team will reserve one, and communicate it to the discoverer and all parties in the pre-disclosure list (see [Pre-disclosure list](#pre-disclosure-list)). +4. As soon as our advisory is available, we will send it, including patches, to members of the pre-disclosure list. In the event that we do not have a patch available 2 working weeks before the disclosure date, we aim to send an advisory that reflects the current state of knowledge to the pre-disclosure list. An updated advisory will be published as soon as available. At this stage, the advisory will be clearly marked with the embargo date. +5. On the day the embargo is lifted, we will publish the advisory and release the new versions containing the fix. +6. If new information and/or better fixes become available, new advisory versions will be released. + +During an embargo period, the OpenSearch Security Team may be required to make potentially controversial decisions in private, since they cannot confer with the community without breaking the embargo. The team will attempt to make such decisions following the guidance of this document and, where necessary, their own best judgement. Following the embargo period, on a best effort basis, the Security Team will disclose any such decisions, including the reasoning behind them, in the interests of transparency and to help provide guidance should a similar decision be required in the future. + +## Embargo Schedule + +Embargo periods will be negotiated on a case-by-case basis depending on the severity of the issue and availability of the fix, where a general starting point is to release an advisory to the pre-disclosure list within 2 weeks of the initial notification, and publicly releasing the advisory within 4 weeks of the advisory pre-release. + +When a discoverer reports a problem to us and requests longer delays than we would consider ideal, we will honor such a request if reasonable. If a discoverer wants an accelerated disclosure compared to what we would prefer, we naturally do not have the power to insist that a discoverer waits for us to be ready and will honor the date specified by the discoverer. +Naturally, if a vulnerability is being exploited in the wild, we will make immediately public release of the patch(es.) + +## Pre-disclosure List + +We maintain a pre-disclosure list of contributors, vendors and operators of OpenSearch for two main reasons: + +1. To apply the fixes to large user populations requiring a significant amount of work post-disclosure +2. To privately collaborate with those who can help write and test the fixes so we can release them as soon as possible and with high confidence in their quality + +If there is an embargo, the pre-disclosure list will receive copies of the advisory and patches, with a clearly marked embargo date, as soon as they are available. The pre-disclosure list will also receive copies of public advisories when they are first issued or updated. + +We expect list members to maintain the confidentiality of the vulnerability up to the embargo date. Specifically, prior to the embargo date, pre-disclosure list members should not make available, even to their own customers and partners: + +* The OpenSearch advisory +* Their own advisory +* The impact, scope, set of vulnerable systems or the nature of the vulnerability +* Revision control commits which are a fix for the problem +* Patched software (even in binary form) + +Without prior consultation with the OpenSearch Security Team, list members may make available to their users only: + +* The existence of an issue +* The assigned OpenSearch advisory number +* The planned disclosure date + +List members may, if (and only if) the OpenSearch Security Team grants permission, deploy fixed versions during the embargo. Permission for deployment, and any restrictions, will be stated in the embargoed advisory text. Where the list member is a service provider who intends to take disruptive action such as rebooting as part of deploying a fix: the list member’s communications to its users about the service disruption may mention that the disruption is to correct a security issue, and relate it to the public information about the issue (as listed above). This applies whether the deployment occurs during the embargo (with permission–see above) or is planned for after the end of the embargo. + +Pre-disclosure list members are allowed to share fixes to embargoed issues, analysis, etc., with the OpenSearch Security Team and security teams of other list members. Technical measures must be taken to prevent non-list-member organizations, or unauthorized staff in list-member organizations, from obtaining the embargoed materials. + +## Pre-disclosure list application process + +Organizations who meet the criteria above (i.e. significant work needed post-disclosure to remediate the issue and/or ability to help create or test the potential fixes) should contact the OpenSearch Security Team via email at security@opensearch.org if they wish to be added to the pre-disclosure list. In the email, you must include: + +* The name of your organization +* How you’re using OpenSearch +* A description of why you fit the criteria (number of users, amount of work needed to remediate, ability to collaborate on fixes...) +* Information about your handling of security problems + * Your invitation to members of the public, who discover security problems with your products/services, to report them in confidence to you + * Specifically, the contact information (email addresses or other contact instructions) which such a member of the public should use +* A statement to the effect that you have read this policy and agree to abide by the terms for inclusion in the list, specifically the requirements to regarding confidentiality during an embargo period +* The email(s) you wish added to the pre-disclosure list + + +The OpenSearch Security Team will review your application and get back to you with their decision.