feat(storage): allow minio to use aws sdk or opendal #15208
Conversation
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 9425213 | Triggered | Generic Password | 6c85ce0 | ci/scripts/regress-test.sh | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secret safely. Learn here the best practices.
- Revoke and rotate this secret.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
Our GitHub checks need improvements? Share your feedbacks!
…nto wcy/switch_minio_to_new_s3
|
@hzxa21 PTAL❤️ |
| let server = server.strip_prefix("minio://").unwrap(); | ||
| let (access_key_id, rest) = server.split_once(':').unwrap(); | ||
| let (secret_access_key, mut rest) = rest.split_once('@').unwrap(); | ||
| let endpoint_prefix = if let Some(rest_stripped) = rest.strip_prefix("https://") { | ||
| rest = rest_stripped; | ||
| "https://" | ||
| } else if let Some(rest_stripped) = rest.strip_prefix("http://") { | ||
| rest = rest_stripped; | ||
| "http://" | ||
| } else { | ||
| "http://" | ||
| }; | ||
| let (address, bucket) = rest.split_once('/').unwrap(); |
There was a problem hiding this comment.
I guess it can be better to use the url crate to parse the URL here instead of doing it manually. Less code and less possibility of security problems.
There was a problem hiding this comment.
Umm just tried, but the here it's not a normal url(127.0.0.1:9301/hummock001, instead of http://127.0.0.1:9301/hummock001), so just keep it the same as before.
There was a problem hiding this comment.
"minio://" is part of the URL, which will result in parsed_url.scheme() == "minio".
There was a problem hiding this comment.
Let's change the use of all state store urls in later PR.
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
keep the same with s3.
Checklist
./risedev check(or alias,./risedev c)Documentation
Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.