Skip to content

Commit

Permalink
Add AWS S3 Sync with Different Access Keys documentation (#195)
Browse files Browse the repository at this point in the history
  • Loading branch information
arpita0911patel authored Aug 22, 2024
2 parents 32f86c2 + 84ba875 commit 2f08bac
Show file tree
Hide file tree
Showing 5 changed files with 99 additions and 2 deletions.
82 changes: 82 additions & 0 deletions docs/services/cloudservices/aws/documentation/aws-s3-sync/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
---
sidebar_position: 4
title: "AWS S3 Sync with Different Access Keys"
description: "AWS S3 Sync with Different Access Keyss"
tags:
- aws
- documentation
- tutorial
- AWS
- s3 sync
- access keys
---


### Step 1: Configure AWS CLI Profiles
First, set up two different profiles in your AWS CLI configuration, one for the source bucket and one for the destination bucket.

```
# Configure source profile
aws configure --profile source-profile
AWS Access Key ID [None]: [Enter source bucket access key]
AWS Secret Access Key [None]: [Enter source bucket secret key]
Default region name [None]: [Enter region, e.g., us-west-2]
Default output format [None]: json
# Configure destination profile
aws configure --profile destination-profile
AWS Access Key ID [None]: [Enter destination bucket access key]
AWS Secret Access Key [None]: [Enter destination bucket secret key]
Default region name [None]: [Enter region, e.g., us-east-1]
Default output format [None]: json
```

### Step 2: Verify Profiles
Verify that your profiles are set up correctly:
```
aws configure list --profile source-profile
aws configure list --profile destination-profile
```

### Step 3: Use AWS S3 Sync Command
Now, you can use the aws s3 sync command with these profiles. Here's the basic syntax:
```
aws s3 sync s3://source-bucket s3://destination-bucket \
--source-profile source-profile \
--profile destination-profile
```
Examples

```
#Sync all contents:
aws s3 sync s3://my-source-bucket s3://my-destination-bucket \
--source-profile source-profile \
--profile destination-profile
#Sync with specific options:
aws s3 sync s3://my-source-bucket s3://my-destination-bucket \
--source-profile source-profile \
--profile destination-profile \
--delete \
--exclude "*.tmp" \
--include "*.jpg"
#Dry run (no actual changes):
aws s3 sync s3://my-source-bucket s3://my-destination-bucket \
--source-profile source-profile \
--profile destination-profile \
--dryrun
```

### Important Notes:
- The --source-profile is used for reading from the source bucket.
- The --profile (or --dest-profile in some AWS CLI versions) is used for writing to the destination bucket.
- Ensure both profiles have the necessary permissions on their respective buckets.
- Be cautious with the --delete flag as it removes files in the destination that don't exist in the source.
- Always test with --dryrun first to ensure the operation will do what you expect.
- Basically, you have to create two access key profile, and attach those appropriate
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 6
sidebar_position: 3
description: "AWS Data Science Tools"
tags:
- AWS
Expand Down
15 changes: 15 additions & 0 deletions docs/services/cloudservices/aws/documentation/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
sidebar_position: 3
title: "Documentation and Tutorial"
description: "AWS documentations"
tags:
- aws
- documentation
- tutorial
- AWS
---


import DocCardList from '@theme/DocCardList';

<DocCardList />
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 1
sidebar_position: 2
title: "Tag Resources on AWS"
description: "AWS tags for cost tracking"
tags:
Expand Down

0 comments on commit 2f08bac

Please sign in to comment.