Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add export data to migrate tool #2610

Merged
merged 2 commits into from
Oct 17, 2023

Conversation

shuiyisong
Copy link
Contributor

@shuiyisong shuiyisong commented Oct 17, 2023

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

This pr mainly adds export-data to cli export tool.

export

e.g

# export create table
./greptime cli export --addr=127.0.0.1:4001 --output-dir=/tmp/export/ --target=create-table --database="greptime-*"

# export table data
./greptime cli export --addr=127.0.0.1:4001 --output-dir=/tmp/export/ --target=table-data --database="greptime-*"

result

/tmp/export
❯ tree
.
├── greptime-public.sql
├── greptime-thello
│   └── monitor.parquet
├── greptime-thello.sql
└── greptime-thello_copy_from.sql

2 directories, 4 files

import

# create database forehand

# psql
# create tables
psql -h 127.0.0.1 -p 4003 -d thello -a -f greptime-thello.sql
# copy from data
psql -h 127.0.0.1 -p 4003 -d thello -a -f greptime-thello_copy_from.sql

# mysql
mysql -h 127.0.0.1 -P 4002 public < greptime-public.sql

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

Copy link
Member

@waynexia waynexia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

src/cmd/src/cli/export.rs Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Oct 17, 2023

Codecov Report

Merging #2610 (2b139cf) into develop (9056c3a) will decrease coverage by 0.46%.
The diff coverage is 0.00%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #2610      +/-   ##
===========================================
- Coverage    85.23%   84.77%   -0.46%     
===========================================
  Files          738      738              
  Lines       118249   118319      +70     
===========================================
- Hits        100794   100310     -484     
- Misses       17455    18009     +554     

@waynexia waynexia enabled auto-merge October 17, 2023 06:18
Copy link
Contributor

@fengys1996 fengys1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@waynexia waynexia added this pull request to the merge queue Oct 17, 2023
Merged via the queue into GreptimeTeam:develop with commit eccad64 Oct 17, 2023
15 checks passed

format!(
"copy {} from '{}' with (format='parquet');\n",
filename.replace(".parquet", ""),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion(non-blocking): Can we assign this to a variable first? I took a long time to realize that it is the table name.

let table_name = filename.replace(".parquet", "");

Comment on lines +342 to +344
})
.collect::<Vec<_>>()
.join("");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we collect it into a string?

Suggested change
})
.collect::<Vec<_>>()
.join("");
})
.collect::<String>();

@shuiyisong shuiyisong deleted the chore/export_data branch November 29, 2023 06:01
@shuiyisong shuiyisong restored the chore/export_data branch November 29, 2023 06:01
@shuiyisong shuiyisong deleted the chore/export_data branch November 29, 2023 06:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants