Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG]在3000万数据进行setup的时候报错 #116

Open
yyf1986 opened this issue Apr 23, 2024 · 6 comments
Open

[BUG]在3000万数据进行setup的时候报错 #116

yyf1986 opened this issue Apr 23, 2024 · 6 comments
Assignees
@6fj
Labels
bug Something isn't working

Comments

@yyf1986
Copy link

yyf1986 commented Apr 23, 2024

Describe the bug
在对3000万数据进行setup的时候报错

2024-04-23 08:16:21.587 [info] [sender_kvdb.cc:InsertOrAssign:728] OPRF FullEvaluate and EncryptLabel Last batch count: 2, item_count:1000000
2024-04-23 08:16:21.590 [info] [sender_kvdb.cc:DispatchInsertOrAssign:517] Launching 1 insert-or-assign worker tasks
2024-04-23 08:16:35.774 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:0
2024-04-23 08:16:36.406 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:1
2024-04-23 08:16:36.880 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:2
2024-04-23 08:16:37.415 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:3
2024-04-23 08:16:38.189 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:4
2024-04-23 08:16:38.881 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:5
2024-04-23 08:16:39.464 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:6
2024-04-23 08:16:40.027 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:7
2024-04-23 08:16:40.573 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:8
2024-04-23 08:16:41.169 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:9
2024-04-23 08:16:41.890 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:10
2024-04-23 08:16:42.621 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:11
2024-04-23 08:16:43.269 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:12
2024-04-23 08:16:43.919 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:13
2024-04-23 08:16:44.596 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:14
2024-04-23 08:16:45.287 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:15
2024-04-23 08:16:45.963 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:16
2024-04-23 08:16:46.668 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:17
2024-04-23 08:16:47.281 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:18
2024-04-23 08:16:47.886 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:19
2024-04-23 08:16:48.537 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:20
2024-04-23 08:16:49.147 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:21
2024-04-23 08:16:49.713 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:22
2024-04-23 08:16:50.116 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:23
2024-04-23 08:16:50.744 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:24
2024-04-23 08:16:51.390 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:25
2024-04-23 08:16:51.984 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:26
2024-04-23 08:16:52.550 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:27
2024-04-23 08:16:53.149 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:28
2024-04-23 08:16:53.719 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:29
2024-04-23 08:16:54.331 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:30
2024-04-23 08:16:55.011 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:31
2024-04-23 08:16:55.600 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:32
2024-04-23 08:16:56.204 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:33
2024-04-23 08:16:56.807 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:34
2024-04-23 08:16:57.329 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:35
2024-04-23 08:16:57.859 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:36
2024-04-23 08:16:58.434 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:37
2024-04-23 08:16:59.026 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:38
2024-04-23 08:16:59.617 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:39
2024-04-23 08:17:00.172 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:40
2024-04-23 08:17:00.730 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:41
2024-04-23 08:17:01.334 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:42
2024-04-23 08:17:01.887 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:43
2024-04-23 08:17:02.535 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:44
2024-04-23 08:17:03.171 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:45
2024-04-23 08:17:03.812 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:46
2024-04-23 08:17:04.427 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:47
2024-04-23 08:17:05.092 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:48
2024-04-23 08:17:05.743 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:49
2024-04-23 08:17:06.360 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:50
2024-04-23 08:17:06.926 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:51
2024-04-23 08:17:07.504 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:52
2024-04-23 08:17:08.078 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:53
2024-04-23 08:17:08.649 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:54
2024-04-23 08:17:09.041 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:55
2024-04-23 08:17:09.584 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:56
2024-04-23 08:17:10.106 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:57
2024-04-23 08:17:10.586 [info] [sender_kvdb.cc:InsertOrAssignWorker:433] *** step leveldb put duration:34812.315606
2024-04-23 08:17:10.671 [info] [sender_kvdb.cc:DispatchInsertOrAssign:535] Finished insert-or-assign worker tasks
2024-04-23 08:17:10.676 [info] [sender_kvdb.cc:InsertOrAssign:841] Finished inserting 1000000 items in SenderDB
2024-04-23 08:17:10.972 [info] [pir.cc:LabeledPirSetup:268] finish bucket:28
2024-04-23 08:17:11.088 [info] [pir.cc:LabeledPirSetup:240] bucket:29 bucket_setup_path:/data/3000w_setup_path/bucket_29
2024-04-23 08:17:11.348 [info] [ecdh_oprf_selector.cc:CreateEcdhOprfServer:33] use fourq
2024-04-23 08:17:11.419 [info] [leveldb_kvstore.cc:Get:73] key not found
2024-04-23 08:17:11.419 [info] [sender_kvdb.cc:SenderKvDB:597] key item_count no value
2024-04-23 08:17:11.590 [info] [sender_kvdb.cc:InsertOrAssign:734] OPRF FullEvaluate and EncryptLabel batch_count: 0
2024-04-23 08:17:38.348 [info] [sender_kvdb.cc:InsertOrAssign:734] OPRF FullEvaluate and EncryptLabel batch_count: 1
2024-04-23 08:17:49.386 [info] [sender_kvdb.cc:InsertOrAssign:728] OPRF FullEvaluate and EncryptLabel Last batch count: 2, item_count:1000000
2024-04-23 08:17:49.389 [info] [sender_kvdb.cc:DispatchInsertOrAssign:517] Launching 1 insert-or-assign worker tasks
2024-04-23 08:18:01.863 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:0
2024-04-23 08:18:02.475 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:1
2024-04-23 08:18:02.978 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:2
2024-04-23 08:18:03.521 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:3
2024-04-23 08:18:04.268 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:4
2024-04-23 08:18:04.927 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:5
2024-04-23 08:18:05.575 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:6
2024-04-23 08:18:06.305 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:7
2024-04-23 08:18:06.978 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:8
2024-04-23 08:18:07.626 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:9
2024-04-23 08:18:08.255 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:10
2024-04-23 08:18:08.830 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:11
2024-04-23 08:18:09.456 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:12
2024-04-23 08:18:10.100 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:13
2024-04-23 08:18:10.683 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:14
2024-04-23 08:18:11.274 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:15
2024-04-23 08:18:11.890 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:16
2024-04-23 08:18:12.543 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:17
2024-04-23 08:18:13.185 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:18
2024-04-23 08:18:13.778 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:19
2024-04-23 08:18:14.323 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:20
2024-04-23 08:18:14.955 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:21
2024-04-23 08:18:15.522 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:22
2024-04-23 08:18:16.102 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:23
2024-04-23 08:18:16.698 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:24
2024-04-23 08:18:17.277 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:25
2024-04-23 08:18:17.829 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:26
2024-04-23 08:18:18.394 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:27
2024-04-23 08:18:18.923 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:28
2024-04-23 08:18:19.452 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:29
2024-04-23 08:18:20.009 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:30
2024-04-23 08:18:20.619 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:31
2024-04-23 08:18:21.157 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:32
2024-04-23 08:18:21.694 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:33
2024-04-23 08:18:22.319 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:34
2024-04-23 08:18:22.933 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:35
2024-04-23 08:18:23.501 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:36
2024-04-23 08:18:24.184 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:37
2024-04-23 08:18:24.841 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:38
2024-04-23 08:18:25.497 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:39
2024-04-23 08:18:26.114 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:40
2024-04-23 08:18:26.744 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:41
2024-04-23 08:18:27.325 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:42
2024-04-23 08:18:27.913 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:43
2024-04-23 08:18:28.492 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:44
2024-04-23 08:18:29.012 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:45
2024-04-23 08:18:29.578 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:46
2024-04-23 08:18:30.162 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:47
2024-04-23 08:18:30.716 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:48
2024-04-23 08:18:31.270 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:49
2024-04-23 08:18:31.854 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:50
2024-04-23 08:18:32.440 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:51
2024-04-23 08:18:33.046 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:52
2024-04-23 08:18:33.626 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:53
2024-04-23 08:18:34.191 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:54
2024-04-23 08:18:34.816 [info] [sender_kvdb.cc:InsertOrAssignWorker:417] Polynomial Interpolate and HE Plaintext Encode, bundle_indx:0, store_idx:55
Traceback (most recent call last):
  File "pir_setup.py", line 61, in <module>
  File "spu/pir.py", line 31, in pir_setup
RuntimeError: tried to interpolate at repeated points
[24825] Failed to execute script 'pir_setup' due to unhandled exception!

在进行3000万数据setup的时候，在第bucket_29的时候出错了。
而且指定bucket_size没有效果，都是固定的100万。
1000万，2000万数据都可以正常进行setup操作。
数据已经进行去重操作。

python /pir_setup.py \
--setup-data /opt/3000w.csv  \
--key-columns 'id' \
--label-column 'fenshu' \
--pir-setup-path /data/3000w_setup_path \
--pir-oprf-key-path /oprf_key.bin

执行命令

The version of some packages
SecretFlow: 1.3.0.dev20231109
SPU: 0.6.0b0
Ray: 2.2.0
Jax: 0.4.12
Jaxlib: 0.4.12
TensorFlow: 2.11.1
PyTorch: 2.0.0+cpu
@yyf1986 yyf1986 added the bug Something isn't working label Apr 23, 2024
@6fj
Copy link
Member

6fj commented Apr 23, 2024

请问可以用一下最新的代码吗，目前最新的secretflow版本是：1.5.0b0

@yyf1986
Copy link
Author

yyf1986 commented Apr 24, 2024

@6fj 我试试

@yyf1986
Copy link
Author

yyf1986 commented Apr 24, 2024

@6fj 新版本的可以了。另外有没有pir的一些性能数据阿

@yyf1986
Copy link
Author

yyf1986 commented Apr 25, 2024

setup预处理的时候，如何降低处理时间呢？

@Chrisdehe
Copy link
Member

我们正在关注此实现，敬请期待

@6fj
Copy link
Member

6fj commented May 14, 2024

可以尝试将label_max_len设置小一点

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@6fj 6fj

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@yyf1986 @6fj @Chrisdehe