what's the test data?

[hitxiaoting]

Hi,

I tried to test the trained model, but I can't find the right test data in the config.py (TEST_DATA = '/datasets/liver/lspig_affine_test_3d.h5'). It seems that you used the preprocessed test data? can you provide the test data?

[hitxiaoting]

2. Why are the init_dice and dice unchanged during the first 100 ep? Is there something wrong with my training?

there is the log:
Number of data in combine-train is 1025
Number of data in sliver-val is 19
Number of data in lits-val is 131
Number of data in lspig-val is 34
start training!
1 0.7367 0.7367
2 0.7424 0.7424
3 0.8181 0.8181
4 0.8623 0.8623
5 0.7958 0.7958
6 0.8404 0.8404
7 0.8432 0.8432
8 0.8039 0.8039
9 0.7727 0.7727
10 0.6873 0.6873
11 0.8171 0.8171
12 0.7196 0.7196
13 0.7786 0.7786
14 0.7673 0.7673
15 0.4266 0.4266
16 0.6368 0.6368
17 0.7970 0.7970
18 0.7799 0.7799
19 0.8095 0.8095
20 0.7448 0.7448
21 0.5320 0.5320
22 0.7305 0.7305
23 0.4475 0.4475
24 0.7091 0.7091
25 0.6955 0.6955
26 0.8044 0.8044
27 0.8705 0.8705
28 0.8099 0.8099
29 0.8085 0.8085
30 0.7590 0.7590
31 0.8467 0.8467
32 0.7659 0.7659
33 0.4205 0.4205
34 0.8080 0.8080
35 0.8509 0.8509
36 0.6484 0.6484
37 0.8460 0.8460
38 0.7490 0.7490
39 0.7785 0.7785
40 0.8021 0.8021
41 0.5664 0.5664
42 0.8024 0.8024
43 0.7085 0.7085
44 0.8203 0.8203
45 0.7671 0.7671
46 0.8082 0.8082
47 0.8068 0.8068
48 0.7761 0.7761
49 0.6964 0.6964
50 0.9018 0.9018
51 0.7247 0.7247
52 0.7643 0.7643
53 0.7813 0.7813
54 0.7564 0.7564
55 0.7361 0.7361
56 0.7870 0.7870
57 0.7829 0.7829
58 0.8236 0.8236
59 0.8188 0.8188
60 0.6168 0.6168
61 0.7109 0.7109
62 0.7636 0.7636
63 0.7866 0.7866
64 0.7220 0.7220
65 0.7810 0.7810
66 0.5895 0.5895
67 0.7467 0.7467
68 0.6633 0.6633
69 0.8002 0.8002
70 0.7011 0.7011
71 0.8119 0.8119
72 0.7412 0.7412
73 0.7666 0.7666
74 0.7417 0.7417
75 0.7154 0.7154
76 0.7312 0.7312
77 0.7254 0.7254
78 0.8062 0.8062
79 0.7590 0.7590
80 0.7032 0.7032
81 0.8269 0.8269
82 0.7603 0.7603
83 0.7347 0.7347
84 0.8269 0.8269
85 0.7470 0.7470
86 0.8150 0.8150
87 0.7413 0.7413
88 0.7485 0.7485
89 0.7555 0.7555
90 0.7575 0.7575
91 0.7978 0.7978
92 0.7066 0.7066
93 0.7734 0.7734
94 0.7871 0.7871
95 0.8375 0.8375
96 0.7074 0.7074
97 0.7436 0.7436
98 0.7733 0.7733
99 0.7960 0.7960
------------------------Saving model------------------
------------------------Model saved!------------------
ep-100- 7: -- critic: 225.4139, alpha: 0.3679, reg: 1.5779, reward: 0.000, score: 0.924, init_score: 0.924
ep-100-15: -- critic: 163.2971, alpha: 0.3679, reg: 1.5373, reward: 0.000, score: 0.924, init_score: 0.924
ep-110- 7: -- critic: 194.0040, alpha: 0.3681, reg: 1.5689, reward: 0.000, score: 0.805, init_score: 0.805
ep-110-15: -- critic: 185.1218, alpha: 0.3681, reg: 1.5527, reward: 0.000, score: 0.805, init_score: 0.805
ep-120- 7: -- critic: 163.0904, alpha: 0.3682, reg: 1.5526, reward: 0.000, score: 0.820, init_score: 0.820
ep-120-15: -- critic: 201.7190, alpha: 0.3683, reg: 1.5674, reward: 0.000, score: 0.820, init_score: 0.820

[Algolzw]

Hi,

I tried to test the trained model, but I can't find the right test data in the config.py (TEST_DATA = '/datasets/liver/lspig_affine_test_3d.h5'). It seems that you used the preprocessed test data? can you provide the test data?

1. For the first question, we just perform a very simple affine transformation to pre-align test images, please refer to this code.

2. The training log is fine because we have set the "bootstrap epoch" to 100 in the config file [EP_BOOTSTRAP]. That means the agent would like to randomly run 100 epochs without learning it. And the learning process will start after BOOTSTRAP epochs.

[Baichenjia]

Hi, Algolzw @Algolzw . I also have some problem about the right test data.

If I set the test data to be e.g., TEST_DATA = 'datasets/lspig_val.h5', then I have error like
KeyError: "Unable to open object (object 'moving_0' doesn't exist)"

It seems that training data cannot be used for testing since the BrainData class in dataloader.py have different key settings compared to the training data. BrainData class is written for some specificed datasets. Should I writte a new BrainData class for testing?

Thank you very much!

[Algolzw]

@Baichenjia

In our work, the testing data are preprocessed and saved in a specific format. You absolutely can save your preprocessed data in your format and rebuild a new dataloader for it.

Good luck!

[Baichenjia]

Thanks for your reply.

Another quick question is why the dice score seems also high even before traning (i.e., 0.7~0.8)?

[Algolzw]

That is caused by the large initial overlapping between fixed and moving 3D images (as shown in Fig.4 in our paper).

[rraayyii]

In your training log, why is the score from epoch 100 higher than that from epoch 120?

Also, why is the reward metric always zero?