-
Notifications
You must be signed in to change notification settings - Fork 4
/
Copy pathREADME
290 lines (204 loc) · 9.19 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
Model for an F Engine used by the Precision Array for Probing the Epoch of
Reionization.
The Model roachf_1024ch_ma.mdl is an 8 input design consisting of Digitizer,
PFB, Packetiser, Loopback, and output 10gbe. 2048 channel PFB.
One can select the total number of inputs in the design (which determines the
packet format). Supported number of inputs is from 16, 32, 64, 128, 256, 512
total inputs.
This mdl file was built using Xilinix 13.3 toolflow and
casper mlib devel library at #4655b346c5f629aa63ed5e50d38a75be4ef33102.
mlib devel can be found at [email protected]:zakiali/mlib_devel.git
REGISTERS OF NOTE
------------------
Monitor
-------
- 'switch_gbe_status' : monitors 10gbe outputs.
bitmap = {25bits of 0 |
gbe_up |
gbe_rx |
gbe_tx |
gbe_a_full |
gbe_tx_oflow |
gbe_rx_badframe |
gbe_rx_over_run}
- 'switch_gbe_badframe' : Counter value of gbe rx bad frame signal.
Note there is no posedge.
- 'switch_gbe_orun' : Counter value of gbe rx overrun signal. Note
there is no posedge.
- 'switch_gbe_oflow' : Counter value of gbe tx over flow signal. Note
there is no posedge.
- 'switch_cnts_data_msb(lsb)' : The cnts block counts the number of
clocks between packets etc.. use
provided software to read. May be
somewhat convoluted to decipher. Not
needed for functionality. Only for
testing 10gbe blocks.
- 'loopback_rx_cnt' : Counts the number of packets output of rx packet
fifo. Reset by 'cnt_rst' signal.
- 'loopback_rx_err_cnt' : Counts the number of rx fifo overfull
signals. Reset by 'cnt_rst' signal.
- 'loopback_loop_err_cnt' : Counts the number of loopback fifo overfull
signals. Reset by 'cnt_rst' signal.
- 'loopback_loop_cnt' : Counts the number of packets output of loop
packet fifo. Reset by 'cnt_rst' signal. This
number should be
(1/number of roaches) * loopback_rx_cnt.
- 'loopback_rx_pkt_too_big' : Count of number to oversize packets
coming from the switch.
- 'loopback_rx_pkt_too_small' : Count of number to undersize packets
coming from the switch.
- 'loopback_rx_over' : Second check of oversize packets from switch.
NOT NECESSARY.
- 'loopback_rx_under' : Second check of undersize packets from switch.
NOT NECESSARY.
- 'loopback_loop_over' : Count of number of oversize packets from
loopback fifo.
- 'loopback_loop_under' : Count of number of undersize packets from
loopback fifo.
- 'loopback_cnts.,,,' : Same as cnts brams in switch block. Counts
differences between gbe_read and loop_read
signals. Was used for debugging. Not needed
anymore.
- 'gpu_mcnt_lsb(msb)' : Instantaneous header of data about to be sent
out over 10gbe. LSB = {bottom 16 of mcnt |
Xid | Fid} MSB = top 32 bits of mcnt.
- 'gpu_gbe_status' : see 'switch_gbe_status.'
- 'adc_rms_levels' : bram of rms values of the inputs. Note that the
ctrl register must be written to in order for it
to run. addresses:
0: 24bit sum of 2^16 samples of input 0.
1: 32bit sum of squares of the samples in 0.
2: 24bit sum of 2^16 samples of input 1.
3: 32bit sum of squares of the samples in 2.
etc.....
LEDs
------
led0 : armed signal
led1 : sync pulse
led2 : eq clip
led3 : adc clip
SNAP BLOCKS
-------------
-'switch_txsnap' + 'bram_msb', 'bram_lsb' , 'bram_oob', 'ctrl', 'addr':
saves meta data in oob, lsb(msb) of data. Triggered by writing
to ctrl signal. See scripts to read from it.
-'switch_rxsnap' + 'bram_msb', 'bram_lsb' , 'bram_oob', 'ctrl', 'addr':
saves meta data in oob, lsb(msb) of data. Triggered by writing
to ctrl signal. See scripts to read from it.
-'gpu_txsnap' + 'bram_msb', 'bram_lsb' , 'bram_oob', 'ctrl', 'addr':
saves meta data in oob, lsb(msb) of data. Triggered by writing
to ctrl signal. See scripts to read from it.
-'adc_snap_0_3'(4_7) : use 'adc_snap_ctrl' to control snaps.
1024 samples of 8bit input data.
Format ={[0123]_0,[0123]_0...,[0123]_1023}
and {[4567]_0,[4567]_1,...,[4567]_1023}
CONTROL BLOCKS
---------------
- 'input_selector' : Selects the type of input. 32bit value. Every
nibble corresponds to an input. lsb is input 0, msb is input 7. If
writtin in hex format, every hex digit is an input. value of
3=digital zero, 2=second digital noise, 1=first digital noise, 0=adc
inputs. e.g. 0x33333210 = input0 has adcs on, input 1 has digital
noise 1, input 2 has digital noise 2 on , inputs3-7 have dig zero.
- 'delay_values' : introduces a course delay into input signal path.
can delay up to 16 fpga clocks. lsb is input 0, msb is input 7.
- 'seed_data' : introduces new seed into digital noise sources. every
byte corresponds to a different wgn generator. note that this value
must be set before we arm/sync up the roaches to one another.
- 'gpu_port' : destination port for gpu host machine. default is 8511.
- 'gpu_ip' : destination ip address for gpu host machine.
- 'gbe_sw_port' : destination port for packets destined for switch.
- 'my_ip' : ipaddress of this f engine.
- 'ip_base' : base ip address of packets destined for switch.
- 'ant_base_offset' : This is actually the F-Engine ID. Determines
which input numbers are on this roach.
- 'fft_shift' : sets shift schedule for ffts.
- 'n_inputs' : Register holds the number of total inputs in the
correlator. This determines which transpose/packetiser to use. It is
a 3 bit value. The number is determined by log2(ninputs) - 4.
- 'feng_ctl'* : sets various control bits. Bit map below...
('gbe_gpu_rst'), #31 reset gbe block going
to gpu machines.
('gbe_sw_rst'), #30 reset gbe block going
to gpu machines.
('loopback_mux_rst'), #29 reset some loopback
specific stuff.
Currently not used,
except for
determining proper
header in mux.
('cnt_rst'), #28 resets various
counters (error and
the such) in the
design.
('fft_preshift',2), #26-27 can preshift (down)
into fft by up to 3
bits. 0=noshift(def)
1=1 bit, etc...
('gpio_monsel',3), #23,24,25 DEPRECATED. Not used
at all.
('fft_tvg2'), #22 use fft test vector
generator 2. Don't
really use this.
Digital noise
sources are better.
('fft_tvg1'), #21 use fft test vector
generator 2. Don't
really use this.
Digital noise
sources are better.
('gbe_gpu_disable'), #20 Disable gpu 10gbe
block. (Valid signal
always low)
('use_qdr_tvg'), #19 (actually called
use_sram_tvg). Use
post transpose test
vector. Never Used.
('gbe_sw_disable'), #18 Disable switch 10gbe
block. (Valid signal
always low)
('arm_rst'), #17 Rearm the roachs
(for syncup). Note
this must go from 0
to 1, for rearm to
take effect.
('sync_rst'), #16 Reset sync pulse
manually. This is
never used, since we
have a 1pps
distributor.
Padding(13), #3-15 nada.
('lb_err_cnt_rst'), #2 Reset the cnts
blocks in switch and
loopback.
Padding(2)) #0-1 nada.
* Some notes about this block : The 'gbe_gpu_disable' and
'gbe_sw_disable' signals need to be high
before the 10gbe cores are reset with. If not,
the cores will not reset and will not
transmit. Also the cores are held in rst
until all the network stuff is set up. Then
the final start up of the F Engine is to take
the 10gbe cores out of reset and and undo the
disable. The gbe disable signals make sure
that no valid data goes into the cores, and
therefore no data gets sent to the 10gbe
cores.
NOTES
=======
1. The 2 10gbe cores in the design are different. The core to the
switch is the 10gbe_v2 core, where as the core to the GPU's is the
10gbe (original) core. It requires a special liscense from Xilinx to
be compiled with.
2. In the initialization scripts, which can be found in the corr
package at [email protected]:zakiali/PAPERCORR.git, both the 10gbe
cores are started the same way (as in I don't use tgtap to start
them up). This was due to the fact that sometimes we did not have
all ROACHs set up for testing, but we were sending all the packets
to the switch. The packets that were destined for non existant
ROACHs were then interpreted as broadcast packets and sent
everywhere, corrupting data integrity. Therefore, I populate the
arp tables manually, including all the mac addresses that are
supposed to be there (non existant ROACHs).
3. The name of the gbe cores relates to the cx4 ports used on the
roachs. The number in the name is the port used (starting from 0).