-
Notifications
You must be signed in to change notification settings - Fork 0
/
search.xml
1139 lines (551 loc) · 527 KB
/
search.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<?xml version="1.0" encoding="utf-8"?>
<search>
<entry>
<title>ApacheDoris异常案例：The statement has been forwarded to master FE(10.x.x.235) and failed to execute</title>
<link href="/2023/05/15/Doris%E5%BC%82%E5%B8%B8-ForwardToMaster/"/>
<url>/2023/05/15/Doris%E5%BC%82%E5%B8%B8-ForwardToMaster/</url>
<content type="html"><![CDATA[<blockquote><p>apache doris fe节点可以通过部署奇数个Follower构成高可用集群，集群中的master角色，会从这些Follower中选举产生。</p></blockquote><blockquote><p>如果你并不十分了解 FE 元数据的运行逻辑，或者没有足够 FE 元数据的运维经验，我们强烈建议在实际使用中，只部署一个 FOLLOWER 类型的 FE 作为 MASTER，其余 FE 都是 OBSERVER，这样可以减少很多复杂的运维问题！后面这句话，来自官方文档：<a href="https://doris.apache.org/zh-CN/docs/dev/admin-manual/maint-monitor/metadata-operation#%E6%9C%80%E4%BD%B3%E5%AE%9E%E8%B7%B5">ApacheDoris元数据运维-最佳实践</a> 说的很中肯，最近连着遇到了两次元数据的问题😂</p></blockquote><h2 id="起因">起因</h2><p>业务方抛过来一张图：<br><img src="/images/20230515/01/01.png" alt="" loading="lazy"><br>表示在通过jdbc执行<code>alter table</code>语句时，发生了如下的异常。让我帮忙排查排查。</p><h2 id="分析">分析</h2><p>因为<code>alter table</code> 语句，涉及到元数据的修改，原则上只能在master节点上修改，难道是master节点出了问题？<code>show proc '/frontends'</code>看看：</p><p><img src="/images/20230515/01/02.png" alt="" loading="lazy"></p><p>图中显示：</p><ul><li>当前的master节点是243不是235</li><li>fe节点状态都是正常的</li></ul><p>看着应该是有节点的信息存在问题，识别错误master节点了。于是从其余的几个fe节点上排查日志，根据<code>The statement has been forwarded to master </code>这个日志，进行查找，没有找到相同的问题。看了一下日志，从Error和Warn也没有看到日有用的信息。那么到底是谁转发的呢？求助与社区，社区建议将235节点drop掉，再加入。这。。。还是自力更生，从代码中看看有没有关键信息吧：</p><pre class="language-java" data-language="java"><code class="language-java"><span class="token comment">//doris查询处理流程：</span><span class="token class-name">ConnectProcessor</span>#dispatch <span class="token operator">-></span> <span class="token class-name">ConnectProcessor</span>#handleQuery <span class="token operator">-></span> <span class="token class-name">StmtExecutor</span>#execute</code></pre><p>快到关键地方了：</p><pre class="language-java" data-language="java"><code class="language-java"><span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">isForwardToMaster</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>isProxy<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token comment">// This is already a stmt forwarded from other FE.</span> <span class="token comment">// If goes here, which means we can't find a valid Master FE(some error happens).</span> <span class="token comment">// To avoid endless forward, throw exception here.</span> <span class="token keyword">throw</span> <span class="token keyword">new</span> <span class="token class-name">UserException</span><span class="token punctuation">(</span><span class="token string">"The statement has been forwarded to master FE("</span> <span class="token operator">+</span> <span class="token class-name">Catalog</span><span class="token punctuation">.</span><span class="token function">getCurrentCatalog</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">getSelfNode</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span>first <span class="token operator">+</span> <span class="token string">") and failed to execute"</span> <span class="token operator">+</span> <span class="token string">" because Master FE is not ready. You may need to check FE's status"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token function">forwardToMaster</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>masterOpExecutor <span class="token operator">!=</span> <span class="token keyword">null</span> <span class="token operator">&&</span> masterOpExecutor<span class="token punctuation">.</span><span class="token function">getQueryId</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">!=</span> <span class="token keyword">null</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> context<span class="token punctuation">.</span><span class="token function">setQueryId</span><span class="token punctuation">(</span>masterOpExecutor<span class="token punctuation">.</span><span class="token function">getQueryId</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">return</span><span class="token punctuation">;</span><span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span> <span class="token constant">LOG</span><span class="token punctuation">.</span><span class="token function">debug</span><span class="token punctuation">(</span><span class="token string">"no need to transfer to Master. stmt: {}"</span><span class="token punctuation">,</span> context<span class="token punctuation">.</span><span class="token function">getStmtId</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">}</span></code></pre><p>判断这条语句是否需要转发的master执行，那么alter语句是需要转到master的，同时本机不是正确的master。继续往下看，到<code>MasterOpExecutor#forward</code>方法：</p><pre class="language-java" data-language="java"><code class="language-java"><span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token keyword">null</span> <span class="token operator">!=</span> ctx<span class="token punctuation">.</span><span class="token function">queryId</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> params<span class="token punctuation">.</span><span class="token function">setQueryId</span><span class="token punctuation">(</span>ctx<span class="token punctuation">.</span><span class="token function">queryId</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">}</span><span class="token constant">LOG</span><span class="token punctuation">.</span><span class="token function">info</span><span class="token punctuation">(</span><span class="token string">"Forward statement {} to Master {}"</span><span class="token punctuation">,</span> ctx<span class="token punctuation">.</span><span class="token function">getStmtId</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> thriftAddress<span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">boolean</span> isReturnToPool <span class="token operator">=</span> <span class="token boolean">false</span><span class="token punctuation">;</span></code></pre><p>出现了一条日志，很关键，按图索骥，一台一台机器查找，最终在150这台observer上见到了：<br><img src="/images/20230515/01/03.png" alt="" loading="lazy"><br>可以看到这台机器将请求转发到了235，可是243才是master节点啊。说明这个节点记录的master信息有问题。</p><h2 id="解决办法">解决办法</h2><p>执行重启大法。重启后，再看日志:<br><img src="/images/20230515/01/04.png" alt="" loading="lazy"><br>转发正常了。</p><h2 id="后续">后续</h2><p>此次解决了问题，但是具体为什么不一致，还没有排查出来。没想到，没过几天，又发生了类似的问题，预知后续如何，且待下回分解。。。</p>]]></content>
<categories>
<category> doris </category>
</categories>
<tags>
<tag> doris </tag>
</tags>
</entry>
<entry>
<title>linux系统No space left on device的几种排查解决方法</title>
<link href="/2023/04/06/Linux%E7%A3%81%E7%9B%98%E6%BB%A1%E4%BA%86%E7%9A%84%E5%87%A0%E7%A7%8D%E6%8E%92%E6%9F%A5%E5%A4%84%E7%90%86%E6%96%B9%E6%B3%95/"/>
<url>/2023/04/06/Linux%E7%A3%81%E7%9B%98%E6%BB%A1%E4%BA%86%E7%9A%84%E5%87%A0%E7%A7%8D%E6%8E%92%E6%9F%A5%E5%A4%84%E7%90%86%E6%96%B9%E6%B3%95/</url>
<content type="html"><![CDATA[<p>在工作中会遇到No space left on device的异常，下面总结一下问题的排查和解决思路：</p><h2 id="磁盘空间真的满了">磁盘空间真的满了</h2><p>执行<code>df -h</code>命令可以看到磁盘的占用情况：</p><pre class="language-none"><code class="language-none">Filesystem Size Used Avail Use% Mounted on/dev/vda1 36G 5.7G 28G 17% /tmpfs 1.9G 209M 1.7G 12% /run/dev/vdb 99G 5.7G 88G 7% /home</code></pre><p>如果磁盘满了，use%会显示为100%。可以执行<code>du -h --max-depth=1</code>继续查看具体哪个目录占用空间大。执行<code> find /home -type f -size +10M -print0 | xargs -0 du -h | sort -nr</code>命令查找出大文件，并排序。</p><p>找到大文件后，可以执行进行删除操作，如果文件删除后，空间还没有变小。需要检查文件是否还被进程持有(<code>lsof 文件名称</code>)，需要杀死进程，才会完全释放空间。</p><h2 id="磁盘空间没有满，但是inode满了">磁盘空间没有满，但是inode满了</h2><p>执行<code>df -i</code>命令可以看到inode的使用情况，inode满了，说明可能存在大量的小文件，此时删除部分文件就可以解决</p><pre class="language-none"><code class="language-none">Filesystem Inodes IUsed IFree IUse% Mounted on/dev/vda1 2359296 142821 2216475 7% /tmpfs 485127 16 485111 1% /sys/fs/cgroup/dev/vdb 6553600 666557 5887043 11% /home</code></pre>]]></content>
<categories>
<category> linux </category>
</categories>
<tags>
<tag> linux </tag>
</tags>
</entry>
<entry>
<title>ApacheDoris快速运维手册</title>
<link href="/2023/02/06/ApacheDoris%E5%BF%AB%E9%80%9F%E8%BF%90%E7%BB%B4%E6%89%8B%E5%86%8C/"/>
<url>/2023/02/06/ApacheDoris%E5%BF%AB%E9%80%9F%E8%BF%90%E7%BB%B4%E6%89%8B%E5%86%8C/</url>
<content type="html"><![CDATA[<blockquote><p>本手册用于快速排查Doris的问题，并进行紧急修复。持续更新中。。。</p></blockquote><h1>基础设置</h1><ul><li><p>调整数据库容量</p><pre class="language-sql" data-language="sql"><code class="language-sql"><span class="token keyword">ALTER</span> <span class="token keyword">DATABASE</span> xxx <span class="token keyword">SET</span> <span class="token keyword">DATA</span> QUOTA <span class="token number">100</span>T<span class="token punctuation">;</span></code></pre></li><li><p>修改数据库链接</p><pre class="language-sql" data-language="sql"><code class="language-sql"><span class="token keyword">SHOW</span> PROPERTY <span class="token keyword">FOR</span> <span class="token string">'xxx'</span> <span class="token operator">LIKE</span> <span class="token string">'%max_user_connections%'</span><span class="token punctuation">;</span><span class="token keyword">SET</span> PROPERTY <span class="token keyword">FOR</span> <span class="token string">'xxx'</span> <span class="token string">'max_user_connections'</span> <span class="token operator">=</span> <span class="token string">'300'</span><span class="token punctuation">;</span></code></pre></li></ul><h1>查询</h1><ul><li><p>修改超时时间</p><pre class="language-sql" data-language="sql"><code class="language-sql"><span class="token keyword">SET</span> query_timeout <span class="token operator">=</span> <span class="token number">1800</span><span class="token punctuation">;</span></code></pre></li></ul><h1>导入</h1><ul><li><p>修改单次导入数据上限</p><pre class="language-sql" data-language="sql"><code class="language-sql">streaming_load_json_max_mb<span class="token operator">=</span><span class="token number">10240</span></code></pre></li></ul><h1>分区&副本</h1><ul><li><p>修改副本数目</p><pre class="language-sql" data-language="sql"><code class="language-sql"><span class="token keyword">ALTER</span> <span class="token keyword">TABLE</span> tb_xxx <span class="token keyword">SET</span><span class="token punctuation">(</span><span class="token string">"dynamic_partition.replication_num"</span> <span class="token operator">=</span> <span class="token string">"3"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></li><li><p>查看分区状态</p><pre class="language-sql" data-language="sql"><code class="language-sql"><span class="token keyword">SHOW</span> DYNAMIC <span class="token keyword">PARTITION</span> <span class="token keyword">TABLES</span><span class="token punctuation">;</span><span class="token keyword">SHOW</span> <span class="token punctuation">[</span><span class="token keyword">TEMPORARY</span><span class="token punctuation">]</span> PARTITIONS <span class="token keyword">FROM</span> <span class="token punctuation">[</span>db_name<span class="token punctuation">.</span><span class="token punctuation">]</span>table_name <span class="token punctuation">[</span><span class="token keyword">WHERE</span><span class="token punctuation">]</span> <span class="token punctuation">[</span><span class="token keyword">ORDER</span> <span class="token keyword">BY</span><span class="token punctuation">]</span> <span class="token punctuation">[</span><span class="token keyword">LIMIT</span><span class="token punctuation">]</span><span class="token punctuation">;</span></code></pre></li><li><p>查看表全部的副本状态</p><pre class="language-sql" data-language="sql"><code class="language-sql">ADMIN <span class="token keyword">SHOW</span> REPLICA <span class="token keyword">STATUS</span> <span class="token keyword">FROM</span> db1<span class="token punctuation">.</span>tbl1<span class="token punctuation">;</span>ADMIN <span class="token keyword">SHOW</span> REPLICA <span class="token keyword">STATUS</span> <span class="token keyword">FROM</span> tbl1 <span class="token keyword">PARTITION</span> <span class="token punctuation">(</span>p1<span class="token punctuation">,</span> p2<span class="token punctuation">)</span> <span class="token keyword">WHERE</span> <span class="token keyword">STATUS</span> <span class="token operator">=</span> <span class="token string">"VERSION_ERROR"</span><span class="token punctuation">;</span>ADMIN <span class="token keyword">SHOW</span> REPLICA <span class="token keyword">STATUS</span> <span class="token keyword">FROM</span> tbl1 <span class="token keyword">WHERE</span> <span class="token keyword">STATUS</span> <span class="token operator">!=</span> <span class="token string">"OK"</span><span class="token punctuation">;</span></code></pre></li><li><p>动态分区删除副本</p><pre class="language-sql" data-language="sql"><code class="language-sql"><span class="token keyword">ALTER</span> <span class="token keyword">TABLE</span> tb_xxx <span class="token keyword">SET</span> <span class="token punctuation">(</span><span class="token string">"dynamic_partition.enable"</span> <span class="token operator">=</span> <span class="token string">"false"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">alter</span> <span class="token keyword">table</span> tb_xxx <span class="token keyword">drop</span> <span class="token keyword">partition</span> p20211129<span class="token keyword">ALTER</span> <span class="token keyword">TABLE</span> tb_xxx <span class="token keyword">SET</span> <span class="token punctuation">(</span><span class="token string">"dynamic_partition.enable"</span> <span class="token operator">=</span> <span class="token string">"true"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></li></ul><h1>杂项</h1><ul><li><p>批量执行sql</p><pre class="language-bash" data-language="bash"><code class="language-bash"><span class="token function">cat</span> xxx.sql <span class="token operator">|</span><span class="token function">xargs</span> <span class="token parameter variable">-d</span> <span class="token string">"<span class="token entity" title="\r">\r</span><span class="token entity" title="\n">\n</span>"</span> ./mysql <span class="token parameter variable">-uxx</span> <span class="token parameter variable">-P9030</span> <span class="token parameter variable">-e</span> <span class="token parameter variable">-h10.xx.xx.xx</span> <span class="token parameter variable">-pxxx</span></code></pre></li></ul><h1>问题处理</h1><ul><li><p>更改表结构时报错：Create replicas failed. Error: Error replicas: xxx=xxx</p><p>原因：更改表结构超时 <a href="https://github.com/apache/doris/issues/2942">https://github.com/apache/doris/issues/2942</a></p><p>解决办法：</p><pre class="language-bash" data-language="bash"><code class="language-bash">ADMIN SET FRONTEND CONFIG <span class="token punctuation">(</span><span class="token string">"max_create_table_timeout_second"</span> <span class="token operator">=</span> <span class="token string">"600"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></li></ul><p></p>]]></content>
<categories>
<category> doris </category>
</categories>
<tags>
<tag> doris </tag>
</tags>
</entry>
<entry>
<title>2021年总结</title>
<link href="/2022/02/01/2021%E5%B9%B4%E6%80%BB%E7%BB%93/"/>
<url>/2022/02/01/2021%E5%B9%B4%E6%80%BB%E7%BB%93/</url>
<content type="html"><![CDATA[<p> 大年初一，这个时候本该回到老家过春节，磕着瓜子唠着嗑，却因为突如其来的疫情，再次被封锁到杭州。老婆孩子以及父母都远在千里之外，自己孤零零在这里度过第三个春节，还是蛮凄凉的。核酸已做了四次，疫情看着有好转的迹象，不得不佩服杭州的行政效率真的很高。闲来无事，想起年终总结尚未做，便仔细回顾记录一下。</p><p> 2021年发生了太多事情，目不暇接，对生活以及活着的意义也有了更多的感悟。</p><p> 先是1月份女儿出生，乌黑浓密的头发，娇小的四肢，嘹亮的哭声，整个病房数她声音最亮，脸色因为黄疸显得黑黑的，和隔壁的小朋友一对比，心都凉透了。好在越长越漂亮，现在长成了一个非常活泼可爱的小姑娘了。</p><p> 紧接着二姨在4月份因为精神问题寻短见去世了，奶奶10月份在床上躺了两年后也去世了。每次猛然想起，浓浓的不敢相信和深深的悲伤。回想和二姨视频通话没还没有多久，人就没了，悲从中来，不能自已。二姨家有三个孩子，大女儿大四了，二女儿在上初中，小儿子在读小学，家里经济压力很大。之前听说精神不太好，到市医院检查吃药，药中途吃了觉得没用，就停了。在农村，所有人都在忙碌，一下子没有看好，人就没了。</p><p> 国庆节回家，补办了因为疫情阻隔的婚礼。二姨坟上花圈还在，只剩下悲伤。奶奶躺在床上，整个人瘦的只剩下皮包骨头，没有意识，不知道他的孙子抱着重孙女来看她了。两年前回家，奶奶还坐在椅子上和我们说话；小学期间，和我、我妹妹、我堂弟三人挤在狭小的出租屋了，给我们做饭的日子仿佛还在昨天。国庆节温度下降比较多，天很冷，奶奶终究没有扛过这个冬天。爷爷也很瘦了，大姑、舅舅也老了，大姑的白头发很多，二爹整个人晒的黝黑黝黑，二妈脸上也全都是风吹日晒的痕迹。在外面工作4年多了，和父母、亲戚聚少离多，有时候想想真的不知道自己究竟在追求些什么，自己离家千里，到底在图些什么。</p><p> 老婆生完孩子，我妈妈在照顾，各种婆媳矛盾，大家都心力憔悴。待了几个月，回老家住了。想想年后，过来上班，我妈妈继续过来带娃，就感觉很头疼。经过这一年，我也在不断反思，希望可以处理好这些事情。</p><p> 工作上，上半年我在基础技术部负责服务访问框架的研发，换了领导、换了小组，再加上家里的矛盾，整个人的状态很不好。下半年，换到数据部门，负责OLAP引擎的调研和落地工作，团队的氛围很不错，精神状态缓过来了。感觉这一年自己的成长总体来说很大，虽然博客写的比较少，但是看了不少书，做了挺多的思考和总结，自己的知识体系逐渐串起来了。接下来重点在于纵深，清理模糊不清楚的地方，在开源社区做一些贡献，提升影响力。</p>]]></content>
<categories>
<category> 随笔 </category>
</categories>
<tags>
<tag> 随笔 </tag>
<tag> 总结 </tag>
</tags>
</entry>
<entry>
<title>读书笔记：《MySql是怎样运行的》 事务及MVCC</title>
<link href="/2021/06/19/%E6%95%B0%E6%8D%AE%E5%BA%93%E4%BA%8B%E5%8A%A1%E5%8F%8AMVCC/"/>
<url>/2021/06/19/%E6%95%B0%E6%8D%AE%E5%BA%93%E4%BA%8B%E5%8A%A1%E5%8F%8AMVCC/</url>
<content type="html"><![CDATA[<h1>事务特性</h1><p>ACID：原子性、一致性、隔离性和持久性</p><p><strong>原子性</strong>：要执行的事务是一个独立的操作单元，要么全部执行，要么全部不执行</p><p><strong>一致性</strong>：事务的一致性是指事务的执行不能破坏数据库的一致性，一致性也称为完整性。一个事务在执行后，数据库必须从一个一致性状态转变为另一个一致性状态。</p><p><strong>隔离性</strong>：多个事务并发执行时，一个事务的执行不应影响其他事务的执行。</p><p><strong>持久性（durability）</strong>：事务一旦提交，则其所有的修改将会保存到数据库当做。即使此时系统崩溃，修改的数据也不会丢失。</p><h1>事务并发执行的一些一致性问题</h1><h2 id="脏写">脏写</h2><p>如果一个事务修改了另一个未提交事务修改过的数据，说明发生了脏写现象。</p><h2 id="脏读">脏读</h2><p>如果一个事务读到了另外一个未提交事务修改过的数据，说明发生了脏读现象。</p><p>1、在<s>事务</s>A执行过程中，事务A对数据资源进行了修改，事务B读取了事务A修改后的数据。</p><p>2、由于某些原因，事务A并没有完成提交，发生了RollBack操作，则事务B读取的数据就是脏数据。</p><p><img src="/images/20210821/01.png" alt="" loading="lazy"></p><h2 id="不可重复读">不可重复读</h2><p>事务B读取了两次数据资源，在这两次读取的过程中事务A修改了数据，导致事务B在这两次读取出来的数据不一致。</p><p>这种在同一个事务中，前后两次读取的数据不一致的现象就是不可重复读(Nonrepeatable Read)。</p><p><img src="/images/20210821/02.png" alt="" loading="lazy"></p><h2 id="幻读">幻读</h2><p>事务B前后两次读取同一个范围的数据，在事务B两次读取的过程中事务A新增了数据，导致事务B后一次读取到前一次查询没有看到的行。</p><p>幻读和不可重复读有些类似，但是幻读强调的是集合的增减，而不是单条数据的更新。</p><p><img src="/images/20210821/03.png" alt="" loading="lazy"></p><h2 id="第一类更新丢失">第一类更新丢失</h2><p>事务A和事务B都对数据进行更新，但是事务A由于某种原因事务回滚了，把已经提交的事务B的更新数据给覆盖了。这种现象就是第一类更新丢失</p><p><img src="/images/20210821/04.png" alt="" loading="lazy"></p><h2 id="第二类更新丢失">第二类更新丢失</h2><p>其实跟第一类更新丢失有点类似，也是两个事务同时对数据进行更新，但是事务A的更新把已提交的事务B的更新数据给覆盖了。这种现象就是第二类更新丢失。</p><p><img src="/images/20210821/05.png" alt="" loading="lazy"></p><h1>事务级别</h1><p><img src="/images/20210821/06.png" alt="" loading="lazy"></p><h2 id="Mysql默认事务级别">Mysql默认事务级别</h2><p>可重复读</p><h1>MVCC原理</h1><blockquote><p>针对读未提交和可重复读级别生效</p></blockquote><h2 id="版本链">版本链</h2><p>聚簇索引记录中包含两个必要的隐藏列，一个是trx_id，另外一个是roll_pointer：</p><p>trx_id：一个事务每次对某条聚簇索引记录进行改动时，都会把该事务的事务id复制给trx_id隐藏列。</p><p>roll_pointer：每次对某条聚簇索引记录进行改动时，都会把旧的版本写入到undo日志中。这个隐藏列相当于一个指针，通过它可以找到该记录修改前的信息。</p><p>每对记录进行一次修改，都会将旧值放到一条undo日志。每条undo日志都有一个roll_pointer属性。（insert操作对应的undo日志没有此操作，因为没有更早的版本）。通过这个属性将undo日志串成一个链表。称为版本链。同时每个版本还包含生成该版本时对应的事务Id。</p><h2 id="ReadView">ReadView</h2><p>读未提交级别：直接读取该记录的最新版本就好。</p><p>序列化级别：采用加锁方式来访问记录。</p><p>读已提交和可重复读级别：读取的都是已提交事务修改的记录。</p><p>MVCC用于读已提交和可重复度级别。核心问题是：需要判断版本链中的哪个版本是当前事务可见的。</p><p>ReadView主要包含四个重要的内容：</p><ul><li><p>m_ids：在生成ReadView时，当前系统中活跃的读写事务的事务id列表</p></li><li><p>min_trx_id：在生成ReadView时，当前系统中活跃的读写事务中最小的事务id，也就是m_ids中的最小值。</p></li><li><p>max_trx_id：在生成ReadView时，系统应该分配给下一个事务的事务Id值。</p></li><li><p>create_trx_id：生成该ReadView的事务的事务Id。</p></li></ul><p>判断某个版本是否可见的步骤：</p><ol><li><p>如果被访问版本的trx_id和ReadView中的create_trx_id相同，说明当前事务在访问它自己修改过的记录，所以该版本可以被当前事务访问。</p></li><li><p>如果被访问版本的trx_id比ReadView中的min_trx_id小，说明生成该版本的事务在当前事务生成ReadView前已提交，所以该版本可以被当前事务访问。</p></li><li><p>如果被访问版本的trx_id属性大于或者等于ReadView中的max_trx_id，表明生成该版本的事务在当前事务生成ReadView后才开启，所以该版本不可以被当前事务访问。</p></li><li><p>如果被访问版本的trx_id属性位于ReadView的min_trx_id和max_trx_id之间，则需要判断trx_id属性是否在m_ids列表中。如果在，说明创建ReadView时生成该版本的事务是活跃的，该版本不可以被访问。如果不在，说明创建ReadView时，该版本已提交，该版本可以被访问。</p></li></ol><p>如果某个版本的数据对当前事务不可见，则沿着版本链找到下一个版本的数据，并沿用上面的步骤判断可见性。</p><h3 id="读已提交生成ReadView的时机">读已提交生成ReadView的时机</h3><p>每次查询开始都会生成一个独立的ReadView</p><h3 id="可重复读生成ReadView的时机">可重复读生成ReadView的时机</h3><p>只会在第一次查询的时候生成一个ReadView，之后的查询就不会重复生成ReadView了。</p><h3 id="二级索引与MVCC">二级索引与MVCC</h3><p>只有聚簇索引记录中才有trx_id和roll_pointer隐藏列，如果某个查询语句通过二级索引查询，需要通过如下步骤：</p><p>步骤一：</p><p>判断ReadView中min_trx_id的值，是否比二级索引页面PageHeader部分的PAGE_MAX_TRX_ID属性值大，如果是，说明可见；否则需要按照步骤二进行回表操作。</p><p>步骤二：</p><p>利用二级索引记录中的主键进行回表操作，得到对应的聚簇索引记录后再按照前面讲过的方式找到对该ReadView可见的第一个版本，然后判断相应的条件是否一致。</p><h1>参考</h1><p><a href="https://book.douban.com/subject/35231266/">MySql是怎样运行的</a></p><p><a href="https://blog.csdn.net/zl1zl2zl3/article/details/88542775"><strong>https://blog.csdn.net/zl1zl2zl3/article/details/88542775</strong></a></p><p><a href="https://zhuanlan.zhihu.com/p/347587789"><strong>https://zhuanlan.zhihu.com/p/347587789</strong></a></p><p><a href="https://zhuanlan.zhihu.com/p/117476959"><strong>https://zhuanlan.zhihu.com/p/117476959</strong></a></p>]]></content>
<categories>
<category> Mysql </category>
</categories>
<tags>
<tag> 数据库 </tag>
<tag> Mysql </tag>
<tag> 读书笔记 </tag>
</tags>
</entry>
<entry>
<title>字节码工具Javassist使用总结</title>
<link href="/2021/03/27/%E5%AD%97%E8%8A%82%E7%A0%81%E5%B7%A5%E5%85%B7Javassist%E4%BD%BF%E7%94%A8%E6%80%BB%E7%BB%93/"/>
<url>/2021/03/27/%E5%AD%97%E8%8A%82%E7%A0%81%E5%B7%A5%E5%85%B7Javassist%E4%BD%BF%E7%94%A8%E6%80%BB%E7%BB%93/</url>
<content type="html"><![CDATA[<h2 id="前言">前言</h2><p>Javassist作为一个字节码生成工具，在链路追踪、性能分析等领域都有广泛的应用。不过<a href="https://www.javassist.org/tutorial/tutorial.html">官方文档</a>只介绍了简单的用法，部分高级特性比如泛型显示、参数名称保留等没有详细介绍，同时网上的相关资料也比较缺乏。本文主要是结合开发过程中对Javassist的一些使用经验，总结了一下Javassist的基本用法和高级特性。</p><h2 id="基本用法">基本用法</h2><p>Javassist对Java的各种概念进行了一层抽象，比如CtClass代表Class、CtMethod代表Method、CtField代表Field等。ClassPool是CtClass的容器，我们创建类、修改类都需要通过ClassPool。</p><h3 id="新增类">新增类</h3><h4 id="示例">示例</h4><pre class="language-java" data-language="java"><code class="language-java"><span class="token annotation punctuation">@Test</span><span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token function">testCreateClass</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">throws</span> <span class="token class-name">Exception</span> <span class="token punctuation">{</span> <span class="token comment">//获取ClassPool实例</span> <span class="token class-name">ClassPool</span> classPool <span class="token operator">=</span> <span class="token class-name">ClassPool</span><span class="token punctuation">.</span><span class="token function">getDefault</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//指定类名</span> <span class="token class-name">CtClass</span> carCtClass <span class="token operator">=</span> classPool<span class="token punctuation">.</span><span class="token function">makeClass</span><span class="token punctuation">(</span><span class="token string">"com.demo.Car"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//新增字段</span> <span class="token class-name">CtField</span> ctField1 <span class="token operator">=</span> <span class="token class-name">CtField</span><span class="token punctuation">.</span><span class="token function">make</span><span class="token punctuation">(</span><span class="token string">"private String brand = \"world\";"</span><span class="token punctuation">,</span> carCtClass<span class="token punctuation">)</span><span class="token punctuation">;</span> carCtClass<span class="token punctuation">.</span><span class="token function">addField</span><span class="token punctuation">(</span>ctField1<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//创建方法形式1</span> <span class="token class-name">CtMethod</span> helloMethod1 <span class="token operator">=</span> <span class="token class-name">CtMethod</span><span class="token punctuation">.</span><span class="token function">make</span><span class="token punctuation">(</span><span class="token string">"public void run(){System.out.println(\"---run---\");}"</span><span class="token punctuation">,</span> carCtClass<span class="token punctuation">)</span><span class="token punctuation">;</span> carCtClass<span class="token punctuation">.</span><span class="token function">addMethod</span><span class="token punctuation">(</span>helloMethod1<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//创建方法形式2</span> <span class="token class-name">CtMethod</span> helloMethod2 <span class="token operator">=</span> <span class="token class-name">CtNewMethod</span><span class="token punctuation">.</span><span class="token function">make</span><span class="token punctuation">(</span><span class="token string">"public void stop(){System.out.println(\"---stop---\");}"</span><span class="token punctuation">,</span> carCtClass<span class="token punctuation">)</span><span class="token punctuation">;</span> carCtClass<span class="token punctuation">.</span><span class="token function">addMethod</span><span class="token punctuation">(</span>helloMethod2<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//创建方法形式3</span> <span class="token class-name">CtMethod</span> helloMethod3 <span class="token operator">=</span> <span class="token class-name">CtNewMethod</span><span class="token punctuation">.</span><span class="token function">make</span><span class="token punctuation">(</span><span class="token class-name">Modifier</span><span class="token punctuation">.</span><span class="token constant">PUBLIC</span><span class="token punctuation">,</span> <span class="token class-name">CtClass</span><span class="token punctuation">.</span>voidType<span class="token punctuation">,</span> <span class="token string">"slow"</span><span class="token punctuation">,</span> <span class="token keyword">null</span><span class="token punctuation">,</span> <span class="token keyword">null</span><span class="token punctuation">,</span> <span class="token string">"{System.out.println(\"---slow---\");}"</span><span class="token punctuation">,</span> carCtClass<span class="token punctuation">)</span><span class="token punctuation">;</span> carCtClass<span class="token punctuation">.</span><span class="token function">addMethod</span><span class="token punctuation">(</span>helloMethod3<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//添加构造器</span> <span class="token class-name">CtConstructor</span> ctConstructor <span class="token operator">=</span> <span class="token class-name">CtNewConstructor</span><span class="token punctuation">.</span><span class="token function">make</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">CtClass</span><span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">{</span>classPool<span class="token punctuation">.</span><span class="token function">getCtClass</span><span class="token punctuation">(</span><span class="token class-name">String</span><span class="token punctuation">.</span><span class="token keyword">class</span><span class="token punctuation">.</span><span class="token function">getName</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token keyword">null</span><span class="token punctuation">,</span> carCtClass<span class="token punctuation">)</span><span class="token punctuation">;</span> ctConstructor<span class="token punctuation">.</span><span class="token function">setBody</span><span class="token punctuation">(</span><span class="token string">"{ $0.brand = $1; }"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> carCtClass<span class="token punctuation">.</span><span class="token function">addConstructor</span><span class="token punctuation">(</span>ctConstructor<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//添加默认构造器</span> carCtClass<span class="token punctuation">.</span><span class="token function">addConstructor</span><span class="token punctuation">(</span><span class="token class-name">CtNewConstructor</span><span class="token punctuation">.</span><span class="token function">defaultConstructor</span><span class="token punctuation">(</span>carCtClass<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//将方法写入文件</span> carCtClass<span class="token punctuation">.</span><span class="token function">writeFile</span><span class="token punctuation">(</span><span class="token string">"/home/lxd/test"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">}</span></code></pre><p>执行上面的代码，我们就可以在<code>/home/lxd/test/com/demo/</code>目录下得到一个<code>Car.class</code>，将此文件拖入Idea，可以看到生成的代码如下：<br><img src="/images/20210327/01.png" alt="" loading="lazy"></p><h4 id="说明">说明</h4><ul><li>字段新增可以直接<code> new CtField()</code>，也可以通过<code>CtField.make()</code>来进行操作，方法以及构造函数的操作新增类似</li><li>方法和构造函数还分别有<code>CtNewMethod</code>和<code>CtNewConstructor</code>，这两个类里面封装了很多有用的静态方法，可以简化操作，比如<code>CtNewMethod</code>内封装有<code>getter</code>以及<code>setter</code>的快捷方法，<code>CtNewConstructor</code>内封装有<code>defaultConstructor</code>等方法</li><li>在javassist中，<code>$0</code>代表this，<code>$1 $2 ...</code>分别对应方法的具体参数，这些信息可以从官方文档中看到。</li></ul><h3 id="编辑类">编辑类</h3><h4 id="示例-2">示例</h4><pre class="language-java" data-language="java"><code class="language-java"><span class="token annotation punctuation">@Test</span><span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token function">modifyClass</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">throws</span> <span class="token class-name">Exception</span> <span class="token punctuation">{</span> <span class="token class-name">ClassPool</span> classPool <span class="token operator">=</span> <span class="token class-name">ClassPool</span><span class="token punctuation">.</span><span class="token function">getDefault</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token class-name">CtClass</span> ctClass <span class="token operator">=</span> classPool<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span><span class="token string">"com.example.Hello"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token class-name">CtMethod</span> ctMethod <span class="token operator">=</span> ctClass<span class="token punctuation">.</span><span class="token function">getDeclaredMethod</span><span class="token punctuation">(</span><span class="token string">"sayHello"</span><span class="token punctuation">,</span> <span class="token keyword">new</span> <span class="token class-name">CtClass</span><span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">{</span>classPool<span class="token punctuation">.</span><span class="token function">getCtClass</span><span class="token punctuation">(</span><span class="token class-name">String</span><span class="token punctuation">.</span><span class="token keyword">class</span><span class="token punctuation">.</span><span class="token function">getName</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> ctMethod<span class="token punctuation">.</span><span class="token function">insertBefore</span><span class="token punctuation">(</span><span class="token string">"System.out.println(\"---before---\");"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> ctMethod<span class="token punctuation">.</span><span class="token function">insertAfter</span><span class="token punctuation">(</span><span class="token string">"System.out.println(\"---after---\");"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//完全替换method内容</span> <span class="token comment">//ctMethod.setBody("{}");</span> ctClass<span class="token punctuation">.</span><span class="token function">writeFile</span><span class="token punctuation">(</span><span class="token string">"/home/lxd/test"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">}</span></code></pre><p>通过上面的代码，我们可以为<code>sayHello</code>方法前后分别插入相关的日志打印功能。</p><p><img src="/images/20210327/02.png" alt="" loading="lazy"></p><h4 id="说明-2">说明</h4><ul><li><code>CtMethod</code>有<code>insertBefore</code>和<code>insertAfter</code>方法，可以很方便的进行代码插桩。</li><li>将一个<code>class</code>类加载为<code>CtClass</code>后，我们就可以用CtField、CtMethod相关的抽象进行代码修改工作。</li><li><code>ClassPool.getDefault()</code>默认和<code>jvm</code>有相同的类检索路径，我们可以通过<code>pool.insertClassPath("xxxx");</code>的形式新增加类检索路径。</li></ul><h2 id="高级特性">高级特性</h2><p>如果我们想用一些比较高级的特性，比如添加注解、泛型展示等，需要对类文件有一个大概的了解。通过对类文件结构的了解，我们可以很方便的从Javassist中找到对应的API。同时有一点需要注意：大部分高级操作都有相对来说比较简便、封装程度比较高的API，不用直接用Javassist的Bytecode类库。Bytecode相对来说较底层，操作的话对字节码了解水平要求较高。</p><p><img src="/images/20210327/03.png" alt="" loading="lazy"></p><p>javassist.jar类文件结构如上图，相关class文件信息都可以从上面的两个包中找到相关抽象。</p><h3 id="类文件">类文件</h3><pre class="language-java" data-language="java"><code class="language-java"><span class="token class-name">ClassFile</span> <span class="token punctuation">{</span> u4 magic<span class="token punctuation">;</span> u2 minor_version<span class="token punctuation">;</span> u2 major_version<span class="token punctuation">;</span> u2 constant_pool_count<span class="token punctuation">;</span> cp_info constant_pool<span class="token punctuation">[</span>constant_pool_count<span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">;</span> u2 access_flags<span class="token punctuation">;</span> u2 this_class<span class="token punctuation">;</span> u2 super_class<span class="token punctuation">;</span> u2 interfaces_count<span class="token punctuation">;</span> u2 interfaces<span class="token punctuation">[</span>interfaces_count<span class="token punctuation">]</span><span class="token punctuation">;</span> u2 fields_count<span class="token punctuation">;</span> field_info fields<span class="token punctuation">[</span>fields_count<span class="token punctuation">]</span><span class="token punctuation">;</span> u2 methods_count<span class="token punctuation">;</span> method_info methods<span class="token punctuation">[</span>methods_count<span class="token punctuation">]</span><span class="token punctuation">;</span> u2 attributes_count<span class="token punctuation">;</span> attribute_info attributes<span class="token punctuation">[</span>attributes_count<span class="token punctuation">]</span><span class="token punctuation">;</span><span class="token punctuation">}</span></code></pre><p>这里给出类文件的基本结构，详细各种属性，可以参考<a href="https://docs.oracle.com/javase/specs/jvms/se8/jvms8.pdf">jvm规范</a>。</p><h3 id="示例类">示例类</h3><p>接下来的高级特性演示主要针对如下的<code>Hello</code>类：</p><pre class="language-java" data-language="java"><code class="language-java"><span class="token keyword">package</span> <span class="token namespace">com<span class="token punctuation">.</span>example</span><span class="token punctuation">;</span><span class="token keyword">public</span> <span class="token keyword">class</span> <span class="token class-name">Hello</span> <span class="token punctuation">{</span> <span class="token keyword">private</span> <span class="token class-name">String</span> lang <span class="token operator">=</span> <span class="token string">"en"</span><span class="token punctuation">;</span> <span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token function">sayHello</span><span class="token punctuation">(</span><span class="token class-name">String</span> name<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token class-name">System</span><span class="token punctuation">.</span>out<span class="token punctuation">.</span><span class="token function">println</span><span class="token punctuation">(</span><span class="token string">"Hello "</span> <span class="token operator">+</span> name<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token function">setLang</span><span class="token punctuation">(</span><span class="token class-name">String</span> lang<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">this</span><span class="token punctuation">.</span>lang <span class="token operator">=</span> lang<span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre><h3 id="添加注解">添加注解</h3><pre class="language-java" data-language="java"><code class="language-java"><span class="token keyword">package</span> <span class="token namespace">com<span class="token punctuation">.</span>example</span><span class="token punctuation">;</span><span class="token keyword">import</span> <span class="token import"><span class="token namespace">java<span class="token punctuation">.</span>lang<span class="token punctuation">.</span>annotation<span class="token punctuation">.</span></span><span class="token class-name">Documented</span></span><span class="token punctuation">;</span><span class="token keyword">import</span> <span class="token import"><span class="token namespace">java<span class="token punctuation">.</span>lang<span class="token punctuation">.</span>annotation<span class="token punctuation">.</span></span><span class="token class-name">ElementType</span></span><span class="token punctuation">;</span><span class="token keyword">import</span> <span class="token import"><span class="token namespace">java<span class="token punctuation">.</span>lang<span class="token punctuation">.</span>annotation<span class="token punctuation">.</span></span><span class="token class-name">Retention</span></span><span class="token punctuation">;</span><span class="token keyword">import</span> <span class="token import"><span class="token namespace">java<span class="token punctuation">.</span>lang<span class="token punctuation">.</span>annotation<span class="token punctuation">.</span></span><span class="token class-name">RetentionPolicy</span></span><span class="token punctuation">;</span><span class="token keyword">import</span> <span class="token import"><span class="token namespace">java<span class="token punctuation">.</span>lang<span class="token punctuation">.</span>annotation<span class="token punctuation">.</span></span><span class="token class-name">Target</span></span><span class="token punctuation">;</span><span class="token annotation punctuation">@Documented</span><span class="token annotation punctuation">@Target</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token class-name">ElementType</span><span class="token punctuation">.</span><span class="token constant">TYPE</span><span class="token punctuation">,</span> <span class="token class-name">ElementType</span><span class="token punctuation">.</span><span class="token constant">METHOD</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token annotation punctuation">@Retention</span><span class="token punctuation">(</span><span class="token class-name">RetentionPolicy</span><span class="token punctuation">.</span><span class="token constant">RUNTIME</span><span class="token punctuation">)</span><span class="token keyword">public</span> <span class="token annotation punctuation">@interface</span> <span class="token class-name">Log</span> <span class="token punctuation">{</span> <span class="token class-name">String</span> <span class="token function">file</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">}</span></code></pre><p>假设我们想为上面的Hello类分别在类级别以及方法级别添加一个Log注解，我们需要创建出 <code>AnnotationsAttribute</code> ， 然后将这个<code>Attribute</code>添加到对应的<code>ClassFile</code>中以及<code>getMethodInfo</code>里面。</p><pre class="language-java" data-language="java"><code class="language-java"><span class="token annotation punctuation">@Test</span><span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token function">addAnnotation</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">throws</span> <span class="token class-name">Exception</span> <span class="token punctuation">{</span> <span class="token class-name">ClassPool</span> classPool <span class="token operator">=</span> <span class="token class-name">ClassPool</span><span class="token punctuation">.</span><span class="token function">getDefault</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token class-name">CtClass</span> ctClass <span class="token operator">=</span> classPool<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span><span class="token string">"com.example.Hello"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//获取到常量池</span> <span class="token class-name">ConstPool</span> constpool <span class="token operator">=</span> ctClass<span class="token punctuation">.</span><span class="token function">getClassFile</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">getConstPool</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//创建类级别的注解属性</span> <span class="token class-name">AnnotationsAttribute</span> clzAnnotationAttr <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">AnnotationsAttribute</span><span class="token punctuation">(</span>constpool<span class="token punctuation">,</span> <span class="token class-name">AnnotationsAttribute</span><span class="token punctuation">.</span>visibleTag<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//类注解</span> <span class="token class-name">Annotation</span> clzLogAnnotation <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">Annotation</span><span class="token punctuation">(</span><span class="token class-name">Log</span><span class="token punctuation">.</span><span class="token keyword">class</span><span class="token punctuation">.</span><span class="token function">getName</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> constpool<span class="token punctuation">)</span><span class="token punctuation">;</span> clzLogAnnotation<span class="token punctuation">.</span><span class="token function">addMemberValue</span><span class="token punctuation">(</span><span class="token string">"file"</span><span class="token punctuation">,</span> <span class="token keyword">new</span> <span class="token class-name">StringMemberValue</span><span class="token punctuation">(</span><span class="token string">"CLASS_LOG"</span><span class="token punctuation">,</span> constpool<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span> clzAnnotationAttr<span class="token punctuation">.</span><span class="token function">setAnnotations</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">Annotation</span><span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">{</span>clzLogAnnotation<span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//添加到类属性中</span> ctClass<span class="token punctuation">.</span><span class="token function">getClassFile</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">addAttribute</span><span class="token punctuation">(</span>clzAnnotationAttr<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//创建方法级别的注解属性</span> <span class="token class-name">AnnotationsAttribute</span> methodAnnotationAttr <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">AnnotationsAttribute</span><span class="token punctuation">(</span>constpool<span class="token punctuation">,</span> <span class="token class-name">AnnotationsAttribute</span><span class="token punctuation">.</span>visibleTag<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token class-name">Annotation</span> methodLogAnnotation <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">Annotation</span><span class="token punctuation">(</span><span class="token class-name">Log</span><span class="token punctuation">.</span><span class="token keyword">class</span><span class="token punctuation">.</span><span class="token function">getName</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> constpool<span class="token punctuation">)</span><span class="token punctuation">;</span> methodLogAnnotation<span class="token punctuation">.</span><span class="token function">addMemberValue</span><span class="token punctuation">(</span><span class="token string">"file"</span><span class="token punctuation">,</span> <span class="token keyword">new</span> <span class="token class-name">StringMemberValue</span><span class="token punctuation">(</span><span class="token string">"METHOD_LOG"</span><span class="token punctuation">,</span> constpool<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span> methodAnnotationAttr<span class="token punctuation">.</span><span class="token function">setAnnotations</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">Annotation</span><span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">{</span>clzLogAnnotation<span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//添加到方法属性中</span> <span class="token class-name">CtMethod</span> ctMethod <span class="token operator">=</span> ctClass<span class="token punctuation">.</span><span class="token function">getDeclaredMethod</span><span class="token punctuation">(</span><span class="token string">"sayHello"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> ctMethod<span class="token punctuation">.</span><span class="token function">getMethodInfo</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">addAttribute</span><span class="token punctuation">(</span>methodAnnotationAttr<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//方法级别的注解类似</span> <span class="token comment">//ctField.getFieldInfo().addAttribute();</span> ctClass<span class="token punctuation">.</span><span class="token function">writeFile</span><span class="token punctuation">(</span><span class="token string">"/home/lxd/test"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">}</span></code></pre><p>执行后，我们就可以看到Hello类添加上了对应的注解。</p><p><img src="/images/20210327/04.png" alt="" loading="lazy"></p><h3 id="泛型显示">泛型显示</h3><p>Java的泛型其实是一个语法糖，在运行态均是Object类型，然后强转为对应的类型。为了使class文件看起来相对有好一些，我们需要通过泛型签名来进行泛型展示。</p><pre class="language-java" data-language="java"><code class="language-java"><span class="token annotation punctuation">@Test</span><span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token function">genericTest</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">throws</span> <span class="token class-name">Exception</span> <span class="token punctuation">{</span> <span class="token class-name">ClassPool</span> classPool <span class="token operator">=</span> <span class="token class-name">ClassPool</span><span class="token punctuation">.</span><span class="token function">getDefault</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token class-name">CtClass</span> ctClass <span class="token operator">=</span> classPool<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span><span class="token string">"com.example.Hello"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//添加字段的泛型签名</span> <span class="token class-name">CtField</span> ctField <span class="token operator">=</span> <span class="token class-name">CtField</span><span class="token punctuation">.</span><span class="token function">make</span><span class="token punctuation">(</span><span class="token string">"private java.util.List exts;"</span><span class="token punctuation">,</span> ctClass<span class="token punctuation">)</span><span class="token punctuation">;</span> ctField<span class="token punctuation">.</span><span class="token function">setGenericSignature</span><span class="token punctuation">(</span><span class="token string">"Ljava/util/List<Ljava/lang/String;>;"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> ctClass<span class="token punctuation">.</span><span class="token function">addField</span><span class="token punctuation">(</span>ctField<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//添加方法的泛型签名</span> <span class="token class-name">CtMethod</span> ctMethod <span class="token operator">=</span> <span class="token class-name">CtNewMethod</span><span class="token punctuation">.</span><span class="token function">make</span><span class="token punctuation">(</span><span class="token string">" public void setExts(java.util.List exts) {$0.exts = $1;}"</span><span class="token punctuation">,</span> ctClass<span class="token punctuation">)</span><span class="token punctuation">;</span> ctMethod<span class="token punctuation">.</span><span class="token function">setGenericSignature</span><span class="token punctuation">(</span><span class="token string">"(Ljava/util/List<Ljava/lang/String;>;)V"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> ctClass<span class="token punctuation">.</span><span class="token function">addMethod</span><span class="token punctuation">(</span>ctMethod<span class="token punctuation">)</span><span class="token punctuation">;</span> ctClass<span class="token punctuation">.</span><span class="token function">writeFile</span><span class="token punctuation">(</span><span class="token string">"/home/lxd/test"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">}</span></code></pre><h4 id="说明-3">说明</h4><ul><li>我们创建方法或者字段，如果直接传入泛型，javassist会报错，只能通过添加泛型签名的形式进行泛型的展示。</li><li>泛型签名中类的全限定名需要用 <code>/</code>代替<code>.</code></li><li>我们还可以通过<code>SignatureAttribute.ClassSignature</code>以及<code>SignatureAttribute.MethodSignature</code>来简化泛型签名的拼写。</li></ul><h3 id="参数名称保留">参数名称保留</h3><pre class="language-java" data-language="java"><code class="language-java"><span class="token annotation punctuation">@Test</span><span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token function">keepParamName</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">throws</span> <span class="token class-name">Exception</span> <span class="token punctuation">{</span> <span class="token class-name">ClassPool</span> classPool <span class="token operator">=</span> <span class="token class-name">ClassPool</span><span class="token punctuation">.</span><span class="token function">getDefault</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token class-name">CtClass</span> ctClass <span class="token operator">=</span> classPool<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span><span class="token string">"com.example.Hello"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token class-name">ConstPool</span> constPool <span class="token operator">=</span> ctClass<span class="token punctuation">.</span><span class="token function">getClassFile</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">getConstPool</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//添加方法</span> <span class="token class-name">CtMethod</span> ctMethod <span class="token operator">=</span> <span class="token class-name">CtNewMethod</span><span class="token punctuation">.</span><span class="token function">make</span><span class="token punctuation">(</span><span class="token string">" public void setExts(java.util.List exts) {}"</span><span class="token punctuation">,</span> ctClass<span class="token punctuation">)</span><span class="token punctuation">;</span> ctMethod<span class="token punctuation">.</span><span class="token function">setGenericSignature</span><span class="token punctuation">(</span><span class="token string">"(Ljava/util/List<Ljava/lang/String;>;)V"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token class-name">String</span><span class="token punctuation">[</span><span class="token punctuation">]</span> paramNames <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">String</span><span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">{</span><span class="token string">"exts"</span><span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token keyword">int</span><span class="token punctuation">[</span><span class="token punctuation">]</span> accessFlags <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token keyword">int</span><span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">{</span><span class="token number">0</span><span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token comment">//添加方法名称Attribute</span> <span class="token class-name">MethodParametersAttribute</span> parametersAttribute <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">MethodParametersAttribute</span><span class="token punctuation">(</span>constPool<span class="token punctuation">,</span> paramNames<span class="token punctuation">,</span> accessFlags<span class="token punctuation">)</span><span class="token punctuation">;</span> ctMethod<span class="token punctuation">.</span><span class="token function">getMethodInfo</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">addAttribute</span><span class="token punctuation">(</span>parametersAttribute<span class="token punctuation">)</span><span class="token punctuation">;</span> ctClass<span class="token punctuation">.</span><span class="token function">addMethod</span><span class="token punctuation">(</span>ctMethod<span class="token punctuation">)</span><span class="token punctuation">;</span> ctClass<span class="token punctuation">.</span><span class="token function">writeFile</span><span class="token punctuation">(</span><span class="token string">"/home/lxd/test"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">}</span></code></pre><p>class文件中默认参数名称会变成<code>var$</code>的形式，原始参数名称不会保留，我们也需要通过添加属性到相关属性表中来保留参数名称。</p><h2 id="总结">总结</h2><p>如果我们只是用来代码插桩或者不需要查看class文件的场景，那么javassist的官方文件已经满足大部分需要了。如果需要更深入的应用，则需要了解Javassist的源码。</p>]]></content>
<categories>
<category> Java </category>
</categories>
<tags>
<tag> Java </tag>
<tag> 字节码 </tag>
</tags>
</entry>
<entry>
<title>Java获取文件ContentType/MimeType</title>
<link href="/2021/01/05/Java%E8%8E%B7%E5%8F%96%E6%96%87%E4%BB%B6ContentType/"/>
<url>/2021/01/05/Java%E8%8E%B7%E5%8F%96%E6%96%87%E4%BB%B6ContentType/</url>
<content type="html"><![CDATA[<p>工作中需要用到根据文件后缀名称判断MimeType的场景，找到几种方式，记录一下。</p><h2 id="判断ContentType">判断ContentType</h2><h3 id="方式一-MimetypesFileTypeMap">方式一 MimetypesFileTypeMap</h3><pre class="language-java" data-language="java"><code class="language-java"><span class="token class-name">MimetypesFileTypeMap</span><span class="token punctuation">.</span><span class="token function">getDefaultFileTypeMap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">getContentType</span><span class="token punctuation">(</span><span class="token string">"xxx.ext"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre><p>此类在javax.activation包下，java11已经移除了javax，不建议使用，同时部分文件无法正确获取到Content-Type。<br>可通过<code>addMimeTypes(String mime_types)</code>方式添加解析。</p><h3 id="方式二-URLConnection">方式二 URLConnection</h3><pre class="language-java" data-language="java"><code class="language-java"><span class="token class-name">URLConnection</span><span class="token punctuation">.</span><span class="token function">getFileNameMap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">getContentTypeFor</span><span class="token punctuation">(</span><span class="token string">"xxx.ext"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre><p>部分后缀获取不到，比如svg格式的图片，无法正确获取到ContentType。</p><h3 id="方式三-ApplicationContext">方式三 ApplicationContext</h3><pre class="language-java" data-language="java"><code class="language-java"><span class="token comment">//HttpServletRequest request</span>request<span class="token punctuation">.</span><span class="token function">getServletContext</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">getMimeType</span><span class="token punctuation">(</span><span class="token string">"xxx.ext"</span><span class="token punctuation">)</span></code></pre><p>此种方式依赖Spring，如果是Spring环境，建议使用此种方式。此方式适配性较好。</p><p>如果部分后缀不识别，可通过以下方式进行添加：</p><pre class="language-java" data-language="java"><code class="language-java"><span class="token annotation punctuation">@Configuration</span><span class="token keyword">public</span> <span class="token keyword">class</span> <span class="token class-name">MimeMappingConfig</span> <span class="token keyword">implements</span> <span class="token class-name">WebServerFactoryCustomizer</span><span class="token generics"><span class="token punctuation"><</span><span class="token class-name">ConfigurableServletWebServerFactory</span><span class="token punctuation">></span></span> <span class="token punctuation">{</span> <span class="token annotation punctuation">@Override</span> <span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token function">customize</span><span class="token punctuation">(</span><span class="token class-name">ConfigurableServletWebServerFactory</span> factory<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token class-name">MimeMappings</span> mappings <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">MimeMappings</span><span class="token punctuation">(</span><span class="token class-name">MimeMappings</span><span class="token punctuation">.</span><span class="token constant">DEFAULT</span><span class="token punctuation">)</span><span class="token punctuation">;</span> mappings<span class="token punctuation">.</span><span class="token function">add</span><span class="token punctuation">(</span><span class="token string">"xxx"</span><span class="token punctuation">,</span> <span class="token string">"application/xml; charset=utf-8"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> factory<span class="token punctuation">.</span><span class="token function">setMimeMappings</span><span class="token punctuation">(</span>mappings<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre>]]></content>
<categories>
<category> Java </category>
</categories>
<tags>
<tag> Java </tag>
</tags>
</entry>
<entry>
<title>2020年总结</title>
<link href="/2021/01/01/2020%E5%B9%B4%E6%80%BB%E7%BB%93/"/>
<url>/2021/01/01/2020%E5%B9%B4%E6%80%BB%E7%BB%93/</url>
<content type="html"><![CDATA[<p> 今天早晨和老婆说起年终总结的事情，老婆觉得年终总结记在心里就好，没必要写出来。我觉得还是需要写出来的，一方面通过写作的过程不断的梳理这一年的得失；另外一方面也可以做一个记录，几年后回头看看，也别有一番滋味。</p><p> 2020年年初疫情爆发，考虑到小命要紧，就没有回家，一个人在杭州过的除夕和大年初一。紧接着老婆在家里和亲戚吵架了，把老婆接到了杭州。疫情封城，每天看着新增的确诊病例和确诊小区，总是战战兢兢。为各个前线的疫情战斗人员默默鼓劲，也为不作为的官员气愤难平。年假结束，也在家里远程上了几个月班。外面寒冷，小家还算温暖的。</p><p> 5月多，老婆检查出来怀孕了，心里很开心，要当爸爸了，感受到了沉甸甸的责任，要更加照顾好老婆。张罗着重新找个安静的房子，这个房子靠近大马路太吵了。端午期间花了3天时间找了个房子完成了搬家，然后发现房子装修一年，里面贴了很多墙纸，甲醛含量可能过高；只好又找了一个房子。期间零零碎碎，各种不顺利，好在熬了过来。老婆怀着孕，还在努力准备会计中级考试，功夫不负有心人，最终也考过了，拿到了中级证书。</p><p> 12月多，老爸和老妈过来了，带着他们逛了逛西湖、雷峰塔，颇多感慨。一辈子没有出过远门，现在可以出看看了。</p><p> 工作上，年初的时候组织架构调整，我被分到中台技术组，做中台治理相关的工作，这块需要单独写一篇文章记录今年思考和感悟。总体来说，工作上还是遇到不少挑战，对业务的了解深了一些。</p><p> 学习上的年初定的目标并没有达到，感觉主要是因为没有坚持下来，有点贪玩了。这个2021年目标需要继续制定、实行。</p><p> 敲下来的都是流水账，平时总结思考还是不够。2020年感觉遗憾大于收获，成长太少了。</p>]]></content>
<categories>
<category> 随笔 </category>
</categories>
<tags>
<tag> 随笔 </tag>
</tags>
</entry>
<entry>
<title>凌晨小记</title>
<link href="/2020/11/18/%E5%87%8C%E6%99%A8%E5%B0%8F%E8%AE%B0/"/>
<url>/2020/11/18/%E5%87%8C%E6%99%A8%E5%B0%8F%E8%AE%B0/</url>
<content type="html"><![CDATA[<blockquote><p>2020年11月18日凌晨记于杭州滨江</p></blockquote><p> 凌晨五点多醒来，想起七八年前的冬天，那时候在读高中，学校每月放两天假。为了在家里多呆一晚上，差不多也是这个时间，父母早早的叫我起来，骑摩托送我到十六里外的镇上，然后再坐上大巴到一百里外的县里读书。家里到镇上的十六里路，有八里地在两山中间，有三里多地，了无人烟。寒风凛冽，山里的夜色浓的像化不开的墨水，朦胧只能看到高处黝黑的山体，四周很寂静，偶尔响起几声夜鸮的叫声和父亲的咳嗽声。那时的我坐在父亲背后的车座上，想着当下的生活，想着即将开始的又一个月的学习，想到了未来，充满了幻想和希冀。如今毕业三年多了，忙忙碌碌。去年因为疫情没有回家，今年因为妻子怀孕，大大的肚子，也没有回家。每次从视频中看到日益年迈的父母，充满了愧疚与无奈。这个冬天，是时候带父母过来看看西湖的景色了。<br> 窗外的天色有点亮了，嘈杂的声音不断传来，新的一天又开始了。</p>]]></content>
<categories>
<category> 随笔 </category>
</categories>
<tags>
<tag> 随笔 </tag>
</tags>
</entry>
<entry>
<title>lombok原理示例</title>
<link href="/2020/01/21/lombok%E5%8E%9F%E7%90%86%E8%A7%A3%E6%9E%90/"/>
<url>/2020/01/21/lombok%E5%8E%9F%E7%90%86%E8%A7%A3%E6%9E%90/</url>
<content type="html"><![CDATA[<p>lombok采用的是编译期注解，依赖jsr269特性，修改的是AST抽象语法树</p><p><img src="/images/20200121/01.png" alt="" loading="lazy"></p><ol><li>添加依赖声明注解</li></ol><pre class="language-markup" data-language="markup"><code class="language-markup"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>dependency</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>groupId</span><span class="token punctuation">></span></span>com.sun<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>groupId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>artifactId</span><span class="token punctuation">></span></span>tools<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>artifactId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>version</span><span class="token punctuation">></span></span>1.8<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>version</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>scope</span><span class="token punctuation">></span></span>system<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>scope</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>systemPath</span><span class="token punctuation">></span></span>${java.home}/../lib/tools.jar<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>systemPath</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>dependency</span><span class="token punctuation">></span></span></code></pre><pre class="language-java" data-language="java"><code class="language-java"><span class="token annotation punctuation">@Target</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token class-name">ElementType</span><span class="token punctuation">.</span><span class="token constant">METHOD</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token annotation punctuation">@Retention</span><span class="token punctuation">(</span><span class="token class-name">RetentionPolicy</span><span class="token punctuation">.</span><span class="token constant">SOURCE</span><span class="token punctuation">)</span><span class="token keyword">public</span> <span class="token annotation punctuation">@interface</span> <span class="token class-name">Expose</span> <span class="token punctuation">{</span><span class="token punctuation">}</span></code></pre><ol start="2"><li>定义处理器</li></ol><pre class="language-java" data-language="java"><code class="language-java"><span class="token annotation punctuation">@SupportedAnnotationTypes</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token string">"org.example.Expose"</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token keyword">public</span> <span class="token keyword">class</span> <span class="token class-name">ExposeProcessor</span> <span class="token keyword">extends</span> <span class="token class-name">AbstractProcessor</span> <span class="token punctuation">{</span> <span class="token keyword">private</span> <span class="token keyword">int</span> r <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> <span class="token annotation punctuation">@Override</span> <span class="token keyword">public</span> <span class="token keyword">synchronized</span> <span class="token keyword">void</span> <span class="token function">init</span><span class="token punctuation">(</span><span class="token class-name">ProcessingEnvironment</span> processingEnv<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">super</span><span class="token punctuation">.</span><span class="token function">init</span><span class="token punctuation">(</span>processingEnv<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token annotation punctuation">@Override</span> <span class="token keyword">public</span> <span class="token keyword">boolean</span> <span class="token function">process</span><span class="token punctuation">(</span><span class="token class-name">Set</span><span class="token generics"><span class="token punctuation"><</span><span class="token operator">?</span> <span class="token keyword">extends</span> <span class="token class-name">TypeElement</span><span class="token punctuation">></span></span> annotations<span class="token punctuation">,</span> <span class="token class-name">RoundEnvironment</span> roundEnv<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token class-name">System</span><span class="token punctuation">.</span>out<span class="token punctuation">.</span><span class="token function">println</span><span class="token punctuation">(</span><span class="token string">"begin process ... "</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token class-name">Set</span><span class="token generics"><span class="token punctuation"><</span><span class="token operator">?</span> <span class="token keyword">extends</span> <span class="token class-name">Element</span><span class="token punctuation">></span></span> elements <span class="token operator">=</span> roundEnv<span class="token punctuation">.</span><span class="token function">getRootElements</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token class-name">Element</span> element <span class="token operator">:</span> elements<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token class-name">System</span><span class="token punctuation">.</span>out<span class="token punctuation">.</span><span class="token function">println</span><span class="token punctuation">(</span><span class="token string">"输入的类:"</span> <span class="token operator">+</span> element<span class="token punctuation">.</span><span class="token function">getSimpleName</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token class-name">Set</span><span class="token generics"><span class="token punctuation"><</span><span class="token operator">?</span> <span class="token keyword">extends</span> <span class="token class-name">Element</span><span class="token punctuation">></span></span> genElements <span class="token operator">=</span> roundEnv<span class="token punctuation">.</span><span class="token function">getElementsAnnotatedWith</span><span class="token punctuation">(</span><span class="token class-name">Expose</span><span class="token punctuation">.</span><span class="token keyword">class</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token class-name">Element</span> genElement <span class="token operator">:</span> genElements<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token class-name">System</span><span class="token punctuation">.</span>out<span class="token punctuation">.</span><span class="token function">println</span><span class="token punctuation">(</span><span class="token string">"开始处理："</span> <span class="token operator">+</span> genElement<span class="token punctuation">.</span><span class="token function">getSimpleName</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token class-name">System</span><span class="token punctuation">.</span>out<span class="token punctuation">.</span><span class="token function">println</span><span class="token punctuation">(</span><span class="token string">"第 "</span> <span class="token operator">+</span> <span class="token punctuation">(</span><span class="token operator">++</span>r<span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token string">" 次循环"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token boolean">true</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre><ol start="3"><li>声明清单文件</li></ol><p>在Resources目录下创建META-INF目录以及services子目录，并创建javax.annotation.processing.Processor文件。</p><pre class="language-none"><code class="language-none">├── java│ └── org│ └── example│ ├── Expose.java│ └── ExposeProcessor.java└── resources └── META-INF └── services └── javax.annotation.processing.Processor</code></pre><p>清单文件内容为</p><pre class="language-none"><code class="language-none">org.example.ExposeProcessor</code></pre><ol start="4"><li>打包</li></ol><pre class="language-markup" data-language="markup"><code class="language-markup"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>build</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>resources</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>resource</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>directory</span><span class="token punctuation">></span></span>src/main/resources<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>directory</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>excludes</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>exclude</span><span class="token punctuation">></span></span>META-INF/**/*<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>exclude</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>excludes</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>resource</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>resources</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>plugins</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>plugin</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>groupId</span><span class="token punctuation">></span></span>org.apache.maven.plugins<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>groupId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>artifactId</span><span class="token punctuation">></span></span>maven-resources-plugin<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>artifactId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>version</span><span class="token punctuation">></span></span>2.6<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>version</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>executions</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>execution</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>id</span><span class="token punctuation">></span></span>process-META<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>id</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>phase</span><span class="token punctuation">></span></span>prepare-package<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>phase</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>goals</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>goal</span><span class="token punctuation">></span></span>copy-resources<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>goal</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>goals</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>configuration</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>outputDirectory</span><span class="token punctuation">></span></span>target/classes<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>outputDirectory</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>resources</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>resource</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>directory</span><span class="token punctuation">></span></span>${basedir}/src/main/resources/<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>directory</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>includes</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>include</span><span class="token punctuation">></span></span>**/*<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>include</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>includes</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>resource</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>resources</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>configuration</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>execution</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>executions</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>plugin</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>plugins</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>build</span><span class="token punctuation">></span></span></code></pre><ol start="5"><li>测试</li></ol><p>新创建一个项目，并添加注解：</p><pre class="language-markup" data-language="markup"><code class="language-markup"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>dependency</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>groupId</span><span class="token punctuation">></span></span>org.example<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>groupId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>artifactId</span><span class="token punctuation">></span></span>jsr269demo<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>artifactId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>version</span><span class="token punctuation">></span></span>1.0-SNAPSHOT<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>version</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>dependency</span><span class="token punctuation">></span></span></code></pre><pre class="language-java" data-language="java"><code class="language-java"><span class="token keyword">public</span> <span class="token keyword">class</span> <span class="token class-name">Test</span> <span class="token punctuation">{</span> <span class="token annotation punctuation">@Expose</span> <span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token function">hello</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token punctuation">}</span> <span class="token annotation punctuation">@Expose</span> <span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token function">world</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre><p>执行mvn clean package时可以看到输出</p><pre class="language-none"><code class="language-none">begin process ... 输入的类:App输入的类:Test开始处理：hello开始处理：world第 1 次循环begin process ... 第 2 次循环</code></pre>]]></content>
<categories>
<category> Java </category>
</categories>
<tags>
<tag> Java </tag>
<tag> Lombok </tag>
</tags>
</entry>
<entry>
<title>Maven插件开发</title>
<link href="/2020/01/21/Maven%E6%8F%92%E4%BB%B6%E5%BC%80%E5%8F%91/"/>
<url>/2020/01/21/Maven%E6%8F%92%E4%BB%B6%E5%BC%80%E5%8F%91/</url>
<content type="html"><![CDATA[<ol><li>创建maven-plugin工程</li></ol><blockquote><p>可以直接使用maven模板<code>maven-archetype-mojo</code></p></blockquote><pre class="language-markup" data-language="markup"><code class="language-markup"> <span class="token comment"><!--打包类型--></span><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>packaging</span><span class="token punctuation">></span></span>maven-plugin<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>packaging</span><span class="token punctuation">></span></span> <span class="token comment"><!--maven依赖--></span><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>dependency</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>groupId</span><span class="token punctuation">></span></span>org.apache.maven<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>groupId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>artifactId</span><span class="token punctuation">></span></span>maven-plugin-api<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>artifactId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>version</span><span class="token punctuation">></span></span>${maven.version}<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>version</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>scope</span><span class="token punctuation">></span></span>provided<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>scope</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>dependency</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>dependency</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>groupId</span><span class="token punctuation">></span></span>org.apache.maven<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>groupId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>artifactId</span><span class="token punctuation">></span></span>maven-core<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>artifactId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>version</span><span class="token punctuation">></span></span>${maven.version}<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>version</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>scope</span><span class="token punctuation">></span></span>provided<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>scope</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>dependency</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>dependency</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>groupId</span><span class="token punctuation">></span></span>org.apache.maven<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>groupId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>artifactId</span><span class="token punctuation">></span></span>maven-artifact<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>artifactId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>version</span><span class="token punctuation">></span></span>${maven.version}<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>version</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>scope</span><span class="token punctuation">></span></span>provided<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>scope</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>dependency</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>dependency</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>groupId</span><span class="token punctuation">></span></span>org.apache.maven.plugin-tools<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>groupId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>artifactId</span><span class="token punctuation">></span></span>maven-plugin-annotations<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>artifactId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>version</span><span class="token punctuation">></span></span>3.6.0<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>version</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>scope</span><span class="token punctuation">></span></span>provided<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>scope</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>dependency</span><span class="token punctuation">></span></span></code></pre><ol start="2"><li>实现execute方法</li></ol><pre class="language-java" data-language="java"><code class="language-java"><span class="token annotation punctuation">@Mojo</span><span class="token punctuation">(</span>name<span class="token operator">=</span><span class="token string">"hello"</span><span class="token punctuation">,</span> defaultPhase <span class="token operator">=</span> <span class="token class-name">LifecyclePhase</span><span class="token punctuation">.</span><span class="token constant">INSTALL</span><span class="token punctuation">)</span><span class="token keyword">public</span> <span class="token keyword">class</span> <span class="token class-name">HelloMojo</span> <span class="token keyword">extends</span> <span class="token class-name">AbstractMojo</span> <span class="token punctuation">{</span> <span class="token annotation punctuation">@Override</span> <span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token function">execute</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">throws</span> <span class="token class-name">MojoExecutionException</span><span class="token punctuation">,</span> <span class="token class-name">MojoFailureException</span> <span class="token punctuation">{</span> <span class="token function">getLog</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">warn</span><span class="token punctuation">(</span><span class="token string">"Hello world mojo"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre><p>执行<code>mvn clean install</code>安装到本地</p><ol start="3"><li>引用自定义插件</li></ol><p>在另外一个工程中引用此maven插件</p><pre class="language-markup" data-language="markup"><code class="language-markup"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>plugin</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>groupId</span><span class="token punctuation">></span></span>org.example<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>groupId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>artifactId</span><span class="token punctuation">></span></span>hello-maven-plugin<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>artifactId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>version</span><span class="token punctuation">></span></span>1.0-SNAPSHOT<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>version</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>executions</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>execution</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>phase</span><span class="token punctuation">></span></span>install<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>phase</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>goals</span><span class="token punctuation">></span></span> <span class="token comment"><!-- 配置执行目标 --></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>goal</span><span class="token punctuation">></span></span>hello<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>goal</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>goals</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>execution</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>executions</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>plugin</span><span class="token punctuation">></span></span></code></pre><p>执行<code>mvn clean install</code>就可以看到打印出来的日志了。</p><pre class="language-bash" data-language="bash"><code class="language-bash"><span class="token punctuation">[</span>INFO<span class="token punctuation">]</span> --- hello-maven-plugin:1.0-SNAPSHOT:hello <span class="token punctuation">(</span>default<span class="token punctuation">)</span> @ <span class="token builtin class-name">test</span> ---<span class="token punctuation">[</span>WARNING<span class="token punctuation">]</span> Hello world mojo<span class="token punctuation">[</span>INFO<span class="token punctuation">]</span> ------------------------------------------------------------------------<span class="token punctuation">[</span>INFO<span class="token punctuation">]</span> BUILD SUCCESS</code></pre>]]></content>
<categories>
<category> Java </category>
</categories>
<tags>
<tag> Java </tag>
<tag> Maven </tag>
</tags>
</entry>
<entry>
<title>Java自定义doclet</title>
<link href="/2020/01/21/Java%E8%87%AA%E5%AE%9A%E4%B9%89doclet/"/>
<url>/2020/01/21/Java%E8%87%AA%E5%AE%9A%E4%B9%89doclet/</url>
<content type="html"><![CDATA[<blockquote><p>调研Doclet主要是因为公司项目中需要自定义接口文档并上传到平台，于是查到了几种方式，在这里记录一下。</p></blockquote><p>javadoc是基于doclet API输出html javadoc文档的默认实现。我们可以通过自定义Doclet来实现自己的文档。在自定义Doclet的实现过程中，我们可以拿到类的完整信息，比如注解，方法等信息。</p><p>官方文档在这里：<a href="https://docs.oracle.com/javase/8/docs/technotes/guides/javadoc/doclet/overview.html">https://docs.oracle.com/javase/8/docs/technotes/guides/javadoc/doclet/overview.html</a></p><p>下面我们通过自定义Doclet来看看具体的流程</p><h3 id="添加依赖包">添加依赖包</h3><p>首先我们需要引入jdk提供的tools包</p><pre class="language-markup" data-language="markup"><code class="language-markup"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>groupId</span><span class="token punctuation">></span></span>org.example<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>groupId</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>artifactId</span><span class="token punctuation">></span></span>OwnDoclet<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>artifactId</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>version</span><span class="token punctuation">></span></span>1.0-SNAPSHOT<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>version</span><span class="token punctuation">></span></span><span class="token comment"><!--省略其它配置--></span><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>dependency</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>scope</span><span class="token punctuation">></span></span>system<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>scope</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>groupId</span><span class="token punctuation">></span></span>com.sun<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>groupId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>artifactId</span><span class="token punctuation">></span></span>tools<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>artifactId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>version</span><span class="token punctuation">></span></span>1.8<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>version</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>systemPath</span><span class="token punctuation">></span></span>${java.home}/../lib/tools.jar<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>systemPath</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>dependency</span><span class="token punctuation">></span></span></code></pre><h3 id="自定义Doclet">自定义Doclet</h3><p>然后，我们自己编写一个简单的Doclet Demo。自定义Doclet需要继承自Doclet类，并实现start方法</p><pre class="language-java" data-language="java"><code class="language-java"><span class="token keyword">public</span> <span class="token keyword">class</span> <span class="token class-name">OwnDoclet</span> <span class="token keyword">extends</span> <span class="token class-name">Doclet</span> <span class="token punctuation">{</span> <span class="token keyword">public</span> <span class="token keyword">static</span> <span class="token keyword">boolean</span> <span class="token function">start</span><span class="token punctuation">(</span><span class="token class-name">RootDoc</span> rootDoc<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token class-name">System</span><span class="token punctuation">.</span>out<span class="token punctuation">.</span><span class="token function">println</span><span class="token punctuation">(</span><span class="token string">"Hello my doclet"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token class-name">ClassDoc</span> classDoc <span class="token operator">:</span> rootDoc<span class="token punctuation">.</span><span class="token function">classes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token class-name">System</span><span class="token punctuation">.</span>out<span class="token punctuation">.</span><span class="token function">println</span><span class="token punctuation">(</span><span class="token string">"ClassName:"</span> <span class="token operator">+</span> classDoc<span class="token punctuation">.</span><span class="token function">qualifiedName</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">return</span> <span class="token boolean">true</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre><p>开发完毕我们需要执行<code>mvn clean install</code>安装到本地</p><p>备注：rootDoc中可以拿到类的完整信息，比如注解、方法、参数名称等信息。因此我们这里在这里做很多事情，比如根据特定的注解提起信息生成API文档并上传到平台中。</p><h3 id="引用并测试">引用并测试</h3><p>在另外一个工程中引入javadoc插件</p><pre class="language-markup" data-language="markup"><code class="language-markup"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>plugin</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>groupId</span><span class="token punctuation">></span></span>org.apache.maven.plugins<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>groupId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>artifactId</span><span class="token punctuation">></span></span>maven-javadoc-plugin<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>artifactId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>version</span><span class="token punctuation">></span></span>3.1.1<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>version</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>configuration</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>encoding</span><span class="token punctuation">></span></span>UTF-8<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>encoding</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>useStandardDocletOptions</span><span class="token punctuation">></span></span>false<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>useStandardDocletOptions</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>doclet</span><span class="token punctuation">></span></span>org.example.OwnDoclet<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>doclet</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>docletArtifacts</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>docletArtifact</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>groupId</span><span class="token punctuation">></span></span>org.example<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>groupId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>artifactId</span><span class="token punctuation">></span></span>OwnDoclet<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>artifactId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>version</span><span class="token punctuation">></span></span>1.0-SNAPSHOT<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>version</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>docletArtifact</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>docletArtifacts</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>configuration</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>plugin</span><span class="token punctuation">></span></span></code></pre><p>执行<code>mvn javadoc:javadoc</code>命令，可以看到输出如下：</p><pre class="language-none"><code class="language-none">正在加载程序包org.example的源文件...正在构造 Javadoc 信息...Hello my docletClassName:org.example.AppClassName:org.example.TestClassName:org.example.RefTestClassName:org.example.Expose</code></pre><h3 id="参考文档">参考文档</h3><p><a href="https://docs.oracle.com/javase/8/docs/technotes/guides/javadoc/doclet/overview.html">Doclet Overview</a></p>]]></content>
<categories>
<category> Java </category>
</categories>
<tags>
<tag> Java </tag>
</tags>
</entry>
<entry>
<title>Etcd学习笔记</title>
<link href="/2020/01/13/Etcd%E5%85%A5%E9%97%A8%E6%95%99%E7%A8%8B/"/>
<url>/2020/01/13/Etcd%E5%85%A5%E9%97%A8%E6%95%99%E7%A8%8B/</url>
<content type="html"><![CDATA[<p>Etcd是CoreOS开发的一个高可用、强一致性的分布式键值对数据库。采用Raft算法。可以用于服务发现。目前k8s的后端存储就用的Etcd。<br>相对于Zookeeper来说，具有以下特点：</p><ul><li>简单：安装部署相对简单，用户友好支持http协议、支持grpc协议</li><li>安全：支持tls认证</li><li>快速：1万次每次的写入速度</li></ul><h3 id="安装">安装</h3><ul><li>环境：macOS Catalina</li><li>版本：etcd v3.4.3</li></ul><p>为了快速体验Etcd，可以直接使用官方编译好的可执行文件：<a href="https://github.com/etcd-io/etcd/releases">Github下载页面</a>。下载后解压可以看到Etcd以及Etcdctl两个可执行文件，前者是Etcd服务端，后者是一个命令行客户端。</p><p>启动一个本机单节点的etcd集群，直接执行<code>./etcd </code>即可：</p><pre class="language-none"><code class="language-none">2020-01-04 17:10:16.654046 I | etcdmain: etcd Version: 3.3.182020-01-04 17:10:16.654162 I | etcdmain: Git SHA: 3c8740a792020-01-04 17:10:16.654166 I | etcdmain: Go Version: go1.12.92020-01-04 17:10:16.654170 I | etcdmain: Go OS/Arch: darwin/amd642020-01-04 17:10:16.654175 I | etcdmain: setting maximum number of CPUs to 8, total number of available CPUs is 82020-01-04 17:10:16.654181 N | etcdmain: failed to detect default host (default host not supported on darwin_amd64)2020-01-04 17:10:16.654187 W | etcdmain: no data-dir provided, using default data-dir ./default.etcd2020-01-04 17:10:16.655724 I | embed: listening for peers on http://localhost:23802020-01-04 17:10:16.655864 I | embed: listening for client requests on localhost:23792020-01-04 17:10:16.670602 I | etcdserver: name = default2020-01-04 17:10:16.670645 I | etcdserver: data dir = default.etcd2020-01-04 17:10:16.670655 I | etcdserver: member dir = default.etcd/member2020-01-04 17:10:16.670659 I | etcdserver: heartbeat = 100ms2020-01-04 17:10:16.670663 I | etcdserver: election = 1000ms2020-01-04 17:10:16.670669 I | etcdserver: snapshot count = 1000002020-01-04 17:10:16.670681 I | etcdserver: advertise client URLs = http://localhost:23792020-01-04 17:10:16.670689 I | etcdserver: initial advertise peer URLs = http://localhost:23802020-01-04 17:10:16.670700 I | etcdserver: initial cluster = default=http://localhost:23802020-01-04 17:10:16.779154 I | etcdserver: starting member 8e9e05c52164694d in cluster cdf818194e3a8c322020-01-04 17:10:16.779652 I | raft: 8e9e05c52164694d became follower at term 02020-01-04 17:10:16.779719 I | raft: newRaft 8e9e05c52164694d [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]2020-01-04 17:10:16.779744 I | raft: 8e9e05c52164694d became follower at term 12020-01-04 17:10:16.850324 W | auth: simple token is not cryptographically signed2020-01-04 17:10:16.888801 I | etcdserver: starting server... [version: 3.3.18, cluster version: to_be_decided]2020-01-04 17:10:16.889057 I | etcdserver: 8e9e05c52164694d as single-node; fast-forwarding 9 ticks (election ticks 10)2020-01-04 17:10:16.889634 E | etcdserver: cannot monitor file descriptor usage (cannot get FDUsage on darwin)2020-01-04 17:10:16.890936 I | etcdserver/membership: added member 8e9e05c52164694d [http://localhost:2380] to cluster cdf818194e3a8c322020-01-04 17:10:17.086142 I | raft: 8e9e05c52164694d is starting a new election at term 12020-01-04 17:10:17.086212 I | raft: 8e9e05c52164694d became candidate at term 22020-01-04 17:10:17.086271 I | raft: 8e9e05c52164694d received MsgVoteResp from 8e9e05c52164694d at term 22020-01-04 17:10:17.086305 I | raft: 8e9e05c52164694d became leader at term 22020-01-04 17:10:17.086322 I | raft: raft.node: 8e9e05c52164694d elected leader 8e9e05c52164694d at term 22020-01-04 17:10:17.087050 I | etcdserver: setting up the initial cluster version to 3.32020-01-04 17:10:17.097255 N | etcdserver/membership: set the initial cluster version to 3.32020-01-04 17:10:17.097308 I | etcdserver: published {Name:default ClientURLs:[http://localhost:2379]} to cluster cdf818194e3a8c322020-01-04 17:10:17.097350 I | embed: ready to serve client requests2020-01-04 17:10:17.097428 I | etcdserver/api: enabled capabilities for version 3.32020-01-04 17:10:17.100090 N | embed: serving insecure client requests on 127.0.0.1:2379, this is strongly discouraged!</code></pre><p>日志中有几个关键的地方：</p><ul><li>name：etcd的名称，如果有多个etcd集群，需要修改名称</li><li>data dir：etcd数据存储的地方</li><li>heartbeat：心跳检测时间</li><li>election：触发投票选举leader的时间</li></ul><p>Etcd有两个版本的API，v2和v3，v3相对v2有很大的性能提升，建议直接使用v3版本。Etcdctl3.4.0版本采用v3版本的API和服务端交互，Etcdctl3.3及更早的版本默认采用v2版本的API和服务端交互，可以通过设置：</p><pre class="language-bash" data-language="bash"><code class="language-bash"><span class="token builtin class-name">export</span> <span class="token assign-left variable">ETCDCTL_API</span><span class="token operator">=</span><span class="token number">3</span></code></pre><p>来采用v3版本的API和服务端交互。</p><p>本地测试可以通过</p><pre class="language-bash" data-language="bash"><code class="language-bash">etcdctl put foo bar</code></pre><p>设置一个key和value。</p><p>通过</p><pre class="language-bash" data-language="bash"><code class="language-bash">etcdctl get fool</code></pre><p>来获取value</p><h3 id="基本操作">基本操作</h3><p>Etcd作为一个数据库，基础的增删改查是少不了的</p><ul><li>新增</li></ul><pre class="language-bash" data-language="bash"><code class="language-bash">etcdctl put foo baretcdctl put foo1 bar1etcdctl put foo2 bar2</code></pre><ul><li>查找</li></ul><pre class="language-bash" data-language="bash"><code class="language-bash"><span class="token comment"># 打印出key和value</span>etcdctl get foo<span class="token comment">#只打印出value</span>etcdctl get foo --print-value-only<span class="token comment">#获取多个从foo到foo2</span>etcdctl get foo foo2<span class="token comment">#根据前缀获取</span>etcdctl get <span class="token parameter variable">--prefix</span> foo</code></pre><ul><li>删除</li></ul><p>删除key和查找key类似，支持各种前缀匹配以及范围匹配</p><pre class="language-bash" data-language="bash"><code class="language-bash">etcdctl del foo</code></pre><ul><li>监控数据变化</li></ul><pre class="language-bash" data-language="bash"><code class="language-bash">etcdctl <span class="token function">watch</span> foo</code></pre><h3 id="总结">总结</h3><p>Etcd使用起来相对非常简单，下次要学习一下Etct 各个语言Client的使用。</p><h3 id="参考文档">参考文档</h3><p><a href="https://etcd.io/docs/v3.4.0/">Etcd官方教程</a><br><a href="https://blog.csdn.net/gaowenhui2008/article/details/95960873">Etcd简介</a><br><a href="https://www.jianshu.com/p/f68028682192">Etcd使用入门</a></p>]]></content>
</entry>
<entry>
<title>2020年规划</title>
<link href="/2020/01/01/2020%E5%B9%B4%E8%A7%84%E5%88%92/"/>
<url>/2020/01/01/2020%E5%B9%B4%E8%A7%84%E5%88%92/</url>
<content type="html"><![CDATA[<h3 id="总体规划">总体规划</h3><ul><li>每天一个小时的读书时间</li><li>两周一篇技术文章</li></ul><h3 id="第一季度">第一季度</h3><ul><li>Pinpoint源码分析</li><li>《k8s in action》阅读</li><li>风控系统总结</li><li>API网关系统总结</li></ul><h3 id="第二季度">第二季度</h3><ul><li>tomcat源码学习</li><li>《云原生服务网格Istio：原理、实践、架构与源码解析》阅读</li></ul><h3 id="第三季度">第三季度</h3><ul><li>Spring源码学习</li></ul><h3 id="第四季度">第四季度</h3><ul><li>jdk8以后的新特性及垃圾回收等整理学习</li></ul><h3 id="文学书籍">文学书籍</h3><ul><li>《平凡的世界》</li></ul>]]></content>
<categories>
<category> 随笔 </category>
</categories>
<tags>
<tag> 随笔 </tag>
</tags>
</entry>
<entry>
<title>2019年总结</title>
<link href="/2019/12/30/2019%E5%B9%B4%E6%80%BB%E7%BB%93/"/>
<url>/2019/12/30/2019%E5%B9%B4%E6%80%BB%E7%BB%93/</url>
<content type="html"><![CDATA[<p> 转眼间又到年底了，虽说过了年又老了一岁，心底里还是期盼过年的。一方面可以回家看看爹妈和爷爷奶奶，一年没有回家，还是蛮想念的，希望高铁通了以后可以常回家看看；另外一方面也借年假休息一下，好好总结过去一年的得与失，计划2020，继续前行。</p><p> 首先总结一下工作方面，2019年的主要工作是继续开发维护风控系统、开发并完善API网关系统、参与了一个技术工作组项目（各个团队出人力共建）-全链路流量标记透传以及最重要的API网关上云。在各个系统的开发维护过程中还是有不少值得总结的地方。</p><p> 风控系统经过2017年和2018年的发展已经比较完善了，不过在规则生成和规则判定方面还有不少值得优化的地方。2019年着重重写了规则生成逻辑和重写了表达式引擎。规则执行更加高效优雅。另外在系统优化之余还加了不少小需求。目前风控系统日渐趋于稳定，需要再单独写一篇文章总结近两年来风控系统的迭代演进以及思考。</p><p> API网关系统大部分功能也已经稳定下来了。2019年上半年参与了一个智能决策平台项目（智能ABT测试系统）的开发，网关侧开发了一个插件实现了主要的实验分流、判定和结果执行逻辑。不过随着决策平台的演进，网关仅仅做了基本的转发功能，大部分功能都没用用到。这块以后项目开发中需要注意，一定要做相对长远一点的规划，分析清楚需求，设计好技术方案。2019年下半年主要完善了网关的监控报警，开发了全链路监控插件，纳入全链路监控体系中。同时发现了Kong的几个比较严重的bug。以后选择开源项目做二次开发的时候要选择已经发布稳定版本的项目同时注意代码质量，</p><p> 2019年的7月至10月份参与了全链路流量标记透传项目。该项目属于全链路压测的一部分。项目做的事情是制订流量标记透传的规范，并基于现有的追踪能力（Pinpoint）开发插件和SDK实现流量属性的传递，存储路由可以根据流量属性路由到影子库或者正常库、影子MQ或者正常MQ等。在此项目中做的比较成功的是充分的资料准备、仔细长远的规划和高效的实施。这三点经验需要好好保持下去。</p><p> 本年度还有一个比较重要的事情就是应用上云了，云内采用Istio作为ServiceMesh，流量入口采用杭研基于Envoy研发的Gateway。云外的Kong网关不再部署到云内，而是复用现有的管理平台，将配置通过中间的一层转换服务转换为k8s的yaml资源下发到pilot，最后再下发到envoy网关。我们团队和杭州团队对接，基本将云外Kong网关到功能和插件迁移到云内的envoy网关。在此项目中经历了种种困难的地方，比如envoy网关不稳定、转换层不稳定、测试不够充分完善以及基建不完善导致的排查问题的艰难等许多问题。总结下来就是：这种共建的项目还是需要找比较靠谱的合作方、做好需求和项目规划、做好测试、对质量要严格要求。</p><p> 生活上最大的事情是买房了，在亲人朋友的帮助下凑够了首付，开启了房奴生活，开启了吃土生活。自己的神州笔记本电脑光荣退休了，花呗分期购入了一台MBP，在这里非常感谢老婆的理解和支持。老婆的会计中级证书也重要考过了一门，开心～。</p><p> 絮絮叨叨记录了挺多，2019年满意的是自己的代码能力、眼界有不少提高，拖延症减轻了很多，执行力也有很大的提高，学习上看了几本技术书做了几次小组内分享，这些还需要保持。不足的在于学习上的规划不够明确，总结做的不够多，接下来的一年需要大力改进。</p>]]></content>
<categories>
<category> 随笔 </category>
</categories>
<tags>
<tag> 随笔 </tag>
</tags>
</entry>
<entry>
<title>记录一次X-Forwarded-For翻转的问题排查</title>
<link href="/2019/12/08/%E8%AE%B0%E5%BD%95%E4%B8%80%E6%AC%A1X-Forwarded-For%E7%BF%BB%E8%BD%AC%E7%9A%84%E9%97%AE%E9%A2%98%E6%8E%92%E6%9F%A5/"/>
<url>/2019/12/08/%E8%AE%B0%E5%BD%95%E4%B8%80%E6%AC%A1X-Forwarded-For%E7%BF%BB%E8%BD%AC%E7%9A%84%E9%97%AE%E9%A2%98%E6%8E%92%E6%9F%A5/</url>
<content type="html"><![CDATA[<h2 id="背景">背景</h2><p> 最近严选的服务在上云，上云后由于IP是不固定的，那么通过IP黑白名单鉴权的手段已经不太能够满足现实情况了。</p><p> 打算通过在Header中放入服务编码的方式来进行鉴权。如果是云内的服务A调用云内的服务B，那么服务A的sidecar(istio-envoy)就可以向请求中注入服务编码信息；如果是从云外访问云内的话，这件事情由边缘网关（基于kong）来做比较合适。</p><p> 网关小组为此开发了一个插件，插件做的事情是拿到downstream的IP，然后从CMDB中查到这个IP对应的服务编码，最终把服务编码信息放到Header中传到业务方。</p><p> 插件运行了一段时间，有一个业务反馈拿不到服务编码。网关小组就是在此情况下开始了问题的排查。</p><h2 id="排查过程">排查过程</h2><p> 我们先看一下请求的流程：</p><p><img src="/images/2019120700.png" alt="" loading="lazy"></p><p> 然后看一下边缘网关的accesslog：</p><p><img src="/images/2019120701.jpg" alt="" loading="lazy"></p><p> 可以看到：remote address是：10.xx.xx.114(入口nginx地址) ， XFF中的信息为：10.xx.xx.120(node层地址)。那么业务方收到的XFF是什么呢？是：10.xx.xx.120,10.xx.xx.114 。</p><p> 到这里可以看到问题有三个：</p><ol><li>为什么网关收到到remote address是：10.xx.xx.114(入口nginx地址)，而不是预想的10.xx.xx.120(node层地址) ？</li><li>为什么网关收到的XFF是10.xx.xx.120(node层地址)，而不是预想的：客户端IP，10.xx.xx.114(入口nginx地址)？</li><li>为什么业务方收到的XFF不对并且是翻转的呢？</li></ol><p> 因为有不少业务都已经上云了，所以第一反应边缘网关应该没有问题。同时根据全链路ID的组成来看：<code>10.xx.xx.36(yanxuan-ianus)^xxxxx^9313374</code>，此ID是由网关生成的，前面的节点没有接入全链路监控（node层是接入了全链路监控的）。</p><p> 猜测：node层（consul-nginx作为sidecar）和边缘网关之间可能还有一层。</p><p> 假设：node层和边缘网关层中间还经过了10.xx.xx.114这台入口nginx</p><p> 结论：第一个和第二个问题勉强说的通了（不过xff还有点问题，没有客户端IP）。</p><p> 悖论：边缘网关收到的Host头是：online.xxx.mail.saas，这种Host应该是不经过入口nginx的。</p><p> 为了验证测试，从SA询问得知：</p><ol><li>这种域名（online.xxx.mail.saas）是不经过入口nginx的</li><li>查了一下这个点的日志，可以看到有这条请求是从客户端直接到node层的，客户端IP是：183.xx.xx.131 。</li></ol><p> 看来node层和边缘网关中间应该没有其他节点了，猜测不成立。</p><p> 查了一下node机器上的consul-nginx日志，consul-nginx作为sidecar，对所有的请求头都是透传的，没有做修改。</p><p> node机器上的consul-nginx如下：<br><img src="/images/2019120702.jpg" alt="" loading="lazy"></p><p> consul-nginx显示：下一跳直接到边缘网关了，没有经过其他节点。consul-nginx收到的xff是：10.xx.xx.120(node本机IP)。</p><p> 这个日志说明:</p><ol><li>网关获取remote address的方式有问题。</li><li>node层可能对xff头做了覆盖操作，不然consul-nginx显示对xff应该是：183.xx.xx.131（客户端IP）。</li></ol><p> 边缘网关日志中打印的是：<code>$remote_addr</code>，仔细查一下官方文档，可以看到这个变量的意思是客户端IP <a href="http://nginx.org/en/docs/http/ngx_http_core_module.html">http://nginx.org/en/docs/http/ngx_http_core_module.html</a> , 没有明确说明一定是上一跳的地址。</p><p> 查了相关资料后发现：real_ip_modul可能会更改client address。同时网关是基于kong的二次开发，kong默认开启了real_ip模块，对比之后可以看出很有可能是real_ip_module修改了remote_addr这个变量的值。</p><blockquote><p>The ngx_http_realip_module module is used to change the client address and optional port to those sent in the specified header field.</p></blockquote><p> 在real_ip_module的文档中，可以看到<a href="http://nginx.org/en/docs/http/ngx_http_realip_module.html">http://nginx.org/en/docs/http/ngx_http_realip_module.html</a></p><blockquote><p>Syntax:real_ip_header field | X-Real-IP | X-Forwarded-For | proxy_protocol;<br>Default:real_ip_header X-Real-IP;<br>Context:http, server, location</p></blockquote><p> 默认是从X-Real-IP中取值赋值到remote_addr中的。为了取到原始的remote_addr值，可以用realip_remote_addr这个变量。这个点以前没有注意到。</p><p> 到这里脉络基本清晰了，猜测如下：</p><ol><li>网关从x-real-ip头中取到了了一个remote_addr，然后追加到xff中，导致应用服务收到了xff是翻转的。</li><li>Node层可能追加了一个x-real-ip，并且对xff做了覆盖重写。</li></ol><p> 验证：</p><ol><li>查看Node层的请求日志：日志很简单，只打印了请求URL以及目标服务的consul-nginx地址，无法看到其他信息。</li><li>从pm2工具隐藏的日志中看到了采样的部分请求的完整请求头，从中明确的看到了，入口nginx传了一个X-real-ip，node层追加了入口nginx的地址到x-real-ip中同时对xff做了清除，赋值本机IP到xff中。</li></ol><p> 结论：上述猜测成立</p><h2 id="综述">综述</h2><ol><li>node层传到网关的xff为本机IP，x-real-ip为：客户端IP，入口nginxIP 。</li><li>边缘网关根据x-real-ip中拿到了入口nginxIP作为客户端IP，然后追加到xff中；导致应用看到的xff好像是翻转的。</li></ol><h2 id="解决办法">解决办法</h2><ol><li>网关需要采用realip_remote_addr变量来获取和网关直连的节点IP。</li><li>网关需要调整remote_addr的获取方法，从xff中获取，同时real_ip_recursive设置为off （取和网关直连的客户端IP）。</li><li>Node层需要明确自身的地位：如果node作为代理层，就需要正确追加xff；如果node层把自己作为应用服务器，就需要完全清除代理服务器相关的Header。</li></ol><h2 id="暴露的问题">暴露的问题</h2><ol><li>对nginx对相关变量和参数对了解程度还不够，其他业务没有反应这个问题主要是因为链路不太一致，也有可能没有看到这个问题，不能靠巧合编程。</li><li>node层接入的全链路sdk应该有问题，导致链路追踪不全面。</li><li>node层打印的accesslog日志也不够清晰，需要补充完善。</li></ol>]]></content>
<categories>
<category> 网关 </category>
</categories>
</entry>
<entry>
<title>javaagent学习</title>
<link href="/2019/12/04/JavaAgent%E5%AD%A6%E4%B9%A0/"/>
<url>/2019/12/04/JavaAgent%E5%AD%A6%E4%B9%A0/</url>
<content type="html"><![CDATA[<h2 id="什么是JavaAgent">什么是JavaAgent ?</h2><p>JavaAgent叫做Java代理，也叫做Java探针。是JVM提供的一种动态修改增强字节码的机制。通过在启动时指定-javaagent参数，或者通过JVMTI（Java Virtual Machine Tool Interface）提供的接口，可以很方便的修改类的字节码。</p><h2 id="JavaAgent可以做什么？">JavaAgent可以做什么？</h2><ol><li>在类加载时对类做修饰，从而增强功能。比如lombok以及APM框架Pinpoint和Skywalking</li><li>动态Attach到进程上面做一些性能分析，比如阿里巴巴的Arthas等</li></ol><h2 id="一个例子🌰">一个例子🌰</h2><p>JavaAgent使用时需要声明一个<code>premain</code>方法，这个方法的签名有两个：</p><ol><li><code>public static void premain(String agentArgs, Instrumentation inst)</code></li><li><code>public static void premain(String agentArgs)</code></li></ol><p>JVM默认会优先加载第一个签名的方法，如果没有的话才会加载第二个签名的方法。</p><p>那么Instrumentation又是什么呢？<br>Instrumentation是对类进行转换的核心，提供了</p><pre class="language-java" data-language="java"><code class="language-java"><span class="token keyword">void</span> <span class="token function">addTransformer</span><span class="token punctuation">(</span><span class="token class-name">ClassFileTransformer</span> transformer<span class="token punctuation">)</span><span class="token keyword">void</span> <span class="token function">addTransformer</span><span class="token punctuation">(</span><span class="token class-name">ClassFileTransformer</span> transformer<span class="token punctuation">,</span> <span class="token keyword">boolean</span> canRetransform<span class="token punctuation">)</span><span class="token class-name">Class</span><span class="token punctuation">[</span><span class="token punctuation">]</span> <span class="token function">getAllLoadedClasses</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token comment">//提供了很多方法</span></code></pre><p>现在我们尝试来写一个javaagent。</p><p>首先是一个普通的Java程序：</p><pre class="language-java" data-language="java"><code class="language-java"><span class="token comment">//org.example.App</span><span class="token keyword">public</span> <span class="token keyword">class</span> <span class="token class-name">App</span> <span class="token punctuation">{</span> <span class="token keyword">public</span> <span class="token keyword">static</span> <span class="token keyword">void</span> <span class="token function">main</span><span class="token punctuation">(</span> <span class="token class-name">String</span><span class="token punctuation">[</span><span class="token punctuation">]</span> args <span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token class-name">System</span><span class="token punctuation">.</span>out<span class="token punctuation">.</span><span class="token function">println</span><span class="token punctuation">(</span> <span class="token string">"Hello World!"</span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre><p>maven打包时需要注意打包为可执行文件</p><pre class="language-markup" data-language="markup"><code class="language-markup"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>plugin</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>artifactId</span><span class="token punctuation">></span></span>maven-jar-plugin<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>artifactId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>version</span><span class="token punctuation">></span></span>3.0.2<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>version</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>configuration</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>archive</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>manifest</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>addClasspath</span><span class="token punctuation">></span></span>true<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>addClasspath</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>classpathPrefix</span><span class="token punctuation">></span></span>lib/<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>classpathPrefix</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>mainClass</span><span class="token punctuation">></span></span>org.example.App<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>mainClass</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>manifest</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>archive</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>configuration</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>plugin</span><span class="token punctuation">></span></span></code></pre><p>其次我们创建一个agent程序：</p><pre class="language-java" data-language="java"><code class="language-java"><span class="token keyword">public</span> <span class="token keyword">class</span> <span class="token class-name">PreApp</span><span class="token punctuation">{</span> <span class="token keyword">public</span> <span class="token keyword">static</span> <span class="token keyword">void</span> <span class="token function">premain</span><span class="token punctuation">(</span><span class="token class-name">String</span> agentArgs<span class="token punctuation">,</span> <span class="token class-name">Instrumentation</span> instrumentation<span class="token punctuation">)</span><span class="token punctuation">{</span> instrumentation<span class="token punctuation">.</span><span class="token function">addTransformer</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">ClassFileTransformer</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token annotation punctuation">@Override</span> <span class="token keyword">public</span> <span class="token keyword">byte</span><span class="token punctuation">[</span><span class="token punctuation">]</span> <span class="token function">transform</span><span class="token punctuation">(</span><span class="token class-name">ClassLoader</span> loader<span class="token punctuation">,</span> <span class="token class-name">String</span> className<span class="token punctuation">,</span> <span class="token class-name">Class</span><span class="token generics"><span class="token punctuation"><</span><span class="token operator">?</span><span class="token punctuation">></span></span> classBeingRedefined<span class="token punctuation">,</span> <span class="token class-name">ProtectionDomain</span> protectionDomain<span class="token punctuation">,</span> <span class="token keyword">byte</span><span class="token punctuation">[</span><span class="token punctuation">]</span> classfileBuffer<span class="token punctuation">)</span> <span class="token keyword">throws</span> <span class="token class-name">IllegalClassFormatException</span> <span class="token punctuation">{</span> <span class="token comment">//此处需要指定类名，不然会打印多次before main。同时类名中的 . 需要替换为 /</span> <span class="token keyword">if</span><span class="token punctuation">(</span><span class="token string">"org/example/App"</span><span class="token punctuation">.</span><span class="token function">equals</span><span class="token punctuation">(</span>className<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">{</span> <span class="token class-name">System</span><span class="token punctuation">.</span>out<span class="token punctuation">.</span><span class="token function">println</span><span class="token punctuation">(</span><span class="token string">"before main--->"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">return</span> classfileBuffer<span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre><p>maven POM中需要指定premain:</p><pre class="language-markup" data-language="markup"><code class="language-markup"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>plugin</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>groupId</span><span class="token punctuation">></span></span>org.apache.maven.plugins<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>groupId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>artifactId</span><span class="token punctuation">></span></span>maven-jar-plugin<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>artifactId</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>version</span><span class="token punctuation">></span></span>3.0.2<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>version</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>configuration</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>archive</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>manifest</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>addClasspath</span><span class="token punctuation">></span></span>true<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>addClasspath</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>manifest</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>manifestEntries</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>Premain-Class</span><span class="token punctuation">></span></span> org.example.PreApp <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>Premain-Class</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>manifestEntries</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>archive</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>configuration</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>plugin</span><span class="token punctuation">></span></span></code></pre><p>然后我们就可以执行：</p><pre class="language-bash" data-language="bash"><code class="language-bash"><span class="token function">java</span> -javaagent:agent-demo-1.0-SNAPSHOT.jar <span class="token parameter variable">-jar</span> demo-1.0-SNAPSHOT.jar </code></pre><p>就可以看到控制台上打印出：</p><pre class="language-none"><code class="language-none">before main--->Hello World!</code></pre><p>说明类增强成功了</p>]]></content>
<categories>
<category> Java </category>
</categories>
<tags>
<tag> Java </tag>
</tags>
</entry>
<entry>
<title>Redis专题分享</title>
<link href="/2019/08/28/Redis%E4%B8%93%E9%A2%98%E5%88%86%E4%BA%AB/"/>
<url>/2019/08/28/Redis%E4%B8%93%E9%A2%98%E5%88%86%E4%BA%AB/</url>
<content type="html"><![CDATA[<p>本文为一次组内的分享，主要介绍了Redis的相关知识</p><p><a href="/files/Redis%E4%B8%93%E9%A2%98%E5%88%86%E4%BA%AB.pdf">Redis使用分享</a></p><embed src="/files/Redis专题分享.pdf" type="application/pdf" width="100%" height="600px">]]></content>
<tags>
<tag> Redis </tag>
</tags>
</entry>
<entry>
<title>Golang小工具分享</title>
<link href="/2019/08/15/Golang%E5%B0%8F%E5%B7%A5%E5%85%B7%E5%88%86%E4%BA%AB/"/>
<url>/2019/08/15/Golang%E5%B0%8F%E5%B7%A5%E5%85%B7%E5%88%86%E4%BA%AB/</url>
<content type="html"><![CDATA[<p>本文为一次组内的分享，主要介绍了Golang的一些小工具，比如：Caddy、Traefik、Gin和Xorm等</p><p><a href="/files/20190521.pdf">Golang小工具分享</a></p><embed src="/files/20190521.pdf" type="application/pdf" width="100%" height="600px">]]></content>
<categories>
<category> 分享 </category>
</categories>
<tags>
<tag> 工具 </tag>
<tag> Golang </tag>
</tags>
</entry>
<entry>
<title>Istio调研</title>
<link href="/2019/07/28/Istio%E8%B0%83%E7%A0%94/"/>
<url>/2019/07/28/Istio%E8%B0%83%E7%A0%94/</url>
<content type="html"><![CDATA[<p>单体应用迁移到微服务的过程中带了不少问题，比如服务注册、服务发现、熔断、鉴权等。Dubbo和SpringCloud虽然简化了微服务间的交互，但是和应用绑定太紧密了，升级和部署部署很方便。同时SpringCloud的一套东西太复杂了，完全掌握需要的成本比较高昂。<br>因此最好是能把应用依赖的SDK下沉到基础设施中，降低微服务接入的复杂度，降低接入成本，使应用更加专注于业务本身。 SeviceMesh翻译过来即是服务网格，做的就是这样的事情。Istio是SeviceMesh的一种实现。本文主要简单介绍一下istio的基本概念和一个示例程序。</p><p>版本:</p><ul><li>minikube: 1.2.0</li><li>kubectl: 1.15</li><li>istio: 1.2.2</li></ul><h2 id="设计">设计</h2><p>Istio主要分为两部分，一部分是数据平面，一部分是控制平面。数据平面主要是作为sidecar和应用部署在一起，拦截应用的流量，sidecar和sidecar进行通信。控制平面主要是控制各种路由策略，将配置下发到sidecar。</p><p><img src="/images/201907281742/arch.svg" alt="" loading="lazy"></p><h3 id="Envoy">Envoy</h3><p>Istio将Envoy作为sidecar和服务部署在一个k8s pod中。Envoy代理应用的所有流量。并且提供负载均衡、服务发现等功能。</p><h3 id="Mixer">Mixer</h3><p>Mixer作为单独的进程部署在服务之外，负责在服务网格上执行访问控制和使用策略，并从 Envoy 代理和其他服务收集遥测数据。因为此种架构会对性能影响比较大。istio1.1版本已经将此功能默认关闭。</p><h3 id="Pilot">Pilot</h3><p>Pilot 为 Envoy sidecar 提供服务发现功能，为智能路由（例如 A/B 测试、金丝雀部署等）和弹性（超时、重试、熔断器等）提供流量管理功能。它将控制流量行为的高级路由规则转换为特定于 Envoy 的配置，并在运行时将它们传播到 sideca</p><h3 id="Galley">Galley</h3><p>Gallery的作用主要是管理验证API配置。随着时间的推移，Galley 将接管 Istio 获取配置、 处理和分配组件的顶级责任。</p><h2 id="概念">概念</h2><h3 id="DestinationRule">DestinationRule</h3><p>DestinationRule的含义相当于Envoy中cluster的概念。用于声明多台相同服务机器的集合。</p><pre class="language-yaml" data-language="yaml"><code class="language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> networking.istio.io/v1alpha3<span class="token key atrule">kind</span><span class="token punctuation">:</span> DestinationRule<span class="token key atrule">metadata</span><span class="token punctuation">:</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> bookinfo<span class="token punctuation">-</span>ratings<span class="token key atrule">spec</span><span class="token punctuation">:</span> <span class="token key atrule">host</span><span class="token punctuation">:</span> ratings.prod.svc.cluster.local <span class="token key atrule">trafficPolicy</span><span class="token punctuation">:</span> <span class="token key atrule">loadBalancer</span><span class="token punctuation">:</span> <span class="token key atrule">simple</span><span class="token punctuation">:</span> LEAST_CONN <span class="token key atrule">subsets</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> testversion <span class="token key atrule">labels</span><span class="token punctuation">:</span> <span class="token key atrule">version</span><span class="token punctuation">:</span> v3 <span class="token key atrule">trafficPolicy</span><span class="token punctuation">:</span> <span class="token key atrule">loadBalancer</span><span class="token punctuation">:</span> <span class="token key atrule">simple</span><span class="token punctuation">:</span> ROUND_ROBIN</code></pre><h3 id="VirtualService">VirtualService</h3><p>VirtualService的含义相当于Envoy中Http Route Table。可以根据Host、Method、Uri等属性将请求路由到不同的服务。</p><pre class="language-yaml" data-language="yaml"><code class="language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> networking.istio.io/v1alpha3<span class="token key atrule">kind</span><span class="token punctuation">:</span> VirtualService<span class="token key atrule">metadata</span><span class="token punctuation">:</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> reviews<span class="token punctuation">-</span>route<span class="token key atrule">spec</span><span class="token punctuation">:</span> <span class="token key atrule">hosts</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> reviews.prod.svc.cluster.local <span class="token key atrule">http</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> <span class="token key atrule">match</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> <span class="token key atrule">uri</span><span class="token punctuation">:</span> <span class="token key atrule">prefix</span><span class="token punctuation">:</span> <span class="token string">"/wpcatalog"</span> <span class="token punctuation">-</span> <span class="token key atrule">uri</span><span class="token punctuation">:</span> <span class="token key atrule">prefix</span><span class="token punctuation">:</span> <span class="token string">"/consumercatalog"</span> <span class="token key atrule">rewrite</span><span class="token punctuation">:</span> <span class="token key atrule">uri</span><span class="token punctuation">:</span> <span class="token string">"/newcatalog"</span> <span class="token key atrule">route</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> <span class="token key atrule">destination</span><span class="token punctuation">:</span> <span class="token key atrule">host</span><span class="token punctuation">:</span> reviews.prod.svc.cluster.local <span class="token key atrule">subset</span><span class="token punctuation">:</span> v2 <span class="token punctuation">-</span> <span class="token key atrule">route</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> <span class="token key atrule">destination</span><span class="token punctuation">:</span> <span class="token key atrule">host</span><span class="token punctuation">:</span> reviews.prod.svc.cluster.local <span class="token key atrule">subset</span><span class="token punctuation">:</span> v1</code></pre><h3 id="ServiceEntry">ServiceEntry</h3><p>DestinationRule定义的是托管在云端集群内部的服务，ServiceEntry主要定义未托管在云端集群内部的服务</p><pre class="language-yaml" data-language="yaml"><code class="language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> networking.istio.io/v1alpha3<span class="token key atrule">kind</span><span class="token punctuation">:</span> ServiceEntry<span class="token key atrule">metadata</span><span class="token punctuation">:</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> external<span class="token punctuation">-</span>svc<span class="token punctuation">-</span>https<span class="token key atrule">spec</span><span class="token punctuation">:</span> <span class="token key atrule">hosts</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> api.dropboxapi.com <span class="token punctuation">-</span> www.googleapis.com <span class="token punctuation">-</span> api.facebook.com <span class="token key atrule">location</span><span class="token punctuation">:</span> MESH_EXTERNAL <span class="token key atrule">ports</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> <span class="token key atrule">number</span><span class="token punctuation">:</span> <span class="token number">443</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> https <span class="token key atrule">protocol</span><span class="token punctuation">:</span> TLS <span class="token key atrule">resolution</span><span class="token punctuation">:</span> DNS</code></pre><h3 id="Gateway">Gateway</h3><p>Gateway常部署在流量的入口。</p><pre class="language-yaml" data-language="yaml"><code class="language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> networking.istio.io/v1alpha3<span class="token key atrule">kind</span><span class="token punctuation">:</span> Gateway<span class="token key atrule">metadata</span><span class="token punctuation">:</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> my<span class="token punctuation">-</span>gateway <span class="token key atrule">namespace</span><span class="token punctuation">:</span> some<span class="token punctuation">-</span>config<span class="token punctuation">-</span>namespace<span class="token key atrule">spec</span><span class="token punctuation">:</span> <span class="token key atrule">selector</span><span class="token punctuation">:</span> <span class="token key atrule">app</span><span class="token punctuation">:</span> my<span class="token punctuation">-</span>gateway<span class="token punctuation">-</span>controller <span class="token key atrule">servers</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> <span class="token key atrule">port</span><span class="token punctuation">:</span> <span class="token key atrule">number</span><span class="token punctuation">:</span> <span class="token number">443</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> https<span class="token punctuation">-</span><span class="token number">443</span> <span class="token key atrule">protocol</span><span class="token punctuation">:</span> HTTPS <span class="token key atrule">hosts</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> uk.bookinfo.com <span class="token punctuation">-</span> eu.bookinfo.com <span class="token key atrule">tls</span><span class="token punctuation">:</span> <span class="token key atrule">mode</span><span class="token punctuation">:</span> SIMPLE <span class="token comment"># enables HTTPS on this port</span> <span class="token key atrule">serverCertificate</span><span class="token punctuation">:</span> /etc/certs/servercert.pem <span class="token key atrule">privateKey</span><span class="token punctuation">:</span> /etc/certs/privatekey.pem</code></pre><h2 id="安装">安装</h2><h3 id="启动minikube集群">启动minikube集群</h3><pre class="language-bash" data-language="bash"><code class="language-bash">minikube start <span class="token parameter variable">--memory</span><span class="token operator">=</span><span class="token number">8192</span> <span class="token parameter variable">--cpus</span><span class="token operator">=</span><span class="token number">4</span></code></pre><h3 id="下载安装包">下载安装包</h3><p><a href="https://github.com/istio/istio/releases">https://github.com/istio/istio/releases</a></p><p><img src="/images/201907281742/istio-dic.png" alt="" loading="lazy"></p><h3 id="安装CRD">安装CRD</h3><blockquote><p>istio自定了50多个资源，包括各种组件以及VirtualService等</p></blockquote><pre class="language-bash" data-language="bash"><code class="language-bash"><span class="token keyword">for</span> <span class="token for-or-select variable">i</span> <span class="token keyword">in</span> install/kubernetes/helm/istio-init/files/crd*yaml<span class="token punctuation">;</span> <span class="token keyword">do</span> kubectl apply <span class="token parameter variable">-f</span> <span class="token variable">$i</span><span class="token punctuation">;</span> <span class="token keyword">done</span></code></pre><h2 id="安装istio">安装istio</h2><pre class="language-bash" data-language="bash"><code class="language-bash">kubectl apply <span class="token parameter variable">-f</span> install/kubernetes/istio-demo.yaml</code></pre><h2 id="启动istio自动注入">启动istio自动注入</h2><pre class="language-bash" data-language="bash"><code class="language-bash">kubectl label namespace default istio-injection<span class="token operator">=</span>enabled</code></pre><p><img src="/images/201907281742/istio-system.png" alt="" loading="lazy"></p><h2 id="示例">示例</h2><h3 id="book服务">book服务</h3><pre class="language-bash" data-language="bash"><code class="language-bash">kubectl apply <span class="token parameter variable">-f</span> samples/bookinfo/platform/kube/bookinfo.yaml</code></pre><h3 id="应用缺省目标规则">应用缺省目标规则</h3><pre class="language-bash" data-language="bash"><code class="language-bash">kubectl apply <span class="token parameter variable">-f</span> samples/bookinfo/networking/destination-rule-all.yaml</code></pre><h2 id="创建网关">创建网关</h2><pre class="language-bash" data-language="bash"><code class="language-bash">kubectl apply <span class="token parameter variable">-f</span> samples/bookinfo/networking/bookinfo-gateway.yaml</code></pre><p><img src="/images/201907281742/book.png" alt="" loading="lazy"></p><h2 id="获取host">获取host</h2><pre class="language-bash" data-language="bash"><code class="language-bash">minikube <span class="token function">ip</span></code></pre><h2 id="获取port">获取port</h2><pre class="language-bash" data-language="bash"><code class="language-bash">kubectl <span class="token parameter variable">-n</span> istio-system get <span class="token function">service</span> istio-ingressgateway <span class="token parameter variable">-o</span> <span class="token assign-left variable">jsonpath</span><span class="token operator">=</span><span class="token string">'{.spec.ports[?(@.name=="http2")].nodePort}'</span></code></pre><p>访问<a href="http://ip">http://ip</a>:port/productpage即可看到页面</p><p><img src="/images/201907281742/bookpage.png" alt="" loading="lazy"></p><p>上面的示例只是简单介绍了应用部署的流程，istio还提供有灰度发布、熔断、限速等其他功能。</p><h2 id="总结">总结</h2><p>Istio将业务逻辑和非业务逻辑分离，非业务逻辑下沉到基础设施，降低了应用接入的成本，带来的好处是巨大的。</p><h2 id="参考文献">参考文献</h2><ul><li><a href="https://istio.io/docs/">Istio官方文档</a></li><li><a href="https://www.cnblogs.com/williamjie/p/9442340.html">使用Istio治理微服务入门</a></li><li><a href="http://mini.eastday.com/mobile/171102113032674.html">Service Mesh：下一代微服务？</a></li><li><a href="https://www.servicemesher.com/blog/201905-servicemesh-development-trend/">ServiceMesh发展趋势：云原生中流砥柱</a></li><li><a href="https://segmentfault.com/a/1190000011118897">康威定律——这个50年前就被提出的微服务概念，你知多少？</a></li><li><a href="https://cloud.tencent.com/developer/article/1409159">Istio 庖丁解牛</a></li><li><a href="https://yq.aliyun.com/articles/606655">浅谈Service Mesh体系中的Envoy</a></li></ul>]]></content>
<categories>
<category> SeviceMesh </category>
</categories>
<tags>
<tag> Istio </tag>
</tags>
</entry>
<entry>
<title>API网关Kong云端部署调研</title>
<link href="/2019/07/10/API%E7%BD%91%E5%85%B3Kong%E4%BA%91%E7%AB%AF%E9%83%A8%E7%BD%B2%E8%B0%83%E7%A0%94/"/>
<url>/2019/07/10/API%E7%BD%91%E5%85%B3Kong%E4%BA%91%E7%AB%AF%E9%83%A8%E7%BD%B2%E8%B0%83%E7%A0%94/</url>
<content type="html"><![CDATA[<p>调研的版本如下：</p><ul><li>Kong: v1.2.1</li><li>KongIngressController: v0.5.0</li><li>Minikube: v1.2.0</li></ul><h2 id="设计思想">设计思想</h2><p>Kong主要通过KongIngressController来支持云端网关。KongIngressController总体设计思想是监听k8s的APIServer，将k8s的Ingress资源转换成Kong的配置。</p><h3 id="无数据库模式">无数据库模式</h3><p>无数据库模式下，kongIngressController作为sidecar和Kong实例一起部署在数据平面。<br><img src="/images/2019071019/dbless-deployment.png" alt="" loading="lazy"></p><h4 id="高可用">高可用</h4><p>在此种模式下，通过增加副本数就可以达到高可用</p><h3 id="依赖数据库模式">依赖数据库模式</h3><p>当有数据库时，KongIngressController单独部署在控制平面，监听ApiServer变更，将数据存储到数据库中。Kong实例依然在数据平面，从数据库中拉取数据。</p><p><img src="/images/2019071019/db-deployment.png" alt="" loading="lazy"></p><h4 id="高可用-2">高可用</h4><p>此种模式，Kong和KongIngressController分别部署在控制平面和数据平面，可以单独扩容。KongIngressController可以存在多个，但是同一时刻只有一个master节点。Kong实例可以部署成DaemonSet，也可以采用自动扩容的策略进行高可用。</p><h2 id="安装使用">安装使用</h2><h3 id="启动k8s集群">启动k8s集群</h3><pre class="language-bash" data-language="bash"><code class="language-bash">minikube startminikube dashboard</code></pre><h3 id="创建KongIngressController">创建KongIngressController</h3><blockquote><p>此种方式创建的是带有数据库模式</p></blockquote><pre class="language-bash" data-language="bash"><code class="language-bash"><span class="token function">curl</span> <span class="token parameter variable">-sL</span> https://bit.ly/kong-ingress <span class="token operator">|</span> kubectl create <span class="token parameter variable">-f</span> -</code></pre><p><img src="/images/2019071019/kong-ingress-k8s.png" alt="" loading="lazy"></p><h3 id="创建服务">创建服务</h3><pre class="language-bash" data-language="bash"><code class="language-bash"><span class="token function">curl</span> <span class="token parameter variable">-sL</span> https://bit.ly/echo-server <span class="token operator">|</span> kubectl apply <span class="token parameter variable">-f</span> -</code></pre><h3 id="创建Ingress">创建Ingress</h3><pre class="language-bash" data-language="bash"><code class="language-bash"> <span class="token builtin class-name">echo</span> <span class="token string">"apiVersion: extensions/v1beta1kind: Ingressmetadata: name: demospec: rules: - http: paths: - path: /foo backend: serviceName: echo servicePort: 80"</span> <span class="token operator">|</span> kubectl apply <span class="token parameter variable">-f</span> -</code></pre><h3 id="查看IP">查看IP</h3><pre class="language-bash" data-language="bash"><code class="language-bash">minikube <span class="token function">service</span> <span class="token parameter variable">-n</span> kong kong-proxy <span class="token parameter variable">--url</span></code></pre><p><img src="/images/2019071019/minikube-ip.png" alt="" loading="lazy"><br>第一个是Http端口，第二个是Https端口</p><h3 id="测试echo服务">测试echo服务</h3><pre class="language-bash" data-language="bash"><code class="language-bash"><span class="token function">curl</span> <span class="token parameter variable">-i</span> http://192.168.99.100:32649/foo</code></pre><p>此时就可以看到返回的集群信息</p><h3 id="添加插件">添加插件</h3><blockquote><p>KongIngressController自定义了许多资源，现在添加一个request-id插件测试一下</p></blockquote><pre class="language-bash" data-language="bash"><code class="language-bash"><span class="token builtin class-name">echo</span> <span class="token string">"apiVersion: configuration.konghq.com/v1kind: KongPluginmetadata: name: request-idconfig: header_name: my-request-idplugin: correlation-id"</span> <span class="token operator">|</span> kubectl apply <span class="token parameter variable">-f</span> -</code></pre><pre class="language-bash" data-language="bash"><code class="language-bash"><span class="token builtin class-name">echo</span> <span class="token string">"apiVersion: extensions/v1beta1kind: Ingressmetadata: name: demo-example-com annotations: plugins.konghq.com: request-idspec: rules: - host: example.com http: paths: - path: /bar backend: serviceName: echo servicePort: 80"</span> <span class="token operator">|</span> kubectl apply <span class="token parameter variable">-f</span> -</code></pre><p>此时访问</p><pre class="language-bash" data-language="bash"><code class="language-bash"><span class="token function">curl</span> <span class="token parameter variable">-i</span> <span class="token parameter variable">-H</span> <span class="token string">"Host:example.com"</span> http://192.168.99.100:32649/bar</code></pre><p>可以看到返回的消息体中就带有my-request-id了</p><p><img src="/images/2019071019/requestId.png" alt="" loading="lazy"></p><p>plugin插件除了可以添加到ingress上面，还可以添加到service上面，这里就不细说了。</p><h2 id="CRD自定义资源">CRD自定义资源</h2><blockquote><p>KongIngressController自定义了一些资源，以便实现自己的功能。主要有KongPlugin、KongIngress、KongConsumer和KongCredential</p></blockquote><h3 id="KongPlugin">KongPlugin</h3><blockquote><p>KongPlugin的含义和非Cloud环境下Kong的含义一样，都是Kong的插件</p></blockquote><pre class="language-yaml" data-language="yaml"><code class="language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> configuration.konghq.com/v1<span class="token key atrule">kind</span><span class="token punctuation">:</span> KongPlugin<span class="token key atrule">metadata</span><span class="token punctuation">:</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> <object name<span class="token punctuation">></span> <span class="token key atrule">namespace</span><span class="token punctuation">:</span> <object namespace<span class="token punctuation">></span> <span class="token key atrule">labels</span><span class="token punctuation">:</span> <span class="token key atrule">global</span><span class="token punctuation">:</span> <span class="token string">"true"</span> <span class="token comment"># 可选，是否为全局插件</span><span class="token key atrule">consumerRef</span><span class="token punctuation">:</span> <name of an existing consumer<span class="token punctuation">></span> <span class="token comment">#可选，指定Consumer</span><span class="token key atrule">disabled</span><span class="token punctuation">:</span> <boolean<span class="token punctuation">></span> <span class="token comment"># 是否开启此插件</span><span class="token key atrule">config</span><span class="token punctuation">:</span> <span class="token comment">#插件本身的配置项</span> <span class="token key atrule">key</span><span class="token punctuation">:</span> value<span class="token key atrule">plugin</span><span class="token punctuation">:</span> <name<span class="token punctuation">-</span>of<span class="token punctuation">-</span>plugin<span class="token punctuation">></span> <span class="token comment"># 插件名称</span></code></pre><h3 id="KongIngress">KongIngress</h3><blockquote><p>KongIngresss在k8s原生的Ingress上面做了扩展，可以更细粒度的控制路由行为。</p></blockquote><pre class="language-yaml" data-language="yaml"><code class="language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> configuration.konghq.com/v1<span class="token key atrule">kind</span><span class="token punctuation">:</span> KongIngress<span class="token key atrule">metadata</span><span class="token punctuation">:</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> configuration<span class="token punctuation">-</span>demo<span class="token key atrule">upstream</span><span class="token punctuation">:</span> <span class="token comment">#后端服务</span> <span class="token key atrule">hash_on</span><span class="token punctuation">:</span> none <span class="token key atrule">hash_fallback</span><span class="token punctuation">:</span> none<span class="token key atrule">proxy</span><span class="token punctuation">:</span> <span class="token key atrule">protocol</span><span class="token punctuation">:</span> http <span class="token key atrule">path</span><span class="token punctuation">:</span> / <span class="token key atrule">connect_timeout</span><span class="token punctuation">:</span> <span class="token number">10000</span> <span class="token key atrule">retries</span><span class="token punctuation">:</span> <span class="token number">10</span> <span class="token key atrule">read_timeout</span><span class="token punctuation">:</span> <span class="token number">10000</span> <span class="token key atrule">write_timeout</span><span class="token punctuation">:</span> <span class="token number">10000</span><span class="token key atrule">route</span><span class="token punctuation">:</span> <span class="token comment">#路由</span> <span class="token key atrule">methods</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> POST <span class="token punctuation">-</span> GET <span class="token key atrule">regex_priority</span><span class="token punctuation">:</span> <span class="token number">0</span> <span class="token key atrule">strip_path</span><span class="token punctuation">:</span> <span class="token boolean important">false</span> <span class="token key atrule">preserve_host</span><span class="token punctuation">:</span> <span class="token boolean important">true</span> <span class="token key atrule">protocols</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> http <span class="token punctuation">-</span> https</code></pre><p>KongIngress需要和Ingress结合使用，主要有两种方式：</p><ul><li><p>创建和Ingress规则同样命名空间，同样名称的KongIngress规则。这种情况下，每个Ingress都需要一个KongIngress，不利于维护。</p></li><li><p>在Ingress配置中添加注解来进行关联：<code>configuration.konghq.com:<KongIngress-resource-name></code> 。 这种方式可以复用KongIngress配置。</p></li></ul><h2 id="annotations">annotations</h2><blockquote><p>KongIngressController在支持k8s原生ingress.class注解的基础上扩展了两个注解：<code>plugins.konghq.com</code>和<code>configuration.konghq.com</code></p></blockquote><h3 id="kubernetes-io-ingress-class"><code>kubernetes.io/ingress.class</code></h3><p>如果一个集群中有多个IngressController，一个Ingress只想被指定的IngressController监听，就需要用到此参数</p><p>Ingress中添加：</p><pre class="language-yaml" data-language="yaml"><code class="language-yaml"><span class="token key atrule">metadata</span><span class="token punctuation">:</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> foo <span class="token key atrule">annotations</span><span class="token punctuation">:</span> <span class="token key atrule">kubernetes.io/ingress.class</span><span class="token punctuation">:</span> <span class="token string">"kong"</span></code></pre><p>KongIngressController中添加：</p><pre class="language-yaml" data-language="yaml"><code class="language-yaml"><span class="token key atrule">spec</span><span class="token punctuation">:</span> <span class="token key atrule">template</span><span class="token punctuation">:</span> <span class="token key atrule">spec</span><span class="token punctuation">:</span> <span class="token key atrule">containers</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> kong<span class="token punctuation">-</span>ingress<span class="token punctuation">-</span>internal<span class="token punctuation">-</span>controller <span class="token key atrule">args</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> /kong<span class="token punctuation">-</span>ingress<span class="token punctuation">-</span>controller <span class="token punctuation">-</span> <span class="token string">'--election-id=ingress-controller-leader-internal'</span> <span class="token punctuation">-</span> <span class="token string">'--ingress-class=kong-internal'</span></code></pre><blockquote><p>如果想在同一个k8s集群内部署多个Kong集群，需要同时调整election-id和ingress-class</p></blockquote><h3 id="plugins-konghq-com"><code>plugins.konghq.com</code></h3><p>这个注解主要用来配置插件</p><pre class="language-none"><code class="language-none">plugins.konghq.com: high-rate-limit, docs-site-cors</code></pre><h3 id="configuration-konghq-com"><code>configuration.konghq.com</code></h3><p>此注解主要用于配置KongIngress</p><h2 id="和Istio结合">和Istio结合</h2><blockquote><p>下图主要参考《IstioHandBook》,可以将Kong放到Cluster的前面，对所有的入口流量进行拦截处理。</p></blockquote><p><img src="/images/2019071019/gateway_istio.jpg" alt="" loading="lazy"></p><h2 id="优点">优点</h2><ol><li><p>Kong在非云环境下的特性，KongIngressController基本都支持了，比如插件机制、负载均衡策略、Router、Service以及Consumer等概念</p></li><li><p>水平扩展比较方便，很多细节都考虑到了</p></li></ol><h2 id="缺点">缺点</h2><ol><li><p>KongIngressController目前还没有发布正式1.0版本，后续版本可能有不兼容的情况存在（类比Kong1.0前和Kong1.0后经常有Breaking Changes）</p></li><li><p>和服务网格Istio的结合思路还不是很清晰</p></li></ol><h2 id="参考文档">参考文档</h2><p><a href="https://www.servicemesher.com/istio-handbook/best-practices/how-to-implement-ingress-gateway.html">为服务网格选择入口网关</a></p><p><a href="https://jimmysong.io/kubernetes-handbook/concepts/ingress.html">Ingress解析</a></p><p><a href="https://www.servicemesher.com/blog/kubernetes-ingress-controller-deployment-and-ha/">Kubernetes Ingress Controller的使用介绍及高可用落地</a></p><p><a href="https://github.com/Kong/kubernetes-ingress-controller/blob/master/docs/design.md">Kong IngressController设计架构</a></p><p><a href="https://github.com/Kong/kubernetes-ingress-controller/tree/master/docs">Kong IngressController Document</a></p><p><a href="https://owelinux.github.io/2018/12/27/article43-k8s-Ingress/">kubernetes 之 Ingress 使用总结</a></p><p><a href="https://jimmysong.io/kubernetes-handbook/">kubernetes Handbook</a></p>]]></content>
<categories>
<category> 网关 </category>
</categories>
</entry>
<entry>
<title>Consul-template基本用法</title>
<link href="/2019/05/11/Consul-template%E5%9F%BA%E6%9C%AC%E7%94%A8%E6%B3%95/"/>
<url>/2019/05/11/Consul-template%E5%9F%BA%E6%9C%AC%E7%94%A8%E6%B3%95/</url>
<content type="html"><![CDATA[<p>consul-template是consul公司开发的一款监控consul服务变化并自动生成配置文件的工具。生成配置文件后，也可以执行指定的一些命令。随着应用微服务化越来越流行，不少公司都开始使用consul了，当consul的监控的服务发生变化时，我们可能会自动重启nginx或者redis等服务。consul-template就是应用在此种场景中。</p><h2 id="安装">安装</h2><p>为了运行<code>consul-template</code>，需要先启动<code>consul</code>服务。<code>consul</code>和<code>consul-template</code>均是采用go语言编写的，可以到<a href="https://www.consul.io/">consul官网</a>以及<a href="https://releases.hashicorp.com/consul-template/">consul-template官网</a>下载二进制包。</p><h2 id="运行">运行</h2><h3 id="启动consul">启动<code>consul</code></h3><pre class="language-bash" data-language="bash"><code class="language-bash">consul agent <span class="token parameter variable">-dev</span> <span class="token comment">#加上-dev参数,consul可以当做master节点运行</span></code></pre><h3 id="启动consul-template">启动<code>consul-template</code></h3><p>首先我们需要编写一个模板文件(in.tpl):</p><pre class="language-none"><code class="language-none">{{ key "foo" }}</code></pre><p>然后执行启动命令:</p><pre class="language-bash" data-language="bash"><code class="language-bash">consul-template <span class="token parameter variable">-template</span> <span class="token string">"in.tpl:out.txt"</span> <span class="token parameter variable">-once</span> <span class="token comment">#添加-once参数后，consul-template运行一次就会退出</span></code></pre><p>接着我们向<code>consul kv</code>存储中添加一对键值:</p><pre class="language-bash" data-language="bash"><code class="language-bash">consul kv put foo bar</code></pre><p>最后，我们可以看到当前目录下生成了一个<code>out.txt</code>文件:</p><pre class="language-txt" data-language="txt"><code class="language-txt">bar</code></pre><h3 id="配置">配置</h3><p>上面的例子是官方提供的，可以供测试使用，实际使用的时候大多会通过指定配置文件的方式启动</p><pre class="language-bash" data-language="bash"><code class="language-bash">consul-template <span class="token parameter variable">-config</span> <span class="token string">"/my/config.hcl"</span></code></pre><p>consul-template的不少配置项需要注意下一下</p><pre class="language-json" data-language="json"><code class="language-json">#config.hclconsul <span class="token punctuation">{</span> #consul集群相关配置 auth <span class="token punctuation">{</span> enabled = <span class="token boolean">true</span> username = <span class="token string">"test"</span> password = <span class="token string">"test"</span> <span class="token punctuation">}</span> address = <span class="token string">"127.0.0.1:8500"</span> token = <span class="token string">"abcd1234"</span> #当consul集群返回异常信息时<span class="token punctuation">,</span>consul-template不会挂掉，而是会不断重试 retry <span class="token punctuation">{</span> enabled = <span class="token boolean">true</span> attempts = <span class="token number">12</span> backoff = <span class="token string">"250ms"</span> max_backoff = <span class="token string">"1m"</span> <span class="token punctuation">}</span>#通常情况下，consul-template会从consul集群的leader节点拉取配置，#当consul特别繁忙时，我们可以选择从follower节点拉取数据，此配置代表最大容忍的数据陈旧时间max_stale = <span class="token string">"10m"</span>#为了防止配置经常修改导致模板不断被渲染，影响系统的稳定#我们可以设置此参数，此参数控制模板两次渲染之间的间隔时间wait <span class="token punctuation">{</span> min = <span class="token string">"5s"</span> max = <span class="token string">"10s"</span><span class="token punctuation">}</span>#下面这个参数很重要。consul-template从consul中取数据时采用的是阻塞的方法#（consul-template设置的默认超时时间为1min，并且不能修改），#假设一个模板依赖<span class="token number">50</span>个资源，一共有<span class="token number">50</span>个consul-template实例的话，就会有<span class="token number">2500</span>个阻塞请求，#会对consul造成比较大的压力#打开deduplicate参数的话，consul-template会自动组成一个集群，leader节点会监控所有的资源，并将整合后的资源再存储到consul中#其他consul-template节点对于这个模板只用监控这一个key即可#不过在<span class="token number">0.18</span>.<span class="token number">5</span>版本，此处有bug<span class="token punctuation">,</span>具体可以看https<span class="token operator">:</span><span class="token comment">//github.com/hashicorp/consul-template/blob/master/CHANGELOG.md</span>deduplicate <span class="token punctuation">{</span> enabled = <span class="token boolean">true</span> prefix = <span class="token string">"consul-template/dedup/"</span><span class="token punctuation">}</span>＃配置模板template <span class="token punctuation">{</span> source = <span class="token string">"/path/on/disk/to/template.ctmpl"</span> destination = <span class="token string">"/path/on/disk/where/template/will/render.txt"</span> create_dest_dirs = <span class="token boolean">true</span> contents = <span class="token string">"{{ keyOrDefault \"service/redis/maxconns@east-aws\" \"5\" }}"</span> command = <span class="token string">"restart service foo"</span> command_timeout = <span class="token string">"60s"</span> error_on_missing_key = <span class="token boolean">false</span> perms = <span class="token number">0600</span> backup = <span class="token boolean">true</span> left_delimiter = <span class="token string">"{{"</span> right_delimiter = <span class="token string">"}}"</span> wait <span class="token punctuation">{</span> min = <span class="token string">"2s"</span> max = <span class="token string">"10s"</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre><h3 id="模板语法">模板语法</h3><p>consul-tempate的语法其实采用是go本身的模板语法，在此基础上添加了不少自定义函数</p><pre class="language-none"><code class="language-none">{{ key "service/redis/maxconns" }}</code></pre><p>key其实是consul-template提供的一个函数，支持从consul中取值。</p><p>同时consul-template也提供了诸如<code>datacenters file keyExists keyOrDefault</code>等函数，具体可以参考官方文档<a href="https://github.com/hashicorp/consul-template#templating-language">https://github.com/hashicorp/consul-template#templating-language</a></p>]]></content>
<categories>
<category> 微服务 </category>
</categories>
<tags>
<tag> Consul </tag>
<tag> Consul-template </tag>
</tags>
</entry>
<entry>
<title>Consul template源码解析</title>
<link href="/2019/05/09/Consul-template%E6%BA%90%E7%A0%81%E8%A7%A3%E6%9E%90/"/>
<url>/2019/05/09/Consul-template%E6%BA%90%E7%A0%81%E8%A7%A3%E6%9E%90/</url>
<content type="html"><![CDATA[<p>上一篇文章简单介绍了一下<a href="/2019/05/11/Consul-template%E5%9F%BA%E6%9C%AC%E7%94%A8%E6%B3%95/">Consul-template基本用法</a>，本篇主要深入看一下consul-template的源码。</p><p>Consul-template的整个流程还是比较清晰的，不过代码中大量运用了goroutine、channel和goto等高级特性，如果不仔细看的话，有些地方可能理不清楚。</p><p>下图即是consul-template的整个执行流程。整个流程开有多个单独的goroutine，分别是监听信号重新加载配置或者停止、监听consul kv存储的变化发送更新通知、监听数据的更新通知以及渲染模板并执行命令等<br><img src="https://xiaodongliu.com/images/2019051201.png" alt="" loading="lazy"></p><h3 id="初始化资源">初始化资源</h3><p>consul-template启动后首先解析配置，并初始化相关资源</p><pre class="language-go" data-language="go"><code class="language-go"><span class="token comment">//cli.go</span><span class="token comment">//dry：dry模式下会将渲染内容展示在stdout中，不会改变生成的文件，方便验证模板内容是否正确</span><span class="token comment">//once:是否只渲染一下，可用于调试</span>runner<span class="token punctuation">,</span> err <span class="token operator">:=</span> manager<span class="token punctuation">.</span><span class="token function">NewRunner</span><span class="token punctuation">(</span>config<span class="token punctuation">,</span> dry<span class="token punctuation">,</span> once<span class="token punctuation">)</span><span class="token keyword">if</span> err <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token function">logError</span><span class="token punctuation">(</span>err<span class="token punctuation">,</span> ExitCodeRunnerError<span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token keyword">go</span> runner<span class="token punctuation">.</span><span class="token function">Start</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token comment">//省略信号监控goroutine</span><span class="token string">``</span><span class="token string">` `</span><span class="token string">``</span><span class="token keyword">go</span><span class="token comment">//runner.go</span><span class="token comment">//NewRunner中调用init方法</span><span class="token keyword">func</span> <span class="token punctuation">(</span>r <span class="token operator">*</span>Runner<span class="token punctuation">)</span> <span class="token function">init</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token builtin">error</span> <span class="token punctuation">{</span><span class="token comment">//...省略配置解析，异常捕获等代码</span><span class="token comment">//根据配置文件创建链接consul的客户端</span>clients<span class="token punctuation">,</span> err <span class="token operator">:=</span> <span class="token function">newClientSet</span><span class="token punctuation">(</span>r<span class="token punctuation">.</span>config<span class="token punctuation">)</span><span class="token comment">//创建监听器</span>watcher<span class="token punctuation">,</span> err <span class="token operator">:=</span> <span class="token function">newWatcher</span><span class="token punctuation">(</span>r<span class="token punctuation">.</span>config<span class="token punctuation">,</span> clients<span class="token punctuation">,</span> r<span class="token punctuation">.</span>once<span class="token punctuation">)</span><span class="token comment">//解析配置的模板</span><span class="token keyword">for</span> <span class="token boolean">_</span><span class="token punctuation">,</span> ctmpl <span class="token operator">:=</span> <span class="token keyword">range</span> <span class="token operator">*</span>r<span class="token punctuation">.</span>config<span class="token punctuation">.</span>Templates <span class="token punctuation">{</span>tmpl<span class="token punctuation">,</span> err <span class="token operator">:=</span> template<span class="token punctuation">.</span><span class="token function">NewTemplate</span><span class="token punctuation">(</span><span class="token operator">&</span>template<span class="token punctuation">.</span>NewTemplateInput<span class="token punctuation">{</span> <span class="token comment">//模板源文件的路径</span> Source<span class="token punctuation">:</span> config<span class="token punctuation">.</span><span class="token function">StringVal</span><span class="token punctuation">(</span>ctmpl<span class="token punctuation">.</span>Source<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment">//模板内容 和上面的路径必须保证有一个存在</span> Contents<span class="token punctuation">:</span> config<span class="token punctuation">.</span><span class="token function">StringVal</span><span class="token punctuation">(</span>ctmpl<span class="token punctuation">.</span>Contents<span class="token punctuation">)</span><span class="token punctuation">,</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token comment">//省略部分初始化代码</span><span class="token keyword">if</span> <span class="token operator">*</span>r<span class="token punctuation">.</span>config<span class="token punctuation">.</span>Dedup<span class="token punctuation">.</span>Enabled <span class="token punctuation">{</span><span class="token keyword">if</span> r<span class="token punctuation">.</span>once <span class="token punctuation">{</span><span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span><span class="token comment">//如果开启了de-dup属性的话，这里会创建de-dup管理器</span>r<span class="token punctuation">.</span>dedup<span class="token punctuation">,</span> err <span class="token operator">=</span> <span class="token function">NewDedupManager</span><span class="token punctuation">(</span>r<span class="token punctuation">.</span>config<span class="token punctuation">.</span>Dedup<span class="token punctuation">,</span> clients<span class="token punctuation">,</span> r<span class="token punctuation">.</span>brain<span class="token punctuation">,</span> r<span class="token punctuation">.</span>templates<span class="token punctuation">)</span><span class="token keyword">if</span> err <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token punctuation">{</span><span class="token keyword">return</span> err<span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token keyword">return</span> <span class="token boolean">nil</span><span class="token punctuation">}</span></code></pre><h3 id="启动de-dup管理器和数据监控goroutine">启动de-dup管理器和数据监控goroutine</h3><p>de-dup主要是为了优化性能，具体可参考上一篇基本用法。</p><pre class="language-go" data-language="go"><code class="language-go"><span class="token keyword">func</span> <span class="token punctuation">(</span>r <span class="token operator">*</span>Runner<span class="token punctuation">)</span> <span class="token function">Start</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token comment">//启动 de-duplication 管理器</span><span class="token keyword">var</span> dedupCh <span class="token operator"><-</span><span class="token keyword">chan</span> <span class="token keyword">struct</span><span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token keyword">if</span> r<span class="token punctuation">.</span>dedup <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token punctuation">{</span><span class="token keyword">if</span> err <span class="token operator">:=</span> r<span class="token punctuation">.</span>dedup<span class="token punctuation">.</span><span class="token function">Start</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> err <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token punctuation">{</span>r<span class="token punctuation">.</span>ErrCh <span class="token operator"><-</span> err<span class="token keyword">return</span><span class="token punctuation">}</span>dedupCh <span class="token operator">=</span> r<span class="token punctuation">.</span>dedup<span class="token punctuation">.</span><span class="token function">UpdateCh</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token keyword">if</span> err <span class="token operator">:=</span> r<span class="token punctuation">.</span><span class="token function">Run</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> err <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token punctuation">{</span>r<span class="token punctuation">.</span>ErrCh <span class="token operator"><-</span> err<span class="token keyword">return</span><span class="token punctuation">}</span><span class="token keyword">for</span> <span class="token punctuation">{</span>NEXT_Q<span class="token punctuation">:</span><span class="token keyword">for</span> <span class="token boolean">_</span><span class="token punctuation">,</span> t <span class="token operator">:=</span> <span class="token keyword">range</span> r<span class="token punctuation">.</span>templates <span class="token punctuation">{</span><span class="token keyword">if</span> <span class="token boolean">_</span><span class="token punctuation">,</span> ok <span class="token operator">:=</span> r<span class="token punctuation">.</span>quiescenceMap<span class="token punctuation">[</span>t<span class="token punctuation">.</span><span class="token function">ID</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">;</span> ok <span class="token punctuation">{</span><span class="token keyword">continue</span> NEXT_Q<span class="token punctuation">}</span><span class="token keyword">for</span> <span class="token boolean">_</span><span class="token punctuation">,</span> c <span class="token operator">:=</span> <span class="token keyword">range</span> r<span class="token punctuation">.</span><span class="token function">templateConfigsFor</span><span class="token punctuation">(</span>t<span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token keyword">if</span> <span class="token operator">*</span>c<span class="token punctuation">.</span>Wait<span class="token punctuation">.</span>Enabled <span class="token punctuation">{</span>r<span class="token punctuation">.</span>quiescenceMap<span class="token punctuation">[</span>t<span class="token punctuation">.</span><span class="token function">ID</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token function">newQuiescence</span><span class="token punctuation">(</span>r<span class="token punctuation">.</span>quiescenceCh<span class="token punctuation">,</span> <span class="token operator">*</span>c<span class="token punctuation">.</span>Wait<span class="token punctuation">.</span>Min<span class="token punctuation">,</span> <span class="token operator">*</span>c<span class="token punctuation">.</span>Wait<span class="token punctuation">.</span>Max<span class="token punctuation">,</span> t<span class="token punctuation">)</span><span class="token keyword">continue</span> NEXT_Q<span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token keyword">if</span> <span class="token operator">*</span>r<span class="token punctuation">.</span>config<span class="token punctuation">.</span>Wait<span class="token punctuation">.</span>Enabled <span class="token punctuation">{</span>r<span class="token punctuation">.</span>quiescenceMap<span class="token punctuation">[</span>t<span class="token punctuation">.</span><span class="token function">ID</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token function">newQuiescence</span><span class="token punctuation">(</span>r<span class="token punctuation">.</span>quiescenceCh<span class="token punctuation">,</span> <span class="token operator">*</span>r<span class="token punctuation">.</span>config<span class="token punctuation">.</span>Wait<span class="token punctuation">.</span>Min<span class="token punctuation">,</span> <span class="token operator">*</span>r<span class="token punctuation">.</span>config<span class="token punctuation">.</span>Wait<span class="token punctuation">.</span>Max<span class="token punctuation">,</span> t<span class="token punctuation">)</span><span class="token keyword">continue</span> NEXT_Q<span class="token punctuation">}</span><span class="token punctuation">}</span>OUTER<span class="token punctuation">:</span><span class="token keyword">select</span> <span class="token punctuation">{</span><span class="token keyword">case</span> view <span class="token operator">:=</span> <span class="token operator"><-</span>r<span class="token punctuation">.</span>watcher<span class="token punctuation">.</span><span class="token function">DataCh</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>r<span class="token punctuation">.</span><span class="token function">Receive</span><span class="token punctuation">(</span>view<span class="token punctuation">.</span><span class="token function">Dependency</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> view<span class="token punctuation">.</span><span class="token function">Data</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token comment">//循环读取数据</span><span class="token keyword">for</span> <span class="token punctuation">{</span><span class="token keyword">select</span> <span class="token punctuation">{</span><span class="token keyword">case</span> view <span class="token operator">:=</span> <span class="token operator"><-</span>r<span class="token punctuation">.</span>watcher<span class="token punctuation">.</span><span class="token function">DataCh</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>r<span class="token punctuation">.</span><span class="token function">Receive</span><span class="token punctuation">(</span>view<span class="token punctuation">.</span><span class="token function">Dependency</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> view<span class="token punctuation">.</span><span class="token function">Data</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token keyword">default</span><span class="token punctuation">:</span><span class="token keyword">break</span> OUTER<span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token keyword">case</span> <span class="token operator"><-</span>dedupCh<span class="token punctuation">:</span><span class="token comment">//接收到de-dup消息</span>log<span class="token punctuation">.</span><span class="token function">Printf</span><span class="token punctuation">(</span><span class="token string">"[INFO] (runner) watcher triggered by de-duplication manager"</span><span class="token punctuation">)</span><span class="token keyword">break</span> OUTER<span class="token punctuation">}</span><span class="token comment">//开始渲染数据</span><span class="token keyword">if</span> err <span class="token operator">:=</span> r<span class="token punctuation">.</span><span class="token function">Run</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> err <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token punctuation">{</span>r<span class="token punctuation">.</span>ErrCh <span class="token operator"><-</span> err<span class="token keyword">return</span><span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre><h3 id="渲染模板">渲染模板</h3><p>consul-template的模板语法其实是采用的golang模板的模板语法，通过自定义函数，来进行数据注入</p><pre class="language-go" data-language="go"><code class="language-go"><span class="token comment">//runner.go</span><span class="token comment">//go runner.Start()</span><span class="token keyword">func</span> <span class="token punctuation">(</span>r <span class="token operator">*</span>Runner<span class="token punctuation">)</span> <span class="token function">Run</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token builtin">error</span> <span class="token punctuation">{</span><span class="token keyword">var</span> newRenderEvent<span class="token punctuation">,</span> wouldRenderAny<span class="token punctuation">,</span> renderedAny <span class="token builtin">bool</span>runCtx <span class="token operator">:=</span> <span class="token operator">&</span>templateRunCtx<span class="token punctuation">{</span>depsMap<span class="token punctuation">:</span> <span class="token function">make</span><span class="token punctuation">(</span><span class="token keyword">map</span><span class="token punctuation">[</span><span class="token builtin">string</span><span class="token punctuation">]</span>dep<span class="token punctuation">.</span>Dependency<span class="token punctuation">)</span><span class="token punctuation">,</span><span class="token punctuation">}</span> <span class="token comment">//渲染模板</span><span class="token keyword">for</span> <span class="token boolean">_</span><span class="token punctuation">,</span> tmpl <span class="token operator">:=</span> <span class="token keyword">range</span> r<span class="token punctuation">.</span>templates <span class="token punctuation">{</span><span class="token comment">//渲染单个模板</span>event<span class="token punctuation">,</span> err <span class="token operator">:=</span> r<span class="token punctuation">.</span><span class="token function">runTemplate</span><span class="token punctuation">(</span>tmpl<span class="token punctuation">,</span> runCtx<span class="token punctuation">)</span><span class="token keyword">if</span> err <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token punctuation">{</span><span class="token keyword">return</span> err<span class="token punctuation">}</span><span class="token punctuation">}</span> <span class="token comment">//渲染模板完毕执行命令</span><span class="token keyword">var</span> errs <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token builtin">error</span><span class="token keyword">for</span> <span class="token boolean">_</span><span class="token punctuation">,</span> t <span class="token operator">:=</span> <span class="token keyword">range</span> runCtx<span class="token punctuation">.</span>commands <span class="token punctuation">{</span>command <span class="token operator">:=</span> config<span class="token punctuation">.</span><span class="token function">StringVal</span><span class="token punctuation">(</span>t<span class="token punctuation">.</span>Exec<span class="token punctuation">.</span>Command<span class="token punctuation">)</span>env <span class="token operator">:=</span> t<span class="token punctuation">.</span>Exec<span class="token punctuation">.</span>Env<span class="token punctuation">.</span><span class="token function">Copy</span><span class="token punctuation">(</span><span class="token punctuation">)</span>env<span class="token punctuation">.</span>Custom <span class="token operator">=</span> <span class="token function">append</span><span class="token punctuation">(</span>r<span class="token punctuation">.</span><span class="token function">childEnv</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> env<span class="token punctuation">.</span>Custom<span class="token operator">...</span><span class="token punctuation">)</span><span class="token keyword">if</span> <span class="token boolean">_</span><span class="token punctuation">,</span> err <span class="token operator">:=</span> <span class="token function">spawnChild</span><span class="token punctuation">(</span><span class="token operator">&</span>spawnChildInput<span class="token punctuation">{</span><span class="token comment">//省略</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> err <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token punctuation">{</span>s <span class="token operator">:=</span> fmt<span class="token punctuation">.</span><span class="token function">Sprintf</span><span class="token punctuation">(</span><span class="token string">"failed to execute command %q from %s"</span><span class="token punctuation">,</span> command<span class="token punctuation">,</span> t<span class="token punctuation">.</span><span class="token function">Display</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span>errs <span class="token operator">=</span> <span class="token function">append</span><span class="token punctuation">(</span>errs<span class="token punctuation">,</span> errors<span class="token punctuation">.</span><span class="token function">Wrap</span><span class="token punctuation">(</span>err<span class="token punctuation">,</span> s<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token keyword">return</span> <span class="token boolean">nil</span><span class="token punctuation">}</span></code></pre><pre class="language-go" data-language="go"><code class="language-go"><span class="token comment">//runner.go</span><span class="token keyword">func</span> <span class="token punctuation">(</span>r <span class="token operator">*</span>Runner<span class="token punctuation">)</span> <span class="token function">runTemplate</span><span class="token punctuation">(</span>tmpl <span class="token operator">*</span>template<span class="token punctuation">.</span>Template<span class="token punctuation">,</span> runCtx <span class="token operator">*</span>templateRunCtx<span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token operator">*</span>RenderEvent<span class="token punctuation">,</span> <span class="token builtin">error</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token comment">// 检查本示例节点是否是leader节点</span>isLeader <span class="token operator">:=</span> <span class="token boolean">true</span><span class="token keyword">if</span> r<span class="token punctuation">.</span>dedup <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token punctuation">{</span>isLeader <span class="token operator">=</span> r<span class="token punctuation">.</span>dedup<span class="token punctuation">.</span><span class="token function">IsLeader</span><span class="token punctuation">(</span>tmpl<span class="token punctuation">)</span><span class="token punctuation">}</span> <span class="token comment">//尝试渲染模板</span>result<span class="token punctuation">,</span> err <span class="token operator">:=</span> tmpl<span class="token punctuation">.</span><span class="token function">Execute</span><span class="token punctuation">(</span><span class="token operator">&</span>template<span class="token punctuation">.</span>ExecuteInput<span class="token punctuation">{</span>Brain<span class="token punctuation">:</span> r<span class="token punctuation">.</span>brain<span class="token punctuation">,</span>Env<span class="token punctuation">:</span> r<span class="token punctuation">.</span><span class="token function">childEnv</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token keyword">if</span> err <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token punctuation">{</span><span class="token keyword">return</span> <span class="token boolean">nil</span><span class="token punctuation">,</span> errors<span class="token punctuation">.</span><span class="token function">Wrap</span><span class="token punctuation">(</span>err<span class="token punctuation">,</span> tmpl<span class="token punctuation">.</span><span class="token function">Source</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token comment">//检查模板渲染所需要的���据是否都满足了，如果不满足，则加入监控列表</span>missing<span class="token punctuation">,</span> used <span class="token operator">:=</span> result<span class="token punctuation">.</span>Missing<span class="token punctuation">,</span> result<span class="token punctuation">.</span>Used<span class="token keyword">for</span> <span class="token boolean">_</span><span class="token punctuation">,</span> d <span class="token operator">:=</span> <span class="token keyword">range</span> used<span class="token punctuation">.</span><span class="token function">List</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token keyword">if</span> isLeader <span class="token operator">&&</span> <span class="token operator">!</span>r<span class="token punctuation">.</span>watcher<span class="token punctuation">.</span><span class="token function">Watching</span><span class="token punctuation">(</span>d<span class="token punctuation">)</span> <span class="token punctuation">{</span>missing<span class="token punctuation">.</span><span class="token function">Add</span><span class="token punctuation">(</span>d<span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token keyword">if</span> <span class="token boolean">_</span><span class="token punctuation">,</span> ok <span class="token operator">:=</span> runCtx<span class="token punctuation">.</span>depsMap<span class="token punctuation">[</span>d<span class="token punctuation">.</span><span class="token function">String</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">;</span> <span class="token operator">!</span>ok <span class="token punctuation">{</span>runCtx<span class="token punctuation">.</span>depsMap<span class="token punctuation">[</span>d<span class="token punctuation">.</span><span class="token function">String</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">]</span> <span class="token operator">=</span> d<span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token keyword">if</span> l <span class="token operator">:=</span> unwatched<span class="token punctuation">.</span><span class="token function">Len</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> l <span class="token operator">></span> <span class="token number">0</span> <span class="token punctuation">{</span><span class="token keyword">for</span> <span class="token boolean">_</span><span class="token punctuation">,</span> d <span class="token operator">:=</span> <span class="token keyword">range</span> unwatched<span class="token punctuation">.</span><span class="token function">List</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token keyword">if</span> isLeader <span class="token operator">||</span> <span class="token operator">!</span>d<span class="token punctuation">.</span><span class="token function">CanShare</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token comment">//注意此处将调用goroutine监控consul kv的变化</span>r<span class="token punctuation">.</span>watcher<span class="token punctuation">.</span><span class="token function">Add</span><span class="token punctuation">(</span>d<span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token keyword">return</span> event<span class="token punctuation">,</span> <span class="token boolean">nil</span><span class="token punctuation">}</span> <span class="token comment">//如果开启了de-duplication模式，并且本示例为leader节点，则更新consul中模板渲染的结果，便于其他节点使用</span><span class="token keyword">if</span> r<span class="token punctuation">.</span>dedup <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token operator">&&</span> isLeader <span class="token punctuation">{</span><span class="token keyword">if</span> err <span class="token operator">:=</span> r<span class="token punctuation">.</span>dedup<span class="token punctuation">.</span><span class="token function">UpdateDeps</span><span class="token punctuation">(</span>tmpl<span class="token punctuation">,</span> used<span class="token punctuation">.</span><span class="token function">List</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span> err <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token punctuation">{</span>log<span class="token punctuation">.</span><span class="token function">Printf</span><span class="token punctuation">(</span><span class="token string">"[ERR] (runner) failed to update dependency data for de-duplication: %v"</span><span class="token punctuation">,</span> err<span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token comment">//如果开启了quiescence特性，则检查一定时间内是否已经更新过了，如果更新过了，不再更新，直接返回</span><span class="token keyword">if</span> q<span class="token punctuation">,</span> ok <span class="token operator">:=</span> r<span class="token punctuation">.</span>quiescenceMap<span class="token punctuation">[</span>tmpl<span class="token punctuation">.</span><span class="token function">ID</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">;</span> ok <span class="token punctuation">{</span>q<span class="token punctuation">.</span><span class="token function">tick</span><span class="token punctuation">(</span><span class="token punctuation">)</span>event<span class="token punctuation">.</span>ForQuiescence <span class="token operator">=</span> <span class="token boolean">true</span><span class="token keyword">return</span> event<span class="token punctuation">,</span> <span class="token boolean">nil</span><span class="token punctuation">}</span><span class="token comment">// 对于每一个模板，将其渲染后的数据写入文件，并存储需要后续执行的命令</span><span class="token keyword">for</span> <span class="token boolean">_</span><span class="token punctuation">,</span> templateConfig <span class="token operator">:=</span> <span class="token keyword">range</span> r<span class="token punctuation">.</span><span class="token function">templateConfigsFor</span><span class="token punctuation">(</span>tmpl<span class="token punctuation">)</span> <span class="token punctuation">{</span>result<span class="token punctuation">,</span> err <span class="token operator">:=</span> renderer<span class="token punctuation">.</span><span class="token function">Render</span><span class="token punctuation">(</span><span class="token operator">&</span>renderer<span class="token punctuation">.</span>RenderInput<span class="token punctuation">{</span>Backup<span class="token punctuation">:</span> config<span class="token punctuation">.</span><span class="token function">BoolVal</span><span class="token punctuation">(</span>templateConfig<span class="token punctuation">.</span>Backup<span class="token punctuation">)</span><span class="token punctuation">,</span>Contents<span class="token punctuation">:</span> result<span class="token punctuation">.</span>Output<span class="token punctuation">,</span>CreateDestDirs<span class="token punctuation">:</span> config<span class="token punctuation">.</span><span class="token function">BoolVal</span><span class="token punctuation">(</span>templateConfig<span class="token punctuation">.</span>CreateDestDirs<span class="token punctuation">)</span><span class="token punctuation">,</span>Dry<span class="token punctuation">:</span> r<span class="token punctuation">.</span>dry<span class="token punctuation">,</span>DryStream<span class="token punctuation">:</span> r<span class="token punctuation">.</span>outStream<span class="token punctuation">,</span>Path<span class="token punctuation">:</span> config<span class="token punctuation">.</span><span class="token function">StringVal</span><span class="token punctuation">(</span>templateConfig<span class="token punctuation">.</span>Destination<span class="token punctuation">)</span><span class="token punctuation">,</span>Perms<span class="token punctuation">:</span> config<span class="token punctuation">.</span><span class="token function">FileModeVal</span><span class="token punctuation">(</span>templateConfig<span class="token punctuation">.</span>Perms<span class="token punctuation">)</span><span class="token punctuation">,</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token keyword">if</span> result<span class="token punctuation">.</span>DidRender <span class="token punctuation">{</span><span class="token comment">//省略模板渲染后执行后续命令的代码</span><span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token keyword">return</span> event<span class="token punctuation">,</span> <span class="token boolean">nil</span><span class="token punctuation">}</span></code></pre><pre class="language-go" data-language="go"><code class="language-go"><span class="token comment">//renderer/renderer.go</span><span class="token keyword">func</span> <span class="token function">Render</span><span class="token punctuation">(</span>i <span class="token operator">*</span>RenderInput<span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token operator">*</span>RenderResult<span class="token punctuation">,</span> <span class="token builtin">error</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>existing<span class="token punctuation">,</span> err <span class="token operator">:=</span> ioutil<span class="token punctuation">.</span><span class="token function">ReadFile</span><span class="token punctuation">(</span>i<span class="token punctuation">.</span>Path<span class="token punctuation">)</span><span class="token keyword">if</span> err <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token operator">&&</span> <span class="token operator">!</span>os<span class="token punctuation">.</span><span class="token function">IsNotExist</span><span class="token punctuation">(</span>err<span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token keyword">return</span> <span class="token boolean">nil</span><span class="token punctuation">,</span> errors<span class="token punctuation">.</span><span class="token function">Wrap</span><span class="token punctuation">(</span>err<span class="token punctuation">,</span> <span class="token string">"failed reading file"</span><span class="token punctuation">)</span><span class="token punctuation">}</span> <span class="token comment">//读取上次渲染后的结果，比较和本次结果是否一致，一致的话就不再重复写入</span><span class="token keyword">if</span> bytes<span class="token punctuation">.</span><span class="token function">Equal</span><span class="token punctuation">(</span>existing<span class="token punctuation">,</span> i<span class="token punctuation">.</span>Contents<span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token keyword">return</span> <span class="token operator">&</span>RenderResult<span class="token punctuation">{</span>DidRender<span class="token punctuation">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span>WouldRender<span class="token punctuation">:</span> <span class="token boolean">true</span><span class="token punctuation">,</span>Contents<span class="token punctuation">:</span> existing<span class="token punctuation">,</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token boolean">nil</span><span class="token punctuation">}</span><span class="token keyword">if</span> i<span class="token punctuation">.</span>Dry <span class="token punctuation">{</span> <span class="token comment">//开启dry模式的话，不写入文件，只打印结果</span>fmt<span class="token punctuation">.</span><span class="token function">Fprintf</span><span class="token punctuation">(</span>i<span class="token punctuation">.</span>DryStream<span class="token punctuation">,</span> <span class="token string">"> %s\n%s"</span><span class="token punctuation">,</span> i<span class="token punctuation">.</span>Path<span class="token punctuation">,</span> i<span class="token punctuation">.</span>Contents<span class="token punctuation">)</span><span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span> <span class="token comment">//否则，确保原子写入文件</span><span class="token keyword">if</span> err <span class="token operator">:=</span> <span class="token function">AtomicWrite</span><span class="token punctuation">(</span>i<span class="token punctuation">.</span>Path<span class="token punctuation">,</span> i<span class="token punctuation">.</span>CreateDestDirs<span class="token punctuation">,</span> i<span class="token punctuation">.</span>Contents<span class="token punctuation">,</span> i<span class="token punctuation">.</span>Perms<span class="token punctuation">,</span> i<span class="token punctuation">.</span>Backup<span class="token punctuation">)</span><span class="token punctuation">;</span> err <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token punctuation">{</span><span class="token keyword">return</span> <span class="token boolean">nil</span><span class="token punctuation">,</span> errors<span class="token punctuation">.</span><span class="token function">Wrap</span><span class="token punctuation">(</span>err<span class="token punctuation">,</span> <span class="token string">"failed writing file"</span><span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token keyword">return</span> <span class="token operator">&</span>RenderResult<span class="token punctuation">{</span>DidRender<span class="token punctuation">:</span> <span class="token boolean">true</span><span class="token punctuation">,</span>WouldRender<span class="token punctuation">:</span> <span class="token boolean">true</span><span class="token punctuation">,</span>Contents<span class="token punctuation">:</span> i<span class="token punctuation">.</span>Contents<span class="token punctuation">,</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token boolean">nil</span><span class="token punctuation">}</span></code></pre><pre class="language-go" data-language="go"><code class="language-go"><span class="token comment">//template/template.go</span><span class="token keyword">type</span> ExecuteInput <span class="token keyword">struct</span> <span class="token punctuation">{</span><span class="token comment">// Brain 存储模板渲染所需要的数据</span>Brain <span class="token operator">*</span>BrainEnv <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token builtin">string</span><span class="token punctuation">}</span><span class="token keyword">type</span> Brain <span class="token keyword">struct</span> <span class="token punctuation">{</span> sync<span class="token punctuation">.</span>RWMutex <span class="token comment">//判断是否收到数据更新</span>receivedData <span class="token keyword">map</span><span class="token punctuation">[</span><span class="token builtin">string</span><span class="token punctuation">]</span><span class="token keyword">struct</span><span class="token punctuation">{</span><span class="token punctuation">}</span> <span class="token comment">//存储具体的数据</span> data <span class="token keyword">map</span><span class="token punctuation">[</span><span class="token builtin">string</span><span class="token punctuation">]</span><span class="token keyword">interface</span><span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token comment">//解析渲染模板：一、需要模板 二需要数据</span><span class="token keyword">func</span> <span class="token punctuation">(</span>t <span class="token operator">*</span>Template<span class="token punctuation">)</span> <span class="token function">Execute</span><span class="token punctuation">(</span>i <span class="token operator">*</span>ExecuteInput<span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token operator">*</span>ExecuteResult<span class="token punctuation">,</span> <span class="token builtin">error</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token keyword">if</span> i <span class="token operator">==</span> <span class="token boolean">nil</span> <span class="token punctuation">{</span>i <span class="token operator">=</span> <span class="token operator">&</span>ExecuteInput<span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token keyword">var</span> used<span class="token punctuation">,</span> missing dep<span class="token punctuation">.</span>Set <span class="token comment">//consul-template使用的go自带的模板渲染引擎</span> tmpl <span class="token operator">:=</span> template<span class="token punctuation">.</span><span class="token function">New</span><span class="token punctuation">(</span><span class="token string">""</span><span class="token punctuation">)</span> <span class="token comment">//自定义分隔符</span> tmpl<span class="token punctuation">.</span><span class="token function">Delims</span><span class="token punctuation">(</span>t<span class="token punctuation">.</span>leftDelim<span class="token punctuation">,</span> t<span class="token punctuation">.</span>rightDelim<span class="token punctuation">)</span> <span class="token comment">//自定义函数，包括需要渲染的数据都在自定义函数里面</span>tmpl<span class="token punctuation">.</span><span class="token function">Funcs</span><span class="token punctuation">(</span><span class="token function">funcMap</span><span class="token punctuation">(</span><span class="token operator">&</span>funcMapInput<span class="token punctuation">{</span>t<span class="token punctuation">:</span> tmpl<span class="token punctuation">,</span>brain<span class="token punctuation">:</span> i<span class="token punctuation">.</span>Brain<span class="token punctuation">,</span>env<span class="token punctuation">:</span> i<span class="token punctuation">.</span>Env<span class="token punctuation">,</span>used<span class="token punctuation">:</span> <span class="token operator">&</span>used<span class="token punctuation">,</span>missing<span class="token punctuation">:</span> <span class="token operator">&</span>missing<span class="token punctuation">,</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token comment">//解析模板</span>tmpl<span class="token punctuation">,</span> err <span class="token operator">:=</span> tmpl<span class="token punctuation">.</span><span class="token function">Parse</span><span class="token punctuation">(</span>t<span class="token punctuation">.</span>contents<span class="token punctuation">)</span><span class="token comment">//开始渲染，返回结果放在b中，这里需要传的数据为nil，是因为数据都在自定义函数里面</span><span class="token keyword">var</span> b bytes<span class="token punctuation">.</span>Buffer<span class="token keyword">if</span> err <span class="token operator">:=</span> tmpl<span class="token punctuation">.</span><span class="token function">Execute</span><span class="token punctuation">(</span><span class="token operator">&</span>b<span class="token punctuation">,</span> <span class="token boolean">nil</span><span class="token punctuation">)</span><span class="token punctuation">;</span> err <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token punctuation">{</span><span class="token keyword">return</span> <span class="token boolean">nil</span><span class="token punctuation">,</span> errors<span class="token punctuation">.</span><span class="token function">Wrap</span><span class="token punctuation">(</span>err<span class="token punctuation">,</span> <span class="token string">"execute"</span><span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token keyword">return</span> <span class="token operator">&</span>ExecuteResult<span class="token punctuation">{</span>Used<span class="token punctuation">:</span> <span class="token operator">&</span>used<span class="token punctuation">,</span>Missing<span class="token punctuation">:</span> <span class="token operator">&</span>missing<span class="token punctuation">,</span>Output<span class="token punctuation">:</span> b<span class="token punctuation">.</span><span class="token function">Bytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token boolean">nil</span><span class="token punctuation">}</span><span class="token keyword">func</span> <span class="token function">funcMap</span><span class="token punctuation">(</span>i <span class="token operator">*</span>funcMapInput<span class="token punctuation">)</span> template<span class="token punctuation">.</span>FuncMap <span class="token punctuation">{</span><span class="token keyword">return</span> template<span class="token punctuation">.</span>FuncMap<span class="token punctuation">{</span><span class="token string">"datacenters"</span><span class="token punctuation">:</span> <span class="token function">datacentersFunc</span><span class="token punctuation">(</span>i<span class="token punctuation">.</span>brain<span class="token punctuation">,</span> i<span class="token punctuation">.</span>used<span class="token punctuation">,</span> i<span class="token punctuation">.</span>missing<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">"file"</span><span class="token punctuation">:</span> <span class="token function">fileFunc</span><span class="token punctuation">(</span>i<span class="token punctuation">.</span>brain<span class="token punctuation">,</span> i<span class="token punctuation">.</span>used<span class="token punctuation">,</span> i<span class="token punctuation">.</span>missing<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment">//所有的值都放在这里面，因此在模板中定义时需要添加上 {{ key xxx}}</span> <span class="token comment">//keyFunc这个函数主要就是从brain中取出数据</span> <span class="token string">"key"</span><span class="token punctuation">:</span> <span class="token function">keyFunc</span><span class="token punctuation">(</span>i<span class="token punctuation">.</span>brain<span class="token punctuation">,</span> i<span class="token punctuation">.</span>used<span class="token punctuation">,</span> i<span class="token punctuation">.</span>missing<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment">//省略其它自定义函数</span><span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre><h3 id="监控consul-kv">监控consul-kv</h3><p>这里监控consul的数据，采用的consul的阻塞get方法，默认会等待1分钟。如果有数据更新，立即返回；否则会超时返回。</p><pre class="language-go" data-language="go"><code class="language-go"><span class="token comment">//watch/watcher.go</span><span class="token keyword">func</span> <span class="token punctuation">(</span>w <span class="token operator">*</span>Watcher<span class="token punctuation">)</span> <span class="token function">Add</span><span class="token punctuation">(</span>d dep<span class="token punctuation">.</span>Dependency<span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token builtin">bool</span><span class="token punctuation">,</span> <span class="token builtin">error</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token comment">//创建监听视图</span>v<span class="token punctuation">,</span> err <span class="token operator">:=</span> <span class="token function">NewView</span><span class="token punctuation">(</span><span class="token operator">&</span>NewViewInput<span class="token punctuation">{</span>Dependency<span class="token punctuation">:</span> d<span class="token punctuation">,</span>Clients<span class="token punctuation">:</span> w<span class="token punctuation">.</span>clients<span class="token punctuation">,</span>MaxStale<span class="token punctuation">:</span> w<span class="token punctuation">.</span>maxStale<span class="token punctuation">,</span>Once<span class="token punctuation">:</span> w<span class="token punctuation">.</span>once<span class="token punctuation">,</span>RetryFunc<span class="token punctuation">:</span> retryFunc<span class="token punctuation">,</span>VaultGrace<span class="token punctuation">:</span> w<span class="token punctuation">.</span>vaultGrace<span class="token punctuation">,</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token keyword">if</span> err <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token punctuation">{</span><span class="token keyword">return</span> <span class="token boolean">false</span><span class="token punctuation">,</span> errors<span class="token punctuation">.</span><span class="token function">Wrap</span><span class="token punctuation">(</span>err<span class="token punctuation">,</span> <span class="token string">"watcher"</span><span class="token punctuation">)</span><span class="token punctuation">}</span>w<span class="token punctuation">.</span>depViewMap<span class="token punctuation">[</span>d<span class="token punctuation">.</span><span class="token function">String</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">]</span> <span class="token operator">=</span> v<span class="token comment">//开启单独的poll</span><span class="token keyword">go</span> v<span class="token punctuation">.</span><span class="token function">poll</span><span class="token punctuation">(</span>w<span class="token punctuation">.</span>dataCh<span class="token punctuation">,</span> w<span class="token punctuation">.</span>errCh<span class="token punctuation">)</span><span class="token keyword">return</span> <span class="token boolean">true</span><span class="token punctuation">,</span> <span class="token boolean">nil</span><span class="token punctuation">}</span></code></pre><pre class="language-go" data-language="go"><code class="language-go"><span class="token comment">// watch/view.go</span><span class="token keyword">func</span> <span class="token punctuation">(</span>v <span class="token operator">*</span>View<span class="token punctuation">)</span> <span class="token function">poll</span><span class="token punctuation">(</span>viewCh <span class="token keyword">chan</span><span class="token operator"><-</span> <span class="token operator">*</span>View<span class="token punctuation">,</span> errCh <span class="token keyword">chan</span><span class="token operator"><-</span> <span class="token builtin">error</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token keyword">var</span> retries <span class="token builtin">int</span><span class="token keyword">for</span> <span class="token punctuation">{</span>doneCh <span class="token operator">:=</span> <span class="token function">make</span><span class="token punctuation">(</span><span class="token keyword">chan</span> <span class="token keyword">struct</span><span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">)</span>successCh <span class="token operator">:=</span> <span class="token function">make</span><span class="token punctuation">(</span><span class="token keyword">chan</span> <span class="token keyword">struct</span><span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">)</span>fetchErrCh <span class="token operator">:=</span> <span class="token function">make</span><span class="token punctuation">(</span><span class="token keyword">chan</span> <span class="token builtin">error</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token comment">//开启单独的goroutine从consul中获取数据</span><span class="token keyword">go</span> v<span class="token punctuation">.</span><span class="token function">fetch</span><span class="token punctuation">(</span>doneCh<span class="token punctuation">,</span> successCh<span class="token punctuation">,</span> fetchErrCh<span class="token punctuation">)</span>WAIT<span class="token punctuation">:</span><span class="token keyword">select</span> <span class="token punctuation">{</span><span class="token comment">//省略接收到数据后的一些充值操作</span>｝<span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre><pre class="language-go" data-language="go"><code class="language-go"><span class="token comment">//watch/view.go</span><span class="token keyword">func</span> <span class="token punctuation">(</span>v <span class="token operator">*</span>View<span class="token punctuation">)</span> <span class="token function">fetch</span><span class="token punctuation">(</span>doneCh<span class="token punctuation">,</span> successCh <span class="token keyword">chan</span><span class="token operator"><-</span> <span class="token keyword">struct</span><span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> errCh <span class="token keyword">chan</span><span class="token operator"><-</span> <span class="token builtin">error</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token keyword">for</span> <span class="token punctuation">{</span><span class="token keyword">case</span> <span class="token operator"><-</span>v<span class="token punctuation">.</span>stopCh<span class="token punctuation">:</span><span class="token keyword">return</span><span class="token keyword">default</span><span class="token punctuation">:</span><span class="token punctuation">}</span>data<span class="token punctuation">,</span> rm<span class="token punctuation">,</span> err <span class="token operator">:=</span> v<span class="token punctuation">.</span>dependency<span class="token punctuation">.</span><span class="token function">Fetch</span><span class="token punctuation">(</span>v<span class="token punctuation">.</span>clients<span class="token punctuation">,</span> <span class="token operator">&</span>dep<span class="token punctuation">.</span>QueryOptions<span class="token punctuation">{</span>AllowStale<span class="token punctuation">:</span> allowStale<span class="token punctuation">,</span> <span class="token comment">//此变量决定是否从consul的flower节点拉取数据</span>WaitTime<span class="token punctuation">:</span> defaultWaitTime<span class="token punctuation">,</span><span class="token comment">//defaultWaitTime = 60 * time.Second 默认超时时间为1分钟，不可以修改</span>WaitIndex<span class="token punctuation">:</span> v<span class="token punctuation">.</span>lastIndex<span class="token punctuation">,</span><span class="token comment">//consul通过此字段来判断客户端和consul的数据是否同步</span>VaultGrace<span class="token punctuation">:</span> v<span class="token punctuation">.</span>vaultGrace<span class="token punctuation">,</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token keyword">select</span> <span class="token punctuation">{</span><span class="token comment">//通知收到了数据</span><span class="token keyword">case</span> successCh <span class="token operator"><-</span> <span class="token keyword">struct</span><span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">:</span><span class="token keyword">default</span><span class="token punctuation">:</span><span class="token punctuation">}</span><span class="token comment">//省略内容</span><span class="token keyword">return</span><span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre><pre class="language-go" data-language="go"><code class="language-go"><span class="token comment">//dependency/kv_get.go</span><span class="token keyword">func</span> <span class="token punctuation">(</span>d <span class="token operator">*</span>KVGetQuery<span class="token punctuation">)</span> <span class="token function">Fetch</span><span class="token punctuation">(</span>clients <span class="token operator">*</span>ClientSet<span class="token punctuation">,</span> opts <span class="token operator">*</span>QueryOptions<span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token keyword">interface</span><span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token operator">*</span>ResponseMetadata<span class="token punctuation">,</span> <span class="token builtin">error</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token comment">//这里会阻塞一分钟，在这一分钟内，如果数据有更新，会立即返回，否则会超时返回</span>pair<span class="token punctuation">,</span> qm<span class="token punctuation">,</span> err <span class="token operator">:=</span> clients<span class="token punctuation">.</span><span class="token function">Consul</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">KV</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">Get</span><span class="token punctuation">(</span>d<span class="token punctuation">.</span>key<span class="token punctuation">,</span> opts<span class="token punctuation">.</span><span class="token function">ToConsulOpts</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token keyword">if</span> err <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token punctuation">{</span><span class="token keyword">return</span> <span class="token boolean">nil</span><span class="token punctuation">,</span> <span class="token boolean">nil</span><span class="token punctuation">,</span> errors<span class="token punctuation">.</span><span class="token function">Wrap</span><span class="token punctuation">(</span>err<span class="token punctuation">,</span> d<span class="token punctuation">.</span><span class="token function">String</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">}</span>rm <span class="token operator">:=</span> <span class="token operator">&</span>ResponseMetadata<span class="token punctuation">{</span>LastIndex<span class="token punctuation">:</span> qm<span class="token punctuation">.</span>LastIndex<span class="token punctuation">,</span>LastContact<span class="token punctuation">:</span> qm<span class="token punctuation">.</span>LastContact<span class="token punctuation">,</span>Block<span class="token punctuation">:</span> d<span class="token punctuation">.</span>block<span class="token punctuation">,</span><span class="token punctuation">}</span>value <span class="token operator">:=</span> <span class="token function">string</span><span class="token punctuation">(</span>pair<span class="token punctuation">.</span>Value<span class="token punctuation">)</span><span class="token keyword">return</span> value<span class="token punctuation">,</span> rm<span class="token punctuation">,</span> <span class="token boolean">nil</span><span class="token punctuation">}</span></code></pre>]]></content>
<categories>
<category> 微服务 </category>
</categories>
<tags>
<tag> Consul </tag>
<tag> Consul-template </tag>
</tags>
</entry>
<entry>
<title>kong0.12.x源码分析(五)---路由匹配策略</title>
<link href="/2019/03/15/kong0-12-x%E6%BA%90%E7%A0%81%E5%88%86%E6%9E%90-%E4%BA%94/"/>
<url>/2019/03/15/kong0-12-x%E6%BA%90%E7%A0%81%E5%88%86%E6%9E%90-%E4%BA%94/</url>
<content type="html"><![CDATA[<blockquote><p>Kong的路由匹配策略基本上是Kong最核心的部分了，逻辑虽然不是很复杂，但是代码还是比较复杂的。</p></blockquote><p>整个路由匹配流程分为两部分：第一部分是系统启动初始化时对路由分类，建立索引。第二部分时请求到来时，根据请求的特征去查找相关路由。具体过程如下：</p><h3 id="1-路由初始化">1. 路由初始化</h3><p>Kong的路由初始化是在<code>init()</code>阶段，调用kong.core.handler中的<code>build_router()</code></p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">local</span> <span class="token keyword">function</span> <span class="token function">build_router</span><span class="token punctuation">(</span>dao<span class="token punctuation">,</span> version<span class="token punctuation">)</span> <span class="token keyword">local</span> apis<span class="token punctuation">,</span> err <span class="token operator">=</span> dao<span class="token punctuation">.</span>apis<span class="token punctuation">:</span><span class="token function">find_all</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token comment">--从数据库中加载所有的API</span> <span class="token keyword">if</span> err <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token string">"could not load APIs: "</span> <span class="token operator">..</span> err <span class="token keyword">end</span> <span class="token keyword">for</span> i <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token operator">#</span>apis <span class="token keyword">do</span> <span class="token comment">-- alias since the router expects 'headers' as a map</span> <span class="token keyword">if</span> apis<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">.</span>hosts <span class="token keyword">then</span> apis<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">.</span>headers <span class="token operator">=</span> <span class="token punctuation">{</span> host <span class="token operator">=</span> apis<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">.</span>hosts <span class="token punctuation">}</span> <span class="token keyword">end</span> <span class="token keyword">end</span> table<span class="token punctuation">.</span><span class="token function">sort</span><span class="token punctuation">(</span>apis<span class="token punctuation">,</span> <span class="token keyword">function</span><span class="token punctuation">(</span>api_a<span class="token punctuation">,</span> api_b<span class="token punctuation">)</span> <span class="token keyword">return</span> api_a<span class="token punctuation">.</span>created_at <span class="token operator"><</span> api_b<span class="token punctuation">.</span>created_at <span class="token comment">-- 根据api的创建时间进行排序</span> <span class="token keyword">end</span><span class="token punctuation">)</span> router<span class="token punctuation">,</span> err <span class="token operator">=</span> Router<span class="token punctuation">.</span><span class="token function">new</span><span class="token punctuation">(</span>apis<span class="token punctuation">)</span> <span class="token comment">-- 创建路由</span> <span class="token keyword">if</span> <span class="token keyword">not</span> router <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token string">"could not create router: "</span> <span class="token operator">..</span> err <span class="token keyword">end</span> <span class="token keyword">if</span> version <span class="token keyword">then</span> router_version <span class="token operator">=</span> version <span class="token keyword">end</span> <span class="token keyword">return</span> <span class="token keyword">true</span><span class="token keyword">end</span></code></pre><p>首先启动时从数据库中加载所有的API,并根据创建时间对API进行排序，然后调用<code>router</code>创建一个新的路由器。</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token comment">-- kong/core/router.lua</span><span class="token keyword">local</span> plain_indexes <span class="token operator">=</span> <span class="token punctuation">{</span> hosts <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> uris <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> methods <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">}</span> <span class="token comment">-- 存储普通的host、uri和methods</span> <span class="token keyword">local</span> uris_prefixes <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span> <span class="token comment">-- 存储前缀匹配的uri</span> <span class="token keyword">local</span> uris_regexes <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span> <span class="token comment">-- 存储含有正则的uri</span> <span class="token keyword">local</span> wildcard_hosts <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span> <span class="token comment">--存储带有正则的host</span> <span class="token keyword">local</span> categories <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span> <span class="token comment">-- 对API进行分类</span> <span class="token keyword">for</span> i <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token operator">#</span>apis <span class="token keyword">do</span> <span class="token comment">-- 第一步：解析API</span> <span class="token keyword">local</span> api_t<span class="token punctuation">,</span> err <span class="token operator">=</span> <span class="token function">marshall_api</span><span class="token punctuation">(</span>apis<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> api_t <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> err <span class="token keyword">end</span> <span class="token comment">-- 第二步：对API进行分组</span> <span class="token function">categorize_api_t</span><span class="token punctuation">(</span>api_t<span class="token punctuation">,</span> api_t<span class="token punctuation">.</span>match_rules<span class="token punctuation">,</span> categories<span class="token punctuation">)</span> <span class="token comment">-- 第三步：创建快速索引</span> <span class="token function">index_api_t</span><span class="token punctuation">(</span>api_t<span class="token punctuation">,</span> plain_indexes<span class="token punctuation">,</span> uris_prefixes<span class="token punctuation">,</span> uris_regexes<span class="token punctuation">,</span> wildcard_hosts<span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token comment">-- uri按照从长到短进行排序</span> table<span class="token punctuation">.</span><span class="token function">sort</span><span class="token punctuation">(</span>uris_prefixes<span class="token punctuation">,</span> <span class="token keyword">function</span><span class="token punctuation">(</span>a<span class="token punctuation">,</span> b<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token operator">#</span>a <span class="token operator">></span> <span class="token operator">#</span>b <span class="token keyword">end</span><span class="token punctuation">)</span> table<span class="token punctuation">.</span><span class="token function">sort</span><span class="token punctuation">(</span>uris_prefixes<span class="token punctuation">,</span> <span class="token keyword">function</span><span class="token punctuation">(</span>uri_t_a<span class="token punctuation">,</span> uri_t_b<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token operator">#</span>uri_t_a<span class="token punctuation">.</span>value <span class="token operator">></span> <span class="token operator">#</span>uri_t_b<span class="token punctuation">.</span>value <span class="token keyword">end</span><span class="token punctuation">)</span></code></pre><p>创建路由器的过程分为三个步骤：解析API，对API进行分组，建立快速索引</p><h4 id="1-1-解析API">1.1 解析API</h4><blockquote><p>解析API做的事情就是把数据库中的格式转换成自己的数据结构，并标记uri是否是前缀匹配，host是否带有通配符等</p></blockquote><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token comment">-- Kong主要采用host uri 和method来匹配路由，每种元素都有自己的编码</span><span class="token keyword">local</span> MATCH_RULES <span class="token operator">=</span> <span class="token punctuation">{</span> HOST <span class="token operator">=</span> <span class="token number">0x01</span><span class="token punctuation">,</span> URI <span class="token operator">=</span> <span class="token number">0x02</span><span class="token punctuation">,</span> METHOD <span class="token operator">=</span> <span class="token number">0x04</span><span class="token punctuation">,</span><span class="token punctuation">}</span><span class="token keyword">local</span> <span class="token keyword">function</span> <span class="token function">marshall_api</span><span class="token punctuation">(</span>api<span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> <span class="token punctuation">(</span>api<span class="token punctuation">.</span>headers <span class="token keyword">or</span> api<span class="token punctuation">.</span>methods <span class="token keyword">or</span> api<span class="token punctuation">.</span>uris<span class="token punctuation">)</span> <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token string">"could not categorize API"</span> <span class="token keyword">end</span> <span class="token comment">-- 解析后的api_t数据结构</span> <span class="token keyword">local</span> api_t <span class="token operator">=</span> <span class="token punctuation">{</span> api <span class="token operator">=</span> api<span class="token punctuation">,</span> <span class="token comment">--原始api</span> strip_uri <span class="token operator">=</span> api<span class="token punctuation">.</span>strip_uri<span class="token punctuation">,</span> <span class="token comment">-- 是否strip uri</span> preserve_host <span class="token operator">=</span> api<span class="token punctuation">.</span>preserve_host<span class="token punctuation">,</span> <span class="token comment">--传给upstream是否保留host</span> match_rules <span class="token operator">=</span> <span class="token number">0x00</span><span class="token punctuation">,</span> <span class="token comment">--当前api所属的匹配策略码，亦即所属类别</span> hosts <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> uris <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> methods <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> upstream_url_t <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">}</span></code></pre><p>解析hosts:</p><blockquote><p>Kong将host、uri和method各分配了一个策略码，然后对他们进行按位与从而达到分类的目的</p></blockquote><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">if</span> api<span class="token punctuation">.</span>headers <span class="token keyword">then</span> <span class="token comment">---省略校验代码</span> <span class="token keyword">local</span> host_values <span class="token operator">=</span> api<span class="token punctuation">.</span>headers<span class="token punctuation">[</span><span class="token string">"Host"</span><span class="token punctuation">]</span> <span class="token keyword">or</span> api<span class="token punctuation">.</span>headers<span class="token punctuation">[</span><span class="token string">"host"</span><span class="token punctuation">]</span> <span class="token keyword">if</span> <span class="token operator">#</span>host_values <span class="token operator">></span> <span class="token number">0</span> <span class="token keyword">then</span> <span class="token comment">-- 采用按位 或 确定策略</span> api_t<span class="token punctuation">.</span>match_rules <span class="token operator">=</span> <span class="token function">bor</span><span class="token punctuation">(</span>api_t<span class="token punctuation">.</span>match_rules<span class="token punctuation">,</span> MATCH_RULES<span class="token punctuation">.</span>HOST<span class="token punctuation">)</span> <span class="token keyword">for</span> _<span class="token punctuation">,</span> host_value <span class="token keyword">in</span> <span class="token function">ipairs</span><span class="token punctuation">(</span>host_values<span class="token punctuation">)</span> <span class="token keyword">do</span> <span class="token keyword">if</span> <span class="token function">find</span><span class="token punctuation">(</span>host_value<span class="token punctuation">,</span> <span class="token string">"*"</span><span class="token punctuation">,</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token keyword">true</span><span class="token punctuation">)</span> <span class="token keyword">then</span> <span class="token comment">-- 查看是否带有通配符</span> <span class="token keyword">local</span> wildcard_host_regex <span class="token operator">=</span> host_value<span class="token punctuation">:</span><span class="token function">gsub</span><span class="token punctuation">(</span><span class="token string">"%."</span><span class="token punctuation">,</span> <span class="token string">"\\."</span><span class="token punctuation">)</span> <span class="token punctuation">:</span><span class="token function">gsub</span><span class="token punctuation">(</span><span class="token string">"%*"</span><span class="token punctuation">,</span> <span class="token string">".+"</span><span class="token punctuation">)</span> <span class="token operator">..</span> <span class="token string">"$"</span> <span class="token function">insert</span><span class="token punctuation">(</span>api_t<span class="token punctuation">.</span>hosts<span class="token punctuation">,</span> <span class="token punctuation">{</span> wildcard <span class="token operator">=</span> <span class="token keyword">true</span><span class="token punctuation">,</span> value <span class="token operator">=</span> host_value<span class="token punctuation">,</span> regex <span class="token operator">=</span> wildcard_host_regex<span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">)</span> <span class="token keyword">else</span> <span class="token function">insert</span><span class="token punctuation">(</span>api_t<span class="token punctuation">.</span>hosts<span class="token punctuation">,</span> <span class="token punctuation">{</span> value <span class="token operator">=</span> host_value<span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">)</span> <span class="token keyword">end</span> api_t<span class="token punctuation">.</span>hosts<span class="token punctuation">[</span>host_value<span class="token punctuation">]</span> <span class="token operator">=</span> host_value <span class="token keyword">end</span> <span class="token keyword">end</span><span class="token keyword">end</span></code></pre><p>解析uri和method的逻辑和解析host的逻辑类似。如果一个api同时配置了host、uri 和method，那么类别为7。</p><h4 id="1-2-API分组">1.2. API分组</h4><p>API分组比较简单，主要将上一步解析出来的API按照类别重新组织一下数据结构</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">local</span> <span class="token keyword">function</span> <span class="token function">categorize_api_t</span><span class="token punctuation">(</span>api_t<span class="token punctuation">,</span> bit_category<span class="token punctuation">,</span> categories<span class="token punctuation">)</span> <span class="token keyword">local</span> category <span class="token operator">=</span> categories<span class="token punctuation">[</span>bit_category<span class="token punctuation">]</span> <span class="token keyword">if</span> <span class="token keyword">not</span> category <span class="token keyword">then</span> category <span class="token operator">=</span> <span class="token punctuation">{</span> apis_by_hosts <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> apis_by_uris <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> apis_by_methods <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> all <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">}</span> categories<span class="token punctuation">[</span>bit_category<span class="token punctuation">]</span> <span class="token operator">=</span> category <span class="token keyword">end</span> <span class="token function">insert</span><span class="token punctuation">(</span>category<span class="token punctuation">.</span>all<span class="token punctuation">,</span> api_t<span class="token punctuation">)</span> <span class="token keyword">for</span> _<span class="token punctuation">,</span> host_t <span class="token keyword">in</span> <span class="token function">ipairs</span><span class="token punctuation">(</span>api_t<span class="token punctuation">.</span>hosts<span class="token punctuation">)</span> <span class="token keyword">do</span> <span class="token keyword">if</span> <span class="token keyword">not</span> category<span class="token punctuation">.</span>apis_by_hosts<span class="token punctuation">[</span>host_t<span class="token punctuation">.</span>value<span class="token punctuation">]</span> <span class="token keyword">then</span> category<span class="token punctuation">.</span>apis_by_hosts<span class="token punctuation">[</span>host_t<span class="token punctuation">.</span>value<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span> <span class="token keyword">end</span> <span class="token function">insert</span><span class="token punctuation">(</span>category<span class="token punctuation">.</span>apis_by_hosts<span class="token punctuation">[</span>host_t<span class="token punctuation">.</span>value<span class="token punctuation">]</span><span class="token punctuation">,</span> api_t<span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token comment">-- 省略uri 和method分组逻辑，和host相同</span><span class="token keyword">end</span></code></pre><h4 id="1-3-建立索引">1.3 建立索引</h4><p>这里说建立索引其实是将<code>host</code>、<code>uri</code>和<code>method</code>分别拆出来放到单独的数组里面</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">local</span> <span class="token keyword">function</span> <span class="token function">index_api_t</span><span class="token punctuation">(</span>api_t<span class="token punctuation">,</span> plain_indexes<span class="token punctuation">,</span> uris_prefixes<span class="token punctuation">,</span> uris_regexes<span class="token punctuation">,</span> wildcard_hosts<span class="token punctuation">)</span> <span class="token keyword">for</span> _<span class="token punctuation">,</span> host_t <span class="token keyword">in</span> <span class="token function">ipairs</span><span class="token punctuation">(</span>api_t<span class="token punctuation">.</span>hosts<span class="token punctuation">)</span> <span class="token keyword">do</span> <span class="token keyword">if</span> host_t<span class="token punctuation">.</span>wildcard <span class="token keyword">then</span> <span class="token function">insert</span><span class="token punctuation">(</span>wildcard_hosts<span class="token punctuation">,</span> host_t<span class="token punctuation">)</span> <span class="token keyword">else</span> plain_indexes<span class="token punctuation">.</span>hosts<span class="token punctuation">[</span>host_t<span class="token punctuation">.</span>value<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token keyword">true</span> <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token keyword">for</span> _<span class="token punctuation">,</span> uri_t <span class="token keyword">in</span> <span class="token function">ipairs</span><span class="token punctuation">(</span>api_t<span class="token punctuation">.</span>uris<span class="token punctuation">)</span> <span class="token keyword">do</span> <span class="token keyword">if</span> uri_t<span class="token punctuation">.</span>is_prefix <span class="token keyword">then</span> plain_indexes<span class="token punctuation">.</span>uris<span class="token punctuation">[</span>uri_t<span class="token punctuation">.</span>value<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token keyword">true</span> <span class="token function">insert</span><span class="token punctuation">(</span>uris_prefixes<span class="token punctuation">,</span> uri_t<span class="token punctuation">)</span> <span class="token keyword">else</span> <span class="token function">insert</span><span class="token punctuation">(</span>uris_regexes<span class="token punctuation">,</span> uri_t<span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token keyword">for</span> method <span class="token keyword">in</span> <span class="token function">pairs</span><span class="token punctuation">(</span>api_t<span class="token punctuation">.</span>methods<span class="token punctuation">)</span> <span class="token keyword">do</span> plain_indexes<span class="token punctuation">.</span>methods<span class="token punctuation">[</span>method<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token keyword">true</span> <span class="token keyword">end</span><span class="token keyword">end</span></code></pre><p>路由的建立上面已经分析完了，主要做的事情是解析API,然后将API根据特征分类，最后再对host、uri和method建立快速索引</p><h3 id="2-路由匹配">2. 路由匹配</h3><p>Kong匹配API是在access阶段的</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">function</span> Kong<span class="token punctuation">.</span><span class="token function">access</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">local</span> ctx <span class="token operator">=</span> ngx<span class="token punctuation">.</span>ctx core<span class="token punctuation">.</span>access<span class="token punctuation">.</span><span class="token function">before</span><span class="token punctuation">(</span>ctx<span class="token punctuation">)</span> <span class="token comment">-- 省略后面执行插件的逻辑</span></code></pre><p>最终会到router的exec方法中</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">function</span> self<span class="token punctuation">.</span><span class="token function">exec</span><span class="token punctuation">(</span>ngx<span class="token punctuation">)</span> <span class="token keyword">local</span> method <span class="token operator">=</span> ngx<span class="token punctuation">.</span>req<span class="token punctuation">.</span><span class="token function">get_method</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">local</span> request_uri <span class="token operator">=</span> ngx<span class="token punctuation">.</span>var<span class="token punctuation">.</span>request_uri <span class="token keyword">local</span> uri <span class="token operator">=</span> request_uri <span class="token keyword">do</span> <span class="token keyword">local</span> idx <span class="token operator">=</span> <span class="token function">find</span><span class="token punctuation">(</span>uri<span class="token punctuation">,</span> <span class="token string">"?"</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">,</span> <span class="token keyword">true</span><span class="token punctuation">)</span> <span class="token keyword">if</span> idx <span class="token keyword">then</span> uri <span class="token operator">=</span> <span class="token function">sub</span><span class="token punctuation">(</span>uri<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> idx <span class="token operator">-</span> <span class="token number">1</span><span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token keyword">local</span> req_host <span class="token operator">=</span> ngx<span class="token punctuation">.</span>var<span class="token punctuation">.</span>http_host <span class="token keyword">or</span> <span class="token string">""</span> <span class="token keyword">local</span> match_t <span class="token operator">=</span> <span class="token function">find_api</span><span class="token punctuation">(</span>method<span class="token punctuation">,</span> uri<span class="token punctuation">,</span> req_host<span class="token punctuation">,</span> ngx<span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> match_t <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span> <span class="token keyword">end</span> <span class="token keyword">return</span> match_t<span class="token keyword">end</span></code></pre><p>再到find_api方法<br>首先检查缓存中是否存在</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">local</span> cache_key <span class="token operator">=</span> <span class="token function">fmt</span><span class="token punctuation">(</span><span class="token string">"%s:%s:%s"</span><span class="token punctuation">,</span> req_method<span class="token punctuation">,</span> req_uri<span class="token punctuation">,</span> req_host<span class="token punctuation">)</span><span class="token keyword">do</span> <span class="token keyword">local</span> match_t <span class="token operator">=</span> cache<span class="token punctuation">:</span><span class="token function">get</span><span class="token punctuation">(</span>cache_key<span class="token punctuation">)</span> <span class="token keyword">if</span> match_t <span class="token keyword">then</span> <span class="token keyword">return</span> match_t <span class="token keyword">end</span><span class="token keyword">end</span></code></pre><p>如果缓存中不存在，根据刚开始建立的索引来判断当前请求所属的类别</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">if</span> plain_indexes<span class="token punctuation">.</span>hosts<span class="token punctuation">[</span>req_host<span class="token punctuation">]</span> <span class="token keyword">then</span> <span class="token comment">-- 通过 按位或计算类别</span> req_category <span class="token operator">=</span> <span class="token function">bor</span><span class="token punctuation">(</span>req_category<span class="token punctuation">,</span> MATCH_RULES<span class="token punctuation">.</span>HOST<span class="token punctuation">)</span> <span class="token keyword">elseif</span> req_host <span class="token keyword">then</span> <span class="token keyword">for</span> i <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token operator">#</span>wildcard_hosts <span class="token keyword">do</span> <span class="token keyword">local</span> from<span class="token punctuation">,</span> _<span class="token punctuation">,</span> err <span class="token operator">=</span> <span class="token function">re_find</span><span class="token punctuation">(</span>req_host<span class="token punctuation">,</span> wildcard_hosts<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">.</span>regex<span class="token punctuation">,</span> <span class="token string">"ajo"</span><span class="token punctuation">)</span> <span class="token keyword">if</span> err <span class="token keyword">then</span> <span class="token function">log</span><span class="token punctuation">(</span>ERR<span class="token punctuation">,</span> <span class="token string">"could not match wildcard host: "</span><span class="token punctuation">,</span> err<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token keyword">end</span> <span class="token keyword">if</span> from <span class="token keyword">then</span> hits<span class="token punctuation">.</span>host <span class="token operator">=</span> wildcard_hosts<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">.</span>value <span class="token comment">-- 通过 按位或计算类别</span> req_category <span class="token operator">=</span> <span class="token function">bor</span><span class="token punctuation">(</span>req_category<span class="token punctuation">,</span> MATCH_RULES<span class="token punctuation">.</span>HOST<span class="token punctuation">)</span> <span class="token keyword">break</span> <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token keyword">end</span><span class="token comment">-- 省略uri和method的匹配过程，和上面的类似</span></code></pre><p>计算出所属的类别后，根据类别值判断出其在应该查找的API集合中的索引。比如一个请求带有host uri和methoud三个参数，那么其应该从 {host、uri、method}(API类别7) {host、uri}{API类别3}依次往下寻找匹配的API。查找时先从一个精简的集合中查找，如果查不到再全量查找。</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">local</span> CATEGORIES <span class="token operator">=</span> <span class="token punctuation">{</span> <span class="token function">bor</span><span class="token punctuation">(</span>MATCH_RULES<span class="token punctuation">.</span>HOST<span class="token punctuation">,</span> MATCH_RULES<span class="token punctuation">.</span>URI<span class="token punctuation">,</span> MATCH_RULES<span class="token punctuation">.</span>METHOD<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token function">bor</span><span class="token punctuation">(</span>MATCH_RULES<span class="token punctuation">.</span>HOST<span class="token punctuation">,</span> MATCH_RULES<span class="token punctuation">.</span>URI<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token function">bor</span><span class="token punctuation">(</span>MATCH_RULES<span class="token punctuation">.</span>HOST<span class="token punctuation">,</span> MATCH_RULES<span class="token punctuation">.</span>METHOD<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token function">bor</span><span class="token punctuation">(</span>MATCH_RULES<span class="token punctuation">.</span>METHOD<span class="token punctuation">,</span> MATCH_RULES<span class="token punctuation">.</span>URI<span class="token punctuation">)</span><span class="token punctuation">,</span> MATCH_RULES<span class="token punctuation">.</span>HOST<span class="token punctuation">,</span> MATCH_RULES<span class="token punctuation">.</span>URI<span class="token punctuation">,</span> MATCH_RULES<span class="token punctuation">.</span>METHOD<span class="token punctuation">,</span><span class="token punctuation">}</span> <span class="token keyword">local</span> categories_len <span class="token operator">=</span> <span class="token operator">#</span>CATEGORIES<span class="token keyword">local</span> CATEGORIES_LOOKUP <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token keyword">for</span> i <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span> categories_len <span class="token keyword">do</span> CATEGORIES_LOOKUP<span class="token punctuation">[</span>CATEGORIES<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">]</span> <span class="token operator">=</span> i<span class="token keyword">end</span><span class="token keyword">local</span> category_idx <span class="token operator">=</span> CATEGORIES_LOOKUP<span class="token punctuation">[</span>req_category<span class="token punctuation">]</span><span class="token keyword">local</span> matched_api<span class="token comment">-- 根据类别值判断出其在应该查找的API集合中的索引。比如一个请求带有host uri和methoud三个参数，那么其应该</span><span class="token comment">-- 从 {host、uri、method}(API类别7) {host、uri}{API类别3}依次往下寻找匹配的API</span><span class="token keyword">while</span> category_idx <span class="token operator"><=</span> categories_len <span class="token keyword">do</span> <span class="token keyword">local</span> bit_category <span class="token operator">=</span> CATEGORIES<span class="token punctuation">[</span>category_idx<span class="token punctuation">]</span> <span class="token keyword">local</span> category <span class="token operator">=</span> categories<span class="token punctuation">[</span>bit_category<span class="token punctuation">]</span> <span class="token keyword">if</span> category <span class="token keyword">then</span> <span class="token comment">-- 从一个精简结合中查找</span> <span class="token comment">-- 如果精简集合中查不到，则全量查找此类别的所有API</span> <span class="token keyword">end</span><span class="token keyword">end</span></code></pre><p>精简集合查找过程</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">local</span> reduced_candidates<span class="token punctuation">,</span> category_candidates <span class="token operator">=</span> <span class="token function">reduce</span><span class="token punctuation">(</span>category<span class="token punctuation">,</span> bit_category<span class="token punctuation">,</span> ctx<span class="token punctuation">)</span> <span class="token keyword">if</span> reduced_candidates <span class="token keyword">then</span> <span class="token keyword">for</span> i <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token operator">#</span>reduced_candidates <span class="token keyword">do</span> <span class="token keyword">if</span> <span class="token function">match_api</span><span class="token punctuation">(</span>reduced_candidates<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">,</span> ctx<span class="token punctuation">)</span> <span class="token keyword">then</span> matched_api <span class="token operator">=</span> reduced_candidates<span class="token punctuation">[</span>i<span class="token punctuation">]</span> <span class="token keyword">break</span> <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token keyword">end</span></code></pre><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">do</span> <span class="token keyword">local</span> reducers <span class="token operator">=</span> <span class="token punctuation">{</span> <span class="token punctuation">[</span>MATCH_RULES<span class="token punctuation">.</span>HOST<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token keyword">function</span><span class="token punctuation">(</span>category<span class="token punctuation">,</span> ctx<span class="token punctuation">)</span> <span class="token keyword">return</span> category<span class="token punctuation">.</span>apis_by_hosts<span class="token punctuation">[</span>ctx<span class="token punctuation">.</span>hits<span class="token punctuation">.</span>host<span class="token punctuation">]</span> <span class="token keyword">end</span><span class="token punctuation">,</span> <span class="token punctuation">[</span>MATCH_RULES<span class="token punctuation">.</span>URI<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token keyword">function</span><span class="token punctuation">(</span>category<span class="token punctuation">,</span> ctx<span class="token punctuation">)</span> <span class="token keyword">return</span> category<span class="token punctuation">.</span>apis_by_uris<span class="token punctuation">[</span>ctx<span class="token punctuation">.</span>hits<span class="token punctuation">.</span>uri <span class="token keyword">or</span> ctx<span class="token punctuation">.</span>req_uri<span class="token punctuation">]</span> <span class="token keyword">end</span><span class="token punctuation">,</span> <span class="token punctuation">[</span>MATCH_RULES<span class="token punctuation">.</span>METHOD<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token keyword">function</span><span class="token punctuation">(</span>category<span class="token punctuation">,</span> ctx<span class="token punctuation">)</span> <span class="token keyword">return</span> category<span class="token punctuation">.</span>apis_by_methods<span class="token punctuation">[</span>ctx<span class="token punctuation">.</span>req_method<span class="token punctuation">]</span> <span class="token keyword">end</span><span class="token punctuation">,</span> <span class="token punctuation">}</span> reduce <span class="token operator">=</span> <span class="token keyword">function</span><span class="token punctuation">(</span>category<span class="token punctuation">,</span> bit_category<span class="token punctuation">,</span> ctx<span class="token punctuation">)</span> <span class="token comment">-- run cached reducer</span> <span class="token keyword">if</span> <span class="token function">type</span><span class="token punctuation">(</span>reducers<span class="token punctuation">[</span>bit_category<span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token string">"function"</span> <span class="token keyword">then</span> <span class="token keyword">return</span> reducers<span class="token punctuation">[</span>bit_category<span class="token punctuation">]</span><span class="token punctuation">(</span>category<span class="token punctuation">,</span> ctx<span class="token punctuation">)</span><span class="token punctuation">,</span> category<span class="token punctuation">.</span>all <span class="token keyword">end</span> <span class="token comment">-- build and cache reducer</span> <span class="token keyword">local</span> reducers_set <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span> <span class="token comment">-- 采用按位与将非1、2、4的API类别转为1、2、4的精简子集</span> <span class="token keyword">for</span> _<span class="token punctuation">,</span> bit_match_rule <span class="token keyword">in</span> <span class="token function">pairs</span><span class="token punctuation">(</span>MATCH_RULES<span class="token punctuation">)</span> <span class="token keyword">do</span> <span class="token keyword">if</span> <span class="token function">band</span><span class="token punctuation">(</span>bit_category<span class="token punctuation">,</span> bit_match_rule<span class="token punctuation">)</span> <span class="token operator">~=</span> <span class="token number">0</span> <span class="token keyword">then</span> reducers_set<span class="token punctuation">[</span><span class="token operator">#</span>reducers_set <span class="token operator">+</span> <span class="token number">1</span><span class="token punctuation">]</span> <span class="token operator">=</span> reducers<span class="token punctuation">[</span>bit_match_rule<span class="token punctuation">]</span> <span class="token keyword">end</span> <span class="token keyword">end</span> reducers<span class="token punctuation">[</span>bit_category<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token keyword">function</span><span class="token punctuation">(</span>category<span class="token punctuation">,</span> ctx<span class="token punctuation">)</span> <span class="token keyword">local</span> min_len <span class="token operator">=</span> <span class="token number">0</span> <span class="token keyword">local</span> smallest_set <span class="token comment">-- 选择集合元素最小的作为结果集返回</span> <span class="token comment">-- 比如当前类别为7,其中国 host集合中有5个API，uri有3个API,method有8个API，那么返回uri这个集合中的API作为查找集</span> <span class="token keyword">for</span> i <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token operator">#</span>reducers_set <span class="token keyword">do</span> <span class="token keyword">local</span> candidates <span class="token operator">=</span> reducers_set<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">(</span>category<span class="token punctuation">,</span> ctx<span class="token punctuation">)</span> <span class="token keyword">if</span> candidates <span class="token operator">~=</span> <span class="token keyword">nil</span> <span class="token keyword">and</span> <span class="token punctuation">(</span><span class="token keyword">not</span> smallest_set <span class="token keyword">or</span> <span class="token operator">#</span>candidates <span class="token operator"><</span> min_len<span class="token punctuation">)</span> <span class="token keyword">then</span> min_len <span class="token operator">=</span> <span class="token operator">#</span>candidates smallest_set <span class="token operator">=</span> candidates <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token keyword">return</span> smallest_set <span class="token keyword">end</span> <span class="token keyword">return</span> reducers<span class="token punctuation">[</span>bit_category<span class="token punctuation">]</span><span class="token punctuation">(</span>category<span class="token punctuation">,</span> ctx<span class="token punctuation">)</span><span class="token punctuation">,</span> category<span class="token punctuation">.</span>all <span class="token keyword">end</span><span class="token keyword">end</span></code></pre><p>得到应该查找的API集合后，就采用匹配器进行比较</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">for</span> i <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token operator">#</span>reduced_candidates <span class="token keyword">do</span> <span class="token keyword">if</span> <span class="token function">match_api</span><span class="token punctuation">(</span>reduced_candidates<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">,</span> ctx<span class="token punctuation">)</span> <span class="token keyword">then</span> matched_api <span class="token operator">=</span> reduced_candidates<span class="token punctuation">[</span>i<span class="token punctuation">]</span> <span class="token keyword">break</span> <span class="token keyword">end</span><span class="token keyword">end</span></code></pre><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">do</span> <span class="token keyword">local</span> matchers <span class="token operator">=</span> <span class="token punctuation">{</span> <span class="token punctuation">[</span>MATCH_RULES<span class="token punctuation">.</span>HOST<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token keyword">function</span><span class="token punctuation">(</span>api_t<span class="token punctuation">,</span> ctx<span class="token punctuation">)</span> <span class="token keyword">local</span> host <span class="token operator">=</span> api_t<span class="token punctuation">.</span>hosts<span class="token punctuation">[</span>ctx<span class="token punctuation">.</span>hits<span class="token punctuation">.</span>host <span class="token keyword">or</span> ctx<span class="token punctuation">.</span>req_host<span class="token punctuation">]</span> <span class="token keyword">if</span> host <span class="token keyword">then</span> ctx<span class="token punctuation">.</span>matches<span class="token punctuation">.</span>host <span class="token operator">=</span> host <span class="token keyword">return</span> <span class="token keyword">true</span> <span class="token keyword">end</span> <span class="token keyword">end</span><span class="token punctuation">,</span> <span class="token comment">-- 省略uri和method的比较过程</span> <span class="token punctuation">}</span> match_api <span class="token operator">=</span> <span class="token keyword">function</span><span class="token punctuation">(</span>api_t<span class="token punctuation">,</span> ctx<span class="token punctuation">)</span> <span class="token comment">-- run cached matcher</span> <span class="token keyword">if</span> <span class="token function">type</span><span class="token punctuation">(</span>matchers<span class="token punctuation">[</span>api_t<span class="token punctuation">.</span>match_rules<span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token string">"function"</span> <span class="token keyword">then</span> <span class="token function">clear_tab</span><span class="token punctuation">(</span>ctx<span class="token punctuation">.</span>matches<span class="token punctuation">)</span> <span class="token keyword">return</span> matchers<span class="token punctuation">[</span>api_t<span class="token punctuation">.</span>match_rules<span class="token punctuation">]</span><span class="token punctuation">(</span>api_t<span class="token punctuation">,</span> ctx<span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token keyword">local</span> matchers_set <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span> <span class="token comment">-- 仍然转化为对host、method和uri的比较</span> <span class="token keyword">for</span> _<span class="token punctuation">,</span> bit_match_rule <span class="token keyword">in</span> <span class="token function">pairs</span><span class="token punctuation">(</span>MATCH_RULES<span class="token punctuation">)</span> <span class="token keyword">do</span> <span class="token keyword">if</span> <span class="token function">band</span><span class="token punctuation">(</span>api_t<span class="token punctuation">.</span>match_rules<span class="token punctuation">,</span> bit_match_rule<span class="token punctuation">)</span> <span class="token operator">~=</span> <span class="token number">0</span> <span class="token keyword">then</span> matchers_set<span class="token punctuation">[</span><span class="token operator">#</span>matchers_set <span class="token operator">+</span> <span class="token number">1</span><span class="token punctuation">]</span> <span class="token operator">=</span> matchers<span class="token punctuation">[</span>bit_match_rule<span class="token punctuation">]</span> <span class="token keyword">end</span> <span class="token keyword">end</span> matchers<span class="token punctuation">[</span>api_t<span class="token punctuation">.</span>match_rules<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token keyword">function</span><span class="token punctuation">(</span>api_t<span class="token punctuation">,</span> ctx<span class="token punctuation">)</span> <span class="token function">clear_tab</span><span class="token punctuation">(</span>ctx<span class="token punctuation">.</span>matches<span class="token punctuation">)</span> <span class="token keyword">for</span> i <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token operator">#</span>matchers_set <span class="token keyword">do</span> <span class="token keyword">if</span> <span class="token keyword">not</span> matchers_set<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">(</span>api_t<span class="token punctuation">,</span> ctx<span class="token punctuation">)</span> <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token keyword">return</span> <span class="token keyword">true</span> <span class="token keyword">end</span> <span class="token keyword">return</span> matchers<span class="token punctuation">[</span>api_t<span class="token punctuation">.</span>match_rules<span class="token punctuation">]</span><span class="token punctuation">(</span>api_t<span class="token punctuation">,</span> ctx<span class="token punctuation">)</span> <span class="token keyword">end</span></code></pre><p>从精简集合中如果找不到的话就全量查找此类别中的所有API</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">if</span> <span class="token keyword">not</span> matched_api <span class="token keyword">then</span> <span class="token keyword">for</span> i <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token operator">#</span>category_candidates <span class="token keyword">do</span> <span class="token keyword">if</span> <span class="token function">match_api</span><span class="token punctuation">(</span>category_candidates<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">,</span> ctx<span class="token punctuation">)</span> <span class="token keyword">then</span> matched_api <span class="token operator">=</span> category_candidates<span class="token punctuation">[</span>i<span class="token punctuation">]</span> <span class="token keyword">break</span> <span class="token keyword">end</span> <span class="token keyword">end</span><span class="token keyword">end</span></code></pre><p>综上所述，Kong的路由匹配主要采用请求的<code>host</code>、<code>url</code>和<code>method</code>三元组来对请求进行分组，从而找到应该匹配的API。第一次匹配可能比较慢，后续存入缓存后，匹配速度就很快了。这种匹配策略有一个缺点，就是无法匹配哪些不仅仅靠host、uri和method区分的API，比如OpenAPI。针对OpenAPI类型的匹配还需要进行定制。</p>]]></content>
<categories>
<category> 网关 </category>
</categories>
<tags>
<tag> Lua </tag>
<tag> 网关 </tag>
<tag> Kong </tag>
</tags>
</entry>
<entry>
<title>kong0.12.x源码分析(四)---插件机制</title>
<link href="/2019/03/15/kong0-12-x%E6%BA%90%E7%A0%81%E5%88%86%E6%9E%90-%E5%9B%9B/"/>
<url>/2019/03/15/kong0-12-x%E6%BA%90%E7%A0%81%E5%88%86%E6%9E%90-%E5%9B%9B/</url>
<content type="html"><![CDATA[<h3 id="Kong插件的目录结构">Kong插件的目录结构</h3><p>一个基本的Kong插件目录结构如下：</p><pre class="language-lua" data-language="lua"><code class="language-lua">simple<span class="token operator">-</span>plugin├── handler<span class="token punctuation">.</span>lua <span class="token comment">--通过实现```init_worker()```、```access()```等方法来实现自己的功能</span>└── schema<span class="token punctuation">.</span>lua <span class="token comment">--存储插件配置的数据结构</span></code></pre><pre class="language-handler.lua```和请求的生命周期相对应" data-language="handler.lua```和请求的生命周期相对应"><code class="language-handler.lua```和请求的生命周期相对应">```lualocal BasePlugin = require "kong.plugins.base_plugin"local CustomHandler = BasePlugin:extend()function CustomHandler:new() CustomHandler.super.new(self, "my-custom-plugin")endfunction CustomHandler:init_worker() CustomHandler.super.init_worker(self)endfunction CustomHandler:certificate(config) CustomHandler.super.certificate(self)endfunction CustomHandler:rewrite(config) CustomHandler.super.rewrite(self)endfunction CustomHandler:access(config) CustomHandler.super.access(self)endfunction CustomHandler:header_filter(config) CustomHandler.super.header_filter(self)endfunction CustomHandler:body_filter(config) CustomHandler.super.body_filter(self)endfunction CustomHandler:log(config) CustomHandler.super.log(self)endCustomHandler.PRIORITY = 10 --控制插件优先级return CustomHandler</code></pre><h3 id="插件的加载执行流程">插件的加载执行流程</h3><blockquote><p>整个插件的加载执行流程主要集中在<code>kong/init.lua</code>和<code>kong/core/plugins_iterator</code>这两个文件中</p></blockquote><h4 id="插件的加载">插件的加载</h4><p>整个加载过程在<code>init.lua</code>文件的<code>load_plugins(kong_conf, dao)</code>方法中。首先校验数据库正在使用的插件，配置文件中是否启用了；然后利用lua动态脚本的特性，将启用的插件元数据保存起来；最后根据优先级对插件进行排序，存储到全局的缓存中</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">for</span> plugin <span class="token keyword">in</span> <span class="token function">pairs</span><span class="token punctuation">(</span>kong_conf<span class="token punctuation">.</span>plugins<span class="token punctuation">)</span> <span class="token keyword">do</span> <span class="token keyword">local</span> ok<span class="token punctuation">,</span> handler <span class="token operator">=</span> utils<span class="token punctuation">.</span><span class="token function">load_module_if_exists</span><span class="token punctuation">(</span><span class="token string">"kong.plugins."</span> <span class="token operator">..</span> plugin <span class="token operator">..</span> <span class="token string">".handler"</span><span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> ok <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> plugin <span class="token operator">..</span> <span class="token string">" plugin is enabled but not installed;\n"</span> <span class="token operator">..</span> handler <span class="token keyword">end</span> <span class="token keyword">local</span> ok<span class="token punctuation">,</span> schema <span class="token operator">=</span> utils<span class="token punctuation">.</span><span class="token function">load_module_if_exists</span><span class="token punctuation">(</span><span class="token string">"kong.plugins."</span> <span class="token operator">..</span> plugin <span class="token operator">..</span> <span class="token string">".schema"</span><span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> ok <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token string">"no configuration schema found for plugin: "</span> <span class="token operator">..</span> plugin <span class="token keyword">end</span> sorted_plugins<span class="token punctuation">[</span><span class="token operator">#</span>sorted_plugins<span class="token operator">+</span><span class="token number">1</span><span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token punctuation">{</span> name <span class="token operator">=</span> plugin<span class="token punctuation">,</span> handler <span class="token operator">=</span> <span class="token function">handler</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> schema <span class="token operator">=</span> schema <span class="token punctuation">}</span><span class="token keyword">end</span></code></pre><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">function</span> _M<span class="token punctuation">.</span><span class="token function">load_module_if_exists</span><span class="token punctuation">(</span>module_name<span class="token punctuation">)</span> <span class="token keyword">local</span> status<span class="token punctuation">,</span> res <span class="token operator">=</span> <span class="token function">pcall</span><span class="token punctuation">(</span>require<span class="token punctuation">,</span> module_name<span class="token punctuation">)</span> <span class="token comment">---省略加载结果校验</span><span class="token keyword">end</span></code></pre><h4 id="插件的执行">插件的执行</h4><p>kong插件的执行比较简单，主要在nginx请求的各个阶段循环调用插件的相应方法即可</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">for</span> plugin<span class="token punctuation">,</span> plugin_conf <span class="token keyword">in</span> <span class="token function">plugins_iterator</span><span class="token punctuation">(</span>singletons<span class="token punctuation">.</span>loaded_plugins<span class="token punctuation">,</span> <span class="token keyword">true</span><span class="token punctuation">)</span> <span class="token keyword">do</span> plugin<span class="token punctuation">.</span>handler<span class="token punctuation">:</span><span class="token function">rewrite</span><span class="token punctuation">(</span>plugin_conf<span class="token punctuation">)</span><span class="token keyword">end</span></code></pre><p><img src="/images/2019031501.png" alt="" loading="lazy"><br>需要注意的点是，插件的实例在<code>plugins_iterator</code>中加载的。插件有全局插件，也有API插件和Consumer插件，而API和ConsumerId是需要在access阶段（access阶段才根据host、method、uri等信息获取到api）才能获取到，因此access阶段之前只能执行全局的插件。</p><h3 id="小结">小结</h3><p>总的来说，Kong的插件还是比较简单和优雅的，基于nginx提供的执行阶段稍作封装就实现了一套灵活的插件系统。</p>]]></content>
<categories>
<category> 网关 </category>
</categories>
<tags>
<tag> Lua </tag>
<tag> 网关 </tag>
<tag> Kong </tag>
</tags>
</entry>
<entry>
<title>精巧的web服务器Caddy</title>
<link href="/2019/01/22/%E7%B2%BE%E5%B7%A7%E7%9A%84web%E6%9C%8D%E5%8A%A1%E5%99%A8Caddy/"/>
<url>/2019/01/22/%E7%B2%BE%E5%B7%A7%E7%9A%84web%E6%9C%8D%E5%8A%A1%E5%99%A8Caddy/</url>
<content type="html"><![CDATA[<p>Caddy是一个开源的，使用Golang编写，支持HTTP/2的web服务器。第一个版本发布于2015年，至今在github上已经有超过2万+的stars。和apache、nginx一样，Caddy也提供基本的静态文件托管、反向代理和负载均衡等基本功能。同时主打易用性，配置比较简单，还具有很多看起来比较现代的特性，比如支持自动https(使用Let’s Encrypt证书，并自动续期)、markdown文件托管、prometheus监控以及同步git代码生成个人博客(hexo、hugo)等等。</p><p>基于Golang语言的特性，Caddy还有以下特性：</p><ul><li>依赖简单，只需要一个二进制文件就可以跑起来</li><li>跨平台，除了Windows, Mac, Linux, BSD, Solaris，还支持Android平台</li><li>天然多核支持</li></ul><p>另外，之所以说Caddy比较精巧，是因为Caddy具有一个强大的插件系统，几乎Caddy提供的所有功能，全部是基于插件机制实现的。</p><p>下面主要介绍一下Caddy的安装、基本用法和插件开发。</p><h3 id="安装">安装</h3><blockquote><p>Caddy采用的是Apache-2.0开源协议，如果我们直接从官网下载带有插件的二进制运行文件，商用的话是需要付费的。不过如果我们自己编译的话，是免费的。</p></blockquote><p>Caddy的编译步骤非常简单，只需要执行如下四条命令:</p><pre class="language-bash" data-language="bash"><code class="language-bash">go get github.com/mholt/caddy/caddygo get github.com/caddyserver/builds<span class="token builtin class-name">cd</span> <span class="token variable">$GOPATH</span>/src/github.com/mholt/caddy/caddygo run build.go</code></pre><p>在当前目录下会生成一个caddy二进制文件，我们可以把caddy放到环境变量里。执行<code>caddy</code>，服务就启动了，打开浏览器，输入<code>http://localhost:2015/</code>，会看到<code>404 Not Found</code>，说明服务启动正常。</p><h3 id="使用">使用</h3><blockquote><p>Caddy的配置文件和指令和nginx很相似，不过配置起来会更简单一些。</p></blockquote><p>一个简单的单站点caddyfile文件如下:</p><pre class="language-nginx" data-language="nginx"><code class="language-nginx">localhost:8080gziplog ./access.log</code></pre><p>或者</p><pre class="language-nginx" data-language="nginx"><code class="language-nginx"><span class="token directive"><span class="token keyword">localhost:8080</span></span> <span class="token punctuation">{</span> gzip log ./access.log<span class="token punctuation">}</span></code></pre><p>caddy的指令也非常丰富</p><h4 id="指令">指令</h4><ul><li>proxy</li></ul><blockquote><p>proxy指令用于代理</p></blockquote><pre class="language-nginx" data-language="nginx"><code class="language-nginx"><span class="token directive"><span class="token keyword">proxy</span> from to...</span> <span class="token punctuation">{</span>policy name [value] <span class="token comment">#负载均衡策略random, least_conn, round_robin, first, ip_hash, uri_hash, or header</span>fail_timeout duration max_fails integer <span class="token comment">#和上面的fail_timeout配合使用，失败多少次认为后端服务不可用，负载均衡时就不再向这台机器发请求</span>max_conns integertry_duration durationtry_interval durationhealth_check path <span class="token comment">#健康检查</span>health_check_port porthealth_check_interval interval_durationhealth_check_timeout timeout_durationfallback_delay delay_durationheader_upstream name valueheader_downstream name valuekeepalive numbertimeout durationwithout prefix <span class="token comment"># 去除前缀</span>except ignored_paths...upstream toinsecure_skip_verifypreset <span class="token comment">#websocket或者transparent</span><span class="token punctuation">}</span></code></pre><ul><li>rewrite</li></ul><blockquote><p>和nginx的rewrite含义一样，重写url</p></blockquote><pre class="language-nginx" data-language="nginx"><code class="language-nginx"><span class="token directive"><span class="token keyword">rewrite</span> [basepath]</span> <span class="token punctuation">{</span>regexp patternext extensions...if a cond bif_op [and|or]to destinations...<span class="token punctuation">}</span></code></pre><ul><li>errors</li></ul><blockquote><p>记录异常，支持自定义返回页面</p></blockquote><pre class="language-nginx" data-language="nginx"><code class="language-nginx"><span class="token directive"><span class="token keyword">errors</span> [logfile]</span> <span class="token punctuation">{</span>code filerotate_size mbrotate_age daysrotate_keep countrotate_compress<span class="token punctuation">}</span></code></pre><ul><li>log</li></ul><blockquote><p>日志记录，支持文件分割</p></blockquote><pre class="language-nginx" data-language="nginx"><code class="language-nginx"><span class="token directive"><span class="token keyword">log</span> path file [format]</span> <span class="token punctuation">{</span>rotate_size mbrotate_age daysrotate_keep countrotate_compressipmask ipv4_mask [ipv6_mask]except paths...<span class="token punctuation">}</span></code></pre><h4 id="插件">插件</h4><p>Caddy的插件分为<code>Server Types </code>,<code>Directives</code>,<code>HTTP Middleware</code>,<code>Caddyfile Loader</code>,<code>DNS Provider</code>,<code>Cluster Support </code>,<code>Listener Middleware</code>以及<code>Event Hook</code>共八种类型。</p><ul><li>Server Types (服务器类型)</li></ul><p>这种类型的插件目前Caddy只有<code>HTTP</code>这一种。</p><ul><li>Directives (指令类型)</li></ul><p>指令类型的插件用于自定义指令。具体步骤如下：</p><ol><li>新建一个go文件，在<code>init</code>方法中填写插件名和setup方法</li></ol><pre class="language-go" data-language="go"><code class="language-go"><span class="token keyword">import</span> <span class="token string">"github.com/mholt/caddy"</span><span class="token keyword">func</span> <span class="token function">init</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>caddy<span class="token punctuation">.</span><span class="token function">RegisterPlugin</span><span class="token punctuation">(</span><span class="token string">"gizmo"</span><span class="token punctuation">,</span> caddy<span class="token punctuation">.</span>Plugin<span class="token punctuation">{</span>ServerType<span class="token punctuation">:</span> <span class="token string">"http"</span><span class="token punctuation">,</span>Action<span class="token punctuation">:</span> setup<span class="token punctuation">,</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">}</span></code></pre><pre class="language-go" data-language="go"><code class="language-go"><span class="token keyword">func</span> <span class="token function">setup</span><span class="token punctuation">(</span>c <span class="token operator">*</span>caddy<span class="token punctuation">.</span>Controller<span class="token punctuation">)</span> <span class="token builtin">error</span> <span class="token punctuation">{</span><span class="token keyword">return</span> <span class="token boolean">nil</span><span class="token punctuation">}</span></code></pre><pre class="language-go" data-language="go"><code class="language-go"><span class="token keyword">for</span> c<span class="token punctuation">.</span><span class="token function">Next</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">if</span> <span class="token operator">!</span>c<span class="token punctuation">.</span><span class="token function">NextArg</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> c<span class="token punctuation">.</span><span class="token function">ArgErr</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">}</span> value <span class="token operator">:=</span> c<span class="token punctuation">.</span><span class="token function">Val</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token comment">//提取配置参数 </span><span class="token punctuation">}</span></code></pre><ol start="2"><li>在<code>run.go</code>的顶部导入即可:</li></ol><pre class="language-go" data-language="go"><code class="language-go"><span class="token keyword">import</span> <span class="token boolean">_</span> <span class="token string">"your/plugin/package/path/here"</span></code></pre><ul><li>HTTP Middleware （HTTP中间件)</li></ul><blockquote><p>HTTP Middleware类型的插件是在指令类型的插件上面实现了<code>httpserver.Handler</code></p></blockquote><pre class="language-go" data-language="go"><code class="language-go"><span class="token keyword">type</span> MyHandler <span class="token keyword">struct</span> <span class="token punctuation">{</span>Next httpserver<span class="token punctuation">.</span>Handler<span class="token punctuation">}</span><span class="token keyword">func</span> <span class="token punctuation">(</span>h MyHandler<span class="token punctuation">)</span> <span class="token function">ServeHTTP</span><span class="token punctuation">(</span>w http<span class="token punctuation">.</span>ResponseWriter<span class="token punctuation">,</span> r <span class="token operator">*</span>http<span class="token punctuation">.</span>Request<span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token builtin">int</span><span class="token punctuation">,</span> <span class="token builtin">error</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token keyword">return</span> h<span class="token punctuation">.</span>Next<span class="token punctuation">.</span><span class="token function">ServeHTTP</span><span class="token punctuation">(</span>w<span class="token punctuation">,</span> r<span class="token punctuation">)</span><span class="token punctuation">}</span></code></pre><pre class="language-go" data-language="go"><code class="language-go"><span class="token keyword">func</span> <span class="token function">setup</span><span class="token punctuation">(</span>c <span class="token operator">*</span>caddy<span class="token punctuation">.</span>Controller<span class="token punctuation">)</span> <span class="token builtin">error</span> <span class="token punctuation">{</span><span class="token keyword">for</span> c<span class="token punctuation">.</span><span class="token function">Next</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token comment">// get configuration</span><span class="token punctuation">}</span>handler <span class="token operator">:=</span> MyHandler<span class="token punctuation">{</span><span class="token punctuation">}</span>httpserver<span class="token punctuation">.</span><span class="token function">GetConfig</span><span class="token punctuation">(</span>c<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">AddMiddleware</span><span class="token punctuation">(</span><span class="token keyword">func</span><span class="token punctuation">(</span>next httpserver<span class="token punctuation">.</span>Handler<span class="token punctuation">)</span> httpserver<span class="token punctuation">.</span>Handler <span class="token punctuation">{</span>handler<span class="token punctuation">.</span>Next <span class="token operator">=</span> next<span class="token keyword">return</span> handler<span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">}</span></code></pre><ul><li>Caddyfile Loader (Caddy配置文件加载器)</li></ul><blockquote><p>caddyfile loader类型的插件主要用于自定义配置文件加载器，比如从数据库中加载配置，通过http接口进行hot-reload等</p></blockquote><pre class="language-go" data-language="go"><code class="language-go"><span class="token keyword">import</span> <span class="token string">"github.com/mholt/caddy"</span><span class="token keyword">func</span> <span class="token function">init</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>caddy<span class="token punctuation">.</span><span class="token function">RegisterCaddyfileLoader</span><span class="token punctuation">(</span><span class="token string">"myloader"</span><span class="token punctuation">,</span> caddy<span class="token punctuation">.</span><span class="token function">LoaderFunc</span><span class="token punctuation">(</span>myLoader<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token keyword">func</span> <span class="token function">myLoader</span><span class="token punctuation">(</span>serverType <span class="token builtin">string</span><span class="token punctuation">)</span> <span class="token punctuation">(</span>caddy<span class="token punctuation">.</span>Input<span class="token punctuation">,</span> <span class="token builtin">error</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token keyword">return</span> <span class="token boolean">nil</span><span class="token punctuation">,</span> <span class="token boolean">nil</span><span class="token punctuation">}</span></code></pre><h3 id="小结">小结</h3><p>从上面我们可以看到，作为一个web服务器，caddy完全可以满足日常使用的需求；同时Caddy可以自行扩展的插件类型非常丰富：从实现一个Server插件、集群插件到实现一个http中间件都是非常方便的。</p>]]></content>
<categories>
<category> 网关 </category>
</categories>
</entry>
<entry>
<title>开源API网关Tyk简单介绍</title>
<link href="/2019/01/15/%E5%BC%80%E6%BA%90API%E7%BD%91%E5%85%B3Tyk%E7%AE%80%E5%8D%95%E4%BB%8B%E7%BB%8D/"/>
<url>/2019/01/15/%E5%BC%80%E6%BA%90API%E7%BD%91%E5%85%B3Tyk%E7%AE%80%E5%8D%95%E4%BB%8B%E7%BB%8D/</url>
<content type="html"><![CDATA[<p>tyk是一款采用golang语言实现的API网关，具有API Gateway、Tyk Dashboard、 Tyk Pumpd和Tyk Identity Broker等几大组件。不过只有API Gateway的源代码是开放的。下面主要介绍一下API Gateway的功能与用法。</p><p><img src="/images/2019011501.png" alt="" loading="lazy"></p><h4 id="安装">安装</h4><p>我们以Ubuntu18.04为例讲一下安装步骤:</p><ol><li>安装redis-server</li></ol><pre class="language-none"><code class="language-none">sudo apt-get install -y redis-server</code></pre><ol start="2"><li>使用一键安装脚本进行安装</li></ol><pre class="language-none"><code class="language-none">curl -s https://packagecloud.io/install/repositories/tyk/tyk-gateway/script.deb.sh | sudo bash</code></pre><ol start="3"><li>配置服务</li></ol><pre class="language-none"><code class="language-none">sudo /opt/tyk-gateway/install/setup.sh --listenport=8080 --redishost=<hostname> --redisport=6379 --domain=""</code></pre><ol start="4"><li>启动服务</li></ol><pre class="language-none"><code class="language-none">sudo service tyk-gateway start</code></pre><h4 id="简单使用">简单使用</h4><p>在tyk中创建API分为两种，一种是在安装目录下的<code>apps</code>目录下创建一个<code>json</code>文件，另一种是使用tyk提供的<code>restful</code>接口<code>/tyk/apis</code>。我们采用第一种方式，在<code>apps</code>目录下创建一个<code>api1.json</code>文件:</p><pre class="language-none"><code class="language-none">{ "name": "Tyk Test API", "api_id": "1", "org_id": "default", "use_keyless":true, "definition": { "location": "header", "key": "x-api-version" }, "version_data": { "not_versioned": true, "versions": { "Default": { "name": "Default", "use_extended_paths": true } } }, "proxy": { "listen_path": "/tyk-api-test/", "target_url": "http://httpbin.org", "strip_listen_path": true }}</code></pre><p>然后调用<code>curl -H "x-tyk-authorization: {your-secret}" -s https://{your-tyk-host}:{port}/tyk/reload/group</code>进行热加载配置文件(<code>x-tyk-authorization</code>必选带上，secret在配置文件中)。这样我们访问<code>/tyk-api-test</code>就会代理到<code>http://httpbin.org</code></p><p>tyk的api支持的配置参数非常多，比如认证、版本号、频率控制和配额控制等，具体可参考<a href="https://tyk.io/docs/tyk-rest-api/api-definition-objects/%60%60%60">Tyk Api定义</a></p><h4 id="插件机制">插件机制</h4><p>tyk的插件功能比较强大，一方面提供了IP黑白名单、参数提取和认证等诸多插件，另一方面也支持js、python、lua语言来自定义插件。插件的执行顺序如下图:</p><p><img src="/images/2019011502.png" alt="" loading="lazy"></p><p>我们用lua开发一个简单的插件测试一下:<br>需要安装依赖:</p><pre class="language-none"><code class="language-none">apt-get install libluajit-5.1-2apt-get install luarocksluarocks install lua-cjson</code></pre><ol><li>创建一个<code>manifest.json</code>文件</li></ol><pre class="language-none"><code class="language-none">{ "file_list": [ "mymiddleware.lua" ], "custom_middleware": { "pre": [ { "name": "MyPreMiddleware" } ], "driver": "lua" }, "checksum": "", "signature": ""}</code></pre><ol start="2"><li>创建一个 <code>mymiddleware.lua</code>文件</li></ol><pre class="language-none"><code class="language-none">function MyPreMiddleware(request, session, spec) tyk.req.set_header("customheader", "customvalue") return request, sessionend</code></pre><p>然后我们需要使用<code>tyk-cli</code>(将<code>/opt/tyk-gateway/utils/</code>添加到<code>path</code>中)进行打包，切换到刚才的插件目录执行<code>tyk-cli bundle build -output bundle-test.zip -y</code>命令，将<code>bundle-test.zip</code>放到一个静态服务器中。<br>修改<code>tyk.conf</code>配置文件</p><pre class="language-none"><code class="language-none">"coprocess_options": { "enable_coprocess": true,},"enable_bundle_downloader": true,"bundle_base_url": "http://my-bundle-server.com/bundles/",</code></pre><p>执行</p><pre class="language-none"><code class="language-none">service tyk-gateway stopservice tyk-gateway-lua start</code></pre><p>最后修改api1.json添加上:</p><pre class="language-none"><code class="language-none">"custom_middleware_bundle": "bundle-test.zip"</code></pre><p>这样插件就生效了</p><h4 id="报表、监控和事件触发器">报表、监控和事件触发器</h4><p>tyk的管理平台提供有一系列详细指标来监控各个API的访问情况，不过相关代码没有开源出来，相关指标的存储访问接口也没有文档中显示，如果想深入了解的话需要学习源代码。<br><img src="/images/2019011503.png" alt="" loading="lazy"></p><p>tyk也内置了两个事件触发器handler,一个是<code>eh_log_handler</code>(这个主要是debug使用)，另外一个是<code>eh_web_hook_handler</code>。内置的事件包括<code>QuotaExceeded</code>,<code>RatelimitExceeded</code>,<code>OrgQuotaExceeded</code>,<code>OrgRateLimitExceeded</code>等，这些事件大都需要启动API认证机制后才会生效。事件触发后后会发送类似如下的数据</p><pre class="language-none"><code class="language-none">{ "event": "{{.Type}}", "message": "{{.Meta.Message}}", "path": "{{.Meta.Path}}", "origin": "{{.Meta.Origin}}", "key": "{{.Meta.Key}}"}</code></pre><p>内置的两个<code>event handler</code>相关来说比较鸡肋，不过也可以自己定义事件类型与<code>event handler</code></p><h4 id="集群部署">集群部署</h4><p>tyk也支持集群部署，需要一个组件MDCB(Tyk Multi Data Centre)与多个slave节点。集群功能是需要付费的，这里就不详细介绍了。</p><p><img src="/images/2019011504.png" alt="" loading="lazy"></p><h4 id="协议支持情况">协议支持情况</h4><p>支持http、gRPC和websocket</p><h4 id="小结">小结</h4><p>tyk总的来说算是一个中规中矩的API网关，丰富的插件、强大的认证机制和多种语言的插件支持给人眼前一亮的感觉，不过开源版本的集群功能、日志监控和灰度发布等���能相对较弱。</p>]]></content>
<categories>
<category> 网关 </category>
</categories>
<tags>
<tag> Golang </tag>
<tag> 网关 </tag>
</tags>
</entry>
<entry>
<title>kong中的trusted_ips、real_ip_header和real_ip_recursive</title>
<link href="/2018/10/28/kong%E4%B8%AD%E7%9A%84trusted-ips%E3%80%81real-ip-header%E5%92%8Creal-ip-recursive/"/>
<url>/2018/10/28/kong%E4%B8%AD%E7%9A%84trusted-ips%E3%80%81real-ip-header%E5%92%8Creal-ip-recursive/</url>
<content type="html"><![CDATA[<blockquote><p>最近在将一个业务流量接入网关时遇到一个问题，本来应该是https请求，经由网关到应用服务器得到重定向url时变成了http请求。问了相关同事，发现他们是根据<code>X-Forwarded-Proto</code>头来判断协议进行转换的。流量从入口nginx到网关再到应用服务器时，入口nginx的XFP头为https，网关把<code>X-Forwarded-Proto</code>协议头覆盖掉了。于是就看了一下kong的XFP头处理逻辑。</p></blockquote><p>kong的nginx_template中发往upstream的location中有这么一句话</p><pre class="language-none"><code class="language-none">proxy_set_header X-Forwarded-Proto $upstream_x_forwarded_proto;</code></pre><p>说明网关传往应用服务器时带了XFP头，不过带的是http,然后再看一下kong是怎么修改upstream_x_forwarded_proto变量的，在kong/core/hander.lua中，</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">local</span> realip_remote_addr <span class="token operator">=</span> var<span class="token punctuation">.</span>realip_remote_addr<span class="token keyword">local</span> trusted_ip <span class="token operator">=</span> singletons<span class="token punctuation">.</span>ip<span class="token punctuation">.</span><span class="token function">trusted</span><span class="token punctuation">(</span>realip_remote_addr<span class="token punctuation">)</span> <span class="token punctuation">......</span> <span class="token keyword">if</span> trusted_ip <span class="token keyword">then</span> var<span class="token punctuation">.</span>upstream_x_forwarded_proto <span class="token operator">=</span> var<span class="token punctuation">.</span>http_x_forwarded_proto <span class="token keyword">or</span> var<span class="token punctuation">.</span>scheme var<span class="token punctuation">.</span>upstream_x_forwarded_host <span class="token operator">=</span> var<span class="token punctuation">.</span>http_x_forwarded_host <span class="token keyword">or</span> var<span class="token punctuation">.</span>host var<span class="token punctuation">.</span>upstream_x_forwarded_port <span class="token operator">=</span> var<span class="token punctuation">.</span>http_x_forwarded_port <span class="token keyword">or</span> var<span class="token punctuation">.</span>server_port <span class="token keyword">else</span> var<span class="token punctuation">.</span>upstream_x_forwarded_proto <span class="token operator">=</span> var<span class="token punctuation">.</span>scheme var<span class="token punctuation">.</span>upstream_x_forwarded_host <span class="token operator">=</span> var<span class="token punctuation">.</span>host var<span class="token punctuation">.</span>upstream_x_forwarded_port <span class="token operator">=</span> var<span class="token punctuation">.</span>server_port <span class="token keyword">end</span></code></pre><p>kong先判断realip_remote_addr是否在可信ip范围内，是的话才会将XFP头设置为从上方带过来的XFP头，否则的话就用var.scheme，由于网关这里没有设置trust_ip，而入口nginx到网关采用的协议为http，从而导致网关将XFP设置为了http。找到了地方，那么我们只需要设置一下trust_ip就ok。trust_ip是在kong.conf文件中设置的。由于kong用的nginx原生的可信Ip模块，所以我们可以参照nginx原生的配置,<br><a href="http://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_recursive">http://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_recursive</a> 这个是nginx的real_ip模块文档。</p><pre class="language-none"><code class="language-none">#kong.conf文件trusted_ips = 192.168.0.1/16, 10.0.0.1/8, 127.0.0.1 # Defines trusted IP addresses blocks that are#real_ip_header = X-Real-IP # Defines the request header field whose value#real_ip_recursive = off # This value sets the ngx_http_realip_module</code></pre><ul><li><p>trusted_ips 决定kong是否会从XFF头或者 X-Real-IP头中获取客户端ip(也决定了是否使用传过来的XFP头)，为了安全，只会从可信IP中传过来的请求才会取这个头</p></li><li><p>real_ip_header 这个决定从哪里获取客户端ip</p></li><li><p>real_ip_recursive real_ip_recursive 是否递归地排除直至得到用户ip（默认为off）</p></li></ul><p>首先，real_ip_header 指定一个http首部名称，默认是X-Real-Ip，假设用默认值的话，nginx在接收到报文后，会查看http首部X-Real-Ip。</p><p>（1）如果有1个IP，它会去核对，发送方的ip是否在set_real_ip_from指定的信任ip列表中。如果是被信任的，它会去认为这个X-Real-Ip中的IP值是前代理告诉自己的，用户的真实IP值，于是，它会将该值赋值给自身的$remote_addr变量；如果不被信任，那么将不作处理，那么$remote_addr还是发送方的ip地址。</p><p>（2）如果X-Real-Ip有多个IP值，比如前一方代理是这么设置的：proxy_set_header X-Real-Ip $proxy_add_x_forwarded_for;</p><p>得到的是一串IP，那么此时real_ip_recursive 的值就至关重要了。nginx将会从ip列表的右到左，去比较set_real_ip_from 的信任列表中的ip。如果real_ip_recursive为off，那么，当最右边一个IP，发现是信任IP，即认为下一个IP（右边第二个）就是用户的真正IP；如果real_ip_recursive为on，那么将从右到左依次比较，知道找到一个不是信任IP为止。然后同样把IP值复制给$remote_addr。</p><p>在本次的问题中，只需要给trusted_ips设置一下信任Ip列表即可。</p>]]></content>
<categories>
<category> 网关 </category>
</categories>
<tags>
<tag> Kong </tag>
<tag> Nginx </tag>
</tags>
</entry>
<entry>
<title>Java中的TreeSet使用过程中的踩坑记录</title>
<link href="/2018/10/28/Java%E4%B8%AD%E7%9A%84TreeSet%E8%A7%A3%E6%9E%90/"/>
<url>/2018/10/28/Java%E4%B8%AD%E7%9A%84TreeSet%E8%A7%A3%E6%9E%90/</url>
<content type="html"><![CDATA[<blockquote><p>最近在使用TreeSet的过程中遇到一个问题，发现对元素覆写compareTo方法后部分元素被吞掉了。于是查了相关资料并看了一下源码，在此记录一下使用过程中的注意事项</p></blockquote><p>原始需求是对一系列规则根据优先级进行排序，从而构成规则链，规则的信息如下：</p><pre class="language-java" data-language="java"><code class="language-java"><span class="token comment">/** * @author 刘晓东 */</span><span class="token keyword">public</span> <span class="token keyword">class</span> <span class="token class-name">RuleInfo</span> <span class="token keyword">implements</span> <span class="token class-name">Comparable</span> <span class="token punctuation">{</span> <span class="token keyword">private</span> <span class="token keyword">long</span> id<span class="token punctuation">;</span> <span class="token keyword">private</span> <span class="token keyword">int</span> priority<span class="token punctuation">;</span><span class="token comment">//规则优先级</span> <span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span> <span class="token keyword">public</span> <span class="token keyword">boolean</span> <span class="token function">equals</span><span class="token punctuation">(</span><span class="token class-name">Object</span> obj<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>obj <span class="token keyword">instanceof</span> <span class="token class-name">RuleInfo</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token class-name">RuleInfo</span> ruleInfo <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token class-name">RuleInfo</span><span class="token punctuation">)</span> obj<span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token keyword">this</span><span class="token punctuation">.</span>id <span class="token operator">==</span> ruleInfo<span class="token punctuation">.</span>id<span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">return</span> <span class="token boolean">false</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token annotation punctuation">@Override</span> <span class="token keyword">public</span> <span class="token keyword">int</span> <span class="token function">compareTo</span><span class="token punctuation">(</span><span class="token class-name">Object</span> o<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>o <span class="token keyword">instanceof</span> <span class="token class-name">RuleInfo</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token class-name">RuleInfo</span> ruleInfo <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token class-name">RuleInfo</span><span class="token punctuation">)</span> o<span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token keyword">this</span><span class="token punctuation">.</span><span class="token function">getPriority</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">-</span> ruleInfo<span class="token punctuation">.</span><span class="token function">getPriority</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">return</span> <span class="token number">0</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre><p>在插入几条具有相同优先级的规则后，发现treeset中只会保留一条规则，其他相同优先级的规则都被吞掉了。然后看了一下TreeSet的源码，发现注释中有这么几句话。</p><pre class="language-none"><code class="language-none">* <p>Note that the ordering maintained by a set (whether or not an explicit* comparator is provided) must be <i>consistent with equals</i> if it is to* correctly implement the {@code Set} interface. (See {@code Comparable}* or {@code Comparator} for a precise definition of <i>consistent with* equals</i>.) This is so because the {@code Set} interface is defined in* terms of the {@code equals} operation, but a {@code TreeSet} instance* performs all element comparisons using its {@code compareTo} (or* {@code compare}) method, so two elements that are deemed equal by this method* are, from the standpoint of the set, equal. The behavior of a set* <i>is</i> well-defined even if its ordering is inconsistent with equals; it* just fails to obey the general contract of the {@code Set} interface.</code></pre><p>翻译过来也就是：Set中各个元素是以equal方法来判断两个元素是否一样，而在TreeSet中是以compareTo方法来进行排序并判断两个元素是否一样的，覆写equal方法在TreeSet中不起作用。因此上面的compareTo方法可以改写为:</p><pre class="language-java" data-language="java"><code class="language-java"><span class="token annotation punctuation">@Override</span> <span class="token keyword">public</span> <span class="token keyword">int</span> <span class="token function">compareTo</span><span class="token punctuation">(</span><span class="token class-name">Object</span> o<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>o <span class="token keyword">instanceof</span> <span class="token class-name">RuleInfo</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token class-name">RuleInfo</span> ruleInfo <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token class-name">RuleInfo</span><span class="token punctuation">)</span> o<span class="token punctuation">;</span> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token keyword">this</span><span class="token punctuation">.</span><span class="token function">getId</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">==</span> ruleInfo<span class="token punctuation">.</span><span class="token function">getId</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token number">0</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">int</span> ret <span class="token operator">=</span> <span class="token keyword">this</span><span class="token punctuation">.</span><span class="token function">getPriority</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">-</span> ruleInfo<span class="token punctuation">.</span><span class="token function">getPriority</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>ret <span class="token operator">==</span> <span class="token number">0</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token punctuation">(</span><span class="token keyword">int</span><span class="token punctuation">)</span> <span class="token punctuation">(</span><span class="token keyword">this</span><span class="token punctuation">.</span>id <span class="token operator">-</span> ruleInfo<span class="token punctuation">.</span>id<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> ret<span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token keyword">return</span> <span class="token number">0</span><span class="token punctuation">;</span> <span class="token punctuation">}</span></code></pre><p>接下来看一下TreeSet的add方法是如何实现的。<br>TreeSet是用TreeMap实现的，其中的add方法直接使用的TreeMap的put方法</p><pre class="language-java" data-language="java"><code class="language-java"><span class="token keyword">public</span> <span class="token keyword">boolean</span> <span class="token function">add</span><span class="token punctuation">(</span><span class="token class-name">E</span> e<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> m<span class="token punctuation">.</span><span class="token function">put</span><span class="token punctuation">(</span>e<span class="token punctuation">,</span> <span class="token constant">PRESENT</span><span class="token punctuation">)</span><span class="token operator">==</span><span class="token keyword">null</span><span class="token punctuation">;</span> <span class="token punctuation">}</span></code></pre><p>TreeMap的put方法为:</p><pre class="language-java" data-language="java"><code class="language-java"><span class="token keyword">public</span> <span class="token class-name">V</span> <span class="token function">put</span><span class="token punctuation">(</span><span class="token class-name">K</span> key<span class="token punctuation">,</span> <span class="token class-name">V</span> value<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token class-name">Entry</span><span class="token generics"><span class="token punctuation"><</span><span class="token class-name">K</span><span class="token punctuation">,</span><span class="token class-name">V</span><span class="token punctuation">></span></span> t <span class="token operator">=</span> root<span class="token punctuation">;</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>t <span class="token operator">==</span> <span class="token keyword">null</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token function">compare</span><span class="token punctuation">(</span>key<span class="token punctuation">,</span> key<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// type (and possibly null) check</span> root <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">Entry</span><span class="token generics"><span class="token punctuation"><</span><span class="token punctuation">></span></span><span class="token punctuation">(</span>key<span class="token punctuation">,</span> value<span class="token punctuation">,</span> <span class="token keyword">null</span><span class="token punctuation">)</span><span class="token punctuation">;</span> size <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">;</span> modCount<span class="token operator">++</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token keyword">null</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">int</span> cmp<span class="token punctuation">;</span> <span class="token class-name">Entry</span><span class="token generics"><span class="token punctuation"><</span><span class="token class-name">K</span><span class="token punctuation">,</span><span class="token class-name">V</span><span class="token punctuation">></span></span> parent<span class="token punctuation">;</span> <span class="token comment">// split comparator and comparable paths</span> <span class="token class-name">Comparator</span><span class="token generics"><span class="token punctuation"><</span><span class="token operator">?</span> <span class="token keyword">super</span> <span class="token class-name">K</span><span class="token punctuation">></span></span> cpr <span class="token operator">=</span> comparator<span class="token punctuation">;</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>cpr <span class="token operator">!=</span> <span class="token keyword">null</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">do</span> <span class="token punctuation">{</span> parent <span class="token operator">=</span> t<span class="token punctuation">;</span> cmp <span class="token operator">=</span> cpr<span class="token punctuation">.</span><span class="token function">compare</span><span class="token punctuation">(</span>key<span class="token punctuation">,</span> t<span class="token punctuation">.</span>key<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>cmp <span class="token operator"><</span> <span class="token number">0</span><span class="token punctuation">)</span> t <span class="token operator">=</span> t<span class="token punctuation">.</span>left<span class="token punctuation">;</span> <span class="token keyword">else</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>cmp <span class="token operator">></span> <span class="token number">0</span><span class="token punctuation">)</span> t <span class="token operator">=</span> t<span class="token punctuation">.</span>right<span class="token punctuation">;</span> <span class="token keyword">else</span> <span class="token keyword">return</span> t<span class="token punctuation">.</span><span class="token function">setValue</span><span class="token punctuation">(</span>value<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">while</span> <span class="token punctuation">(</span>t <span class="token operator">!=</span> <span class="token keyword">null</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>key <span class="token operator">==</span> <span class="token keyword">null</span><span class="token punctuation">)</span> <span class="token keyword">throw</span> <span class="token keyword">new</span> <span class="token class-name">NullPointerException</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token annotation punctuation">@SuppressWarnings</span><span class="token punctuation">(</span><span class="token string">"unchecked"</span><span class="token punctuation">)</span> <span class="token class-name">Comparable</span><span class="token generics"><span class="token punctuation"><</span><span class="token operator">?</span> <span class="token keyword">super</span> <span class="token class-name">K</span><span class="token punctuation">></span></span> k <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token class-name">Comparable</span><span class="token generics"><span class="token punctuation"><</span><span class="token operator">?</span> <span class="token keyword">super</span> <span class="token class-name">K</span><span class="token punctuation">></span></span><span class="token punctuation">)</span> key<span class="token punctuation">;</span> <span class="token keyword">do</span> <span class="token punctuation">{</span> parent <span class="token operator">=</span> t<span class="token punctuation">;</span> cmp <span class="token operator">=</span> k<span class="token punctuation">.</span><span class="token function">compareTo</span><span class="token punctuation">(</span>t<span class="token punctuation">.</span>key<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>cmp <span class="token operator"><</span> <span class="token number">0</span><span class="token punctuation">)</span> t <span class="token operator">=</span> t<span class="token punctuation">.</span>left<span class="token punctuation">;</span> <span class="token keyword">else</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>cmp <span class="token operator">></span> <span class="token number">0</span><span class="token punctuation">)</span> t <span class="token operator">=</span> t<span class="token punctuation">.</span>right<span class="token punctuation">;</span> <span class="token keyword">else</span> <span class="token keyword">return</span> t<span class="token punctuation">.</span><span class="token function">setValue</span><span class="token punctuation">(</span>value<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">while</span> <span class="token punctuation">(</span>t <span class="token operator">!=</span> <span class="token keyword">null</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token class-name">Entry</span><span class="token generics"><span class="token punctuation"><</span><span class="token class-name">K</span><span class="token punctuation">,</span><span class="token class-name">V</span><span class="token punctuation">></span></span> e <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">Entry</span><span class="token generics"><span class="token punctuation"><</span><span class="token punctuation">></span></span><span class="token punctuation">(</span>key<span class="token punctuation">,</span> value<span class="token punctuation">,</span> parent<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>cmp <span class="token operator"><</span> <span class="token number">0</span><span class="token punctuation">)</span> parent<span class="token punctuation">.</span>left <span class="token operator">=</span> e<span class="token punctuation">;</span> <span class="token keyword">else</span> parent<span class="token punctuation">.</span>right <span class="token operator">=</span> e<span class="token punctuation">;</span> <span class="token function">fixAfterInsertion</span><span class="token punctuation">(</span>e<span class="token punctuation">)</span><span class="token punctuation">;</span> size<span class="token operator">++</span><span class="token punctuation">;</span> modCount<span class="token operator">++</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token keyword">null</span><span class="token punctuation">;</span><span class="token punctuation">}</span></code></pre><p>可以看到如果元素自定义了比较器，那么当返回比较值为0时，直接将值覆盖了，从而相同优先级的元素只会保留一个。所以使用TreeSet自定义比较器时一定要注意这一点。</p>]]></content>
<categories>
<category> Java </category>
</categories>
<tags>
<tag> Java </tag>
</tags>
</entry>
<entry>
<title>kong0.12.x源码分析(三)----kong事件通知机制</title>
<link href="/2018/07/31/kong0-12-x%E6%BA%90%E7%A0%81%E5%88%86%E6%9E%90-%E4%B8%89/"/>
<url>/2018/07/31/kong0-12-x%E6%BA%90%E7%A0%81%E5%88%86%E6%9E%90-%E4%B8%89/</url>
<content type="html"><![CDATA[<p>最近一直在搞网关插件，没来的及写博客。不过也有好处，就是对代码的熟悉程度更高了一点。<br>今天我们来看一下kong的事件通知机制。<br>kong的事件通知基于<a href="https://github.com/Kong/lua-resty-worker-events">lua-resty-worker-events</a> 这个库，这个库也是Kong开发的。<br>Kong的事件通知主要涉及两个方面，一个方面是单个节点内的事件通知，比如api信息更新，插件更新等；另一个方面是集群内的信息同步</p><p>我们先看单个节点内的事件通知：<br>单个节点内的事件主要采用发布订阅的方式，大家首先都到事件中心注册感兴趣的事件，然后当这些事件发生时，事件中心再进行回调处理。那么lua-resty-worker-events这个库就扮演着事件中心的角色。</p><p>事件注册的相关代码主要在 kong/core/router.lua文件里：</p><pre class="language-lua" data-language="lua"><code class="language-lua">worker_events<span class="token punctuation">.</span><span class="token function">register</span><span class="token punctuation">(</span><span class="token keyword">function</span><span class="token punctuation">(</span>data<span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> data<span class="token punctuation">.</span>schema <span class="token keyword">then</span> <span class="token function">log</span><span class="token punctuation">(</span>ngx<span class="token punctuation">.</span>ERR<span class="token punctuation">,</span> <span class="token string">"[events] missing schema in crud subscriber"</span><span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token keyword">end</span> <span class="token keyword">if</span> <span class="token keyword">not</span> data<span class="token punctuation">.</span>entity <span class="token keyword">then</span> <span class="token function">log</span><span class="token punctuation">(</span>ngx<span class="token punctuation">.</span>ERR<span class="token punctuation">,</span> <span class="token string">"[events] missing entity in crud subscriber"</span><span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token keyword">end</span> <span class="token keyword">local</span> cache_key <span class="token operator">=</span> dao<span class="token punctuation">[</span>data<span class="token punctuation">.</span>schema<span class="token punctuation">.</span>table<span class="token punctuation">]</span><span class="token punctuation">:</span><span class="token function">entity_cache_key</span><span class="token punctuation">(</span>data<span class="token punctuation">.</span>entity<span class="token punctuation">)</span> <span class="token keyword">if</span> cache_key <span class="token keyword">then</span> cache<span class="token punctuation">:</span><span class="token function">invalidate</span><span class="token punctuation">(</span>cache_key<span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token keyword">if</span> data<span class="token punctuation">.</span>old_entity <span class="token keyword">then</span> cache_key <span class="token operator">=</span> dao<span class="token punctuation">[</span>data<span class="token punctuation">.</span>schema<span class="token punctuation">.</span>table<span class="token punctuation">]</span><span class="token punctuation">:</span><span class="token function">entity_cache_key</span><span class="token punctuation">(</span>data<span class="token punctuation">.</span>old_entity<span class="token punctuation">)</span> <span class="token keyword">if</span> cache_key <span class="token keyword">then</span> cache<span class="token punctuation">:</span><span class="token function">invalidate</span><span class="token punctuation">(</span>cache_key<span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token keyword">if</span> <span class="token keyword">not</span> data<span class="token punctuation">.</span>operation <span class="token keyword">then</span> <span class="token function">log</span><span class="token punctuation">(</span>ngx<span class="token punctuation">.</span>ERR<span class="token punctuation">,</span> <span class="token string">"[events] missing operation in crud subscriber"</span><span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token keyword">end</span> <span class="token keyword">local</span> entity_channel <span class="token operator">=</span> data<span class="token punctuation">.</span>schema<span class="token punctuation">.</span>table <span class="token keyword">local</span> entity_operation_channel <span class="token operator">=</span> <span class="token function">fmt</span><span class="token punctuation">(</span><span class="token string">"%s:%s"</span><span class="token punctuation">,</span> data<span class="token punctuation">.</span>schema<span class="token punctuation">.</span>table<span class="token punctuation">,</span> data<span class="token punctuation">.</span>operation<span class="token punctuation">)</span> <span class="token comment">-- crud:apis</span> <span class="token keyword">local</span> _<span class="token punctuation">,</span> err <span class="token operator">=</span> worker_events<span class="token punctuation">.</span><span class="token function">post_local</span><span class="token punctuation">(</span><span class="token string">"crud"</span><span class="token punctuation">,</span> entity_channel<span class="token punctuation">,</span> data<span class="token punctuation">)</span> <span class="token keyword">if</span> err <span class="token keyword">then</span> <span class="token function">log</span><span class="token punctuation">(</span>ngx<span class="token punctuation">.</span>ERR<span class="token punctuation">,</span> <span class="token string">"[events] could not broadcast crud event: "</span><span class="token punctuation">,</span> err<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token keyword">end</span> <span class="token comment">-- crud:apis:create</span> _<span class="token punctuation">,</span> err <span class="token operator">=</span> worker_events<span class="token punctuation">.</span><span class="token function">post_local</span><span class="token punctuation">(</span><span class="token string">"crud"</span><span class="token punctuation">,</span> entity_operation_channel<span class="token punctuation">,</span> data<span class="token punctuation">)</span> <span class="token keyword">if</span> err <span class="token keyword">then</span> <span class="token function">log</span><span class="token punctuation">(</span>ngx<span class="token punctuation">.</span>ERR<span class="token punctuation">,</span> <span class="token string">"[events] could not broadcast crud event: "</span><span class="token punctuation">,</span> err<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token keyword">end</span><span class="token keyword">end</span><span class="token punctuation">,</span> <span class="token string">"dao:crud"</span><span class="token punctuation">)</span></code></pre><p>从上面的代码我们可以看到，当数据库发生更新操作时，主要执行清除缓存的操作（缓存清除了，下次请求到来就会从数据库中重新拉取数据）<br>然后剩余的一些事件注册也在这个文件里，我们看一下主要也都是关于数据库变更的事件通知，做的主要操作也是清除缓存。另外还有一些事件会通知到集群的其它节点，比如：<code>cluster_events:broadcast("balancer:targets", key)</code>。这个集群事件我们随后再分析。<br>接下来我们看一下事件的生产者：<br>从上面的事件订阅那里我们可以看到事件生产者主要应该在dao层这里，我们打开 kong/dao/dao.lua文件，在 update、insert和delete方法可以很明显的看到类似的语句。说明我们的判断是正确的。</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">local</span> _<span class="token punctuation">,</span> err <span class="token operator">=</span> self<span class="token punctuation">.</span>events<span class="token punctuation">.</span><span class="token function">post_local</span><span class="token punctuation">(</span><span class="token string">"dao:crud"</span><span class="token punctuation">,</span> <span class="token string">"delete"</span><span class="token punctuation">,</span> <span class="token punctuation">{</span> schema <span class="token operator">=</span> self<span class="token punctuation">.</span>schema<span class="token punctuation">,</span> operation <span class="token operator">=</span> <span class="token string">"delete"</span><span class="token punctuation">,</span> entity <span class="token operator">=</span> row<span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">)</span></code></pre><p>总的来说单个node内的集群主要就是采用发布订阅的方式来进行事件的更新。<br>那么集群范围内又是如何更新节点的呢？<br>我们看一下kong/cluster_events.lua文件，可以很明显的看到有一个subscribe方法，也有一个broadcast方法，难道也是采用的发布订阅模式？其实的确是这样的:</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">function</span> _M<span class="token punctuation">:</span><span class="token function">broadcast</span><span class="token punctuation">(</span>channel<span class="token punctuation">,</span> data<span class="token punctuation">,</span> nbf<span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token function">type</span><span class="token punctuation">(</span>channel<span class="token punctuation">)</span> <span class="token operator">~=</span> <span class="token string">"string"</span> <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token string">"channel must be a string"</span> <span class="token keyword">end</span> <span class="token keyword">if</span> <span class="token function">type</span><span class="token punctuation">(</span>data<span class="token punctuation">)</span> <span class="token operator">~=</span> <span class="token string">"string"</span> <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token string">"data must be a string"</span> <span class="token keyword">end</span> <span class="token keyword">if</span> nbf <span class="token keyword">and</span> <span class="token function">type</span><span class="token punctuation">(</span>nbf<span class="token punctuation">)</span> <span class="token operator">~=</span> <span class="token string">"number"</span> <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token string">"nbf must be a number"</span> <span class="token keyword">end</span> <span class="token comment">-- insert event row</span> <span class="token comment">--log(DEBUG, "broadcasting on channel: '", channel, "' data: ", data,</span> <span class="token comment">-- " with nbf: ", nbf and nbf or "none")</span> <span class="token keyword">local</span> ok<span class="token punctuation">,</span> err <span class="token operator">=</span> self<span class="token punctuation">.</span>strategy<span class="token punctuation">:</span><span class="token function">insert</span><span class="token punctuation">(</span>self<span class="token punctuation">.</span>cluster_name<span class="token punctuation">,</span> self<span class="token punctuation">.</span>node_id<span class="token punctuation">,</span> channel<span class="token punctuation">,</span> <span class="token function">ngx_now</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> data<span class="token punctuation">,</span> nbf<span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> ok <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> err <span class="token keyword">end</span> <span class="token keyword">return</span> <span class="token keyword">true</span><span class="token keyword">end</span></code></pre><p>可以看到发布事件是主要就是往数据库中插入一条数据<br>然后我们看一下订阅相关的代码</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">function</span> _M<span class="token punctuation">:</span><span class="token function">subscribe</span><span class="token punctuation">(</span>channel<span class="token punctuation">,</span> cb<span class="token punctuation">,</span> start_polling<span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token function">type</span><span class="token punctuation">(</span>channel<span class="token punctuation">)</span> <span class="token operator">~=</span> <span class="token string">"string"</span> <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token function">error</span><span class="token punctuation">(</span><span class="token string">"channel must be a string"</span><span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token keyword">if</span> <span class="token function">type</span><span class="token punctuation">(</span>cb<span class="token punctuation">)</span> <span class="token operator">~=</span> <span class="token string">"function"</span> <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token function">error</span><span class="token punctuation">(</span><span class="token string">"callback must be a function"</span><span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token keyword">if</span> <span class="token keyword">not</span> self<span class="token punctuation">.</span>callbacks<span class="token punctuation">[</span>channel<span class="token punctuation">]</span> <span class="token keyword">then</span> self<span class="token punctuation">.</span>callbacks<span class="token punctuation">[</span>channel<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token punctuation">{</span> cb <span class="token punctuation">}</span> <span class="token function">insert</span><span class="token punctuation">(</span>self<span class="token punctuation">.</span>channels<span class="token punctuation">,</span> channel<span class="token punctuation">)</span> <span class="token keyword">else</span> <span class="token function">insert</span><span class="token punctuation">(</span>self<span class="token punctuation">.</span>callbacks<span class="token punctuation">[</span>channel<span class="token punctuation">]</span><span class="token punctuation">,</span> cb<span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token keyword">if</span> start_polling <span class="token operator">==</span> <span class="token keyword">nil</span> <span class="token keyword">then</span> start_polling <span class="token operator">=</span> <span class="token keyword">true</span> <span class="token keyword">end</span> <span class="token keyword">if</span> <span class="token keyword">not</span> self<span class="token punctuation">.</span>polling <span class="token keyword">and</span> start_polling <span class="token keyword">then</span> <span class="token comment">-- start recurring polling timer</span> <span class="token keyword">local</span> ok<span class="token punctuation">,</span> err <span class="token operator">=</span> <span class="token function">timer_at</span><span class="token punctuation">(</span>self<span class="token punctuation">.</span>poll_interval<span class="token punctuation">,</span> poll_handler<span class="token punctuation">,</span> self<span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> ok <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token string">"failed to start polling timer: "</span> <span class="token operator">..</span> err <span class="token keyword">end</span> self<span class="token punctuation">.</span>polling <span class="token operator">=</span> <span class="token keyword">true</span> <span class="token keyword">end</span> <span class="token keyword">return</span> <span class="token keyword">true</span><span class="token keyword">end</span><span class="token keyword">local</span> <span class="token keyword">function</span> <span class="token function">poll</span><span class="token punctuation">(</span>self<span class="token punctuation">)</span> <span class="token comment">-- get events since last poll</span> <span class="token keyword">local</span> min_at<span class="token punctuation">,</span> err <span class="token operator">=</span> self<span class="token punctuation">.</span>shm<span class="token punctuation">:</span><span class="token function">get</span><span class="token punctuation">(</span>CURRENT_AT_KEY<span class="token punctuation">)</span> <span class="token keyword">if</span> err <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token string">"failed to retrieve 'at' in shm: "</span> <span class="token operator">..</span> err <span class="token keyword">end</span> <span class="token keyword">if</span> <span class="token keyword">not</span> min_at <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token string">"no 'at' in shm"</span> <span class="token keyword">end</span> <span class="token comment">-- apply grace period</span> min_at <span class="token operator">=</span> min_at <span class="token operator">-</span> self<span class="token punctuation">.</span>poll_offset <span class="token operator">-</span> <span class="token number">0.001</span> <span class="token keyword">local</span> max_at <span class="token operator">=</span> <span class="token function">ngx_now</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token function">log</span><span class="token punctuation">(</span>DEBUG<span class="token punctuation">,</span> <span class="token string">"polling events from: "</span><span class="token punctuation">,</span> min_at<span class="token punctuation">,</span> <span class="token string">" to: "</span><span class="token punctuation">,</span> max_at<span class="token punctuation">)</span> <span class="token keyword">for</span> rows<span class="token punctuation">,</span> err<span class="token punctuation">,</span> page <span class="token keyword">in</span> self<span class="token punctuation">.</span>strategy<span class="token punctuation">:</span><span class="token function">select_interval</span><span class="token punctuation">(</span>self<span class="token punctuation">.</span>channels<span class="token punctuation">,</span> self<span class="token punctuation">.</span>cluster_name<span class="token punctuation">,</span> min_at<span class="token punctuation">,</span> max_at<span class="token punctuation">)</span> <span class="token keyword">do</span> <span class="token keyword">if</span> err <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token string">"failed to retrieve events from DB: "</span> <span class="token operator">..</span> err <span class="token keyword">end</span> <span class="token keyword">if</span> page <span class="token operator">==</span> <span class="token number">1</span> <span class="token keyword">then</span> <span class="token keyword">local</span> ok<span class="token punctuation">,</span> err <span class="token operator">=</span> self<span class="token punctuation">.</span>shm<span class="token punctuation">:</span><span class="token function">safe_set</span><span class="token punctuation">(</span>CURRENT_AT_KEY<span class="token punctuation">,</span> max_at<span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> ok <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token string">"failed to set 'at' in shm: "</span> <span class="token operator">..</span> err <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token keyword">for</span> i <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token operator">#</span>rows <span class="token keyword">do</span> <span class="token keyword">local</span> row <span class="token operator">=</span> rows<span class="token punctuation">[</span>i<span class="token punctuation">]</span> <span class="token keyword">if</span> row<span class="token punctuation">.</span>node_id <span class="token operator">~=</span> self<span class="token punctuation">.</span>node_id <span class="token keyword">then</span> <span class="token keyword">local</span> ran<span class="token punctuation">,</span> err <span class="token operator">=</span> self<span class="token punctuation">.</span>events_shm<span class="token punctuation">:</span><span class="token function">get</span><span class="token punctuation">(</span>row<span class="token punctuation">.</span>id<span class="token punctuation">)</span> <span class="token keyword">if</span> err <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token string">"failed to probe if event ran: "</span> <span class="token operator">..</span> err <span class="token keyword">end</span> <span class="token keyword">if</span> <span class="token keyword">not</span> ran <span class="token keyword">then</span> <span class="token function">log</span><span class="token punctuation">(</span>DEBUG<span class="token punctuation">,</span> <span class="token string">"new event (channel: '"</span><span class="token punctuation">,</span> row<span class="token punctuation">.</span>channel<span class="token punctuation">,</span> <span class="token string">"') data: '"</span><span class="token punctuation">,</span> row<span class="token punctuation">.</span>data<span class="token punctuation">,</span> <span class="token string">"' nbf: '"</span><span class="token punctuation">,</span> row<span class="token punctuation">.</span>nbf <span class="token keyword">or</span> <span class="token string">"none"</span><span class="token punctuation">,</span> <span class="token string">"'"</span><span class="token punctuation">)</span> <span class="token keyword">local</span> exptime <span class="token operator">=</span> self<span class="token punctuation">.</span>poll_interval <span class="token operator">+</span> self<span class="token punctuation">.</span>poll_offset <span class="token comment">-- mark as ran before running in case of long-running callbacks</span> <span class="token keyword">local</span> ok<span class="token punctuation">,</span> err <span class="token operator">=</span> self<span class="token punctuation">.</span>events_shm<span class="token punctuation">:</span><span class="token function">set</span><span class="token punctuation">(</span>row<span class="token punctuation">.</span>id<span class="token punctuation">,</span> <span class="token keyword">true</span><span class="token punctuation">,</span> exptime<span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> ok <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token string">"failed to mark event as ran: "</span> <span class="token operator">..</span> err <span class="token keyword">end</span> <span class="token keyword">local</span> cbs <span class="token operator">=</span> self<span class="token punctuation">.</span>callbacks<span class="token punctuation">[</span>row<span class="token punctuation">.</span>channel<span class="token punctuation">]</span> <span class="token keyword">if</span> cbs <span class="token keyword">then</span> <span class="token keyword">for</span> j <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token operator">#</span>cbs <span class="token keyword">do</span> <span class="token keyword">if</span> <span class="token keyword">not</span> row<span class="token punctuation">.</span>nbf <span class="token keyword">then</span> <span class="token comment">-- unique callback run without delay</span> <span class="token keyword">local</span> ok<span class="token punctuation">,</span> err <span class="token operator">=</span> <span class="token function">pcall</span><span class="token punctuation">(</span>cbs<span class="token punctuation">[</span>j<span class="token punctuation">]</span><span class="token punctuation">,</span> row<span class="token punctuation">.</span>data<span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> ok <span class="token keyword">and</span> <span class="token keyword">not</span> ngx_debug <span class="token keyword">then</span> <span class="token function">log</span><span class="token punctuation">(</span>ERR<span class="token punctuation">,</span> <span class="token string">"callback threw an error: "</span><span class="token punctuation">,</span> err<span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token keyword">else</span> <span class="token comment">-- unique callback run after some delay</span> <span class="token keyword">local</span> delay <span class="token operator">=</span> <span class="token function">max</span><span class="token punctuation">(</span>row<span class="token punctuation">.</span>nbf <span class="token operator">-</span> <span class="token function">ngx_now</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">)</span> <span class="token function">log</span><span class="token punctuation">(</span>DEBUG<span class="token punctuation">,</span> <span class="token string">"delaying nbf event by "</span><span class="token punctuation">,</span> delay<span class="token punctuation">,</span> <span class="token string">"s"</span><span class="token punctuation">)</span> <span class="token keyword">local</span> ok<span class="token punctuation">,</span> err <span class="token operator">=</span> <span class="token function">timer_at</span><span class="token punctuation">(</span>delay<span class="token punctuation">,</span> nbf_cb_handler<span class="token punctuation">,</span> cbs<span class="token punctuation">[</span>j<span class="token punctuation">]</span><span class="token punctuation">,</span> row<span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> ok <span class="token keyword">then</span> <span class="token function">log</span><span class="token punctuation">(</span>ERR<span class="token punctuation">,</span> <span class="token string">"failed to schedule nbf event timer: "</span><span class="token punctuation">,</span> err<span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token keyword">return</span> <span class="token keyword">true</span><span class="token keyword">end</span></code></pre><p>可以看到订阅这里主要就是维持了一个定时器，不断根据间隔去数据库中poll数据，发现有数据更新了，执行一下数据更新回调（kong源代码中没有cluster_name字段，这个是我为了公司业务新添加的字段不影响阅读）</p><p>kong的事件更新机制主要还是发布订阅模型。</p>]]></content>
<categories>
<category> 网关 </category>
</categories>
<tags>
<tag> Lua </tag>
<tag> 网关 </tag>
<tag> Kong </tag>
</tags>
</entry>
<entry>
<title>kong0.12.x源码分析(二)----kong启动流程分析</title>
<link href="/2018/05/01/kong0-12-x%E6%BA%90%E7%A0%81%E5%88%86%E6%9E%90-%E4%BA%8C/"/>
<url>/2018/05/01/kong0-12-x%E6%BA%90%E7%A0%81%E5%88%86%E6%9E%90-%E4%BA%8C/</url>
<content type="html"><![CDATA[<p>今天我们看一下kong的启动流程，kong启动时调用kong start命令，kong的启动脚本内容如下：</p><pre class="language-bash" data-language="bash"><code class="language-bash"><span class="token shebang important">#!/usr/bin/env resty</span>require <span class="token string">"luarocks.loader"</span>package.path <span class="token operator">=</span> <span class="token string">"./?.lua;./?/init.lua;"</span> <span class="token punctuation">..</span> package.pathrequire<span class="token punctuation">(</span><span class="token string">"kong.cmd.init"</span><span class="token punctuation">)</span><span class="token punctuation">(</span>arg<span class="token punctuation">)</span></code></pre><p>在这个脚本里，通过require命令，调用kong/cmd/init.lua脚本，接下来我们看一下init脚本里放了些什么</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">local</span> pl_app <span class="token operator">=</span> require <span class="token string">"pl.lapp"</span><span class="token keyword">local</span> log <span class="token operator">=</span> require <span class="token string">"kong.cmd.utils.log"</span><span class="token keyword">local</span> options <span class="token operator">=</span> <span class="token string">[[ --v verbose --vv debug]]</span><span class="token keyword">local</span> cmds_arr <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token keyword">local</span> cmds <span class="token operator">=</span> <span class="token punctuation">{</span> start <span class="token operator">=</span> <span class="token keyword">true</span><span class="token punctuation">,</span> stop <span class="token operator">=</span> <span class="token keyword">true</span><span class="token punctuation">,</span> quit <span class="token operator">=</span> <span class="token keyword">true</span><span class="token punctuation">,</span> restart <span class="token operator">=</span> <span class="token keyword">true</span><span class="token punctuation">,</span> reload <span class="token operator">=</span> <span class="token keyword">true</span><span class="token punctuation">,</span> health <span class="token operator">=</span> <span class="token keyword">true</span><span class="token punctuation">,</span> check <span class="token operator">=</span> <span class="token keyword">true</span><span class="token punctuation">,</span> prepare <span class="token operator">=</span> <span class="token keyword">true</span><span class="token punctuation">,</span> migrations <span class="token operator">=</span> <span class="token keyword">true</span><span class="token punctuation">,</span> version <span class="token operator">=</span> <span class="token keyword">true</span><span class="token punctuation">,</span> roar <span class="token operator">=</span> <span class="token keyword">true</span><span class="token punctuation">}</span><span class="token keyword">for</span> k <span class="token keyword">in</span> <span class="token function">pairs</span><span class="token punctuation">(</span>cmds<span class="token punctuation">)</span> <span class="token keyword">do</span> cmds_arr<span class="token punctuation">[</span><span class="token operator">#</span>cmds_arr<span class="token operator">+</span><span class="token number">1</span><span class="token punctuation">]</span> <span class="token operator">=</span> k<span class="token keyword">end</span>table<span class="token punctuation">.</span><span class="token function">sort</span><span class="token punctuation">(</span>cmds_arr<span class="token punctuation">)</span><span class="token keyword">local</span> help <span class="token operator">=</span> string<span class="token punctuation">.</span><span class="token function">format</span><span class="token punctuation">(</span><span class="token string">[[Usage: kong COMMAND [OPTIONS]The available commands are: %sOptions:%s]]</span><span class="token punctuation">,</span> table<span class="token punctuation">.</span><span class="token function">concat</span><span class="token punctuation">(</span>cmds_arr<span class="token punctuation">,</span> <span class="token string">"\n "</span><span class="token punctuation">)</span><span class="token punctuation">,</span> options<span class="token punctuation">)</span><span class="token keyword">return</span> <span class="token keyword">function</span><span class="token punctuation">(</span>args<span class="token punctuation">)</span> <span class="token keyword">local</span> cmd_name <span class="token operator">=</span> table<span class="token punctuation">.</span><span class="token function">remove</span><span class="token punctuation">(</span>args<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> cmd_name <span class="token keyword">then</span> <span class="token function">pl_app</span><span class="token punctuation">(</span>help<span class="token punctuation">)</span> pl_app<span class="token punctuation">.</span><span class="token function">quit</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">elseif</span> <span class="token keyword">not</span> cmds<span class="token punctuation">[</span>cmd_name<span class="token punctuation">]</span> <span class="token keyword">then</span> <span class="token function">pl_app</span><span class="token punctuation">(</span>help<span class="token punctuation">)</span> pl_app<span class="token punctuation">.</span><span class="token function">quit</span><span class="token punctuation">(</span><span class="token string">"No such command: "</span> <span class="token operator">..</span> cmd_name<span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token keyword">local</span> cmd <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">"kong.cmd."</span> <span class="token operator">..</span> cmd_name<span class="token punctuation">)</span> <span class="token keyword">local</span> cmd_lapp <span class="token operator">=</span> cmd<span class="token punctuation">.</span>lapp <span class="token keyword">local</span> cmd_exec <span class="token operator">=</span> cmd<span class="token punctuation">.</span>execute <span class="token keyword">if</span> cmd_lapp <span class="token keyword">then</span> cmd_lapp <span class="token operator">=</span> cmd_lapp <span class="token operator">..</span> options <span class="token comment">-- append universal options</span> args <span class="token operator">=</span> <span class="token function">pl_app</span><span class="token punctuation">(</span>cmd_lapp<span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token comment">-- check sub-commands</span> <span class="token keyword">if</span> cmd<span class="token punctuation">.</span>sub_commands <span class="token keyword">then</span> <span class="token keyword">local</span> sub_cmd <span class="token operator">=</span> table<span class="token punctuation">.</span><span class="token function">remove</span><span class="token punctuation">(</span>args<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> sub_cmd <span class="token keyword">then</span> pl_app<span class="token punctuation">.</span><span class="token function">quit</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">elseif</span> <span class="token keyword">not</span> cmd<span class="token punctuation">.</span>sub_commands<span class="token punctuation">[</span>sub_cmd<span class="token punctuation">]</span> <span class="token keyword">then</span> pl_app<span class="token punctuation">.</span><span class="token function">quit</span><span class="token punctuation">(</span><span class="token string">"No such command for "</span> <span class="token operator">..</span> cmd_name <span class="token operator">..</span> <span class="token string">": "</span> <span class="token operator">..</span> sub_cmd<span class="token punctuation">)</span> <span class="token keyword">else</span> args<span class="token punctuation">.</span>command <span class="token operator">=</span> sub_cmd <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token function">xpcall</span><span class="token punctuation">(</span><span class="token keyword">function</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token function">cmd_exec</span><span class="token punctuation">(</span>args<span class="token punctuation">)</span> <span class="token keyword">end</span><span class="token punctuation">,</span> <span class="token keyword">function</span><span class="token punctuation">(</span>err<span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> <span class="token punctuation">(</span>args<span class="token punctuation">.</span>v <span class="token keyword">or</span> args<span class="token punctuation">.</span>vv<span class="token punctuation">)</span> <span class="token keyword">then</span> err <span class="token operator">=</span> err<span class="token punctuation">:</span>match <span class="token string">"^.-:.-:.(.*)$"</span> io<span class="token punctuation">.</span>stderr<span class="token punctuation">:</span><span class="token function">write</span><span class="token punctuation">(</span><span class="token string">"Error: "</span> <span class="token operator">..</span> err <span class="token operator">..</span> <span class="token string">"\n"</span><span class="token punctuation">)</span> io<span class="token punctuation">.</span>stderr<span class="token punctuation">:</span><span class="token function">write</span><span class="token punctuation">(</span><span class="token string">"\n Run with --v (verbose) or --vv (debug) for more details\n"</span><span class="token punctuation">)</span> <span class="token keyword">else</span> <span class="token keyword">local</span> trace <span class="token operator">=</span> debug<span class="token punctuation">.</span><span class="token function">traceback</span><span class="token punctuation">(</span>err<span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">)</span> io<span class="token punctuation">.</span>stderr<span class="token punctuation">:</span><span class="token function">write</span><span class="token punctuation">(</span><span class="token string">"Error: \n"</span><span class="token punctuation">)</span> io<span class="token punctuation">.</span>stderr<span class="token punctuation">:</span><span class="token function">write</span><span class="token punctuation">(</span>trace <span class="token operator">..</span> <span class="token string">"\n"</span><span class="token punctuation">)</span> <span class="token keyword">end</span> pl_app<span class="token punctuation">.</span><span class="token function">quit</span><span class="token punctuation">(</span><span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token keyword">true</span><span class="token punctuation">)</span> <span class="token keyword">end</span><span class="token punctuation">)</span><span class="token keyword">end</span></code></pre><p>init脚本我把一些日志去掉了，看的更清晰一些。pl.lapp是一个处理命令行输入的库，<a href="https://stevedonovan.github.io/Penlight/api/libraries/pl.lapp.html">pl.lapp</a>。这个脚本的主要逻辑就是，先对输入的命令参数做校验，符合要求的话，讲会调用kong.cmd.${cmdname}脚本，那么我们启动时调用的就会是kong.cmd.start脚本。那么我们再看一下start脚本里面写的什么。</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">local</span> prefix_handler <span class="token operator">=</span> require <span class="token string">"kong.cmd.utils.prefix_handler"</span><span class="token keyword">local</span> nginx_signals <span class="token operator">=</span> require <span class="token string">"kong.cmd.utils.nginx_signals"</span><span class="token keyword">local</span> conf_loader <span class="token operator">=</span> require <span class="token string">"kong.conf_loader"</span><span class="token keyword">local</span> DAOFactory <span class="token operator">=</span> require <span class="token string">"kong.dao.factory"</span><span class="token keyword">local</span> kill <span class="token operator">=</span> require <span class="token string">"kong.cmd.utils.kill"</span><span class="token keyword">local</span> log <span class="token operator">=</span> require <span class="token string">"kong.cmd.utils.log"</span><span class="token keyword">local</span> <span class="token keyword">function</span> <span class="token function">execute</span><span class="token punctuation">(</span>args<span class="token punctuation">)</span> <span class="token keyword">local</span> conf <span class="token operator">=</span> <span class="token function">assert</span><span class="token punctuation">(</span><span class="token function">conf_loader</span><span class="token punctuation">(</span>args<span class="token punctuation">.</span>conf<span class="token punctuation">,</span> <span class="token punctuation">{</span> prefix <span class="token operator">=</span> args<span class="token punctuation">.</span>prefix <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token comment">--加载配置文件</span> <span class="token function">assert</span><span class="token punctuation">(</span><span class="token keyword">not</span> kill<span class="token punctuation">.</span><span class="token function">is_running</span><span class="token punctuation">(</span>conf<span class="token punctuation">.</span>nginx_pid<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">"Kong is already running in "</span> <span class="token operator">..</span> conf<span class="token punctuation">.</span>prefix<span class="token punctuation">)</span> <span class="token comment">--检查nginx是否已经在运行</span> <span class="token keyword">local</span> err <span class="token keyword">local</span> dao <span class="token operator">=</span> <span class="token function">assert</span><span class="token punctuation">(</span>DAOFactory<span class="token punctuation">.</span><span class="token function">new</span><span class="token punctuation">(</span>conf<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token comment">--通过数据库配置创建数据库</span> <span class="token function">xpcall</span><span class="token punctuation">(</span><span class="token keyword">function</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token function">assert</span><span class="token punctuation">(</span>prefix_handler<span class="token punctuation">.</span><span class="token function">prepare_prefix</span><span class="token punctuation">(</span>conf<span class="token punctuation">,</span> args<span class="token punctuation">.</span>nginx_conf<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token keyword">if</span> args<span class="token punctuation">.</span>run_migrations <span class="token keyword">then</span> <span class="token function">assert</span><span class="token punctuation">(</span>dao<span class="token punctuation">:</span><span class="token function">run_migrations</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token comment">--检查是否是数据库升级命令</span> <span class="token keyword">end</span> <span class="token keyword">local</span> ok<span class="token punctuation">,</span> err <span class="token operator">=</span> dao<span class="token punctuation">:</span><span class="token function">are_migrations_uptodate</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token comment">--检查数据库版本是否升级到最新</span> <span class="token keyword">if</span> err <span class="token keyword">then</span> <span class="token comment">-- error correctly formatted by the DAO</span> <span class="token function">error</span><span class="token punctuation">(</span>err<span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token keyword">if</span> <span class="token keyword">not</span> ok <span class="token keyword">then</span> <span class="token comment">-- we cannot start, throw a very descriptive error to instruct the user</span> <span class="token function">error</span><span class="token punctuation">(</span><span class="token string">"the current database schema does not match\n"</span> <span class="token operator">..</span> <span class="token string">"this version of Kong.\n\nPlease run `kong migrations up` "</span> <span class="token operator">..</span> <span class="token string">"first to update/initialise the database schema.\nBe aware "</span> <span class="token operator">..</span> <span class="token string">"that Kong migrations should only run from a single node, and "</span> <span class="token operator">..</span> <span class="token string">"that nodes\nrunning migrations concurrently will conflict "</span> <span class="token operator">..</span> <span class="token string">"with each other and might corrupt\nyour database schema!"</span><span class="token punctuation">)</span> <span class="token keyword">end</span> ok<span class="token punctuation">,</span> err <span class="token operator">=</span> dao<span class="token punctuation">:</span><span class="token function">check_schema_consensus</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token comment">-- 这个主要针对 cassandra 做检查，psql不做检查</span> <span class="token keyword">if</span> err <span class="token keyword">then</span> <span class="token comment">-- error correctly formatted by the DAO</span> <span class="token function">error</span><span class="token punctuation">(</span>err<span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token keyword">if</span> <span class="token keyword">not</span> ok <span class="token keyword">then</span> <span class="token function">error</span><span class="token punctuation">(</span><span class="token string">"Cassandra has not reached cluster consensus yet"</span><span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token function">assert</span><span class="token punctuation">(</span>nginx_signals<span class="token punctuation">.</span><span class="token function">start</span><span class="token punctuation">(</span>conf<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token comment">--通过指定的配置启动nginx</span> <span class="token function">log</span><span class="token punctuation">(</span><span class="token string">"Kong started"</span><span class="token punctuation">)</span> <span class="token keyword">end</span><span class="token punctuation">,</span> <span class="token keyword">function</span><span class="token punctuation">(</span>e<span class="token punctuation">)</span> err <span class="token operator">=</span> e <span class="token comment">-- cannot throw from this function</span> <span class="token keyword">end</span><span class="token punctuation">)</span> <span class="token keyword">if</span> err <span class="token keyword">then</span> log<span class="token punctuation">.</span><span class="token function">verbose</span><span class="token punctuation">(</span><span class="token string">"could not start Kong, stopping services"</span><span class="token punctuation">)</span> <span class="token function">pcall</span><span class="token punctuation">(</span>nginx_signals<span class="token punctuation">.</span><span class="token function">stop</span><span class="token punctuation">(</span>conf<span class="token punctuation">)</span><span class="token punctuation">)</span> log<span class="token punctuation">.</span><span class="token function">verbose</span><span class="token punctuation">(</span><span class="token string">"stopped services"</span><span class="token punctuation">)</span> <span class="token function">error</span><span class="token punctuation">(</span>err<span class="token punctuation">)</span> <span class="token comment">-- report to main error handler</span> <span class="token keyword">end</span><span class="token keyword">end</span><span class="token keyword">local</span> lapp <span class="token operator">=</span> <span class="token string">[[Usage: kong start [OPTIONS]Start Kong (Nginx and other configured services) in the configuredprefix directory.Options: -c,--conf (optional string) configuration file -p,--prefix (optional string) override prefix directory --nginx-conf (optional string) custom Nginx configuration template --run-migrations (optional boolean) optionally run migrations on the DB]]</span><span class="token keyword">return</span> <span class="token punctuation">{</span> lapp <span class="token operator">=</span> lapp<span class="token punctuation">,</span> execute <span class="token operator">=</span> execute<span class="token punctuation">}</span></code></pre><p>我们看到这块脚本主要的功能就是先加载配置，然后检查数据库是否更新到最新版本（这个插件更新和相关更新数据库的操作都需要执行run migrations命令，使数据库结构同步到最新），而后检查执行nginx启动命令</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token comment">-- nginx_signals.lua</span><span class="token keyword">function</span> _M<span class="token punctuation">.</span><span class="token function">start</span><span class="token punctuation">(</span>kong_conf<span class="token punctuation">)</span> <span class="token keyword">local</span> nginx_bin<span class="token punctuation">,</span> err <span class="token operator">=</span> _M<span class="token punctuation">.</span><span class="token function">find_nginx_bin</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> nginx_bin <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> err <span class="token keyword">end</span> <span class="token keyword">if</span> kill<span class="token punctuation">.</span><span class="token function">is_running</span><span class="token punctuation">(</span>kong_conf<span class="token punctuation">.</span>nginx_pid<span class="token punctuation">)</span> <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token string">"nginx is already running in "</span> <span class="token operator">..</span> kong_conf<span class="token punctuation">.</span>prefix <span class="token keyword">end</span> <span class="token keyword">local</span> cmd <span class="token operator">=</span> <span class="token function">fmt</span><span class="token punctuation">(</span><span class="token string">"%s -p %s -c %s"</span><span class="token punctuation">,</span> nginx_bin<span class="token punctuation">,</span> kong_conf<span class="token punctuation">.</span>prefix<span class="token punctuation">,</span> <span class="token string">"nginx.conf"</span><span class="token punctuation">)</span> log<span class="token punctuation">.</span><span class="token function">debug</span><span class="token punctuation">(</span><span class="token string">"starting nginx: %s"</span><span class="token punctuation">,</span> cmd<span class="token punctuation">)</span> <span class="token keyword">local</span> ok<span class="token punctuation">,</span> _<span class="token punctuation">,</span> _<span class="token punctuation">,</span> stderr <span class="token operator">=</span> pl_utils<span class="token punctuation">.</span><span class="token function">executeex</span><span class="token punctuation">(</span>cmd<span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> ok <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> stderr <span class="token keyword">end</span> log<span class="token punctuation">.</span><span class="token function">debug</span><span class="token punctuation">(</span><span class="token string">"nginx started"</span><span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token keyword">true</span><span class="token keyword">end</span><span class="token keyword">function</span> _M<span class="token punctuation">.</span><span class="token function">stop</span><span class="token punctuation">(</span>kong_conf<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token function">send_signal</span><span class="token punctuation">(</span>kong_conf<span class="token punctuation">,</span> <span class="token string">"TERM"</span><span class="token punctuation">)</span><span class="token keyword">end</span><span class="token keyword">function</span> _M<span class="token punctuation">.</span><span class="token function">quit</span><span class="token punctuation">(</span>kong_conf<span class="token punctuation">,</span> graceful<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token function">send_signal</span><span class="token punctuation">(</span>kong_conf<span class="token punctuation">,</span> <span class="token string">"QUIT"</span><span class="token punctuation">)</span><span class="token keyword">end</span><span class="token keyword">function</span> _M<span class="token punctuation">.</span><span class="token function">reload</span><span class="token punctuation">(</span>kong_conf<span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> kill<span class="token punctuation">.</span><span class="token function">is_running</span><span class="token punctuation">(</span>kong_conf<span class="token punctuation">.</span>nginx_pid<span class="token punctuation">)</span> <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> <span class="token string">"nginx not running in prefix: "</span> <span class="token operator">..</span> kong_conf<span class="token punctuation">.</span>prefix <span class="token keyword">end</span> <span class="token keyword">local</span> nginx_bin<span class="token punctuation">,</span> err <span class="token operator">=</span> _M<span class="token punctuation">.</span><span class="token function">find_nginx_bin</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> nginx_bin <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> err <span class="token keyword">end</span> <span class="token keyword">local</span> cmd <span class="token operator">=</span> <span class="token function">fmt</span><span class="token punctuation">(</span><span class="token string">"%s -p %s -c %s -s %s"</span><span class="token punctuation">,</span> nginx_bin<span class="token punctuation">,</span> kong_conf<span class="token punctuation">.</span>prefix<span class="token punctuation">,</span> <span class="token string">"nginx.conf"</span><span class="token punctuation">,</span> <span class="token string">"reload"</span><span class="token punctuation">)</span> log<span class="token punctuation">.</span><span class="token function">debug</span><span class="token punctuation">(</span><span class="token string">"reloading nginx: %s"</span><span class="token punctuation">,</span> cmd<span class="token punctuation">)</span> <span class="token keyword">local</span> ok<span class="token punctuation">,</span> _<span class="token punctuation">,</span> _<span class="token punctuation">,</span> stderr <span class="token operator">=</span> pl_utils<span class="token punctuation">.</span><span class="token function">executeex</span><span class="token punctuation">(</span>cmd<span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> ok <span class="token keyword">then</span> <span class="token keyword">return</span> <span class="token keyword">nil</span><span class="token punctuation">,</span> stderr <span class="token keyword">end</span> <span class="token keyword">return</span> <span class="token keyword">true</span><span class="token keyword">end</span></code></pre><p>nginx_signals脚本主要就是找到nginx的执行文件，执行start stop等命令<br>分析到这里我们可能就有点迷惑了，难道这样就完了，没见到有什么特别的地方呀。别急，kong的主要逻辑都写在ngix.conf文件中呢。上篇文章我们分析了kong会将templates里的文件编译成最终的nginx.conf，那么接下来我们看一下nginx.conf文件变成什么样子了（已经去除了一些代码）。</p><pre class="language-none"><code class="language-none">worker_processes auto;daemon on;worker_rlimit_nofile 500000;events { worker_connections 16384; multi_accept on;}http { charset UTF-8; # used by traffic limit lua_shared_dict limit_conn_store 100m; # used by redis cluster lua_shared_dict redis_cluster_slot_locks 1m; lua_socket_log_errors off; init_by_lua_block { kong = require 'kong' kong.init() } init_worker_by_lua_block { kong.init_worker() } upstream kong_upstream { server 0.0.0.1; balancer_by_lua_block { kong.balancer() } keepalive 60; } server { server_name kong; listen 0.0.0.0:9000; error_page 400 404 408 411 412 413 414 417 /kong_error_handler; error_page 500 502 503 504 /kong_error_handler; client_body_buffer_size 60m; real_ip_header X-Real-IP; real_ip_recursive off; location / { default_type text/html; rewrite_by_lua_block { kong.rewrite() } access_by_lua_block { kong.access() } header_filter_by_lua_block { kong.header_filter() } body_filter_by_lua_block { kong.body_filter() } log_by_lua_block { kong.log() } } location = /kong_error_handler { internal; content_by_lua_block { kong.handle_error() } } } server { server_name kong_admin; listen 127.0.0.1:9001; client_max_body_size 10m; client_body_buffer_size 10m; listen 127.0.0.1:8444 ssl; location / { default_type application/json; content_by_lua_block { kong.serve_admin_api() } } location /nginx_status { internal; access_log off; stub_status; } location /robots.txt { return 200 'User-agent: *\nDisallow: /'; } }}</code></pre><p>通过分析我们看到，Kong主要是在nginx的生命周期的每个阶段插入了自己的代码，从而起到增到增强nginx的作用。那么nginx的每个请求的生命周期和kong对每个请求处理的生命周期应该是一致的。事实也正是这样的，kong的启动相关的代码主要在kong/init.lua目录中的init方法和init_worker方法</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">function</span> Kong<span class="token punctuation">.</span><span class="token function">init</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">local</span> pl_path <span class="token operator">=</span> require <span class="token string">"pl.path"</span> <span class="token keyword">local</span> conf_loader <span class="token operator">=</span> require <span class="token string">"kong.conf_loader"</span> <span class="token comment">-- retrieve kong_config</span> <span class="token keyword">local</span> conf_path <span class="token operator">=</span> pl_path<span class="token punctuation">.</span><span class="token function">join</span><span class="token punctuation">(</span>ngx<span class="token punctuation">.</span>config<span class="token punctuation">.</span><span class="token function">prefix</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">".kong_env"</span><span class="token punctuation">)</span> <span class="token keyword">local</span> config <span class="token operator">=</span> <span class="token function">assert</span><span class="token punctuation">(</span><span class="token function">conf_loader</span><span class="token punctuation">(</span>conf_path<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token keyword">local</span> dao <span class="token operator">=</span> <span class="token function">assert</span><span class="token punctuation">(</span>DAOFactory<span class="token punctuation">.</span><span class="token function">new</span><span class="token punctuation">(</span>config<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token comment">-- instantiate long-lived DAO</span> <span class="token keyword">local</span> ok<span class="token punctuation">,</span> err_t <span class="token operator">=</span> dao<span class="token punctuation">:</span><span class="token function">init</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> ok <span class="token keyword">then</span> <span class="token function">error</span><span class="token punctuation">(</span><span class="token function">tostring</span><span class="token punctuation">(</span>err_t<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token function">assert</span><span class="token punctuation">(</span>dao<span class="token punctuation">:</span><span class="token function">are_migrations_uptodate</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token comment">-- populate singletons</span> singletons<span class="token punctuation">.</span>ip <span class="token operator">=</span> ip<span class="token punctuation">.</span><span class="token function">init</span><span class="token punctuation">(</span>config<span class="token punctuation">)</span> singletons<span class="token punctuation">.</span>dns <span class="token operator">=</span> <span class="token function">dns</span><span class="token punctuation">(</span>config<span class="token punctuation">)</span> singletons<span class="token punctuation">.</span>loaded_plugins <span class="token operator">=</span> <span class="token function">assert</span><span class="token punctuation">(</span><span class="token function">load_plugins</span><span class="token punctuation">(</span>config<span class="token punctuation">,</span> dao<span class="token punctuation">)</span><span class="token punctuation">)</span> singletons<span class="token punctuation">.</span>dao <span class="token operator">=</span> dao singletons<span class="token punctuation">.</span>configuration <span class="token operator">=</span> config <span class="token function">assert</span><span class="token punctuation">(</span>core<span class="token punctuation">.</span><span class="token function">build_router</span><span class="token punctuation">(</span>dao<span class="token punctuation">,</span> <span class="token string">"init"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token keyword">end</span></code></pre><p>kong的init方法里做的事情很简单，主要是加载配置，检查数据库结构有没有更新到最新，然后加载所有的插件，最后建立路由。</p><pre class="language-lua" data-language="lua"><code class="language-lua"><span class="token keyword">function</span> Kong<span class="token punctuation">.</span><span class="token function">init_worker</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token comment">-- init DAO</span> <span class="token keyword">local</span> ok<span class="token punctuation">,</span> err <span class="token operator">=</span> singletons<span class="token punctuation">.</span>dao<span class="token punctuation">:</span><span class="token function">init_worker</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> ok <span class="token keyword">then</span> <span class="token function">ngx_log</span><span class="token punctuation">(</span>ngx_CRIT<span class="token punctuation">,</span> <span class="token string">"could not init DB: "</span><span class="token punctuation">,</span> err<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token keyword">end</span> <span class="token comment">-- init inter-worker events</span> <span class="token keyword">local</span> worker_events <span class="token operator">=</span> require <span class="token string">"resty.worker.events"</span> <span class="token keyword">local</span> ok<span class="token punctuation">,</span> err <span class="token operator">=</span> worker_events<span class="token punctuation">.</span><span class="token function">configure</span> <span class="token punctuation">{</span> shm <span class="token operator">=</span> <span class="token string">"kong_process_events"</span><span class="token punctuation">,</span> <span class="token comment">-- defined by "lua_shared_dict"</span> timeout <span class="token operator">=</span> <span class="token number">5</span><span class="token punctuation">,</span> <span class="token comment">-- life time of event data in shm</span> interval <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token comment">-- poll interval (seconds)</span> wait_interval <span class="token operator">=</span> <span class="token number">0.010</span><span class="token punctuation">,</span> <span class="token comment">-- wait before retry fetching event data</span> wait_max <span class="token operator">=</span> <span class="token number">0.5</span><span class="token punctuation">,</span> <span class="token comment">-- max wait time before discarding event</span> <span class="token punctuation">}</span> <span class="token keyword">if</span> <span class="token keyword">not</span> ok <span class="token keyword">then</span> <span class="token function">ngx_log</span><span class="token punctuation">(</span>ngx_CRIT<span class="token punctuation">,</span> <span class="token string">"could not start inter-worker events: "</span><span class="token punctuation">,</span> err<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token keyword">end</span> <span class="token comment">-- init cluster_events</span> <span class="token keyword">local</span> dao_factory <span class="token operator">=</span> singletons<span class="token punctuation">.</span>dao <span class="token keyword">local</span> configuration <span class="token operator">=</span> singletons<span class="token punctuation">.</span>configuration <span class="token keyword">local</span> cluster_events<span class="token punctuation">,</span> err <span class="token operator">=</span> kong_cluster_events<span class="token punctuation">.</span><span class="token function">new</span> <span class="token punctuation">{</span> dao <span class="token operator">=</span> dao_factory<span class="token punctuation">,</span> poll_interval <span class="token operator">=</span> configuration<span class="token punctuation">.</span>db_update_frequency<span class="token punctuation">,</span> poll_offset <span class="token operator">=</span> configuration<span class="token punctuation">.</span>db_update_propagation<span class="token punctuation">,</span> <span class="token punctuation">}</span> <span class="token keyword">if</span> <span class="token keyword">not</span> cluster_events <span class="token keyword">then</span> <span class="token function">ngx_log</span><span class="token punctuation">(</span>ngx_CRIT<span class="token punctuation">,</span> <span class="token string">"could not create cluster_events: "</span><span class="token punctuation">,</span> err<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token keyword">end</span> <span class="token comment">-- init cache</span> <span class="token keyword">local</span> cache<span class="token punctuation">,</span> err <span class="token operator">=</span> kong_cache<span class="token punctuation">.</span><span class="token function">new</span> <span class="token punctuation">{</span> cluster_events <span class="token operator">=</span> cluster_events<span class="token punctuation">,</span> worker_events <span class="token operator">=</span> worker_events<span class="token punctuation">,</span> propagation_delay <span class="token operator">=</span> configuration<span class="token punctuation">.</span>db_update_propagation<span class="token punctuation">,</span> ttl <span class="token operator">=</span> configuration<span class="token punctuation">.</span>db_cache_ttl<span class="token punctuation">,</span> neg_ttl <span class="token operator">=</span> configuration<span class="token punctuation">.</span>db_cache_ttl<span class="token punctuation">,</span> resty_lock_opts <span class="token operator">=</span> <span class="token punctuation">{</span> exptime <span class="token operator">=</span> <span class="token number">10</span><span class="token punctuation">,</span> timeout <span class="token operator">=</span> <span class="token number">5</span><span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">}</span> <span class="token keyword">if</span> <span class="token keyword">not</span> cache <span class="token keyword">then</span> <span class="token function">ngx_log</span><span class="token punctuation">(</span>ngx_CRIT<span class="token punctuation">,</span> <span class="token string">"could not create kong cache: "</span><span class="token punctuation">,</span> err<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token keyword">end</span> <span class="token keyword">local</span> ok<span class="token punctuation">,</span> err <span class="token operator">=</span> cache<span class="token punctuation">:</span><span class="token function">get</span><span class="token punctuation">(</span><span class="token string">"router:version"</span><span class="token punctuation">,</span> <span class="token punctuation">{</span> ttl <span class="token operator">=</span> <span class="token number">0</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token keyword">function</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token string">"init"</span> <span class="token keyword">end</span><span class="token punctuation">)</span> <span class="token keyword">if</span> <span class="token keyword">not</span> ok <span class="token keyword">then</span> <span class="token function">ngx_log</span><span class="token punctuation">(</span>ngx_CRIT<span class="token punctuation">,</span> <span class="token string">"could not set router version in cache: "</span><span class="token punctuation">,</span> err<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token keyword">end</span> singletons<span class="token punctuation">.</span>cache <span class="token operator">=</span> cache singletons<span class="token punctuation">.</span>worker_events <span class="token operator">=</span> worker_events singletons<span class="token punctuation">.</span>cluster_events <span class="token operator">=</span> cluster_events singletons<span class="token punctuation">.</span>dao<span class="token punctuation">:</span><span class="token function">set_events_handler</span><span class="token punctuation">(</span>worker_events<span class="token punctuation">)</span> core<span class="token punctuation">.</span>init_worker<span class="token punctuation">.</span><span class="token function">before</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token comment">-- run plugins init_worker context</span> <span class="token keyword">for</span> _<span class="token punctuation">,</span> plugin <span class="token keyword">in</span> <span class="token function">ipairs</span><span class="token punctuation">(</span>singletons<span class="token punctuation">.</span>loaded_plugins<span class="token punctuation">)</span> <span class="token keyword">do</span> plugin<span class="token punctuation">.</span>handler<span class="token punctuation">:</span><span class="token function">init_worker</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">end</span><span class="token keyword">end</span></code></pre><p>因为nginx是单线程的，所以很多事情只能放到定时器里来做，而nginx的定时器只可以放到init_worker阶段里做。init_worker做的主要是初始化数据库相关，然后最重要的是初始化集群管理相关逻辑（因为是采用的轮询数据库的方式，只能放到这里来做），最后调用每个插件的init_worker方法。<br>然后kong/init.lua还有一些其他方法，和nginx的生命周期相对应，这个我们后面再分析。</p>]]></content>
<categories>
<category> 网关 </category>
</categories>
<tags>
<tag> Lua </tag>
<tag> 网关 </tag>
<tag> Kong </tag>
</tags>
</entry>
<entry>
<title>kong0.12.x源码分析(一)----kong目录结构解析</title>
<link href="/2018/04/30/kong0-12-x%E6%BA%90%E7%A0%81%E5%88%86%E6%9E%90-%E4%B8%80/"/>
<url>/2018/04/30/kong0-12-x%E6%BA%90%E7%A0%81%E5%88%86%E6%9E%90-%E4%B8%80/</url>
<content type="html"><![CDATA[<p>最近需要在kong的基础上做api网关，因此有不少功能需要定制，以便更符合业务需要。然后发现kong在0.12以后变化挺大的，网上现有的kong的源码解析文章比较老了，于是打算做一系列kong的源码解析文章，一方面可以对自己的理解做一下梳理，另一方面也可以给其他同事做一些参考。因本人水平有限，错误在所难免，欢迎各位在下面留言，也可以通过邮箱和我交流: <a href="mailto:[email protected]">[email protected]</a> <a href="mailto:[email protected]">[email protected]</a><br>kong的目录结构如下</p><pre class="language-bash" data-language="bash"><code class="language-bash">kong<span class="token operator">|</span>__api<span class="token operator">|</span>__cluster_events<span class="token operator">|</span>__cmd<span class="token operator">|</span>__core<span class="token operator">|</span>__dao<span class="token operator">|</span>__plugins<span class="token operator">|</span>__templates<span class="token operator">|</span>__tools<span class="token operator">|</span>__vendor<span class="token operator">|</span>__cache.lua<span class="token operator">|</span>__cluster_events.lua<span class="token operator">|</span>__conf_loader.lua<span class="token operator">|</span>__constants.lua<span class="token operator">|</span>__init.lua<span class="token operator">|</span>__meta.lua<span class="token operator">|</span>__mlcache.lua<span class="token operator">|</span>__singletons.lua</code></pre><p>kong的目录主要就这些，看起来蛮清晰的。kong是基于OpenResty，而OpenResty又是基于nginx的，所以，kong的启动流程和nginx很相似，也有start，stop，reload等命令。</p><h4 id="api目录">api目录</h4><p>这个目录主要的作用是基于lapis开发的一个简单web服务，提供一系列restful api借口，供管理平台使用</p><pre class="language-bash" data-language="bash"><code class="language-bash">├── api_helpers.lua ├── crud_helpers.lua├── init.lua└── routes ├── apis.lua ├── cache.lua ├── certificates.lua ├── consumers.lua ├── kong.lua ├── plugins.lua ├── snis.lua └── upstreams.lua</code></pre><h4 id="cmd目录">cmd目录</h4><p>cmd目录下主要关注一下init.lua 。这个文件负责根据不同的指令分别调用start，stop，reload等lua脚本，这些脚本中则通过调用nginx的start，stop等命令完成服务的启动。</p><pre class="language-bash" data-language="bash"><code class="language-bash">├── check.lua├── health.lua├── init.lua├── migrations.lua├── prepare.lua├── quit.lua├── reload.lua├── restart.lua├── roar.lua├── start.lua├── stop.lua├── utils│ ├── kill.lua│ ├── log.lua│ ├── nginx_signals.lua│ └── prefix_handler.lua└── version.lua</code></pre><h4 id="core目录">core目录</h4><p>core目录下面放的是一些kong的核心操作。router.lua负责创建到upstreams的路由，采用hash表存储，提高查找效率。reports文件是向udp logger发送日志。hander.lua负责集群事件的发布和订阅处理。plugins_iterator.lua负责加载插件。balancer.lua负责做负载均衡。</p><pre class="language-bash" data-language="bash"><code class="language-bash">├── balancer.lua├── certificate.lua├── error_handlers.lua├── globalpatches.lua├── handler.lua├── plugins_iterator.lua├── reports.lua└── router.lua</code></pre><h4 id="dao目录">dao目录</h4><p>dao目录下主要放置的是数据库访问相关的操作。kong目前只支持psql和cassandra两种数据库，这点比较坑。db目录下的文件主要是封装了对具体数据库访问的增删改查操作。外面的dao.lua文件通过传入的db类型，生成链接，负责增删改查操作。也就是说dao.lua不关心具体的数据库实现，只是对具体的数据库访问做了一层封装。factory.lua是一个工厂类，负责管理生成数据库实例，初始化，更新数据等操作。migrations目录下则是具体的创建更新表的动作，每次升级都需要执行一次migrations操作。schemas下面放的是model文件，类似java中的bean文件。</p><pre class="language-bash" data-language="bash"><code class="language-bash">├── dao.lua├── db│ ├── cassandra.lua│ ├── init.lua│ └── postgres.lua├── errors.lua├── factory.lua├── migrations│ ├── cassandra.lua│ ├── helpers.lua│ └── postgres.lua├── model_factory.lua├── schemas│ ├── apis.lua│ ├── consumers.lua│ ├── plugins.lua│ ├── ssl_certificates.lua│ ├── ssl_servers_names.lua│ ├── targets.lua│ └── upstreams.lua└── schemas_validation.lua</code></pre><h4 id="cluster-events">cluster_events</h4><p>这个目录下放的是集群管理相关的事件存储，kong0.12.x后的集群管理改变很大。kong的集群管理主要是采用的发布订阅模式，中间会以数据库为媒介，定时去poll一下，以此维持心跳，看看有没有事件更新。集群管理后面会进行详细的分析。</p><pre class="language-bash" data-language="bash"><code class="language-bash">└── strategies ├── cassandra.lua └── postgres.lua</code></pre><h4 id="tools">tools</h4><p>tools目录下放置的是一些常用的工具类，比如加密、dns服务、ip服务等</p><pre class="language-bash" data-language="bash"><code class="language-bash">├── ciphers.lua├── dns.lua├── ip.lua├── printable.lua├── public.lua├── responses.lua├── timestamp.lua└── utils.lua</code></pre><h4 id="vendor">vendor</h4><p>这个目录下只有一个文件，主要是给lua提供面向对象的能力。kong的插件机制会用到面向对象的继承思想，这个后面我们会详细分析。</p><pre class="language-bash" data-language="bash"><code class="language-bash"><span class="token builtin class-name">.</span>└── classic.lua</code></pre><h4 id="templates">templates</h4><p>templates顾名思义就是模板啦，下面主要放一些模板文件，kong_defaults.lua下面放的���一些kong的默认配置文件；nginx.lua主要放置nginx的一些启动参数；nginx_kong.lua放的是nginx的配置文件模板，kong启动时会以此为模板生成nginx.conf文件</p><pre class="language-bash" data-language="bash"><code class="language-bash"><span class="token builtin class-name">.</span>├── kong_defaults.lua├── nginx_kong.lua└── nginx.lua</code></pre>]]></content>
<categories>
<category> 网关 </category>
</categories>
<tags>
<tag> Lua </tag>
<tag> 网关 </tag>
<tag> Kong </tag>
</tags>
</entry>
<entry>
<title>微服务网关kong编译安装教程</title>
<link href="/2018/03/13/%E5%BE%AE%E6%9C%8D%E5%8A%A1%E7%BD%91%E5%85%B3kong%E7%BC%96%E8%AF%91%E5%AE%89%E8%A3%85%E6%95%99%E7%A8%8B/"/>
<url>/2018/03/13/%E5%BE%AE%E6%9C%8D%E5%8A%A1%E7%BD%91%E5%85%B3kong%E7%BC%96%E8%AF%91%E5%AE%89%E8%A3%85%E6%95%99%E7%A8%8B/</url>
<content type="html"><![CDATA[<p><a href="https://github.com/Kong/kong">Kong</a>是一个高性能的微服务网关。依赖于OpenResty、lua、luarocks和postgresql或者Cassandra。在服务器上安装时可能没有sudo权限，因此相关依赖我们都需要从源码安装</p><h4 id="首先我们安装OpenResty">首先我们安装OpenResty</h4><p>之前我写过<a href="http://xiaodongliu.com/2017/11/27/OpenResty%E5%AE%89%E8%A3%85%E8%AE%B0%E5%BD%95/">一篇博客</a>进行OpenResty安装。这次主要是添加一些配置参数</p><pre class="language-none"><code class="language-none">wget https://openresty.org/download/openresty-1.13.6.1.tar.gztar -xzvf openresty-1.13.6.1.tar.gzcd openresty-1.13.6.1./configure \ --with-pcre-jit \ --with-ipv6 \ --with-http_realip_module \ --with-http_ssl_module \ --with-http_stub_status_module \ --with-http_v2_module \ --prefix=/home/webedit/openresty make make install</code></pre><p>同时需要配置环境变量</p><pre class="language-none"><code class="language-none">export PATH="$PATH:/home/webedit/openresty/bin"export PATH="$PATH:/home/webedit/openresty/luajit/lib"export PATH="$PATH:/home/webedit/openresty/luajit/include/luajit-2.1"</code></pre><h4 id="其次我们需要安装luarocks-OpenResty中已经包含有luajit了">其次我们需要安装luarocks(OpenResty中已经包含有luajit了)</h4><pre class="language-none"><code class="language-none">wget https://github.com/luarocks/luarocks/archive/2.4.3.zipunzip luarocks-2.4.3.zipcd uarocks-2.4.3./configure \ --lua-suffix=jit \ --prefix=/home/webedit/luarocks \ --with-lua=/home/webedit/openresty/luajit \ --with-lua-include=/home/webedit/openresty/luajit/include/luajit-2.1</code></pre><p>配置环境变量</p><pre class="language-none"><code class="language-none">export PATH="$PATH:/home/webedit/luarocks/bin"</code></pre><h4 id="接下来我们需要下载kong">接下来我们需要下载kong</h4><pre class="language-none"><code class="language-none">wget https://github.com/Kong/kong/archive/0.12.3.zipunzip kong-0.12.3.zipcd kong-0.12.3/home/webedit/luarocks/bin/luarocks make</code></pre><p>配置环境变量</p><pre class="language-none"><code class="language-none">export LUA_PATH="/home/webedit/luarocks/share/lua/5.1/?.lua"export PATH="$PATH:/home/webedit/kong-src/bin"</code></pre><h4 id="然后下载postgresql-tar-gz包">然后下载postgresql tar.gz包</h4><pre class="language-none"><code class="language-none">mkdir data ./pgsql/bin/initdb -D data/ --locale=en_US.UTF-8 -U postgres -W ./pgsql/bin/pg_ctl -D data/ start ./pgsql/bin/psql -U postgres postgres=# Alter USER postgres WITH PASSWORD '***密码**'; //添加密码 ALTER ROLE //出现这个才算成功，第一次操作没成功，pgadmin连不上 </code></pre><p>进入命令行工具创建user</p><pre class="language-none"><code class="language-none">CREATE USER kong; CREATE DATABASE kong OWNER kong;</code></pre><h4 id="最后修改Kong的配置文件-home-webedit-kong-src-kong-conf-如下">最后修改Kong的配置文件( /home/webedit/kong-src/kong.conf)如下</h4><pre class="language-none"><code class="language-none"># -----------------------# Kong configuration file# -----------------------prefix = /home/webedit/kong/ # Working directory. Equivalent to Nginx's # prefix path, containing temporary files # and logs. # Each Kong process must have a separate # working directory.proxy_listen = 0.0.0.0:9000 # Address and port on which Kong will acceptadmin_listen = 127.0.0.1:9001 # Address and port on which Kong will exposepg_host = 127.0.0.1 # The PostgreSQL host to connect to.pg_port = 5432 # The port to connect to.pg_user = kong # The username to authenticate if required.pg_password = # The password to authenticate if required.pg_database = kong # The database name to connect to.</code></pre><p>这样就可以启动kong了</p><pre class="language-none"><code class="language-none">kong migrations up -c /home/webedit/kong-src/kong.confkong start -c /home/webedit/kong-src/kong.confcurl -i http://localhost:9001/</code></pre>]]></content>
<categories>
<category> 网关 </category>
</categories>
<tags>
<tag> Lua </tag>
<tag> 网关 </tag>
<tag> Kong </tag>
</tags>
</entry>
<entry>
<title>logback打印mybatis sql语句</title>
<link href="/2018/01/24/logback%E6%89%93%E5%8D%B0mybatis-sql%E8%AF%AD%E5%8F%A5/"/>
<url>/2018/01/24/logback%E6%89%93%E5%8D%B0mybatis-sql%E8%AF%AD%E5%8F%A5/</url>
<content type="html"><![CDATA[<p>最近想在mybatis中打印一下sql语句，发现官网提供的方法都不适合，最终在os china找到了，现在记录一下，以作备份</p><h4 id="1-在mybatis的configuration中增加setting配置">1. 在mybatis的configuration中增加setting配置</h4><pre class="language-none"><code class="language-none"><settings> <setting name="logPrefix" value="dao."/></settings></code></pre><h4 id="2-在logback配置文件中增加logger">2. 在logback配置文件中增加logger</h4><pre class="language-none"><code class="language-none"><logger name="dao" level="DEBUG"/></code></pre><p>参考:<br><a href="http://www.mybatis.org/mybatis-3/logging.html">http://www.mybatis.org/mybatis-3/logging.html</a><br><a href="https://my.oschina.net/u/2263802/blog/956588">https://my.oschina.net/u/2263802/blog/956588</a></p>]]></content>
</entry>
<entry>
<title>logrus日志按照时间分割</title>
<link href="/2017/12/11/logrus%E6%97%A5%E5%BF%97%E6%8C%89%E7%85%A7%E6%97%B6%E9%97%B4%E5%88%86%E5%89%B2/"/>
<url>/2017/12/11/logrus%E6%97%A5%E5%BF%97%E6%8C%89%E7%85%A7%E6%97%B6%E9%97%B4%E5%88%86%E5%89%B2/</url>
<content type="html"><![CDATA[<p>最近学习了一下go语言，发现和Java的语法差别好大啊，不过用起来的确很方便。<br>golang官方的日志库不太好用，就从github上找了一个比较火的logrus日志库，这个库和java的库用法还是有不少区别的，今天主要想做的是将日志文件按照时间进行切割，logrus借助了<code>go-file-rotatelogs</code> 这个库，所以首先我们应该安装相关依赖</p><h5 id="1-安装依赖库">1. 安装依赖库</h5><pre class="language-bash" data-language="bash"><code class="language-bash">go get github.com/sirupsen/logrusgo get github.com/lestrrat/go-file-rotatelogsgo get github.com/rifflock/lfshook</code></pre><h5 id="2-添加hook">2. 添加hook</h5><pre class="language-go" data-language="go"><code class="language-go"><span class="token keyword">package</span> log<span class="token keyword">import</span> <span class="token punctuation">(</span><span class="token string">"github.com/lestrrat/go-file-rotatelogs"</span><span class="token string">"github.com/pkg/errors"</span><span class="token string">"github.com/rifflock/lfshook"</span>log <span class="token string">"github.com/sirupsen/logrus"</span><span class="token string">"path"</span><span class="token string">"time"</span><span class="token punctuation">)</span><span class="token keyword">func</span> <span class="token function">init</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>log<span class="token punctuation">.</span><span class="token function">SetLevel</span><span class="token punctuation">(</span>log<span class="token punctuation">.</span>InfoLevel<span class="token punctuation">)</span>log<span class="token punctuation">.</span><span class="token function">AddHook</span><span class="token punctuation">(</span><span class="token function">newRotateHook</span><span class="token punctuation">(</span><span class="token string">""</span><span class="token punctuation">,</span> <span class="token string">"stdout.log"</span><span class="token punctuation">,</span> <span class="token number">7</span><span class="token operator">*</span><span class="token number">24</span><span class="token operator">*</span>time<span class="token punctuation">.</span>Hour<span class="token punctuation">,</span> <span class="token number">24</span><span class="token operator">*</span>time<span class="token punctuation">.</span>Hour<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token keyword">func</span> <span class="token function">newRotateHook</span><span class="token punctuation">(</span>logPath <span class="token builtin">string</span><span class="token punctuation">,</span> logFileName <span class="token builtin">string</span><span class="token punctuation">,</span> maxAge time<span class="token punctuation">.</span>Duration<span class="token punctuation">,</span> rotationTime time<span class="token punctuation">.</span>Duration<span class="token punctuation">)</span> <span class="token operator">*</span>lfshook<span class="token punctuation">.</span>LfsHook <span class="token punctuation">{</span>baseLogPath <span class="token operator">:=</span> path<span class="token punctuation">.</span><span class="token function">Join</span><span class="token punctuation">(</span>logPath<span class="token punctuation">,</span> logFileName<span class="token punctuation">)</span>writer<span class="token punctuation">,</span> err <span class="token operator">:=</span> rotatelogs<span class="token punctuation">.</span><span class="token function">New</span><span class="token punctuation">(</span>baseLogPath<span class="token operator">+</span><span class="token string">".%Y-%m-%d"</span><span class="token punctuation">,</span>rotatelogs<span class="token punctuation">.</span><span class="token function">WithLinkName</span><span class="token punctuation">(</span>baseLogPath<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment">// 生成软链，指向最新日志文</span>rotatelogs<span class="token punctuation">.</span><span class="token function">WithMaxAge</span><span class="token punctuation">(</span>maxAge<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment">// 文件最大保存时间</span>rotatelogs<span class="token punctuation">.</span><span class="token function">WithRotationTime</span><span class="token punctuation">(</span>rotationTime<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment">// 日志切割时间间隔</span><span class="token punctuation">)</span><span class="token keyword">if</span> err <span class="token operator">!=</span> <span class="token boolean">nil</span> <span class="token punctuation">{</span>log<span class="token punctuation">.</span><span class="token function">Errorf</span><span class="token punctuation">(</span><span class="token string">"config local file system logger error. %+v"</span><span class="token punctuation">,</span> errors<span class="token punctuation">.</span><span class="token function">WithStack</span><span class="token punctuation">(</span>err<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token keyword">return</span> lfshook<span class="token punctuation">.</span><span class="token function">NewHook</span><span class="token punctuation">(</span>lfshook<span class="token punctuation">.</span>WriterMap<span class="token punctuation">{</span>log<span class="token punctuation">.</span>DebugLevel<span class="token punctuation">:</span> writer<span class="token punctuation">,</span> <span class="token comment">// 为不同级别设置不同的输出目的</span>log<span class="token punctuation">.</span>InfoLevel<span class="token punctuation">:</span> writer<span class="token punctuation">,</span>log<span class="token punctuation">.</span>WarnLevel<span class="token punctuation">:</span> writer<span class="token punctuation">,</span>log<span class="token punctuation">.</span>ErrorLevel<span class="token punctuation">:</span> writer<span class="token punctuation">,</span>log<span class="token punctuation">.</span>FatalLevel<span class="token punctuation">:</span> writer<span class="token punctuation">,</span>log<span class="token punctuation">.</span>PanicLevel<span class="token punctuation">:</span> writer<span class="token punctuation">,</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token operator">&</span>log<span class="token punctuation">.</span>TextFormatter<span class="token punctuation">{</span>DisableColors<span class="token punctuation">:</span> <span class="token boolean">true</span><span class="token punctuation">,</span> TimestampFormat<span class="token punctuation">:</span> <span class="token string">"2006-01-02 15:04:05.000"</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">}</span></code></pre><h5 id="3-使用">3. 使用</h5><blockquote><p>logrus默认自带了一个log实例，步骤二给默认的实例添加了日志轮转，因此我们应该在其它包里面使用默认的log实例，而不应该直接使用<code> logrus.New()</code></p></blockquote><pre class="language-go" data-language="go"><code class="language-go">log <span class="token string">"github.com/sirupsen/logrus"</span></code></pre>]]></content>
</entry>
<entry>
<title>electron安装卡node install.js的解决方法</title>
<link href="/2017/12/10/electron%E5%AE%89%E8%A3%85%E5%8D%A1node-install-js%E7%9A%84%E8%A7%A3%E5%86%B3%E6%96%B9%E6%B3%95/"/>
<url>/2017/12/10/electron%E5%AE%89%E8%A3%85%E5%8D%A1node-install-js%E7%9A%84%E8%A7%A3%E5%86%B3%E6%96%B9%E6%B3%95/</url>
<content type="html"><![CDATA[<p>最近想用electron加上vue开发一个客户端软件，于是就打算采用simulatedgreg的脚手架，结果执行<code>npm install</code>时，一直卡在electron node install.js上；我把npm切换到taobao的源依然不行，后来查到一种方法，现在记录下来：<br>具体的操作是:</p><h5 id="1-初始化脚手架">1. 初始化脚手架</h5><pre class="language-javascript" data-language="javascript"><code class="language-javascript">vue init simulatedgreg<span class="token operator">/</span>electron<span class="token operator">-</span>vue my<span class="token operator">-</span>project# Install dependencies and run your appcd my<span class="token operator">-</span>project</code></pre><h5 id="2-删除package-json中的electron依赖，如果执行过npm-install的话，还需要删除-node-modules目录下的electron目录">2. 删除package.json中的electron依赖，如果执行过<code>npm install</code>的话，还需要删除 node_modules目录下的electron目录</h5><h5 id="3-执行npm-install命令，这样会把除electron外的依赖全都安装好了">3. 执行<code>npm install</code>命令，这样会把除electron外的依赖全都安装好了</h5><h5 id="4-我们最后安装electron">4. 我们最后安装electron</h5><pre class="language-none"><code class="language-none">ELECTRON_MIRROR=http://npm.taobao.org/mirrors/electron/ npm install --save-dev electron</code></pre><p>因为electron的包比较大，所以还会卡install.js 不过我们看一下网速就发现，速度已经提上去了，20s左右就安装好了</p>]]></content>
<categories>
<category> 前端 </category>
</categories>
</entry>
<entry>
<title>nuxt使用element ui2.x踩坑记录</title>
<link href="/2017/12/01/nuxt%E4%BD%BF%E7%94%A8element-ui2-x%E8%B8%A9%E5%9D%91%E8%AE%B0%E5%BD%95/"/>
<url>/2017/12/01/nuxt%E4%BD%BF%E7%94%A8element-ui2-x%E8%B8%A9%E5%9D%91%E8%AE%B0%E5%BD%95/</url>
<content type="html"><![CDATA[<h3 id="1-第一步我们使用nuxt官方提供的starter模板创建项目">1. 第一步我们使用nuxt官方提供的starter模板创建项目</h3><pre class="language-bash" data-language="bash"><code class="language-bash">vue init nuxt-community/starter-template <span class="token operator"><</span>project-name<span class="token operator">></span></code></pre><h3 id="2-安装相关依赖">2. 安装相关依赖</h3><pre class="language-bash" data-language="bash"><code class="language-bash"><span class="token builtin class-name">cd</span> <span class="token operator"><</span>project-name<span class="token operator">></span><span class="token function">npm</span> <span class="token function">install</span></code></pre><h3 id="3-安装element-ui-v2-0-7">3. 安装element-ui (v2.0.7)</h3><pre class="language-bash" data-language="bash"><code class="language-bash"><span class="token function">npm</span> i element-ui <span class="token parameter variable">-S</span><span class="token function">npm</span> <span class="token function">install</span> babel-plugin-component <span class="token parameter variable">-D</span></code></pre><h3 id="4-我们需要配置一下nuxt-config-js">4. 我们需要配置一下nuxt.config.js</h3><pre class="language-javascript" data-language="javascript"><code class="language-javascript">module<span class="token punctuation">.</span>exports <span class="token operator">=</span> <span class="token punctuation">{</span> <span class="token comment">/* ** Headers of the page */</span> <span class="token literal-property property">head</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token literal-property property">title</span><span class="token operator">:</span> <span class="token string">'web'</span><span class="token punctuation">,</span> <span class="token literal-property property">meta</span><span class="token operator">:</span> <span class="token punctuation">[</span> <span class="token punctuation">{</span> <span class="token literal-property property">charset</span><span class="token operator">:</span> <span class="token string">'utf-8'</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">{</span> <span class="token literal-property property">name</span><span class="token operator">:</span> <span class="token string">'viewport'</span><span class="token punctuation">,</span> <span class="token literal-property property">content</span><span class="token operator">:</span> <span class="token string">'width=device-width, initial-scale=1'</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">{</span> <span class="token literal-property property">hid</span><span class="token operator">:</span> <span class="token string">'description'</span><span class="token punctuation">,</span> <span class="token literal-property property">name</span><span class="token operator">:</span> <span class="token string">'description'</span><span class="token punctuation">,</span> <span class="token literal-property property">content</span><span class="token operator">:</span> <span class="token string">'Nuxt.js project'</span> <span class="token punctuation">}</span> <span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token literal-property property">link</span><span class="token operator">:</span> <span class="token punctuation">[</span> <span class="token punctuation">{</span> <span class="token literal-property property">rel</span><span class="token operator">:</span> <span class="token string">'icon'</span><span class="token punctuation">,</span> <span class="token literal-property property">type</span><span class="token operator">:</span> <span class="token string">'image/x-icon'</span><span class="token punctuation">,</span> <span class="token literal-property property">href</span><span class="token operator">:</span> <span class="token string">'/favicon.ico'</span> <span class="token punctuation">}</span> <span class="token punctuation">]</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token comment">/* ** Customize the progress bar color */</span> <span class="token literal-property property">loading</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token literal-property property">color</span><span class="token operator">:</span> <span class="token string">'#3B8070'</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token comment">/* ** Build configuration */</span> <span class="token literal-property property">plugins</span><span class="token operator">:</span> <span class="token punctuation">[</span> <span class="token punctuation">{</span> <span class="token literal-property property">src</span><span class="token operator">:</span> <span class="token string">'~plugins/element-ui'</span><span class="token punctuation">,</span> <span class="token literal-property property">ssr</span><span class="token operator">:</span> <span class="token boolean">true</span> <span class="token punctuation">}</span> <span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token literal-property property">css</span><span class="token operator">:</span> <span class="token punctuation">[</span> <span class="token string">'element-ui/lib/theme-chalk/index.css'</span> <span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token literal-property property">build</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token literal-property property">vendor</span><span class="token operator">:</span> <span class="token punctuation">[</span> <span class="token string">'element-ui'</span> <span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token literal-property property">babel</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token string-property property">"plugins"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token punctuation">[</span><span class="token string">"component"</span><span class="token punctuation">,</span> <span class="token punctuation">[</span> <span class="token punctuation">{</span> <span class="token string-property property">"libraryName"</span><span class="token operator">:</span> <span class="token string">"element-ui"</span><span class="token punctuation">,</span> <span class="token string-property property">"styleLibraryName"</span><span class="token operator">:</span> <span class="token string">"theme-chalk"</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token string">'transform-async-to-generator'</span><span class="token punctuation">,</span> <span class="token string">'transform-runtime'</span> <span class="token punctuation">]</span><span class="token punctuation">]</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token literal-property property">comments</span><span class="token operator">:</span> <span class="token boolean">true</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre><h3 id="5-在plugins目录中新建一个element-ui-js文件-文件内容为">5. 在plugins目录中新建一个element-ui.js文件,文件内容为:</h3><pre class="language-javascript" data-language="javascript"><code class="language-javascript"><span class="token keyword">import</span> Vue <span class="token keyword">from</span> <span class="token string">'vue'</span><span class="token keyword">import</span> ElementUI <span class="token keyword">from</span> <span class="token string">'element-ui'</span>Vue<span class="token punctuation">.</span><span class="token function">use</span><span class="token punctuation">(</span>ElementUI<span class="token punctuation">)</span></code></pre><h3 id="6-ok-这样我们的环境就配好了，在pages目录中新建一个login-vue">6. ok,这样我们的环境就配好了，在pages目录中新建一个login.vue</h3> <pre class="language-javascript" data-language="javascript"><code class="language-javascript"><span class="token operator"><</span>template<span class="token operator">></span><span class="token operator"><</span>div<span class="token operator">></span> <span class="token operator"><</span>el<span class="token operator">-</span>button<span class="token operator">></span>默认按钮<span class="token operator"><</span><span class="token operator">/</span>el<span class="token operator">-</span>button<span class="token operator">></span> <span class="token operator"><</span>el<span class="token operator">-</span>button type<span class="token operator">=</span><span class="token string">"primary"</span><span class="token operator">></span>主要按钮<span class="token operator"><</span><span class="token operator">/</span>el<span class="token operator">-</span>button<span class="token operator">></span> <span class="token operator"><</span>el<span class="token operator">-</span>button type<span class="token operator">=</span><span class="token string">"success"</span><span class="token operator">></span>成功按钮<span class="token operator"><</span><span class="token operator">/</span>el<span class="token operator">-</span>button<span class="token operator">></span> <span class="token operator"><</span>el<span class="token operator">-</span>button type<span class="token operator">=</span><span class="token string">"info"</span><span class="token operator">></span>信息按钮<span class="token operator"><</span><span class="token operator">/</span>el<span class="token operator">-</span>button<span class="token operator">></span> <span class="token operator"><</span>el<span class="token operator">-</span>button type<span class="token operator">=</span><span class="token string">"warning"</span><span class="token operator">></span>警告按钮<span class="token operator"><</span><span class="token operator">/</span>el<span class="token operator">-</span>button<span class="token operator">></span> <span class="token operator"><</span>el<span class="token operator">-</span>button type<span class="token operator">=</span><span class="token string">"danger"</span><span class="token operator">></span>危险按钮<span class="token operator"><</span><span class="token operator">/</span>el<span class="token operator">-</span>button<span class="token operator">></span><span class="token operator"><</span><span class="token operator">/</span>div<span class="token operator">></span><span class="token operator"><</span><span class="token operator">/</span>template<span class="token operator">></span><span class="token operator"><</span>script<span class="token operator">></span><span class="token keyword">export</span> <span class="token keyword">default</span> <span class="token punctuation">{</span> <span class="token function">data</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token literal-property property">methods</span><span class="token operator">:</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token punctuation">;</span><span class="token operator"><</span><span class="token operator">/</span>script<span class="token operator">></span><span class="token operator"><</span>style <span class="token operator">></span><span class="token operator"><</span><span class="token operator">/</span>style<span class="token operator">></span></code></pre><h3 id="7-执行npm-run-dev-打开-http-localhost-3000-login，">7.执行<code>npm run dev </code> 打开<code> http://localhost:3000/login</code>，</h3><p>这样就看到一排element-ui样式的按钮了</p>]]></content>
<categories>
<category> 前端 </category>
</categories>
</entry>
<entry>
<title>Redis4.0安装及集群搭建</title>
<link href="/2017/11/29/Redis4-0%E5%AE%89%E8%A3%85%E5%8F%8A%E9%9B%86%E7%BE%A4%E6%90%AD%E5%BB%BA/"/>
<url>/2017/11/29/Redis4-0%E5%AE%89%E8%A3%85%E5%8F%8A%E9%9B%86%E7%BE%A4%E6%90%AD%E5%BB%BA/</url>
<content type="html"><![CDATA[<p>环境:Deepin15 (Debian8)</p><h2 id="Redis4-0安装">Redis4.0安装</h2><h3 id="1-下载redis">1. 下载redis</h3><pre class="language-none"><code class="language-none">wget http://download.redis.io/releases/redis-4.0.2.tar.gz</code></pre><h3 id="2-编译安装">2. 编译安装</h3><pre class="language-none"><code class="language-none">cp ~/redis-4.0.2.tar.gz /usr/local/tar -xzvf redis-4.0.2.tar.gzcd redis-4.0.2make</code></pre><p>编译完成后会提示执行 <code>make test</code> 命令，执行后可能报错，提示缺少tcl，那么我们需要安装一下</p><pre class="language-none"><code class="language-none">apt-get install tcl</code></pre><p>不报错之后，执行 <code>make install</code> 命令;这样就安装好了</p><h3 id="3-测试">3. 测试</h3><p>执行 <code>redis-server</code>命令,就采用默认配置启动了,而后，我们执行<code>redis-cli</code>进入交互命令行，可以通过<code>ping</code>命令看看是否安装成功，通过<code>shutdown</code>命令关闭redis服务</p><h2 id="集群配置">集群配置</h2><p>我们在单机上采用三主三从配置</p><h3 id="1-在redis4-0目录下建立cluster文件夹，并建立６个redis-conf文件夹">1. 在redis4.0目录下建立cluster文件夹，并建立６个redis-conf文件夹</h3><pre class="language-none"><code class="language-none">mkdir clustercd clustermkdir 7000 7001 7002 7003 7004 7005cp ../redis.conf 7000/cp ../redis.conf 7001/cp ../redis.conf 7002/cp ../redis.conf 7003/cp ../redis.conf 7004/cp ../redis.conf 7005/</code></pre><h3 id="2-对每个redis-conf文件，修改以下几个条目，注意端口号和nodes文件名称要和文件夹保持一致">2 . 对每个redis.conf文件，修改以下几个条目，注意端口号和nodes文件名称要和文件夹保持一致</h3><pre class="language-none"><code class="language-none">port 7000cluster-enabled yescluster-config-file nodes_7000.confcluster-node-timeout 5000appendonly yesdaemonize yes</code></pre><p>修改完成后切换到每个目录中，执行</p><pre class="language-none"><code class="language-none">redis-server redis.conf</code></pre><p>命令（可以写个bash文件批量处理）</p><h3 id="３-安装ruby环境">３. 安装ruby环境</h3><pre class="language-none"><code class="language-none">apt-get install rubygem install redis</code></pre><h3 id="４-切换到redis-4-0-2的src目录中，执行以下命令">４. 切换到redis-4.0.2的src目录中，执行以下命令</h3><pre class="language-bash" data-language="bash"><code class="language-bash">./redis-trib.rb create <span class="token parameter variable">--replicas</span> <span class="token number">1</span> <span class="token number">127.0</span>.0.1:7000 <span class="token number">127.0</span>.0.1:7001 <span class="token number">127.0</span>.0.1:7002 <span class="token number">127.0</span>.0.1:7003 <span class="token number">127.0</span>.0.1:7004 <span class="token number">127.0</span>.0.1:7005</code></pre><p>执行<code>./redis-trib.rb check 127.0.0.1:7000 </code>来判断测试是否正常<br>最后我们可以通过<code> redis-cli -c -p 7000</code> 来连接集群</p><h3 id="5-外网连接">5. 外网连接</h3><p>注释掉这一行(为了安全,redis默认开启)：<code>bind 127.0.0.1</code><br>调整<code>protected-mode yes</code>为<code>protected-mode no</code><br>同时创建集群时也需要指定本机IP：</p> <pre class="language-bash" data-language="bash"><code class="language-bash">./redis-trib.rb create <span class="token parameter variable">--replicas</span> <span class="token number">1</span> xxx:7000 xxx:7001 xxx:7002 xxx:7003 xxx:7004 xxx:7005</code></pre>]]></content>
<categories>
<category> Redis </category>
</categories>
<tags>
<tag> Redis </tag>
</tags>
</entry>
<entry>
<title>OpenResty 安装记录</title>
<link href="/2017/11/27/OpenResty%E5%AE%89%E8%A3%85%E8%AE%B0%E5%BD%95/"/>
<url>/2017/11/27/OpenResty%E5%AE%89%E8%A3%85%E8%AE%B0%E5%BD%95/</url>
<content type="html"><![CDATA[<p>最近风控项目需要用到OpenResty，于是打算在本地环境中搭建一套，然而搭建过程并没有那么顺利<br>官网的安装步骤很简单</p><h3 id="1-安装相关依赖包">1.安装相关依赖包</h3><pre class="language-none"><code class="language-none">apt-get install libreadline-dev libncurses5-dev libpcre3-dev libssl-dev perl make build-essential</code></pre><h3 id="2-下载源码并解压，然后编译安装">2.下载源码并解压，然后编译安装</h3><pre class="language-none"><code class="language-none">tar -xzvf openresty-VERSION.tar.gzcd openresty-VERSION/./configuremakesudo make install</code></pre><p>我的环境是Deepin系统，Deepin15采用的Debian8内核，按照官网的步骤时，系统会默认安装openssl1.1.0，然而OpenResty１.13版本暂时不支持openssl1.1.0，所以在安装过程中会报错，所以我们需要先安装openssl 1.0.2</p><h3 id="0-如果本机已经有openssl了，需要先卸载">0. 如果本机已经有openssl了，需要先卸载</h3><pre class="language-none"><code class="language-none">openssl versionapt-get purge opensslrm -rf /etc/ssl #删除配置文件 </code></pre><h3 id="1-到openssl官网下载openssl1-0-2版本">1. 到openssl官网下载openssl1.0.2版本</h3><p><a href="https://www.openssl.org/source/">OpenSSl 1.0.2</a></p><h3 id="2-编译安装">2. 编译安装</h3><pre class="language-none"><code class="language-none">./config --prefix=/usr/local --openssldir=/usr/local/sslmake && make install./config shared --prefix=/usr/local --openssldir=/usr/local/sslmake cleanmake && make install</code></pre><h3 id="3-安装好之后需要将nginx放到环境变量中">3. 安装好之后需要将nginx放到环境变量中</h3><pre class="language-none"><code class="language-none">PATH=/usr/local/openresty/nginx/sbin:$PATHexport PATH</code></pre>]]></content>
<categories>
<category> 网关 </category>
</categories>
</entry>
<entry>
<title>git常用命令</title>
<link href="/2017/10/11/git%E5%B8%B8%E7%94%A8%E5%91%BD%E4%BB%A4/"/>
<url>/2017/10/11/git%E5%B8%B8%E7%94%A8%E5%91%BD%E4%BB%A4/</url>
<content type="html"><![CDATA[<h4 id="更新分支信息">更新分支信息</h4><pre class="language-git" data-language="git"><code class="language-git">git fetch origin --prune</code></pre><h4 id="设置远端分支-远端分支存在">设置远端分支(远端分支存在)</h4><pre class="language-git" data-language="git"><code class="language-git">git branch -u origin/[远程分支]</code></pre><h4 id="设置远端分支-远端分支不存在">设置远端分支(远端分支不存在)</h4><pre class="language-git" data-language="git"><code class="language-git">git push origin [远程分支]</code></pre>]]></content>
<categories>
<category> 工具 </category>
</categories>
</entry>
<entry>
<title>SpringBoot部署war包到tomcat</title>
<link href="/2017/09/26/SpringBoot%E9%83%A8%E7%BD%B2war%E5%8C%85%E5%88%B0tomcat/"/>
<url>/2017/09/26/SpringBoot%E9%83%A8%E7%BD%B2war%E5%8C%85%E5%88%B0tomcat/</url>
<content type="html"><![CDATA[<h4 id="pom文件中打包方式由jar修改为war">pom文件中打包方式由jar修改为war</h4><pre class="language-none"><code class="language-none"><packaging>war</packaging></code></pre><h4 id="将内置tomcat设为provided">将内置tomcat设为provided</h4><pre class="language-none"><code class="language-none"><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-tomcat</artifactId><scope>provided</scope></dependency></code></pre><h4 id="需要集成SpringBootServletInitializer并重写configure方法">需要集成SpringBootServletInitializer并重写configure方法</h4><pre class="language-java" data-language="java"><code class="language-java"><span class="token keyword">import</span> <span class="token import"><span class="token namespace">org<span class="token punctuation">.</span>springframework<span class="token punctuation">.</span>boot<span class="token punctuation">.</span></span><span class="token class-name">SpringApplication</span></span><span class="token punctuation">;</span><span class="token keyword">import</span> <span class="token import"><span class="token namespace">org<span class="token punctuation">.</span>springframework<span class="token punctuation">.</span>boot<span class="token punctuation">.</span>autoconfigure<span class="token punctuation">.</span></span><span class="token class-name">SpringBootApplication</span></span><span class="token punctuation">;</span><span class="token keyword">import</span> <span class="token import"><span class="token namespace">org<span class="token punctuation">.</span>springframework<span class="token punctuation">.</span>boot<span class="token punctuation">.</span>builder<span class="token punctuation">.</span></span><span class="token class-name">SpringApplicationBuilder</span></span><span class="token punctuation">;</span><span class="token keyword">import</span> <span class="token import"><span class="token namespace">org<span class="token punctuation">.</span>springframework<span class="token punctuation">.</span>boot<span class="token punctuation">.</span>web<span class="token punctuation">.</span>support<span class="token punctuation">.</span></span><span class="token class-name">SpringBootServletInitializer</span></span><span class="token punctuation">;</span><span class="token annotation punctuation">@SpringBootApplication</span><span class="token keyword">public</span> <span class="token keyword">class</span> <span class="token class-name">App</span> <span class="token keyword">extends</span> <span class="token class-name">SpringBootServletInitializer</span> <span class="token punctuation">{</span><span class="token keyword">public</span> <span class="token keyword">static</span> <span class="token keyword">void</span> <span class="token function">main</span><span class="token punctuation">(</span><span class="token class-name">String</span><span class="token punctuation">[</span><span class="token punctuation">]</span> args<span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token class-name">SpringApplication</span><span class="token punctuation">.</span><span class="token function">run</span><span class="token punctuation">(</span><span class="token class-name">App</span><span class="token punctuation">.</span><span class="token keyword">class</span><span class="token punctuation">,</span> args<span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">}</span><span class="token annotation punctuation">@Override</span><span class="token keyword">protected</span> <span class="token class-name">SpringApplicationBuilder</span> <span class="token function">configure</span><span class="token punctuation">(</span><span class="token class-name">SpringApplicationBuilder</span> application<span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token keyword">return</span> application<span class="token punctuation">.</span><span class="token function">sources</span><span class="token punctuation">(</span><span class="token class-name">App</span><span class="token punctuation">.</span><span class="token keyword">class</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre>]]></content>
<categories>
<category> Java </category>
</categories>
</entry>
<entry>
<title>Java序列化的一些问题</title>
<link href="/2017/09/24/Java%E5%BA%8F%E5%88%97%E5%8C%96%E7%9A%84%E4%B8%80%E4%BA%9B%E9%97%AE%E9%A2%98/"/>
<url>/2017/09/24/Java%E5%BA%8F%E5%88%97%E5%8C%96%E7%9A%84%E4%B8%80%E4%BA%9B%E9%97%AE%E9%A2%98/</url>
<content type="html"><![CDATA[<h3 id="1-静态变量不会被序列化">1. 静态变量不会被序列化</h3><p>序列化保存的是对象的状态，静态变量属于类的信息，因此不会被序列化的</p><h3 id="2-父类的序列化与-Transient-关键字">2. 父类的序列化与 Transient 关键字</h3><p>要想将父类对象也序列化，就需要让父类也实现Serializable 接口。如果父类不实现的话的，就需要有默认的无参的构造函数。<br>Transient 关键字的作用是控制变量的序列化，在变量声明前加上该关键字，可以阻止该变量被序列化到文件中，在被反序列化后，transient 变量的值被设为初始值，如 int 型的是 0，对象型的是 null。</p>]]></content>
<categories>
<category> Java </category>
</categories>