BERT CVE Model doesn't optimally converge over epochs

[dipanjanS]

Current BERT model for probable CVEs works decently but some how doesn't seem to converge optimally as we increase the epochs.

False positives suddenly increase\fluctuate over epochs. The idea is to:

• Remove data bias
• Train with lower learning rate, better optimizers (https://arxiv.org/abs/1904.00962v1)
• Validate predictions of trained model with last 2-3 weeks of triaged data

This would be a blocker into retraining the BERT model also because we need to know some level of optimal configuration before we start retraining