ioc scan business logic #5

eirsep · 2024-06-12T22:57:59Z

Description

[Describe what this change achieves]

Issues Resolved

[List any issues this PR will resolve]

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Chase Engelbrecht <[email protected]>

* notification for alerting in correlation * correlation alerts mapping change * working code Signed-off-by: Riya Saxena <[email protected]> * alertsInCorrelation without notifciations Signed-off-by: Riya Saxena <[email protected]> * alertsInCorrelation without notifciations Signed-off-by: Riya Saxena <[email protected]> * alertsInCorrelation without notifciations Signed-off-by: Riya Saxena <[email protected]> * alerts in correlations notification service added Signed-off-by: Riya Saxena <[email protected]> * addressing the comments Signed-off-by: Riya Saxena <[email protected]> * addressing the comments Signed-off-by: Riya Saxena <[email protected]> * address the design changes discussed Signed-off-by: Riya Saxena <[email protected]> * address the design changes discussed Signed-off-by: Riya Saxena <[email protected]> * fixed tests Signed-off-by: Riya Saxena <[email protected]> --------- Signed-off-by: Riya <[email protected]> Signed-off-by: Riya Saxena <[email protected]>

* notification for alerting in correlation * correlation alerts mapping change * working code Signed-off-by: Riya Saxena <[email protected]> * alertsInCorrelation without notifciations Signed-off-by: Riya Saxena <[email protected]> * alertsInCorrelation without notifciations Signed-off-by: Riya Saxena <[email protected]> * alertsInCorrelation without notifciations Signed-off-by: Riya Saxena <[email protected]> * alerts in correlations notification service added Signed-off-by: Riya Saxena <[email protected]> * addressing the comments Signed-off-by: Riya Saxena <[email protected]> * addressing the comments Signed-off-by: Riya Saxena <[email protected]> * getCorrelationAlerts API changes Signed-off-by: Riya Saxena <[email protected]> * APIs added for Alerts in Correlations Signed-off-by: Riya Saxena <[email protected]> * update alerts with an errorMessage when correlationRule is deleted Signed-off-by: Riya Saxena <[email protected]> * address the design changes discussed Signed-off-by: Riya Saxena <[email protected]> * address the design changes discussed Signed-off-by: Riya Saxena <[email protected]> * fixed tests Signed-off-by: Riya Saxena <[email protected]> * minor fixes due to merge Signed-off-by: Riya Saxena <[email protected]> * alerts API changes Signed-off-by: Riya Saxena <[email protected]> * klint fixes Signed-off-by: Riya Saxena <[email protected]> * license headers added Signed-off-by: Riya Saxena <[email protected]> * fixed format violations Signed-off-by: Riya Saxena <[email protected]> --------- Signed-off-by: Riya <[email protected]> Signed-off-by: Riya Saxena <[email protected]>

Signed-off-by: Riya Saxena <[email protected]>

* Removed unused imports. Removed redundant helper function. Signed-off-by: AWSHurneyt <[email protected]> * Added note about system index refactoring. Signed-off-by: AWSHurneyt <[email protected]> * Implemented draft of IocService. Signed-off-by: AWSHurneyt <[email protected]> * Made changes based on PR feedback. Signed-off-by: AWSHurneyt <[email protected]> * Fixed test helper function. Signed-off-by: AWSHurneyt <[email protected]> * Removed unused imports. Signed-off-by: AWSHurneyt <[email protected]> * Adjusted mappings based on PR feedback. Signed-off-by: AWSHurneyt <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]>

…#1073) * wip index monitor still fails * fix remote monitor setup in security-analytics Signed-off-by: Subhobrata Dey <[email protected]> * wip threat intel trigger * add remote monitor triggers Signed-off-by: Surya Sashank Nistala <[email protected]> --------- Signed-off-by: Subhobrata Dey <[email protected]> Signed-off-by: Surya Sashank Nistala <[email protected]> Co-authored-by: Subhobrata Dey <[email protected]>

* source and store Signed-off-by: Joanne Wang <[email protected]> * search feeds api Signed-off-by: Joanne Wang <[email protected]> * cleanup Signed-off-by: Joanne Wang <[email protected]> * address comments Signed-off-by: Joanne Wang <[email protected]> * rest of comments --------- Signed-off-by: Joanne Wang <[email protected]>

* delete api Signed-off-by: Joanne Wang <[email protected]> * clean up Signed-off-by: Joanne Wang <[email protected]> * delete api integ test Signed-off-by: Joanne Wang <[email protected]> * added validation logic Signed-off-by: Joanne Wang <[email protected]> * respond to comments Signed-off-by: Joanne Wang <[email protected]> * fix merge conflicts Signed-off-by: Joanne Wang <[email protected]> * fix merge conflicts Signed-off-by: Joanne Wang <[email protected]> --------- Signed-off-by: Joanne Wang <[email protected]>

* Removed unused imports. Removed redundant helper function. Signed-off-by: AWSHurneyt <[email protected]> * Added note about system index refactoring. Signed-off-by: AWSHurneyt <[email protected]> * Implemented draft of IocService. Signed-off-by: AWSHurneyt <[email protected]> * Made changes based on PR feedback. Signed-off-by: AWSHurneyt <[email protected]> * Fixed test helper function. Signed-off-by: AWSHurneyt <[email protected]> * Removed unused imports. Signed-off-by: AWSHurneyt <[email protected]> * Adjusted mappings based on PR feedback. Signed-off-by: AWSHurneyt <[email protected]> * Continuation of fetch IOC service implementation. Signed-off-by: AWSHurneyt <[email protected]> * Continuation of fetch IOC service implementation. Signed-off-by: AWSHurneyt <[email protected]> * Implemented ListtIOCs API. Signed-off-by: AWSHurneyt <[email protected]> * Removed "enabled" field from ListIOCs API as that will not be configured at the IOC level. Signed-off-by: AWSHurneyt <[email protected]> * Renamed response keys. Signed-off-by: AWSHurneyt <[email protected]> * Removed "enabled" field mapping as that will not be configured at the IOC level. Signed-off-by: AWSHurneyt <[email protected]> * Updated fetch service. Signed-off-by: AWSHurneyt <[email protected]> * Removed ListIOCs API assets. Those will be included in separate PR. Signed-off-by: AWSHurneyt <[email protected]> * Updated IOC mappings. Signed-off-by: AWSHurneyt <[email protected]> * Removed unused import. Signed-off-by: AWSHurneyt <[email protected]> * Refactored NO_VERSION. Signed-off-by: AWSHurneyt <[email protected]> * Removed dev logs. Signed-off-by: AWSHurneyt <[email protected]> * Removed TODO. Signed-off-by: AWSHurneyt <[email protected]> * Added junit-jupiter dependency so EnabledIfSystemProperty annotation can be used to disable S3-related integ tests. Signed-off-by: AWSHurneyt <[email protected]> * Removed dev code. Signed-off-by: AWSHurneyt <[email protected]> * Added bug fix TODO. Signed-off-by: AWSHurneyt <[email protected]> * Added support for generating test IOCs of a specific type. Signed-off-by: AWSHurneyt <[email protected]> * Refactored factory used for connecting to S3. Added duration to fetchIOC response. Signed-off-by: AWSHurneyt <[email protected]> * Added integ test for fetching from s3. Signed-off-by: AWSHurneyt <[email protected]> * Fixed indexExists check. Signed-off-by: AWSHurneyt <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]>

* Removed unused imports. Removed redundant helper function. Signed-off-by: AWSHurneyt <[email protected]> * Added note about system index refactoring. Signed-off-by: AWSHurneyt <[email protected]> * Implemented draft of IocService. Signed-off-by: AWSHurneyt <[email protected]> * Made changes based on PR feedback. Signed-off-by: AWSHurneyt <[email protected]> * Fixed test helper function. Signed-off-by: AWSHurneyt <[email protected]> * Removed unused imports. Signed-off-by: AWSHurneyt <[email protected]> * Adjusted mappings based on PR feedback. Signed-off-by: AWSHurneyt <[email protected]> * Continuation of fetch IOC service implementation. Signed-off-by: AWSHurneyt <[email protected]> * Implemented ListtIOCs API. Signed-off-by: AWSHurneyt <[email protected]> * Removed "enabled" field from ListIOCs API as that will not be configured at the IOC level. Signed-off-by: AWSHurneyt <[email protected]> * Renamed response keys. Signed-off-by: AWSHurneyt <[email protected]> * Removed "enabled" field mapping as that will not be configured at the IOC level. Signed-off-by: AWSHurneyt <[email protected]> * Added feedId as a filter for LiistIOCs API. Added handling for IndexNotFoundException when calling ListIOCs API. Signed-off-by: AWSHurneyt <[email protected]> * Implemented ListtIOCs API. Signed-off-by: AWSHurneyt <[email protected]> * Removed "enabled" field from ListIOCs API as that will not be configured at the IOC level. Signed-off-by: AWSHurneyt <[email protected]> * Renamed response keys. Signed-off-by: AWSHurneyt <[email protected]> * Removed unused test suite. Signed-off-by: AWSHurneyt <[email protected]> * Added feedId as a filter for LiistIOCs API. Added handling for IndexNotFoundException when calling ListIOCs API. Signed-off-by: AWSHurneyt <[email protected]> * Added feedId as a filter for ListIOCs API. Signed-off-by: AWSHurneyt <[email protected]> * Fixed merge conflict. Signed-off-by: AWSHurneyt <[email protected]> * Removed unused test suite. Signed-off-by: AWSHurneyt <[email protected]> * Fixed test case. Signed-off-by: AWSHurneyt <[email protected]> * Fixed test index mappings. Signed-off-by: AWSHurneyt <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]>

Signed-off-by: Subhobrata Dey <[email protected]>

…ct#1078) * refresh and update Signed-off-by: Joanne Wang <[email protected]> * clean up Signed-off-by: Joanne Wang <[email protected]> * change ID generation Signed-off-by: Joanne Wang <[email protected]> * comments Signed-off-by: Joanne Wang <[email protected]> * index create state and other comments Signed-off-by: Joanne Wang <[email protected]> * set states outside func Signed-off-by: Joanne Wang <[email protected]> * renamed model fields Signed-off-by: Joanne Wang <[email protected]> * lowercase s Signed-off-by: Joanne Wang <[email protected]> * added TODOs Signed-off-by: Joanne Wang <[email protected]> * respond to TODOs Signed-off-by: Joanne Wang <[email protected]> * remove file Signed-off-by: Joanne Wang <[email protected]> --------- Signed-off-by: Joanne Wang <[email protected]>

… names (opensearch-project#1080) * Implemented logic to update the IocStoreConfig with the saTifSourceConfig ID and IOC index names. Signed-off-by: AWSHurneyt <[email protected]> * Removed unused test suite. Signed-off-by: AWSHurneyt <[email protected]> * Added configId to error logs. Signed-off-by: AWSHurneyt <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]>

* fix mappings Signed-off-by: Joanne Wang <[email protected]> * comment Signed-off-by: Joanne Wang <[email protected]> * fix comment Signed-off-by: Joanne Wang <[email protected]> * added java doc and todo Signed-off-by: Joanne Wang <[email protected]> * remove duplicate index names from mapping Signed-off-by: Joanne Wang <[email protected]> --------- Signed-off-by: Joanne Wang <[email protected]>

Signed-off-by: Joanne Wang <[email protected]>

* fix integ test Signed-off-by: Joanne Wang <[email protected]> * fix mapping Signed-off-by: Joanne Wang <[email protected]> * add todo Signed-off-by: Joanne Wang <[email protected]> * change user type Signed-off-by: Joanne Wang <[email protected]> * change state and type to keyword Signed-off-by: Joanne Wang <[email protected]> * minor refactoring Signed-off-by: Joanne Wang <[email protected]> * fix existing tests Signed-off-by: Joanne Wang <[email protected]> * add serialization tests for tifsource config object Signed-off-by: Joanne Wang <[email protected]> --------- Signed-off-by: Joanne Wang <[email protected]>

* Moved "feed" variables from generic STIX2 model in SA-commons to STIX2IOC model as those variables are specific to security analytics functionality. Added feedName variables to STIX2IOC. Signed-off-by: AWSHurneyt <[email protected]> * Moved "feedId" variables back to generic STIX2 model in SA-commons. Moved "feedName" variables to generic STIX2 model in SA-commons. Signed-off-by: AWSHurneyt <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]>

…#1085) * Addressing PR comments. Signed-off-by: AWSHurneyt <[email protected]> * Removed IOC type from the search bar param since we will offer a filter for it. Signed-off-by: AWSHurneyt <[email protected]> * Made feedId, and type params of ListIOCsActionRequest support lists of strings. Signed-off-by: AWSHurneyt <[email protected]> * Addressed PR feedback. Signed-off-by: AWSHurneyt <[email protected]> * Implemented DetailedSTIX2IOCDto for ListIOCs API. Signed-off-by: AWSHurneyt <[email protected]> * DetailedSTIX2IOCDto no longer extends STIX2IOCDto. Signed-off-by: AWSHurneyt <[email protected]> * Implemented basic unit tests for DetailedSTIX2IOCDto data model. Signed-off-by: AWSHurneyt <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]>

* Implemented API to test s3 connection. Signed-off-by: AWSHurneyt <[email protected]> * Fixed comment. Signed-off-by: AWSHurneyt <[email protected]> * Updated permissions for communication with S3. Signed-off-by: AWSHurneyt <[email protected]> * Refactored TestS3ConnectionRequest to parse from an S3Source. Improved error message handling for failed connection attempts. Implemented integ tests. Signed-off-by: AWSHurneyt <[email protected]> * Removed unnecessary permissions from policy file. Signed-off-by: AWSHurneyt <[email protected]> * Revised S3 connection URI, and ListIOC API URI. Signed-off-by: AWSHurneyt <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]>

Signed-off-by: Subhobrata Dey <[email protected]>

…generic entity to re-use for threat intel alert Signed-off-by: Surya Sashank Nistala <[email protected]>

Signed-off-by: Surya Sashank Nistala <[email protected]>

Signed-off-by: Joanne Wang <[email protected]>

…ct#1094) * wip Signed-off-by: Joanne Wang <[email protected]> * comments Signed-off-by: Joanne Wang <[email protected]> * working Signed-off-by: Joanne Wang <[email protected]> * delete ioc indices for delete api Signed-off-by: Joanne Wang <[email protected]> * working rn Signed-off-by: Joanne Wang <[email protected]> * cleanup Signed-off-by: Joanne Wang <[email protected]> * comments Signed-off-by: Joanne Wang <[email protected]> --------- Signed-off-by: Joanne Wang <[email protected]>

Signed-off-by: Surya Sashank Nistala <[email protected]>

* add search ioc findings api Signed-off-by: Subhobrata Dey <[email protected]> add search ioc findings api Signed-off-by: Subhobrata Dey <[email protected]> add search ioc findings api Signed-off-by: Subhobrata Dey <[email protected]> add search ioc findings api Signed-off-by: Subhobrata Dey <[email protected]> * fix review comments for ioc findings api Signed-off-by: Subhobrata Dey <[email protected]> --------- Signed-off-by: Subhobrata Dey <[email protected]>

Signed-off-by: Surya Sashank Nistala <[email protected]>

.Threatintel security

engechas and others added 4 commits June 10, 2024 14:58

Add 2.15 release notes (opensearch-project#1061)

c3476fe

Signed-off-by: Chase Engelbrecht <[email protected]>

updates the release notes 2.15 (opensearch-project#1070)

362f0d6

Signed-off-by: Riya Saxena <[email protected]>

eirsep force-pushed the tim branch from f2229e2 to 3b2f65b Compare June 13, 2024 00:01

AWSHurneyt and others added 4 commits June 12, 2024 17:31

eirsep force-pushed the tim branch from 3b2f65b to 2b7ea96 Compare June 14, 2024 01:09

eirsep force-pushed the tim branch 2 times, most recently from 7eedfda to 8c2c428 Compare June 19, 2024 07:44

AWSHurneyt and others added 5 commits June 19, 2024 00:44

pass integ tests (opensearch-project#1082)

b99121e

Signed-off-by: Subhobrata Dey <[email protected]>

fix (opensearch-project#1086)

8a6a389

eirsep force-pushed the tim branch 5 times, most recently from 4fd974c to ec14132 Compare June 20, 2024 10:46

eirsep force-pushed the tim branch from 3015ee1 to 2a4a3fd Compare June 20, 2024 21:06

Change search config api request (opensearch-project#1089)

1927009

Signed-off-by: Joanne Wang <[email protected]>

eirsep force-pushed the tim branch 3 times, most recently from 8ef1f30 to c6c7ab2 Compare June 21, 2024 17:19

AWSHurneyt added 3 commits June 21, 2024 15:03

eirsep force-pushed the tim branch from 89e85f9 to e370d5d Compare June 25, 2024 00:18

eirsep and others added 5 commits June 25, 2024 00:37

ioc scan business logic

f4bc11a

add search ioc findings api

28b8ff4

Signed-off-by: Subhobrata Dey <[email protected]>

refactor iocFinding model and service to pull out CRUD operations to …

ab94cfe

…generic entity to re-use for threat intel alert Signed-off-by: Surya Sashank Nistala <[email protected]>

threat intel alert model and crud operations

127bde5

Signed-off-by: Surya Sashank Nistala <[email protected]>

threat intel trigger execution logic

2e072f0

eirsep force-pushed the tim branch from e370d5d to 2e072f0 Compare June 25, 2024 07:41

jowg-amazon and others added 7 commits June 25, 2024 10:42

fix user mappings (opensearch-project#1095)

4d5df3a

Signed-off-by: Joanne Wang <[email protected]>

Merge remote-tracking branch 'origin/main' into tim

f8bab1e

fix threat intel monitor request in indexing flow

ad80024

Signed-off-by: Surya Sashank Nistala <[email protected]>

Merge remote-tracking branch 'origin/feature/threat_intel' into tim

34bc5a2

wire in ioc findings

07d5c13

eirsep force-pushed the tim branch from 4b814de to 07d5c13 Compare June 26, 2024 11:41

eirsep added 2 commits June 26, 2024 18:26

get threat intel monitor alerts API

721abca

Signed-off-by: Surya Sashank Nistala <[email protected]>

revert commented out code

784d9fa

Signed-off-by: Surya Sashank Nistala <[email protected]>

eirsep pushed a commit that referenced this pull request Jul 10, 2024

Merge pull request #5 from eirsep/threatintel_security

03edc5d

.Threatintel security

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ioc scan business logic #5

ioc scan business logic #5

eirsep commented Jun 12, 2024

ioc scan business logic #5

Are you sure you want to change the base?

ioc scan business logic #5

Conversation

eirsep commented Jun 12, 2024

Description

Issues Resolved

Check List