Supervised or Semi-Supervised machine learning for deduplication

[maxkadel]

Introduction

The team has been asked by leadership to look into possible applications of AI (writ large) in the library.

Problem Statement

Right now, the catalog includes Princeton records from Alma, and partner records from SCSB. There can be duplication between these collections, which can be confusing for users of the Catalog.

Can we identify when a SCSB record is a duplicate of a record in Alma using supervised or semi-supervised machine learning?

Acceptance criteria

• If there is no research directory, create one.

• Update an existing markdown document or add a new one in the research directory.

• It has introduction: Explains the goals and purpose of this research work.

• It lists methods: Describe what you did to research the question.

• It has a conclusion: Includes a summary of what was discovered in the research process.

• It has a step by step list of potential next steps that build upon the research.

• It includes references: References any related resources that have assisted in the research process (links to other tickets, online articles etc.).

• It includes any artifacts (charts, notes, code samples etc.) that were produced during the work.