Code

Feature/clip similarity filter#69

Code

Merged

zhijianma merged 17 commits intomainmodelscope/data-juicer:mainfrom feature/clip_similarity_filtermodelscope/data-juicer:feature/clip_similarity_filterCopy head branch name to clipboardNov 15, 2023

+418-10

Commits

Commits on Nov 13, 2023

• feature: add clip_similarity_filter

  zhijianma

  committed
  ceada34

• docs: pre-commit style

  zhijianma

  committed
  a293e71

• fix: update clip model for test

  zhijianma

  committed
  80381f4

• chore: add torch

  zhijianma

  committed
  d986ffa

Commits on Nov 14, 2023

• docs: add default clip model

  zhijianma

  committed
  af7101d

• 1. truncation text inputs.

  Show description for a7e2b2e
  zhijianma

  committed
  a7e2b2e

• change num_proc to 1 in test

  zhijianma

  committed
  7415502

• change num_proc to 1 in test

  zhijianma

  committed
  6c48771

• Merge branch 'main' into feature/clip_similarity_filter

  zhijianma

  committed
  f2d4a8c

• typo: merge pre-commit

  zhijianma

  committed
  000ed35

• docs: update filter number in Operators_ZH.md

  zhijianma

  committed
  8288a4a

• fixz: set torch num threads to 1

  zhijianma

  committed
  74a094e

Commits on Nov 15, 2023

• style: change param names

  zhijianma

  committed
  c84d1fa

• typo

  zhijianma

  committed
  57cc584

• Merge branch 'main' into feature/clip_similarity_filter

  zhijianma

  committed
  ec6b978

• docs: update filter numbers

  zhijianma

  committed
  87a8675

• sytle: pre-commit filter

  zhijianma

  committed
  551fa6e