Finer Beat Timings

[ljendrusch]

Greetings,
First off, incredible tool, thanks for making it available. Awesome job representing KAIST.
I'm doing a project to calculate audio similarity of song segments over a database of songs.
Allin1 has proven invaluable and I've done some heavy testing to prove its boundary detection is very accurate.
Is the output from the Beats head originally 100ths of a second, or is it finer? If it is finer, how could I access the finer timings?
Finer timings would increase my accuracy by a good deal because I'm comparing raw sample arrays.

Example code:

y_a, _ = librosa.load(song_a_path, sr=SR)
y_b, _ = librosa.load(song_b_path, sr=SR)

seg_a = y_a[ a1 : a2 ]
seg_b = y_b[ b1 : b2 ]

# normalize BMP
seg_a = librosa.effects.timestretch( seg_a, rate = song_b_BPM / song_a_BPM )

# calculate highest cosine similarity by sliding the shorter segment
# across slices of the larger segment, aligning on downbeats
scores = []
for downbeat_a in seg_a_downbeats:
    score = cosine_similarity_of( seg_a[ downbeat_a : ], seg_b )
    scores.append( score )
    # with some other stuff to ensure same-size sample arrays and align properly
return max(scores)

I'm getting segment bounds a1, a2, b1, b2 and downbeats from allin1 AnalysisResults.
But since seg_a and seg_b are 44.1khz I'm afraid aligning on 100ths of seconds will have a large effect on accuracy.
Any help or advice you may provide would be greatly appreciated. Again, awesome model, thanks for your hard work.

[tae-jun]

Hi, I'm glad that you use this model 🤩

Sadly, the temporal resolution is fixed to 100 FPS (frames per second).

In detail, it extracts 100 embeddings per second and then they are fed into DBNDownBeatTrackingProcessor, which finally decides the positions of beats and downbeats.

But, I think 100 FPS should be enough for many cases so I'm curious why you need higher temporal resolution.

Maybe we can think together and find another solution.

Thanks!