Skip to content

Commit

Permalink
Read preskip and gain from codec private
Browse files Browse the repository at this point in the history
Looks like (unlike WebM) MP4 doesn't use `OpusHead` description from the encoder directly and has its own `dOps` format that differs in byte layout.

So far preskip was hard-coded to 3840. If there's a significant difference in the value in the `OpusHead` private data from the encoder, can lead to more than necessary data being cut off from the beginning and possibly a slight desync with video.

Effects of the shift can be observed in this example: https://codepen.io/brainshave/pen/QWXxKmW. It compares original source with `webm-muxer` (5.0.0) and `mp4-muxer` (5.0.4). WebM file is moslty in sync (but still not prefect but this could be either specific to decoder or encoder) while MP4 is shifted 2880 samples left.

To fix this, add reading of the `OpusHead` format for creating the `dOps` box. Apart from preskip this also adds handling for gain field.

Being conservative WRT other fields like number of channels and sample rate that are be provided from other sources. Also, keeping the original hard-coded values if the `description` field is missing to avoid regressions.
  • Loading branch information
szymonrw committed Sep 2, 2024
1 parent d572599 commit cc8fa1d
Show file tree
Hide file tree
Showing 5 changed files with 73 additions and 40 deletions.
37 changes: 23 additions & 14 deletions build/mp4-muxer.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions build/mp4-muxer.min.js

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions build/mp4-muxer.min.mjs

Large diffs are not rendered by default.

37 changes: 23 additions & 14 deletions build/mp4-muxer.mjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

31 changes: 23 additions & 8 deletions src/box.ts
Original file line number Diff line number Diff line change
Expand Up @@ -445,14 +445,29 @@ export const esds = (track: Track) => {
};

/** Opus Specific Box. */
export const dOps = (track: AudioTrack) => box('dOps', [
u8(0), // Version
u8(track.info.numberOfChannels), // OutputChannelCount
u16(3840), // PreSkip, should be at least 80 milliseconds worth of playback, measured in 48000 Hz samples
u32(track.info.sampleRate), // InputSampleRate
fixed_8_8(0), // OutputGain
u8(0) // ChannelMappingFamily
]);
export const dOps = (track: AudioTrack) => {
// Default PreSkip, should be at least 80 milliseconds worth of playback, measured in 48000 Hz samples
let preskip = 3840;
let gain = 0;

// Read preskip and from codec private data from the encoder
// https://www.rfc-editor.org/rfc/rfc7845#section-5
const description = track.info.decoderConfig.description;
if (description) {
const view = new DataView(ArrayBuffer.isView(description) ? description.buffer : description);
preskip = view.getUint16(10, true);
gain = view.getInt16(14, true);
}

return box('dOps', [
u8(0), // Version
u8(track.info.numberOfChannels), // OutputChannelCount
u16(preskip),
u32(track.info.sampleRate), // InputSampleRate
fixed_8_8(gain), // OutputGain
u8(0) // ChannelMappingFamily
]);
};

/**
* Time-To-Sample Box: Stores duration information for a media's samples, providing a mapping from a time in a media
Expand Down

0 comments on commit cc8fa1d

Please sign in to comment.