Optionally dropping frames

[fjcaetano]

My application needed to create a "slideshow" of three frames to be shown with one second each and those frames should be captured one second after the previous. I "achieved" it with the following:

recordSession.suggestedMaxRecordDuration = CMTimeMake(299, 100)
recordSession.videoMaxFrameRate = 1
recordSession.videoTimeScale = 3

However, it captured 3 frames, extended their output length to 1 second, and thus, created a 3 second video with three frames of one second each, however, the recording took a fraction of a second. The frames weren't being captured with one second distance of each other.

My solution adds the boolean property shouldDropFrames to SCRecordSession. It's default value is YES, so nothing needs to be changed for the default behavior to continue, however, if this property is set to NO, the session will drop the frames until the full length of the recording is over.

I had to customize the framework in order to achieve what I needed. I do realise this behavior may not be intended, but, nevertheless, I thought it would be interesting to submit a pull-request for your appreciation.

[rFlex]

This implementation breaks the videoTimeScale property if shouldDropFrames is not set to true. I believe you can very easily implement that feature by doing the following on the latest SCRecordSession impl. Could you tell me if that worked?

if (_videoTimeScale != 1.0) {
      computedFrameDuration = CMTimeMultiplyByFloat64(computedFrameDuration, _videoTimeScale);
      _timeOffset = CMTimeAdd(_timeOffset, CMTimeSubtract(duration, computedFrameDuration));
}

// Here is the condition to add        
// If the computed time of the buffer is less than the lastTime (which is presentation time + duration), skip the buffer
if (_shouldDropFrame && CMTIME_COMPARE_INLINE(lastTimeVideo, <, _lastTimeVideo)) {
    return NO;
}

lastTimeVideo = CMTimeAdd(lastTimeVideo, computedFrameDuration);

[fjcaetano]

@rFlex I'm sorry for the delay. I couldn't came back to this till now. Unfortunately, your solution doesn't work for me. What I need is a 3 second long video with 1 fps, but the recording must also last 3 seconds. That is the main issue, making the recording last those 3 seconds.

Your solution brings two issues:

• It may not have been clear, but the default behavior has shouldDropFrames set to YES, as was previously documented in the videoMaxFrameRate property.
• lastTimeVideo must not appended computedFrameDuration if the user explicitly states that shouldDropFrames = NO because it interferes with how the library computes if the recording should be completed or not. This is the main issue. It was easy to create a 3s video with 1fps, the hard part was making the recording last 3s.
• Last, but not least, because of reason 2, I can't simply compare lastTimeVideo to _lastTimeVideo since the last one will only change when the buffer gets appended. I explicitly compared it to the calculated offset to forcefully append the buffer when the offset surpasses the elapsed time.

I'll try to reproduce the issue you mentioned with videoTimeScale, but, as I said before, I do understand if you cannot merge this pull-request because it may not be what the library seeks to do.

[fjcaetano]

By the way, in this new implementation, setting a different value to maxFrameRate in SCVideoConfiguration has no effect at all. I searched for occurrences if maxFrameRate and I realised it is not being used.

[fjcaetano]

I got it, and updated to the new updates. The SCVideoConfiguration's maxFrameRate really wasn't being used. Now I've set it to work as the maximum input framerate accepted by the library.

Conceptually, we could translate videoTimeScale as being the inverse proportion of how long the recording should be extended. If it's set to 0.5, the recording should take twice the maxRecordDuration hence the whole think will be shrunk to half.

When working together, maxFrameRate sets how many fps should be appended to the buffer while recording. If set to 10, approximately one every three frames will be appended. If we set videoTimescale to 1/3, a 5 second recording will take 15 seconds to record and produce a 30fps video of 15 seconds shrunk into 5 seconds.

I had to discard the old implementation because the updates since then and the necessary fixes were to large to consider.

[MMasterson]

Hey @rFlex

was this ever merged? or implemented? I'm looking for a way to drop frames for a very similar effect as @fjcaetano

[fjcaetano]

There are so many commits after this that I don't think this would be wise.

@MMasterson, perhaps you could cherry-pick this out of my fork?