Enhancement: Explicit frame rate control

[Bgimet]

Hi,

Good job for this plugin. One good thing will be to be able to change the framerate of the stream send, is that possible in any ways ? Like 50/60hz will be great.

Thanks !

[keijiro]

It's possible by changing the refresh rate of your local computer via the display adaptor settings.

[Bgimet]

Hi,

I'm in 1920x1080 60hz on my display adaptator and the stream received from NewTek Studios Monitor is 1080p 29.97. The framerate is not the same...

[keijiro]

It also depends on the frame rate of Unity. Do your project keep 60 fps?

[Bgimet]

Yes, according to my stats, my project is always upper than 80fps.

[twosky2000]

The setting can be set in "void sendFrame" sender.h.
29.97 is default by the ndi sdk. Read the sdk documentation for infos. The below code is for 50fps

        void sendFrame(void* data, int width, int height, uint32_t fourCC)
        {
            static NDIlib_video_frame_v2_t frame;

            frame.xres = width;
            frame.yres = height;
            frame.FourCC = static_cast<NDIlib_FourCC_type_e>(fourCC);
            frame.frame_format_type = NDIlib_frame_format_type_progressive;
            frame.frame_rate_D = 1200;
            frame.frame_rate_N = 60000;
            frame.p_data = static_cast<uint8_t*>(data);
            frame.line_stride_in_bytes = width * 2;

            NDIlib_send_send_video_async_v2(instance_, &frame);
        }

Be mindfull with the settings. I have some problems, but also changed a lot of other things..

[keijiro]

I thought that the frame rate setting is only meaningful when clock_video is true. I disabled video clocking so that I ignored the frame rate setting.

When I tested the plugin with NDI Monitor and OBS, it worked at 60 fps. I wonder how I can reproduce the problem with my environment.

[twosky2000]

I don't have much knowledge. Only puzzle pieces together.
I don't see a fps display directly , i interpreted it from the window title of studio monitor, and that i think is the meta data.
Edit
Confirmed: I rewrote the the SDK C++ receiver tool and with a fps counter an i get about screen refresh rate numbers.
I also made the change with 60hz refresh rate, changed my display settings to 48hz. I receive 48fps in my test receiver. I will test some things an hope to get the fps from unity to the plugin and write the metadata right. Not sure atm how to check the receiver, maybe it scales up?

[keijiro]

I found that the behavior was changed in the latest version (3.5) of the NDI SDK. It also causes a frame freeze issue with the Windows version of Studio Monitor (issue #9).

I changed the sender code to explicitly set the frame rate to 60. This change has been applied in the following update of the plugin.

https://github.com/keijiro/KlakNDI/releases/tag/v0.0.5

This is just a hot fix for the urgent critical issue. I'm thinking of adding a frame rate property to the sender component.

[Bgimet]

Nice, thank you !

I'll try that and send you a feeback.

Regards,
Benjamin

[decowboy]

It would be great if the frame rate property could be added to the sender component.
The statically set framerate of 60 is hurting performance for my specific application.

Love the plugin by the way :)

[keijiro]

@decowboy Just curious but I wonder how you want to use it. Running game at 60fps while sending frames at 30fps?

[decowboy]

Hi @keijiro
I've developed a mini map for (real life) races in Unity, to be used as an overlay on a video stream.
Given that our video broadcast is at 30 fps, we are now running our Unity application at 30fps as well (with a render texture), but we noticed the NDI output is still 60 fps.
I'm thinking the NDI output is now just encoding each frame twice, but that's merely a guess.

In short, we'd be running both the "game" and the NDI at 30 fps.

[keijiro]

@decowboy In that case, I think frames are sent at 30 fps even though the receiver app says it's 60 fps. Anyway, thanks for providing the information about your use case. I'll consider it in future updates.

[teganscott]

Hi @keijiro I am having a similar issue here.
My Unity application is running at 35fps, but I keep getting calls back from NDI saying that "GPU readback error was detected" and "Too many GPU readback requests". Is there a way I can set the NDI recording to 30fps instead of 60fps? thanks!

[ericmarodon]

Hi @keijiro, thanks a lot for this wonderful plugin. It works wonders at 60p... but I'd really need to change the frame rate to 25 fps for video output. Could you tell where to do that in the code, if that's feasible?

I tried changing the setting in sender.h but sources still appear at 60p in Studio Monitor and the video mixing software I use (Vmix), and it creates all sorts of problems, lags, hicups when trying to mix it with 25 fps footage (I'm in Europe) or other framerates.

Thanks a lot, that would be wonderful.

[ericmarodon]

Support for NDI 4.0 and NDI HX is great, but the framerate still looks hard-locked to 60fps. Has anyone found a way to change that? Thanks for your help.

[keijiro]

I still can't make time for doing research on this topic, but I just wonder why the changes @ericmarodon made didn't take any effect.

I tried changing the setting in sender.h but sources still appear at 60p in Studio Monitor

Did you change the following line to 25, right?

https://github.com/keijiro/KlakNDI/blob/master/Plugin/Include/Sender.h#L28

I think you also have to change the display refresh rate on your PC to 25Hz. Then it should work.

[ericmarodon]

Did you change the following line to 25, right?

Yes, that's exactly what I did. I had numerator = 25 and denominator = 1, but it looks like it's being ignored. So I was wondering if there was a hard-coded 60 fps value somewhere else I couldn't find.

I'll try messing with the display refresh rate, but if this was a factor, being in in Europe, I would have thought this would output a multiple or 25 or 50hz, not 60. Thanks for the great work on the plugin nonetheless.

[ericmarodon]

FYI, I've tried to change the display setting from 60 hz to 75 hz (the only other option I have) but it doesn't change anything. Also I've created a script to set the application.targetframerate to 25 fps, and despite it working in the editor, in both cases, the NDI stream is a steady 60 fps.

[v2k]

control over the output frame rate would be great. I too am looking to send output ranging from 29.97, 25, 50, to 60 etc.

[0-fox]

Hi @keijiro thanks very much for this plugin.
I don't suppose you had a chance to look at framerates? I need to work at 50fps and it would be great to be able to set the NDI stream to the same.

[ruudboon]

I experienced the same issues last week. Changing screen resolution didn't effect the ndi output. It was always 60fps.

[eric29]

Any progress or update on this? I would also need to make the NDI output be 30fps for broadcast tv. It would be nice if I can run Unity separately at 60fps, though, but not a big deal if it matches the NDI output. The sender.h file has a variable for changing the frame rate, but it's under "iOS" folder, so I'm assuming that's why it doesn't work on Windows?

Studio Monitor is always receiving 60fps, even if I change my game's target framerate to 30fps.

[keijiro]

@eric29 That's old information. Now it's on the C# side. You can modify NdiSender.cs to add frame rate attributes.
https://github.com/keijiro/KlakNDI/blob/main/jp.keijiro.klak.ndi/Runtime/Component/NdiSender.cs#L149

[eric29]

Thanks keijiro. I've got an old version of Klak NDI, gotta update.

[johannesklaer]

@keijiro You wrote: You can modify NdiSender.cs to add frame rate attributes. Can you please tell us how? Can I modify the frameData e.g. for 30/25 fps ?

[keijiro]

@johannesklaer Please check this reply: #8 (comment)

You can add FrameRateN and FrameRateD there.
https://github.com/keijiro/KlakNDI/blob/main/jp.keijiro.klak.ndi/Runtime/Interop/Structs.cs#L61

The usage is not clearly documented, but you can guess how they work from the comments.
https://github.com/Palakis/obs-ndi/blob/master/lib/ndi/Processing.NDI.structs.h#L221

[johannesklaer]

@keijiro Hi, We have a hard time setting the frame rate in a NDI Stream. we tried to set FrameRateN and FrameRateD, it seems that the framerate is lower (eg. 30 fps), the NDI Monitor shows 30fps but the Datarate increases somehow. Is it possible that we have to feed at a lower rate as well? Thanks for your help.

[keijiro]

but the Datarate increases somehow.

What is the Datarate in this context?

[johannesklaer]

we have different Streams, one of them has around 200 MBit its 3840 * 1024 and streams from a RenderTexture. If we could make the stream play at 30 FPS it would have only 100 MBit, that would be great. Also would be nice if we could switch to YUV for the stream, I tried, but then you have to convert the RenderTexture with the encode shader, I dont know how to change it.

[keijiro]

Have you changed the target frame rate of Unity to a proper value?

[johannesklaer]

Yes, I tried, but no difference. (Application.TragetFramerate=30). It increases / doubles the datarate. But I found out something weird. If I clamp the Application to 30fps and set the metadata to 300p
FrameRateN = 300000,
FrameRateD = 1000
It seems to half the datarate, just as I wanted to.

[johannesklaer]

Do you think I could set the TargetFramerate to 60 and render only every second frame to the NDI Stream?

[keijiro]

FrameRateN = 300000,
FrameRateD = 1000
It seems to half the datarate, just as I wanted to.

Then, use it. That kind of weirdness is acceptable because the NDI SDK is a closed blackbox. Or you can ask NewTek about it.

Do you think I could set the TargetFramerate to 60 and render only every second frame to the NDI Stream?

You have to modify the sender code.

[johannesklaer]

Do you have an Idea how I could switch to a different FOURCC? I can switch it, but it creates just garbage, I think I have to convert It the right way. In Encoder.compute are different conversions, but how can I use them?

[keijiro]

@johannesklaer The discussion is getting off topic. Also I'm afraid that it sounds like free personal technical consulting rather than contribution to the open source project. If you'd like to continue the discussion about FourCC, please create a new issue for it.

[johannesklaer]

That is fine with me. But can you confirm that the NDI Sender depends on the Games Framerate when streaming? It seems to me the case, but framerate control in Unity is quite awkward (sorry another topic). And if I find out more about NDI I may contribute to this wonderful project.

[keijiro]

But can you confirm that the NDI Sender depends on the Games Framerate when streaming?

It depends on the NDI SDK implementation, and that hasn't been disclosed.