Trying to maximize training speed on Linux (New Linux User)

[poisenbery]

Heyo, I'm somewhat new to doing these things on Linux and I'm not really sure where I should be looking for guidance (or what I should even be looking up).

I originally was using this on a Windows setup with an Rtx3090Ti and followed the additional steps for the CUDNN 8.6 dll's and was really happy with the training speeds I was getting.

After a few months of experience, I began to see more and more things that implied that Linux was the superior operating system for all things related to machine learning and AI. I installed Linux on a spare drive and set up all of the AI related things (Textgen, Automatic1111 WebUI, and Koyha_ss).

The speeds I am getting with textgen and image gen are MUCH faster (around 5x) than I was getting on windows, but the training speed I get seems to be exactly the same (~1.08s/it).

This has been making me feel like I'm making some type of noob mistake, or I'm overlooking something that a linux user should be aware of.

Is it wrong for me to assume that my training speeds should be faster on linux, or is there something obvious that I just don't know about yet? I'm not very familiar with these types of things but am very willing to learn.

I've seen stuff about something called "DeepSpeed" and "Dockers" but I'm not smart enough to know if that is what I should be trying to do.

If I used https://github.com/P2Enjoy/kohya_ss-docker and compiled the things, would I get faster speeds?

Sorry if this is a stupid question. I'm not very smart at linux.