Initial load time drastically increased when using FIELD models + crash *IMPORTANT-READ*

[fixapc]

Describe the bug
Once field models is activated when starting FF7, load time is drastically increased regardless of mods. When a random assortment of mods is installed and sorted there is almost a guarantee that the field models will cause an even longer load delay upon initial start leading to a crash. Once this crash and log is generated; the FF7 folder and 7TH heaven folder must be deleted and reinstalled to get the game to post again. When activating a random assortment of mods with field models the same scenario will present it self.

** Versions **
[ 2.2.3.522, 2.3.1.0, 2.3.1.66] Tested on latest stable version

To Reproduce
Steps to reproduce the behavior:

1. Fresh installation of steam based FF7 (2021-07-06)
2. Installation of 7TH heaven mod manager (Any stated version post 2.2.3.522)
3. Activation of random assortment of mods with field models

Expected behavior
2.2.3.522
Drastically increased load behavior but rare if at all leads to a crash

2.3.1.0
Drastically increased load behavior that eventually leads to a crash, once this error is generated. The game always crashes until you delete the folders and reinstall.

2.3.1.66
Drastically increased load behavior that eventually leads to a crash, once this error is generated. The game always crashes until you delete the folders and reinstall.

Screenshots
If applicable, add screenshots to help explain your problem.

When the below error is presented folders must be deleted to start FF7 with field models enabled again.

Desktop (please complete the following information):

• OS: [Windows 10 Pro]
• Version: [21H1]
• Build: [19043.1055]

Additional context

1. I have noticed that the field models is one of the only mods if not the only one that drastically increased load time.
2. When cmh175 battle models is activated a cloud battle model is also applied to clouds field model, regardless of other mods activated.

---

Recommendations

1. Disable auto activation of mods upon download completion or import (This is not the best thing during testing when loading up a new 7th heaven installation directory)
2. Increase or add sleep delay between mod installs when downloading in bulk
3. Have ninostyle models selected by default in the mods configuration menu (Users want to do a simple 2 step install and start the game with an upgraded feel)
4. Merge all models into 1 package to prevent over lapping of models. This way users only have to go to one place to adjust all models for the game. it makes more sense and is less confusing. The sub directories of the models mod should be labeled as field, battle and world.
5. Merge all textures into 1 package to prevent overlapping and confusion of end users.
6. Merge all spells into 1 package
7. Merge all audio into 1 package
8. Have the game start in full screen mode by default.
9. When activating any new sound track mods, have the mod manager automatically switch to VGMStream and vice versa. Many people as my self don't like to read too much documentation and may think the audio mod is not properly applied.

Extra notes

1. Everything else seems near perfectly stable even maxing out the graphics and changing mod configurations frequently. I am sure there is a bug or 2 i am not finding.
2. I have been using Qhimm forums for 15+ years and 7TH Heaven since its near initial release.
3. I am also working on a review of the mod manager and hope it may generate you guys some more traffic, more people need to know about the great work you have done on the original FF7. I randomly play through the entire game every once in awhile.

LAST LINES OF FFNX.log before crash
[00000001] TRACE: 601704EE (FFNx): (filename not available): open_lgp_file
[00000001] TRACE: 00675532 (FF7): (filename not available): (function-name not available)
[00000001] TRACE: 00408BF9 (FF7): (filename not available): (function-name not available)
[00000001] TRACE: 6016B7B2 (FFNx): (filename not available): common_create_window
[00000001] TRACE: 0067DC2B (FF7): (filename not available): (function-name not available)
[00000001] TRACE: 0040B832 (FF7): (filename not available): (function-name not available)
[00000001] TRACE: 7663FA29 (KERNEL32): (filename not available): BaseThreadInitThunk
[00000001] TRACE: 77527A9E (ntdll): (filename not available): RtlGetAppContainerNamedObjectPath
[00000001] TRACE: 77527A6E (ntdll): (filename not available): RtlGetAppContainerNamedObjectPath

[AnsgarTOdinson]

Had this same issue, reverted everything back. Should I wait for a fix before changing over the new 7th Heaven and using FFXN?

[julianxhokaxhiu]

I'm not even sure where to start in this issue. Please post the entire log file instead of the last lines. Then we can double check what is going on. Additionally this looks like a crash that is not related to "loading times" which may vary, depending on your HW configuration.

7th Heaven was never meant to scale with the volume of mods, and quantity of data we have today. Remember that this is still a 1998 game with the very sufferings and limits of that era, including the 4GB memory. Therefore, choose your mods carefully and make sure you find a balance within your set.

Finally, we do support only either the latest stable or canary, you can find in the Release page. Any other version is not supported ( by us ).

Thank you in advance.

[fixapc]

Had this same issue, reverted everything back. Should I wait for a fix before changing over the new 7th Heaven and using FFXN?

Did you install the 2.3.1.82 that released a few days ago. I have had no crash issues with .78 and .82. Although there is still the huge load delay and sometimes a not responding window that presents it self at the start when field models are enabled. If you just wait it seems to always continue.

[fixapc]

I'm not even sure where to start in this issue. Please post the entire log file instead of the last lines. Then we can double check what is going on. Additionally this looks like a crash that is not related to "loading times" which may vary, depending on your HW configuration.

7th Heaven was never meant to scale with the volume of mods, and quantity of data we have today. Remember that this is still a 1998 game with the very sufferings and limits of that era, including the 4GB memory. Therefore, choose your mods carefully and make sure you find a balance within your set.

Finally, we do support only either the latest stable or canary, you can find in the Release page. Any other version is not supported ( by us ).

Thank you in advance.

It seems the issue is gone, but the drastic load delay is still present with field models. As a fellow dev i know feedback is essential to working at improving the quality of software. Anyway what ever you did seems to have stopped the issue. When .78 was released that error message disappeared.

[julianxhokaxhiu]

It seems the issue is gone, but the drastic load delay is still present with field models. As a fellow dev i know feedback is essential to working at improving the quality of software. Anyway what ever you did seems to have stopped the issue. When .78 was released that error message disappeared.

The only difference which might come up into my mind is the Code 5 Fix which is now no more necessary. Try ensuring it is disabled, might make a difference :)

Thanks for the feedback and we really appreciate that, although it is difficult to tackle something by having no basis to work on, that's why we ask for logs which should at least help in grasping the potential issue, if any.

Nevertheless, glad to know it works better now. So, if it's fine by you please close this issue if it's not impacting you anymore.

Cheers

[barkermn01]

The drastic load delay is actually something i have noticed and it's not just field models ESUI adds a large amount of load time lag, i think is a problem due to the 4GB memory limit and 32Bit memory cleaner operation while the game is loading everything it needs (this is my best guess)

@julianxhokaxhiu I would like to talk to you about an idea i had that might help resolve this but it would be a lot of work and I'm not sure if it would work with FFNx, but a quick overview would be to create a container application for each mod that contains a file list tree, and loads assets into there own memory and is able to give the pointers out, then in 7th heaven and the driver we instead check each application if it has the asset if it does we refer to mod priority order and we use the assets from them. while I know in theory this is plausible I'm not sure if it's possible within the game's code or other wise just as an idea to try and get around this 4GB limit in the game if we can move all the asset memory out of the game and into there own contained applications.

[julianxhokaxhiu]

To be honest this is a topic I'd like to be tackled in 7th only as the way 7th hooks APIs to inject content in the driver, is not the way FFNx was meant to be used, is mainly a workaround.

What we could do to speed up the loading times, assuming the issue is somewhere on fetching the data from IROs files and unpacking them, would be to double check the lifecycle of each chunk of data and find a way to either "keep it in memory until needed" or "improve loading speed". In either cases, this is purely a 7th limitation and issue on the way the game is currently being hooked.

The real solution would be to throw away this pattern and re-organize how 7th integrates with FFNx ( aka, extract in the real paths monitored by the driver the content that matches the mod load order, and then run the game with 0 windows API hooking ). Then when the game is closed, wipe all that data and leave a clean installation ( optional maybe through a flag in settings ).

By doing this we might also solve #19 too.

[Kaldarasha]

I can confirm that this issue is related to 7H. An earlier built of it (1.5.7.11) does run well together with FFNx.

[barkermn01]

I can confirm that this issue is related to 7H. An earlier built of it (1.5.7.11) does run well together with FFNx.

With the lastest copy of FFNx? or the version that was designed for that version of 7th heaven?

Separately:
As it, the only changes I made to that code have been reverted so what else has been done to it...

[Kaldarasha]

I used at that time the latest FFNx canary. The problem did pop up a longer time ago and happened in conjunction with an FFNx update I guess, but since people reporting the issue on the more open Qhimm discord, I assume they used the current public 7H. So it might be a problem with something unabomb has added.

[chrcoluk]

Ideally on a bug report of this nature a list of activated mods should be provided, saying a random list makes it difficult to try and confirm the problem.

Inevitably with a bigger mod list there will be a longer delay loading the game and a increased chance of instability, please remember this game was made in 1998 and we are doing things which push it to the limits.

The bundling of mods together was a decision made by the modding community to keep them separated, so it is wise to only enable mods you want loaded, rather than just reordering and keeping them all turned on.

I will do some testing with the older 1.5 version of 7th Heaven to see if there really is a regression.

[julianxhokaxhiu]

Please give it a spin again using the latest canary for both 7th/FFNx and let us know how it goes. We've worked on so many things since this issue has been reported that might be it got much better on your end.

[barkermn01]

I will confirm this is still an issue in the new 7th heaven. Some mods that don't do anything at game start-up, or need anything loading at game start-up, are causing huge delays for no apparent reason and i also slightly disagree with inevitably with a bigger mod list there will be a longer delay loading the game, as that should not be true mods are supposed to be loaded when they need to be most mods are not needed at launch ESUI is an example of needed at launch so might rightly cause delays in loading to a usable state. But, mods like Cetra are not they have no content that is loaded until the 3d world is being rendered so should not cause start load time issues but may cause issues when loading between scenes

[julianxhokaxhiu]

I will confirm this is still an issue in the new 7th heaven. Some mods that don't do anything at game start-up, or need anything loading at game start-up, are causing huge delays for no apparent reason and i also slightly disagree with inevitably with a bigger mod list there will be a longer delay loading the game, as that should not be true mods are supposed to be loaded when they need to be most mods are not needed at launch ESUI is an example of needed at launch so might rightly cause delays in loading to a usable state. But, mods like Cetra are not they have no content that is loaded until the 3d world is being rendered so should not cause start load time issues but may cause issues when loading between scenes

If you run 7th with TurBolog opened you'll notice 7th does a lot of things before even the game starting as you hit Play. That's what is causing the delay. We need to find out what is happening and why that is being done.

[tangtang95]

I discovered that the main culprit of mods that increase load time drastically are mods using folders that ends with .lgp. Apparently 7th uses a sort of virtual lgp loaded into memory and when the .lgp folder is too big it cause this slowdown.

The solution is to use direct folders used by FFNx. In general, the direct folder name is just the one without .lgp, but not for world, high and other folders. In this way, it gains a lot in performance.
Notice that direct folders and lgp folders are not fully compatible, there are some features only available in lgp folders such as reading flevel chunks

[barkermn01]

I discovered that the main culprit of mods that increase load time drastically are mods using folders that ends with .lgp. Apparently 7th uses a sort of virtual lgp loaded into memory and when the .lgp folder is too big it cause this slowdown.

The solution is to use direct folders used by FFNx. In general, the direct folder name is just the one without .lgp, but not for world, high and other folders. In this way, it gains a lot in performance. Notice that direct folders and lgp folders are not fully compatible, there are some features only available in lgp folders such as reading flevel chunks

Hrm would be possible to start a thread with 7th heaven to start building that in the background without the game start up and then it's built for game start up?

[julianxhokaxhiu]

@fixapc can you please try with the latest canary? We had recently implemented a huge overhaul. Let us know

PLEASE NOTE: Make sure you're on the latest FFNx Stable or Canary also, while testing the latest 7th Canary.

[julianxhokaxhiu]

Closing because of missing feedback. Most probably this has been fixed by the Canary.