Reducing startup time

[al12rs]

I understand, if you say it would be too hard then I will drop this.

Just to give you some context though this is what happens on my end:


I have 30 seconds of operation at one core one thread (which I assume are patching + compilation), it takes me the same amount of time to complete the compilation of the vanilla shaders (No ENB) without the Nvidia cache, but in that case at least I can actually interact with the game after the first 5 seconds.
To be able to not do the hole 30 seconds of capped work because we do it once and then use that for the next runs would be an already incredible improvement.
I mean you talk about only 8 seconds of patching for example but for people with weaker CPUs (I have a i5 3450) that just goes up, and I would argue that 8 seconds is actually already incredibly long considering it's already twice the time vanilla Skyrim takes to load.
It's just insane that there are 30 seconds of not utilized resources each time the game is started, while if ENB is disabled the game loads in just 4 seconds.

Btw this was with a completely vanilla SkyrimSE with just ENB installed on SSD.

If you manage to find some kind of solution to this problem then I can guarantee you people would be incredibly grateful for it.

[ENBSeries]

The only solution i know is to not create shaders when game do it, but put in delay, some number of them per frame, but this is very risky cause game use multithreaded calls, so stuttering guaranteed. And i was wrong about timing, that was old info before i patched most shaders to hdr. Now checked again, patching takes 18 seconds and creation of cached shaders 0.88 seconds (if cache is used in driver and mod setting didnt change). Process includes creating original shader to turn off the mod, dissasembling, patching text by signatures, assembling, creating modified shader. Patching itself is around 1900 lines of code for pixel shader, not including dissasembling and assembling. This can't be done fast, so only put this as a job for future and return to game fake shader and if it try to use it, force to compile and to have stutter. I am not so interested to place critical sections in runtime code just to check if shaders processed or not, this shit not worth peformance loss.

[ENBSeries]

Have one idea, let me think.

[al12rs]

One of the suggestions was to allow users to give up the ability to change or disable effects in-game for better performance once somebody is satisfied with his setup. That could let you skip recompiling the vanilla shaders part of the patching step for example, and some other things could be changed to only need a one time work.
People would change and tweak their presets in game until satisfied and then they would gladly sacrifice their ability to change stuff for better loadtimes and performance. We are talking about a x6 ~ x7 load time increase over vanilla performance here, so yeah anything to reduce it, short of sacrificing in game performance.

Well I'm just voicing some of the ideas I could get without knowing the code.

I hope you got a good idea for a solution.

[ENBSeries]

The problem is to not forget keep track of changes for every depending parameters which influence patching. I'm very much unfocused person and can do mistakes. But the idea is to make special file with parameters unabled by presets, if its data is not equal to what is enabled, then remove all cache files and make new. Also instead of dump all shaders, just differencies data patched and non patched, this should save lot of disk space and not make people angry. But i have to do a lot for this and pray that not gonna break up something cause forgot to add to cache validator any new variable.

[al12rs]

I don't know how many settings we are talking about, or if the changes in-game of ENB have anything to do with this, but if you have to compare just configurations at startup then just compute hashing the entire setting file and comparing the result it to older one might be faster than checking individual values.

To tell you the truth I'm still not sure what your plan is.

Like, only patching what is strictly necessary for the loaded preset or something?
My idea was to simply have a switch to use fast boot in the setting that would: check if custom cashe is up and if yes use it. If not, save current configuration and make cache for it then block changes from users (ignore them in favour of custom stored configuration). If user disables the switch in the settings, invalidate custom cache and settings and instead revert to current behaviour.
This is very naive implementation and would basically be to the benefit of exclusively end users and not preset creators or tweakers.

From what I understood that you are trying to do, your solution would be more tolerant to changes until one of the important settings is touched and in that case you need to make custom cache for that one again? I didn't understand there much.

[ENBSeries]

I dont like idea to force something, people will come here to complain. So i must do something which going to be compromise or Jack of all trades style. Regarding hash of files, nice idea, didnt thought about it, thanks.

[al12rs]

I dont like idea to force something, people will come here to complain.

Yeah you are right, I can totally understand that.

I'm glad I could be of help for the hash thing.

I'm currently investigating the vanilla game (without ENB I mean but with other mods) loadtimes with a couple of other devs to see if there is room for improvement there as well, and it seems that the most time is spent on getPrivateProfileStringA, which is incredibly inefficient. One dev is working on a solution that would hook those calls and get the actual INI strings from a cached version of the file in memory, instead of opening the file, parsing it completely, and then return the value of the given key for each of the 10626 calls. I tested his work and I can see a 5-6 seconds speed up in startup time on a moderately modded setup. Hopefully this can be made into a SKSE plugin.

Let me know if there are developments on your end, I would be glad to hear of them.

[al12rs]

I saw the SSE ENB update that mentions a startup time decrease:


I tested it out on my modded setup on an SSD using MO2 with 80 plugins installed.
I was also using the mod I talked about in my previous post that reduces the game startup time by caching the INI files in memory for faster consecutive access.
The mod is now available and people have been reporting big decreases in startup time so I recommend it: https://www.nexusmods.com/skyrimspecial ... ods/18860/

The results of the testing gave a consistent 8 seconds decrease in startup time, from 44s to 36s.

Another user reported similar results (7-8 seconds) on much bigger setup.

Is that the implementation of cache system you were talking about? Or something else?

[ENBSeries]

No, its another trick. Game creates shaders twice (create, delete, then create again), so i decided to create copies from previously created and saved half of the time for processing them. Everything else are internal and external mod shaders compilation (done in multiple threads, but created in single thread). What you say most likely means some complex external shaders used with long cycles which takes much time to compile, you can try my official files to check if time is different. But if its about the same, must be game itself mostly. I cant improve performance of creating mod shaders, cause they are already in multiple threads and i cant cache them, because they can be made from multiple included files and i will fail to handle all possible scenarious to find such files to see if they are changed. The more reasonable way is to compile such shaders not at the moment game starts, but in parallel with something else, but this require many code to write which will not lead to crash if shaders not yet created, but but game is started, so i prefer to find elegant solution instead of to put test of their existance everywhere.