I've made a video that demonstrates the effect. Please watch (and make sure to enable HD playback, full screen and annotations):

https://youtu.be/ItLnPosez3I

As you can see, during loading the level, the VRAM is already flushed once. And then, while I'm running around, the VRAM usage is capped at around 4.8 GB. It seems 0.8 GB are used for the framebuffer and stuff while the textures managed by ENB seem to be capped at 4 GB. As this threshold is reached, some contents are flushed out of VRAM (clearly visible as the VRAM usage drops a bit) and you can see as it is filled up again, causing the usual frame drops.

If this is indeed working better on other machines, I'd really like to see a similar video of that. And then I'd really be curious what the differences to my setup are...

May be you just need to install mods which have higher resolution textures? I can't tell anything more than already said.

_________________
i5-4690k, 16Gb RAM, GTX 1060 6Gb, X-Fi Titanium, Win7 x128
I am INFP, not the brutal, godamnit.

I already installed practically every gfx and high res texture mod available ...

I wonder why the VRAM usage keeps dropping, especially during the loading of the save game and then later while running around. Looks to me like some limit is hit and it has to free some room so it unloads data from the VRAM.

Do you have any way to track the flushing of textures from the VRAM? Is it done by the driver or by ENBHost itself? What difference does it make if I change teh setting "DisableDriverMemoryManager" since I couldn't see any difference in the behaviour.

BTW, I'm not blaming you or whatever, I just want to know what's going on and if there's some problem on my side, I'd really like to fix it. Especially since I hate the framerate drops caused by loading texture into the VRAM is quite and it's one of the reasons I bought this expensive gfx card to never have to experience that again...

If you want to do some tests or if I can be of any assistance, I'd be happy to help. I guess it's not that easy to find a system with more than 4 GB VRAM to test on.

enbhost.exe do nothing to vram, it's just memory container. D3d9.dll flush data from vram when it unable to allocate memory any more in vram (but it flushes only dynamic memory ReservedMemorySizeMb) or flushing everything when device is reset (alt+tab in full screen mode for example). It removes some amount of resources of any type from vram when game creating render targets (for drawing shadows for example), as they must be located in fast real video memory (NVidia driver can place them in ram and draw to ram, so performance will be awful, read what i wrote in news page about gf970 several months ago). Flush at loading game is normal to Skyrim, because this stupid engine (or rather it's developers) do not take care about order of creating vram resources.
DisableDriverMemoryManager is totally driver and hardware dependent thing, seems set it to true is better for AMD/ATI.

_________________
i5-4690k, 16Gb RAM, GTX 1060 6Gb, X-Fi Titanium, Win7 x128
I am INFP, not the brutal, godamnit.