Custom shader for hairs

[kebrus]

Hi, i'm really sorry if i'm asking this on the wrong place but i can't find a better one.

I want to create a custom hair shader for Skyrim (maybe kajiya kay or Scheuermann technique) but i'm not even sure if it's possible, from what i understood i can create custom post-processing effects but this is not really the case, i would like to have my own shader applied on hairs only (per object). Is it possible?

[ENBSeries]

No, this hardly possible the way you think. First problem is that hairs are not strictly detached objects with own shaders after engine preparations to drawing. Second, it's not just one shader, but 106 (on my pc for nvidia only) in assembly, pre-generated by uber shader method, so it's a huge pain to convert them back to single, also these shaders used for other objects which are not hairs at all. The only way to do custom hairs is to modify mesh (nif) or material properties to declare some unique state which ENBSeries detect and replace by another shader. If you are author of some hairs or have permission to do that, i'll help with providing specific shaders after testing modified model of hairs (detection of it as unique). Some mods already use such trick.

[kebrus]

indeed i made a few hairs before: http://www.nexusmods.com/skyrim/mods/28491/

i got really busy and never continued the project, and now after coming back i just can't stop looking at them and feel like it was a waste of time, i was trying to improve on the existing skyrim hairs while maintaining a low polygon count and keep the original feel, but that would only really matter if it was possible (at least easily) to change all NPCs in the game, since it's mainly for the PC i thought i could start a new project with lore friendly hairs hence the idea to create a new shader for them, that way i would have new meshes, new texture and better control over how they are rendered with a new shader

anyway, it's sad there isn't a easy way to do it (it might mean two versions of the mod, with and without ENB), still i'm glad there is indeed a way
you have my permission to use the models from that mod in the link, since i was still in a preparation phase i don't have any new mesh, if you are willing to do it then i'll start working on a new mesh, maybe a simple short hair with a easy texture to work with for the shader

[ENBSeries]

Okay, i'll try to find solution for you after experimenting with shaders.

[ENBSeries]

Did changes to emissive color to visualize what can be wrong when making custom shaders for hairs. As you see hairs split in to different shaders, far not the same properties when drawing them. Alpha test enabled is red hair which is work fine with depth buffer effects, but below is alpha blending hairs, which can't normally work with ssao, depth of field, reflection, image based lighting and other depth based effects. Alpha blend hair even drawed after alpha test hairs, almost at the end of frame and together with brows. Making single hair model to handle one shader only? Or somehow design one main shader which generate several? This hard to decide for me, because don't know native modding of TES games, i even have no idea where is the mesh of that transparent hair below red one.

[kebrus]

the mesh below the hair is the hairline mesh, usually with the same name (doesn't have to be but usually it's something line hair01.nif and hairline01.nif), there's three purposes for that mesh, one is because when you use a helmet only the top piece is removed, so in theory you could have the hairline showing when using a helmet, i say theory because it isn't really practical creating a mesh that shows fine using any helmet AND works fine with the normal hair on top. the second reason is alpha sorting, because there are so many problems with alpha sorting having a separate mesh that always renders first (hairline) helps in those weird angles where z-testing and alpha-blending don't work very well and to smooth edges, this way you are guaranteed to always show a bit of hair, the third reason is because the hairline mesh uses alpha blending to create a smooth transition between the head and the hair

the top piece is then rendered with alpha testing at some value like 0.5 (in a 0 - 1 range). Some modders put the same mesh on the hairline mesh so they can use alpha blending at the tips of the hair for a smoother hair, others delete it altogether, there isn't a "best" way to deal with this problem and it's almost a matter of taste

having alpha blending in one layer and alpha testing on top is actually a technique you can achieve with shaders and it's probably the standard one, see this explanation from the wolfire guys http://blog.wolfire.com/2009/02/renderi ... oth-edges/

bethesda actually places the hairline mesh on the hair mesh, i guess they do it because they want to make sure the same effect from the top mesh is also applied on the hairline mesh.

in conclusion you can pretty much ignore the bottom hair because in that particular case i was just imitating bethesda, i can simply remove the hairline mesh altogether, but i was hoping to be able to use alpha-blending in the shader, as you can see from the image you posted if the hairline didn't had alpha blending you would see a brown hard edge between the forehead and the hairline, if i can't it means i have to always use alpha testing which isn't pretty at all in a lot of places

from my tests in game the alpha blending in the hairline isn't much of a problem because it exists only to correct some details, it only means a bit more work to me to be careful on where to use it
in either case, if you can, it was best if i could change both the hair shader and the hairline shader separately and deal with the alpha blending problem in the modelling itself

here are the default settings for the vanilla hair01
http://s13.postimg.org/dfe2pp7n9/hairsettings.png

[ENBSeries]

I haven't forgot about this, just trying to find out some solution for all shader combinations possible at different lighting conditions and skyrimprefs.ini changes, very tricky.

[kebrus]

thx

i've been doing some 3d modelling research myself, trying different techniques and topology combinations, take your time as i am taking mine

cheers

[ENBSeries]

I'm totally stuck with this task, don't see any solution, too many shaders have similar properties to your hair model, definetly don't want to convert them in generic one with toggling values dynamically, it's too much boring work for lazy ass me. Inside interiors shaders are different, by amount of active lights, if shadow enabled - these give a lot combinations. Thought about another simpler solution, to use absolutely different shader type which don't have all these variations, but such seems not exist with shadow enabled, so even if i can manually apply shadow and lights, this object will not be rendered to shadow at all, so trick not works. I'll keep looking for solution.

[kebrus]

I know I'm necroing this but how is the development on this part? Considering that everything runs with some shader I find it really strange how hard it is to replace or simply use another shader. Even with CK itself or other tools I can't seem to find any way to tell my mesh to use a different shader. I wouldn't even mind to write it in assembly code as long as I could right something.

But I understand your problem tho, it seems to be a lot harder to inject code than to natively use a pre-existing one with some flags activated. I suspect the shadow issue is due alpha testing vs alpha blending.

Whatever's the case, do you think this you could look at this again or have you abandoned the idea completely?