You really want to be using a separate prefix for each game, to keep any game-specific tweaks and settings separate.
Managing that without something like lutris or bottles IS A HUGE PAIN IN THE ASS, DO NOT EVEN TRY IT. Literally no-one does this. You will not find a guide.
But if you insist, yes, you'll need a version of wine, as well as dxvk. Both you can install yourself.
Vulkan you should have already on most distros.
Then there's a litany of optional windows dependencies that you can/have to install within a prefix, using winetricks, some of which will be necessary to run some things, while others will break other things.
But seriously. If you want to fiddle with the details, while not going insane with tedious prefix management, just use Bottles.
My issue with lutris is that when I used it before and something didn't work, or used to work but broke, I was completely at a loss, because I did not understand what all it is doing exactly. It just felt that what lutris is doing is a bit too obfuscated or unclear to me.
Yes. I find prefixes in bottles much easier to configure. It's all the same stuff, but the UI is simply better IMO.
But if that's the problem you were having, setting off to manage wine yourself will only make it worse. You would be massively increasing the amount of legwork you'd have to do, not decreasing it.
Also grab protonup while you're at it. It can download specific proton/wine versions for you and put the files in all the right places for the extra versions to be selectable in bottles/steam.
This what I do and it works pretty flawlessly. I've never used Bottles or Lutris since I started adding the games to Steam. It also takes care of controller support.
From my understanding, at least one other necessary component is dxvk, and that wine is not enough.
dxvk is not necessary but it massively improves game performance on a lot of games, also keep in mind some games will actually not run if you use dxvk, so you sometimes (very rarely) have to use OpenGL instead and the only way I really found to do that was to have a wine prefix without dxvk. (I might've just been stupid though)
for the record only games that I had to use open GL for so far was Starcraft 2's Galaxy Editor (although the actual game itself runs fine with dxvk) and I had some problems with alt tabbing in really old versions of Warcraft 3 (1.27 and older)
Anyway, for a time I used Lutris a bit but now I always run wine through terminal because Lutris was great when it worked but there were seemingly no solutions for when it didn't (while running wine directly has never given me a problem).
And what's great with running wine directly through the terminal is that either it will run and you're good or you'll get some error messages saying that some .dll or whatever is missing (usually .net, visual studio, msvcr100+.dll, mscvcp100+.dll, ms*.dll, etc.) and you just use winetricks and go through the list until you find what you need.
for example when I play Need For Speed Most Wanted 2005 I start it with this command: WINEDLLOVERRIDES="dinput8=n,b" wine speed.exe
another thing is sometimes you want to run games with some arguments, for instance before I got XCOM 2 on Steam and used AML I used to run it with this: wine XCom2.exe -noRedScreens -review
Now here's a pretty big caveat, which is that I use FISH instead of Bash which adds some big QoL improvements (in this case it's mainly about tab completion).
If I actually had to manually type in the commands or spend minutes going through history file every time I would never do it.
Although I suppose you could make an alias for each game.
Oh yeah and always run the games from the same directory as their .exe file is located in, more often than not it won't work if you just do wine /path/to/directory/game.exe instead of cd /path/to/directory && wine game.exe
Edit: some typos, and I just want to note that the && aren't supposed to include the amp; parts
If you don't mind me asking, how would one go about diagnosing why a game/mod doesn't work?
Specifically im working on nitrox for subnautica, which does some DLL funkery - but has reports of users getting it working on older editions, and radio silence for the later. I figured out from a reddit post that a ribbon.DLL was needed, to even boo the launcher (and the error code was pretty descriptive about it). Once it's added the launcher doesn't ever actually launch the game though, and the console is pretty radio silent about it all.
Again ignore all this if youre unsure/busy, but any tips are appreciated!
Why a game (unmodded) doesn't work will usually be pretty obvious just from the errors you get
I might've just been lucky, but so far the only time I had to look up a fix was for battle.net launcher, which ever since a while back has to be done after every update.
But when it comes to mods it can be a lot trickier, and I don't really have a general purpose solution. Hopefully the game is popular enough that someone else has a guide on what to do.
I've never played subnautica so I don't know, but if the mods have their own modified DLLs then you're probably supposed to use DLLOVERRIDES, so maybe WINEDLLOVERRIDES="ribbon=n,b" since it's called ribbon.dll
Some other issues I've had with modding is that the mod might expect the game to be in a certain directory or for directories to have certain names or there might be some problem because Windows isn't case sensitive while Linux is.
Honestly the mods are generally not causing issues from my experience, it's the mod loaders and what not.
Didn't use Wine much lately, but when I do i use usually 2 prefixes; one for 32, one for 64 bit.
Winetricks is often helpful; so is the appdb on WineHQ.
Don't use vanilla wine, use Wine-GE, which is basically Proton-GE, but designed for non-Steam launchers like Lutris.
Wine-GE (and Proton-GE) already includes DXVK and VKD3D btw.
You may also need to install proprietary codecs. Look up how to install the full ffmpeg and gstreamer plugins, or just "codecs" for your distro.
As to what else you require specifically, that really depends on your distro, graphics card, and how you're launching your games.
The Arch Wiki covers some of this stuff, but note that some of it may not be applicable to you if you're not using Arch (or an Arch-based distro). In saying that, a lot of it is applicable, but the package names may be different in your distro so you'll need to look that up on your own.
The classic way of using Wine is as simple as running any Windows binary using Wine program loader. If you've got Wine installed, you can likely just "Open with" in the Dolphin explorer, or simply wine program.exe in terminal. That is not a good idea however, because you just have a single Wine prefix (which by default is in ~/.wine and is controllable with env variable WINEPREFIX`) and also you will likely miss at least some dependencies for your game.
Dependencies required for launching a game will be different depending on what that game was built with in what period. If the game uses Vulkan or OpenGL natively, you don't need any translators, but it still might need .NET Runtime, VC Redistributables, some other Windows libraries that are not (and cannot be due to legality) shipped with stock Wine. If it's DX 9/10/11 game, you need DXVK. if it's DX12 you nees VKD3D-Proton. You can install these using Winetricks.
To automate and ease all of that, I recommend Bottles. The app is focused on providing you with generic way to run Windows software instead of collecting scripts, it manages prefixes, install most needed dependences automatically and provide a way to manually install everything else.
I think this is also a good idea to mess with native way pf running things with Wine if you just want to learn more on how it works.
While creating a Bottle, you choose a profile for "Application" or "Gaming". It then createa a prefix with most common dependencies for that purpose. It might not install all you need though, but it has menu to install whatever you need manually within few clicks.
dxvk is available through the distro, if you're on an ubuntu or debian-based distro just run sudo apt install dxvk. vulkan should come pre-installed on any modern distro.
i'd highly recommend using some kind of manager, if you don't want lutris than use playonlinux which is more minimal. this keeps things organized and lets you use different wine versions and tweaks for each game. for example, some newer games need proton-experimental or an otherwise very recent version to run, but you may not want to replace your system wine with this.
afaik you need dxvk in the actual wine prefix, just having it on your system doesn't do anything.
I had installed dxvk from AUR and didn't get why it wasn't doing anything, turns out the point of having dxvk on your system is for symlinking it into your wine prefixes so you don't have to get a dxvk installation for each prefix individually.
when I found that out I just removed dxvk package and got installed dxvk directly to the my prefix with winetricks, because I only use two wine prefixes anyway (one for vulkan and another for games that have to use opengl)
I used to manually port games to macOS using wine/wine tricks/wine bottler, and honestly it’s not that bad. Especially if using GUI versions of all of those. Yes it takes fiddling and tweaking and a decent amount of failure, but I found it worth it once I succeeded in doing it once. That was almost a decade ago, and while I haven’t tried it since (I rely mostly on steam at the moment) I have a few games from say GoG and other places I’m planning to manually port. Ignore the haters, and just go for. Read/watch a tutorial, fail miserably for a while, and eventually you’ll tweak enough parameters and change enough values you’ll find what works. Once you do that for one game all the others will be easier. Every game is a bit different, but the freedom and knowledge you’ll gain is great. If my 13 year old ass could do it on a school issue MacBook to share Skyrim to all my friends you can definitely do it with more modern tooling.
EDIT: I think at one point I ported the windows version of steam, and then could use the “add non steam game” option and then could simply hit run. Didn’t work for everything, but there was a handful of games that was a quick and dirty work around for.
Like others have posted, I use Lutris. I usually have to supplement it with game posts on the protondb website so I know what fixes might be needed. Often they have some environment variables that might be useful or they list certain items that need to be installed via winetricks(or in steam's case protontricks). Some games just don't run but you can usually get a lot of them to run.