/dev: Random Optimization Stuff

[Vanilla]

Hello everyone, Vanilla here.  The following post is me just talking about some optimization of the server. So whether you're interested in developing or not, I hope you find it interesting at the very least.

 

As more and more content is added to the server, developers constantly face the task of going through existing content to try and bring the performance cost required to play on the server at an all time low. Now because Garry's Mod is a 16 year old game, built on 16 year old technologies, this task is usually somewhat difficult. Over the years, developers such as Moose, Badger, etc. have assisted the server with sucking every tiny little droplet of performance that can be extracted from Garry's Mod itself. Whether it be the tickrate being extremely low or PAC3 restrictions, they all have their reasons for existing, (even if most of us don't know why).

So with optimizing Garry's Mod and the server itself out of the question, we try and go through the content and optimize that instead. That's because the content is made by various different people with various different skill levels. Some content hasn't been touched since it was added (whether it be well optimized already or simply isn't big enough to make a significant impact) and some content changes almost daily with several fixes and optimizations (good examples would be TFA base and the augment system).

Now it's all well and good to optimize every little thing, it's the big content with the big functions that actually make the difference. For example, one of (if not the most) significant server bottlenecks is the TFA weapons base. Now while I wish we could just remove weapons altogether and enjoy that sweet sweet performance, it's obviously not an option on such a combat focused server. That's why we have developers like @FireCup constantly slaving away at cutting unused features and optimizing used ones all for your enjoyment. Another big performance tank is the recently (by recent I mean like almost a year now) implemented AUGMENT + QUEST SYSTEM.

 

Originally, the quest system pipeline of operations/structure looked like this:

Now while this isn't a professional flow diagram, it should help you understand exactly how it works. Let me explain specifically what each square does.

1. Has the player completed a quest action (e.g. press the screenshot button)

2. If they have, read their file from the server. (imagine the file to be a txt file with all the quests you have and how much you have progressed them)

3. Check through the file, do they have the quest that corresponds to the action they just did?)

4. If they do, add 1 to their quest file. (e.g. 45/100 --> 46/100)

5. Save their player data and exit the function.

Now, that seems pretty standard and somewhat efficient right? All the actions are being done straight after each other with little to no conditions being used.

WRONG.

You see, when I originally wrote that, I was but an innocent human. Oblivious to server performance and functionality, I hastily wrote the above function to quickly wrap up the progress of the quest system.

 

To see just how flawed this really is, take a look at the following:

We've all had this quest. It's simple, just walk x amount of meters and collect your credits.

In practice, it's just three clean lines of code all beautifully contained within the hook.

Now, think about it expanding this out for a second.

Every time a player takes a step, their file is read, checked, incremented, saved.

EVERY TIME.

EVERY SINGLE TIME. EVERY SINGLE PLAYER.

60 PLAYERS ALL RUNNING, ALL READING FILES, ALL SAVING FILES, ALL INCREMENTING, ETC.

 

Every damn time you take a footstep, your file gets read, checked, incremented, saved, 3 times for each quest, another time if you're sprinting. And that's just ONE/TWO quests. Imagine the disk usage costs!?!?!

 

This made it onto the live server. And only about a week later, @Kumo slapped me on the wrists and told me how much of an idiot I was. So, how would you fix something like this? The answer is a lot of orange juice, and a lot of patience.

Instead of reading your file every time you complete a quest action, the file would be read ONCE; When you first connect. Then, the data is saved to a table/array assigned to your player entity. So, whenever you take a footstep or complete a question action, all the system has to do is check if you have the quest, and increment accordingly. Finally, when you disconnect or after five minutes pass, your quest data is saved to the safety of the server. This however means that if you increment a hard quest (kill an event character) and the server crashes within that 5 minute window, that one kill is lost. Though, that is the sacrifice of a much needed performance boost.

 

Now, enough of that little story. I'm going to tell you how we (or at least, I) optimize the server today. I would like to say the developers follow a strict optimization routine with steps and instructions on each situation, but really, it's just bursts of energy that makes us want to optimize things. Anyway, there Is a general procedure followed to find and optimize content on the server. In three simple steps!!

1. Identification

2. Comprehension

3. Implementation

 

STEP 1: Identification

First, we have to know. What's lagging the server/client?? We use a neat little tool on the server called 'FProfiler'. What it does is record for a set period of time all the code/functions that is running and determines what is taking the longest to run. The profiler can be run on both server and client side. You'll notice when we use it on the server side as the server will start chugging and lag quiet dramatically. Whereas when we use it on the client side, it (as you guessed) only affects OUR own client and thus you won't be affected.

Here is a screenshot of a profile run on the server. In the first column, you can see the function name. Second column is the path. This tells us what addon the function is from and where we can find it. Third column is the lines of code the function can be found. And finally, the fourth column tells us how many milliseconds it takes for the function to run.

As you can probably guess, the higher the number is, the longer it takes to run, and therefore, the less performing it is.

Basically, bigger = badder.

 

STEP 2: Comprehension

Now, Comprehension. Is the function that is causing lag, something that we can really touch? Is it already optimized to it's limit? Is there something we can do? If we fix this, what will it break? These are all questions that must have an answer before we can really do anything to optimize. Commonly, things like TFA and the gamemode have been optimized to it's limit already and so are mainly glossed over. However, things like our custom content normally has some room for improvement and so these are commonly the types of things we look to optimize.

 

STEP 3: Implementation

In the implementation stage, the code is simply, well, optimized! It's hard to explain specifically as there is a million things that come with optimizing functions. So instead, I'll be explaining the process of how I optimized the augment system for clients on the 08/05.

 

Step 1:

This step is simple, first I noticed (and got many reports for) people crashing and generally low fps. So, I ran the FProfiler for myself and took a look. I don't have the screenshot now, but what I saw was that my augment system was in shocking NUMBER 2 (Number 1 being PAC3 of course). So of course, I quickly took to the Atom Editor and got to work.

 

Step 2:
The function in question, was the one that draws little transparent icons at your feet when you have that specific augment active. Obviously, since this was my own addon I knew that it could be optimized further and can be touched. So, I quickly took to the third step.

Step 3:

First, some prior understanding that you need to know. We are going to be touching a 'Draw' function, this type of function runs every single frame and displays an output to the screen. Because of this, bad code practice can lead to a lot of frame drops and thus must be optimized!!

Originally when I first wrote this section of code, there was really only one augment that could make use of the icon at the feet feature and so I wrote it with little room for expansion. As more augments came, the code got more and more spaghetti-fied.

 

As simply as I can put it, Cam3D2D is a way of drawing 2D things onto a 3D space. (This is what is used to draw the icons at your feet). When you use Cam3D2D, you must first 'start' the draw space, draw what you want, then end it. Originally, the code looked something like this:

Basically, it was a lot of code that didn't need to be repeated. So, the first thing I did was factorize the whole thing and removed the unnecessary repeats. A visualization of this would be something like this.

Ahhh. Much better! But, that's not it. Let's take a look at what specifically is inside the 'if has augment' section.

What this means is that for every entity, we check if they are the target of the 'Wookie Arms' augment. If they are, draw the icon. Think about this for a second. Remember when I said that this draw function runs EVERY SINGLE FRAME. That means, for every frame, every single entity is being checked if it is a wookie arms target. Also, by every entity I mean EVERY ENTITY. Every NPC, every player, every prop, everything. Obviously, this is what is doing the bulk of the lag.

How can we fix this? Well, we essentially have to make the loop run as little as possible. Ok. How do we reduce the size of the loop? Well, let's look at it this way. Why are we scanning every player to check if they are a target in the first place? Can't everytime someone becomes a target, they get added to a list and just loop through that list instead? Well, that's exactly right!!

Instead of looping through every damn entity, and then checking if they are a target, we simply just have to make a list of people who are confirmed to be a target! Something of that scope would look a little like this:

Obviously after x amount of seconds, they must be removed from the list and then sent to the player again. But BAM! Optimized! Now all that's left is to save it to the server an-

 

Oh....

I crashed the server when it was 60 players.....

because I misspelled wookie...

 

 

sorry!

[Galle]

You could of been optimizing stuff instead of writing this

 

[Billy_Barrel]

Very cool 

[TomCos]

I dont understand this tech mumbo jumbo! Make the server 100 FPS when 128 peolple please! i cant oplkay the server faehen its lakging becasunbre poor eokptimisation@!!!!!111

[Hamish]

why does my 4090ti @ 25Gsharts run at 20fps please fix or im telling garry newman 

[Fire]

Something I've been messing around with is different weapon bases, but they've been more experiments and probably won't leave the kitchen. But it's been interesting comparing codebases and taking inspiration from them. Unfortunately the TFA codebase isn't as clean as some others so it's been taking a while to learn the inner workings of it.

https://drive.google.com/file/d/1ouGM3zMPJHfR5aXminkQZkbVthGISNfA/view?usp=sharing

https://drive.google.com/file/d/1ZE4AX5iPAF5FfSD5pikR0AwvDCHoBbHK/view?usp=sharing

[Banks]

wow good post

[Twist]

@BanksWhens the next optimisation post?

[Banks]

3 hours ago, Twist said:

@BanksWhens the next optimisation post?

When you finish fishing