EZ Server

General Category => Updates => Topic started by: Akkadius on November 04, 2015, 12:25:52 pm



Title: Wednesday
Post by: Akkadius on November 04, 2015, 12:25:52 pm
Today I will be shutting off DL as expected, EZ Server will be moved from my desktop back to its rightful place in the server rack.

I've made a tremendous amount of performance related changes and on top of that fixed issues on the server end and now is the true test to see if the server will still continue to have issues. Some of those you can see here: https://raw.githubusercontent.com/EQEmu/Server/master/changelog.txt

This is my ask for you guys to keep your eyes peeled in regards to lag issues that could come back up, if we experience issues again I will have to evaluate the next step, but we should be good to go.

Halloween will be shut off as well.

I may re-enabled DL for Friday or Saturday as well to stress things on the server again but we will have to see.

Akka


Title: Re: Wednesday
Post by: Brannyn on November 04, 2015, 12:36:13 pm
You are too kind to us


Title: Re: Wednesday
Post by: Drep on November 04, 2015, 01:07:03 pm
I read through the list you posted in the link.  Did at some point in time all these things get changed?    You already had the server running great with very little performance issues.    These performance issues just popped up in the last month or so which leads me to think  something specific being updated and/or starting to get a hardware failure.  performance updates are always good but since these issues just popped up rather recently seems something went haywire.    maybe a backup with old settings etc.   

just very odd since it used to run great.


Title: Re: Wednesday
Post by: Akkadius on November 04, 2015, 01:22:01 pm
I read through the list you posted in the link.  Did at some point in time all these things get changed?    You already had the server running great with very little performance issues.    These performance issues just popped up in the last month or so which leads me to think  something specific being updated and/or starting to get a hardware failure.  performance updates are always good but since these issues just popped up rather recently seems something went haywire.    maybe a backup with old settings etc.  

just very odd since it used to run great.

The root issue that was causing the server to have "lag" issues at primetime was related to a virtual machine running on the same host (EZ Dev server) that was running hard at 100% CPU for whatever reason, I just shut it off. This was bleeding into the resources available for EZ production to the point where the server would start to trip over itself during primetime.

While I had the server on my desktop, I took the opportunity to deep dive all of the zone processes and find inefficient code paths and took out a big chunk of overhead. It's just not something that anyone else is doing in EQEmu and I had the ripe opportunity to do so with 700 players on. So to eliminate this footprint, all of EQEmu benefits as a result.

Yes the server was handling the load prior just fine but there are things that also add up and contribute to the overall issue. So while there was the root issue, there are the compounding multiplied issues of zone processes bringing the server to that point as well.

Before we ran 10-30% CPU on a normal load, but that started getting out of the average trend with many various factors playing a part in this. Some zones consistently stay at high CPU loads and I've found the issues behind those as well.

People make code changes over time, script changes, without realizing the negative affects of these changes and they need to be fixed/optimized.

The Perl issue has been an age old issue forever and is a huge win for EZ and EQEmu, though EZ will benefit largely because it is very script heavy.

All of the other overhead issues resolved help in a multiplied sense.

Large zones present issues in themselves that people arent often aware of (oldcommons, frozenshadow, sunderock (T10)) that these fixes also tremendously benefit the overall latency and responsiveness of the zone.

Here are the commits as well: https://github.com/EQEmu/Server/commits/master


Title: Re: Wednesday
Post by: balidet on November 04, 2015, 02:02:57 pm
"Thank you"

really I don't know what else to say..

but I mean it!




Title: Re: Wednesday
Post by: Drep on November 04, 2015, 02:12:04 pm
Yeah,  very much appreciated Akk!


Title: Re: Wednesday
Post by: Noot on November 06, 2015, 11:37:53 am
Much Appreciated!