I read through the list you posted in the link. Did at some point in time all these things get changed? You already had the server running great with very little performance issues. These performance issues just popped up in the last month or so which leads me to think something specific being updated and/or starting to get a hardware failure. performance updates are always good but since these issues just popped up rather recently seems something went haywire. maybe a backup with old settings etc.
just very odd since it used to run great.
The root issue that was causing the server to have "lag" issues at primetime was related to a virtual machine running on the same host (EZ Dev server) that was running hard at 100% CPU for whatever reason, I just shut it off. This was bleeding into the resources available for EZ production to the point where the server would start to trip over itself during primetime.
While I had the server on my desktop, I took the opportunity to deep dive all of the zone processes and find inefficient code paths and took out a big chunk of overhead. It's just not something that anyone else is doing in EQEmu and I had the ripe opportunity to do so with 700 players on. So to eliminate this footprint, all of EQEmu benefits as a result.
Yes the server was handling the load prior just fine but there are things that also add up and contribute to the overall issue. So while there was the root issue, there are the compounding multiplied issues of zone processes bringing the server to that point as well.
Before we ran 10-30% CPU on a normal load, but that started getting out of the average trend with many various factors playing a part in this. Some zones consistently stay at high CPU loads and I've found the issues behind those as well.
People make code changes over time, script changes, without realizing the negative affects of these changes and they need to be fixed/optimized.
The Perl issue has been an age old issue forever and is a huge win for EZ and EQEmu, though EZ will benefit largely because it is very script heavy.
All of the other overhead issues resolved help in a multiplied sense.
Large zones present issues in themselves that people arent often aware of (oldcommons, frozenshadow, sunderock (T10)) that these fixes also tremendously benefit the overall latency and responsiveness of the zone.
Here are the commits as well:
https://github.com/EQEmu/Server/commits/master