EZ Server

Yesterday we crashed 5 times on MCP... We could do 2 spawns and would crash on the 3rd spawn everytime.

Now today its doing the same thing on t3/t4 bosses.

Anyone else having this problem?

I used to have that problem consistently in POA until I greatly reduced the amount of characters I would bring to the fights.  It seems that stability goes out the window after 12 characters, so unless I'm doing MCP I bring 6-8 only and it hasn't crashed since.  The sad thing is that POA is designated as a raid zone, and if it is not able to support a traditional raid size force then something is definitely wrong.

Or we just normally crashed when it was my turn to snag a book...  :)

But yes like Bone said lots of crashes in air yesterday, poor Fugi ate alot of purples.

Yum Yum, meh I'm thinking kill 2 zone out zone in.. repeat I know it sounds.. dumb but like clockwork every 3rd boss no matter tier = crash..oh well @ least it is better then before how I'm looking at it...

Or maybe crashes rate increases with too many instances up? for some reason i get less crashes when less ppl doing POA regardless of being instances or public

So tonight poa instance has been crashing every second mob.... I've done with with and with out and 3rd party programs.... Seems like i'm spending more time loading goons then I am playing... Also 3 different people in 3 different states all get the crash.

Seems if you rezone every mob ... it works somewhat with less crashes. but shrug

Dont mind losing 64 to MCP ie wipe or whatever.... dont enjoy losing them to zone crash....

also I know this has all been addressed just keeping it all bumped to let people know..

Need to find out why. I know some people can do airplane and almost never crash. Maybe certain spells or 3rd party programs? I'm not the developer, so I'm feeling kinda helpless on this until we figure a pattern as to why its happening.

Not sure how hard it would be but what about changing to a different zone? I know even on Live Airplane was buggy might just be the zone.

NP Hunter, Hope one day we can find a pattern for ya , I know you do your best always just keeping everyone informed to farm stones and don't get to discouraged when or if the zone crashes..


Thanks as always..

 ;D ;D

Just some more information

Today Thebone and I, sat in PoA and ran through a quick 14 bosses (7 T3) (7T4) + 2 other sets that were testing.

We spawned 1 T3, then followed by T4 the engagements were less then 10 mins. Clicked PoA port rings to zone to public PoA and re-entered (after 2). We did this of course 7 times, we decided to try a third boss and boom zone crash when mob was about 20%, and again @ third mob this happened x2. (this happened on the 8th and 9th set of t3/t4 + 3rd spawns)

So basically I would zone out every 2 or 1 mob to prevent crashing if you are a victim of crashing, if not /meh lucky you..

All this post is for to gather and post more information no bitching  ;D

If the zone is crashing and not just clients, perhaps there's something in the logs that might give a clue.

I feel ya, Fugitive.

http://ezserver.online/forums/index.php?topic=1399.msg18322;topicseen#new

I really am sorry guys but this problem has been around along long time.  Dunno what to tell you other than I'm sorry.

yeah, I know what ya mean..

RNG sux... it seems to never favor anyone haha!!!

I'll post 20 T4 soon... just to post.. looking more at the crashes.. which do seem to be a lot less do to the zoning in and out.. but /shrug

Bump.  Fix this for the love of God.  This is a widespread issue among the veteran player base and nothing is being done.  It wastes so much time in lost stones and morale in seeing loot disappear before your eyes.  The problem is that after so many fights the zone starts to get sluggish and then just drops.  Take out he AE root and knockback effects.  They do absolutely nothing to make the encounter difficult.  They are a nuisance at best, and there is a high possibility that they lead to the zone instability.  Take it out to see what happens.  Nobody wants to make the fights easier, and ironically this will not change their difficulty at all.  Design content that the players can actually play and enjoy, as opposed to that which is buggy to the extent that they want to put their fist through the wall when they see 3 warlords in 1 day disappear due to zone crashes. 

Also....when a zone like HoH crashes, the zone remembers the static spawn that were up and where they were.  For example, if I clear half of the earth avatars and my instance crashes, I come back to half of the avatars left.  Apply this to the named mobs in Airplane.  Fix it so that they do not poof when the zone crashes.  Your player base will thank you 1000000000000x times over.  These two issues are creating an artificial cockblock against progression through t3/t4 that does not need to be there.  People want to play your game and enjoy the encounters without being kicked in the teeth. 

I couldn't have said it better...

Between the knock backs and the Root with illusion AEs... I would like to see them gone for a day or two just to see if they are the culprits..

yay a fresh zone...

and crashed again during kb/roots... yet another 64(MCP) wasted down the drain..

=/

I've seen other people able to do it with the kb/root without crashing the zone. I'm guessing its either your client and/or 3rd party programs. I might temp replace kb/root with a different damage spell later when I get time.

I am how ever considering reducing the stones by 1 color in the near future.

Thanks as always for the follow up =)

Sometimes it's no issues sometimes it's a crash fest with out without the 3rd party crap...

oh well

a stone reduction would help =P


 ;D

What does this even mean?  Other people are able to do it without crashing the zone?  Of course...you can successfully complete fights without crashing the zone.  But the zone is going to eventually crash.  It. will. crash.   For me it is around the 5th or 6th in a row.  Then boom goes the zone.  Please figure out a way to leave the bosses up after a zone crash.  It will greatly reduce the anger and frustration that this problem (which affects every single person that does t3/t4/4.0) causes.   

Thank you for considering reducing the stone cost.  It would help alleviate some of the time wasted.  Also please consider making each mob have the same chance to spawn.  I have some data I'll post soon about t4, but the warlord is frustratingly scarce.

That's not all true. I used to fight for hours and hours 20+ spawns in a row. I can't ever recall crashing the zone when it was just me or me and Shy. Not a single time - only when other players were in the zone did it seem prone to crashes to me. Which seems to indicate something about what they are running/doing.

I don't doubt it, my guess is some plugin in mq2 probably has an impact on it, and also potentially things like spells you cast/use or what scripts you may be using in mq2.

Try fighting without mq2 just with vanilla eq loaded and see if you crash as often.

Were these 20+ spawns prior to instancing?  Were you running MQ2?

So what you are saying is that you and shy played au natural and everyone who is playing now must be using a 3rd party program or something else that everyone else is universally doing that is causing the problem? I am not trying to attack you but to suggest that you and shy went up there from the git go and never used 3rd party programs is to suggest that everyone reading this is extremely naive. Sure you can go up there now and 2 box MCP now or whatever but you did not start out that way. When you first started going up there, the source of heals came from clerics and clerics alone. And it didn't come from one or 2.

Yea because a program I run on MY computer can crash the zone hosted on the SERVER.
Come on crab...
If I had a local client crash thats something I could go "oh shit MQ2 crashed my stuff"

That is an incredibly good point.

Jeez, everyone over react a bit. That wasn't what I was contending. However I think plugins that people run can absolutely crash zones - it was more than true on live and it's true here too. I think it has to do with the timing of how spells land/hit and the lag that is created with larger groups of people. MQ2 plugins were always capable of crashing zones - so testing without running mq2 at all is a great way of helping troubleshoot it is all. Hence my suggestion - could people read any deeper into that?

I fought both pre and post instancing Crawford - still no issues.

Oh and we did have a group test it that was crashing very often - and they crashed much less often without it. So it is certainly worth a try for you too.

I've done it with out any add ons no mq2 or isboxer and same happens

See now that's valuable input. I encourage anyone crashing to do the same and report back - see what kind of impact it has on your crashing issues. The more information that is provided the easier it gets. Do you honestly think if it was something easy/apparent Hunter/Basher/Secrets wouldn't have found it already and fixed it?

I know some people seem to be vastly aided in less crashed when running clear - this is actually the first I've heard of someone that isn't impacted. How many toons are you using Thebone on how many machines - and is it just you in the zone? How often are you crashing?

I can get crashes running vanilla 6 or 12 on same machine..
I can get crashes running 6 or 12 with isboxxer
I can get crashes with a default Global file
I can get crashes with a skinny Global file
I can get crashes using 2 machines with 6 on each.

The bottom line is the crashes happen..fresh zone.. 4-5 mobs in.. crashes happen with or with out 3rd party programs..

I know you have had good luck...
I've had decent nights then I have nights I crash on ever mob I pop.. with or without vanilla or mq/isbox

I was under the assumption that you knew that tests had been done, Crab, to determine that people crash without running any 3rd party program.  Fugitive (and others that I can't think of off the top of my head) has tested it all quite extensively.  All thanks in the world to Fugitive, but it would be nice if Hunter/Basher/Secrets would test as extensively as Fugitive has to figure this out.

I wish I knew more about coding and zone stuff.. but =/ I suck.. thats about the limit of my testing..

and

Ran it on a xp machine , Vista and Win7

xp machine was 32
vista and 7 were 64


on and Crashing on Shadow who has almost the same spell set lacking Shout Kills...

I crash on shadow normally every 2-3 shadows.. or depends on steps..


ironic close to same spell set..

Haven't tested shadow out as much as poa though..


btw this post is not bashing or bitching just information Im passing along

Well it's been a few months since I've been online. Believe me lots of tests were run back then, long before most people had even fought in airplane let alone MCP/T4. What we saw is people running with less addons and small groups were crashing far less if at all. If that's changed its information I didn't have. I do suggest everyone that's having the issue try it out though to see how it impacts them, it's certainly valuable info and individual users may see different results.

Methods to lessen the problem are always welcome and are definitely worth looking into.  That being said, they don't remove the problem.  Something is causing the crashes.  Either the zone needs to be re-designed from the bottom up or some serious work needs to be put into discovering where the problem lies (and not just work done by the player base).

We shouldnt have to not play a group cuz of crashes... Then why have a guild??? Also the stun and root cause high amounts of lag try removing them and see maybe?? If you do this maybe make there dots hit harder to make up for no stuns? But as far as roots its just a waste...

See i think the spell load has already been changed by Hunter. T4/MCP didn't root back then only t3 did. That tells me hunter/basher/secrets must have already tried changing up the spells to see how it impacted the crashing.

I would like to see just the KB/Root removed for 1 day let us test and post our results.. tomorrow could be a great day for testing =) I got stones..

and login server seems to be playing fair..

I might be wrong but i do think t4 roots.. stuns and DT also dot....

Spell load is definitely different if this is the case. Anyone confirm been a while since I did T4/MCP - if the load has changed which components are the same and which are different that would help narrow it down if it indeed it is spell related.

T3 does Stun/KB and Root.  T4 does Stun/KB/Root/deathtouch/ae poison.  MCP does all of the above, though I don't remember if it does AE dot.  The effects are negligible so I don't pay it much attention. 

Definitly the loadout has been changed then. Hunter would have to pull up the old loadout, but I know no root was in there at all and no knockback either that I can recall. Some people had the issue from the beginning too - so if it was spell load related it would have to be in the parts that haven't changed not the ones that have.

Guess that's my point, those two effects weren't there at the beginning and some groups still had zone crash issues. So it's likely to be the effects that were there not those 2, or it's unrelated.

At this point, since it is a zone crash, it would be really nice to know that someone has looked at those zone logs and seen if there were any clues there.  I know when my personal server has a zone crash I can usually find some clue by examining the logs and looking for things they have in common.

even in all the instances?

Especially since it seems that when zones crash it tends to wipe the instances. (Atleast I have lost a couple instances to zone crashes). That's 100k down the drain everytime the zone crashes.

I give up.  6th warlord of the past 2 days just poofed to a zone crash.  6 lost warlords.  Granted that's 12 rot Berserker breastplates, but it's still infuriating.  If there is no rot bot instituted in T4 and the T5 flag requires every piece, I'm done.  Hunter, you can't imagine how frustrating this is if you never experience it.  Move plane of air to a more stable zone or re-do the encounters, please.  You have players that want to play your game, will you fix it for them so they can enjoy it???

For a limited time, I've removed all spells from T3/T4/Epic4.0 Bosses to see if that has any effect.

In exchange, I've given the mobs a faster attack speed, with same damage per hit.

Let me know how that affects anything.

When T5 is released, or possibly earlier, I might consider lowering the stone color requirements, as stated before.

Ok i'm about to put it to test i'll let you know my findings

"Thebones" findings.. = dead monk dead monk dead monk...

/shrug


Results:

His monk sux


Thanks Hunter! Will run it through the motions..

ok so far....


20 PoA fights to include T3/T4/MCP


1 - crash Warlord
1 - crash H1N1

so.... 2 for 20.. looking better..
but why for the love of jesus does it have to crash on a bp mob !!! lol =)

I'm still trucking I'll edit or quote this post to keep updates..

Well 1/10 or 1/6 not outside of the realm of statistical probability to be same/same. Meaning it isn't related to the spells that the bosses cast .

I'm still running though...

and yesterday we were crashing every 2 ....

and 4 MCP crashes !! <----  >:(

so..

Today has been a lot better


Why don't you get back in game and help collect more data...


As I see it will the small amount collect
I am crashing far less.. but I intend to get about 50 done today..

hopefully

unless wife faction becomes unrecoverable.

Zones not stable, removing as many potential memory leaks as possible is an improvement.
I only did a couple before I decided to finish some back gearing but I will be up there later tonight and most of tomorrow.

well we have went from crashing every other boss to 1/20 i'd say it helped alot

MCP still has his spells. Didn't know if this was intended since you said above that you were removing spells from him as well.

Did 42
crashed 5


So about every 8.5 mobs.. or so

That's a long way from where it was yesterday..


Thanks Hunter for letting us get some data.

If stones price were dropped this wouldn't even be a problem I could eat 8/16/32 easier then where it stands now.. I know that is just bubblegum on a bullet hole, but I think that's about all the info I can give you..

There has been improvement for Thebone and myself, I hope others post their data so we can see.


Thanks for double drop too..

Doing only t3 mobs, I am also crashing every 7-8 mob or so.  One group, plus one above to hand in for spawn, in guild instance.  However, it did not seem to make a difference if I had two groups in, I had the same crash rate. 

Thank you for the modification so that we could check.

Talked to Crawford he was telling me he was zoning every three to four and his crashes was few and far between..


Yeah ours was T3/ T4 and MCP spawns... didn't matter if it was 12 or more...

Can't wait to see what craw posts..

Like I said its still a far cry from where it was, so im happy... I wish I knew more about what's happening in the back ground so I could look or offer more data..

Yesterday, during this "experiment", I killed 67 T4 mobs and 3 T3 mobs.  I crashed 2 times.  That is a far cry from the day before when I crashed 10+ times and lost 3 Warlords (Still bitter.  What's his coded spawn rate, seriously?  I'll have enough data soon for a prediction).  For the majority of the day I zoned out every 2 spawns.  For my last 20 kills I zoned out after the 10th and then of course I left after the 20th.  No crashes in that time.  The zone seemed to run more smoothly yesterday and I had very few issues.

Thanks for the chance to test this.  It helped morale in a big way. 

Just did 19  crashed on 20,


This is way better... was zoning on every 3rd... got greedy after 18 and was seeing if I could get more without zoning in and out.. 19 good.. 20... not so good.. lol  ;D

oh and btw  I think 8 of those were the priime healer with many back to back spawns..

Where the hell is the back to back Warlord spawns 9 -15?!

 yeah thought so..

and the burning prince was like the majority of the rest..

RNG my ass..

=)

They come after 20 in a row duh!

I know Im going to get laughed at for my lame math skills but the coded spawn rate I know I have somewhere, was posted on the web for a short while.  I seem to remember it being like .10 or something like that.

glad to see this is finally getting a good looking into to get this fixed.

Most of their spells are the SAME as PoD (T1/T2) zone bosses, and/or with higher damage values. Does airplane crash and not PoD? Cause its the same boss spells. At least we're trying some new things out. Could just be we're getting too high HP/Damage for the system to handle, shrug.

Don't know I can box upward 18 in pod for fun and never hiccup...

Hunter, btw thanks for being open to suggestions and trying out different things from poa to the new assassin quest to fixing spells, I know I appreciate it when I see weekly or daily improvements made to the server for your cult ..er minions...er playerbase..!!

Dont drink the grapejuice Fugitive

I know this is a different sort of PoA crash but I cannot zone in there on my second SoD install even though it was a direct copy of the first which zones in fine.  They both have the spell file and I have recopied all the PoA files with no success. 

I crash about half way on the loading bar every time.

Using win7? because i use it, and when i had it in program files, it would ask fro access to copy/replace so i would give it to it... but woudnt work properly...

So I moved folder to desktop, works like charm.

Have you messed with your golbel file at all?

Yeah Zart, it sounds like a modded global file.

This. I've seen problems before where standard folder length would be too long for the system to handle. You could put the item there, but not open it. Is your folder length too long? Try moving it to root if it is, see if that helps.

Vista and Windows 7 have special protections on files and directories under Program Files.  Since EQ isn't aware of how to handle those the OS does some monkey business to virtualize things.  Moving the EQ folder out of Program Files saves you a ton of headaches.

Yup, so i was told, so therefore put my EQ file onto desktop.

So basically based on what I am reading here it seems that the main issue is the server file getting filled up and hitting a max amount of information, or a set file size limit.  The zoning out and back in forces the system to create a new file.  Is there a way to set it so the file has an unlimited size and is recreated on server reboot?  Seeing as there are more and more getting flagged for T3/T4 it is becoming more of an issue.  Especially if you are fighting in public instead of guild instance.  Since in public you cannot control other players zoning out to reset the file. 

/shrug

really don't know sometimes lucky as hell on crashes other times /crash-mega-explosion.


It's just like HoH and Shadow if I don't go after him the zone never crashes, when I engage him 5-9 times the zone 100% crashes..

and just get a guild instance it's cheap

There is a registry key I've been using for a while that lets me Take Ownership of anything I need to in Win7/Vista. I wonder if doing that for the EQ folder in Win7 would be helpful?

I can link to the registry on my ftp server if it is helpful?

If you know the steps, you don't need a registry key to take ownership of a directory.  I don't know if that avoids the VirtualStore issue anyway.  Frankly, it's easier to just install somewhere else rather than fiddle with it.

Hoh and Air instances are crashing on me, I run 12 char and HoH will last about 30 or 40 mobs then crash. Air tonight crashed twice, I spawned H-1N-1 then spawned another and @ 40% crash. Came back and spawned another H-1N-1 and at 40% air crashed again. I know this has been around awhile but I sure wish it could get addressed somehow.

I usually stick to 6 of my best toons when it comes to Air // HoH I rarely ever crash only using 6. Some people say its Shamans and kraken, but I use one in my 6 man crew and I crash about every.. 5-6 hours on a normal day inside a GI

While this is possible, I believe it's more likely an instantaneous overload. I crash far more often during prime time than during slow parts of the day, if it was a data storage problem of this type it should stay consistent. Perhaps when the server is busy and it takes longer to cycle through the data, something grows out of its type.

Though if there was a specific variable or file being used for all instances of the same class..maybe. Anyone with more knowledge on emu/ez code have an insight?

Came across something interesting today. I was logging scenarios to try and narrow something down with these crashes, air specifically. For all the characters, a zone crash resulted in the log file ending seemingly at random. But my warrior logged this very final line after one of the crashes:

[Sat Oct 15 16:18:15 2011] ERROR: String not found. (2088400502)

2,088,400,502 error code number
2,147,483,647 max range of a signed integer

On the same factor with a small difference less than 60 million.

My notes show the zone crashed when my 7th The Warlord was around 50%. My rogue is my lead dps and on average does 25 million damage per warlord. With the other 11 other classes there, the error code number aligns almost perfectly with a variable saved after the last kill before the final npc that crashed it. (Total damage - current warlord) Based on average dps, the number predicts almost perfectly when that value would overflow and when the zone would crash. (Total damage -> overflow)

Sorry if I'm wasting your time with more of the same, hope it helps,
Blarr

Paldail,
I tried this last night and crashed after one H-1N-1. I used my war, pal, pal, mnk, rng, brd. I checked dps out beforehand and this group is putting out just over 100kdps. I was buffed with pets buff's and used shm hp buff and chanter epic 3.0 haste and vampiric thunder buff.

What classes do you run?

Blarr,

I emailed your post to hunter and this was his response, maybe you could post your data there?

As of right now I run Pal, Pal, War, Monk, Mage, Shaman as my team that I pushed to 5.0 with. But keep in mind the 4 melee have UC paired with a shaman for Kraken which makes the shaman somewhat viable. Mage is ok dps could probably be replaced. But either way I take down T3 bosses under 3min I think and T4 bosses is like 4-5min. Of coruse H1N1 and Warlord take a bit longer since they are stacked with HP.

Bottom line is no matter what you choose, stay with it dont make to many alts finish your main Team UCs and AUGs as soon as you can of course all DPS is decent.

Enjoy your game no matter what you choose...

I'm good toons, was just trying to understand if I am using a diff char that may be causing an increase in crashing

I posted this issue on the emu forums and there were some interesting responses about a 4k memory leak.

http://www.eqemulator.org/forums/showthread.php?p=204174

Neat!

Good find and Good fix Lerxt,

Just some thoughts I was having.

If this 4k Leak was a culprit why in other zones say PoD we can have 18-30 toons with out it ever crashing?

But put 18 toons in PoA, HoH(sometimes), Abyss the Crash down timer starts tick tick tick..

These are just questions. I may not have conveyed my thoughts right but hope you get what I mean.

I do think this is diffently in the right direction.

Anyone know if Hunter is going to push that part of code?

Would be outstanding if this is on the right road to get all these High end Tier Zones to stop /Q'ing out on us haha!!

I'll try to update the source code this weekend and see if that fixes things.

Thanks for the updates everyone!

No Thank you for everything you do Hunter!

I thought that too.  I haven't hunted in PoA, but maybe there are more messages generated there or more people filter things out when they are there. *shrug*

It's really just guessing without seeing a callstack for where the crash happens.  Either way, it won't hurt and maybe it will help a bit.