EZ Server

General Category => Updates => Topic started by: Hunter on November 28, 2012, 11:41:39 pm



Title: ROLL BACK 11-28-12
Post by: Hunter on November 28, 2012, 11:41:39 pm
Update: Server is back to normal!

Figured out the problems. When updating the source and database, I also had updated some plugins.

One of them would crash the zone if you hailed an NPC that was checking to see if you had an item on you, for example a T2 BP for flag or Incomplete Epic Book for the pop up warning to combine first.

Second one would eat your items that you handed due to the check_handin plugins new code.

Both plugins, as well as some others have been reverted back to their original. The updated ones were causing the issues all this time.

Secrets put a fix in the new source code (which we are not using yet) to prevent the zone crash plugin bug, meaning we could upgrade the plugin without crashing the zone after we upgrade our source.

We will probably do double loot for Saturday and Sunday due to the down times we've had and the short roll back we had.

When we get the new loot system working correctly on a test server, then I will back up the EZ Server just before doing the update so any roll backs would be minimum. Will post about it too when doing it. Probably won't attempt it though for at least a week.

The people that have emailed me so far about Epic's getting eaten will be reimbursed soon through email.

Everything should be safe to play and continue your gaming as normal.

Enjoy!



Old / Original post below about roll back:

EZ Server was rolled back by 18 hours.

Originally tested the new source code including the new loot system (tables) on a test server and it appeared to work fine.

Then updated the live EZ Server and still appeared fine.

Went to bed and woke up later to find out certain loot list are broken, and zones are crashing.

Hateborne and Akkadius are trying to help. Currently no solutions yet.

I don't know when I'm going to try to update the source again. Might try to hammer this out on the test server since we know what to look for (crashes and loot problems).

Feel free to play and grind now since we are rolled back to a working copy.

If I update the source code again, then I will make another backup before doing the update again, so if this problem happens again we can roll back to a very recent update again.

For the majority of players work lost, it was only 16 hours total during mostly non-peek hours. Things could be worse. Luckily the roll backs we've had to do are very rare.

Hope that answers most questions. Feel free to ask anything that is not clear.


Title: Re: ROLL BACK 11-28-12
Post by: Akkadius on November 28, 2012, 11:47:25 pm
The only standing issue were the intermittent crashes which are undetected and not reproduced.

Once these issues get addressed from not just EZ servers standpoint from the rest of the community things should be fine.

There were a lot of updates cumulative to when he upgraded.


Title: Re: ROLL BACK 11-28-12
Post by: Hunter on November 29, 2012, 01:06:29 am
Rolled back again.

Players were leaving loot on their corpses.

Now in addition to rolling back the database AND source, I rolled back the new quest I created for logging into a db table instead of write() to text file.

So far I can't seem to duplicate leaving loot on my corpse, as I died a few times already.

After all zones are up, then I'll unlock and we'll see if the zones continue crashing.



Title: Re: ROLL BACK 11-28-12
Post by: Hunter on November 29, 2012, 01:40:18 am
Trying to pin point where the crashes were coming from.

I removed Epic Vendor, Keeper of the Ages, and AA per hour logging, and now we're not crashing anymore.

Corpses and Regen are working fine too.

Will probably add 1 quest at a time back to find out where the problem was. For now we're doing ok.


Title: Re: ROLL BACK 11-28-12
Post by: Strix on November 29, 2012, 03:10:12 am
Hi Hunter

I've been trying to login for the last hour with no luck.  Occassionally get to the character select screen but that's about it.  Most attempts give the "Unknown Error when trying to connect to the server" message.

Strix


Title: Re: ROLL BACK 11-28-12
Post by: Hunter on November 29, 2012, 05:09:42 am
Secrets may have fixed our bug!

Not tested yet but we're 99.9% sure.



Title: Re: ROLL BACK 11-28-12
Post by: Rotic on November 29, 2012, 05:15:37 am
Good to hear! Cheers.


Title: Re: ROLL BACK 11-28-12
Post by: Mattnaik on November 29, 2012, 07:10:44 am
Is it safe to farm some SLS's? I spent all day yesterday while at work (shh don't tell me boss!) farming them and they're gone. No biggie I'd just rather not do it again if there's a possibility of rollback.


Title: Re: ROLL BACK 11-28-12
Post by: wolfegunr on November 29, 2012, 07:25:37 am
Good to hear, ty both for your hard work, Secrets good to have you back in action around here you rock dude!


Title: Re: ROLL BACK 11-28-12
Post by: Secrets on November 29, 2012, 07:27:07 am
Is it safe to farm some SLS's? I spent all day yesterday while at work (shh don't tell me boss!) farming them and they're gone. No biggie I'd just rather not do it again if there's a possibility of rollback.

Should be fine but I'm not the one to make that kind of call. The server was rolled back because of the loot changes, and the loot changes were reverted, so it should be 'safe' perse.


Title: Re: ROLL BACK 11-28-12
Post by: Blurring on November 29, 2012, 12:01:59 pm
Good to hear, ty both for your hard work, Secrets good to have you back in action around here you rock dude!

+1, thanks Hunter and everybody else helping :)


Title: Re: ROLL BACK 11-28-12
Post by: Akkadius on November 29, 2012, 01:20:41 pm
So ultimately the issue was a null pointer in a Perl script that has to do with item hand in hand-lings, which has nothing to do with the updates. But the plugin for handling item turn ins was was updated to support some safety netting and sounds to me from the players that item turn ins are completely failing altogether right now


Title: Re: ROLL BACK 11-28-12
Post by: Akkadius on November 29, 2012, 01:55:20 pm
So ultimately the issue was a null pointer in a Perl script that has to do with item hand in hand-lings, which has nothing to do with the updates. But the plugin for handling item turn ins was was updated to support some safety netting and sounds to me from the players that item turn ins are completely failing altogether right now

Pretty safe to say any 'crashes' are taken care of and are identified, just a matter of time for Hunter to wake up and clean the rest of the mess up.


Title: Re: ROLL BACK 11-28-12
Post by: Fliker on November 29, 2012, 03:52:56 pm
So ultimately the issue was a null pointer in a Perl script that has to do with item hand in hand-lings, which has nothing to do with the updates. But the plugin for handling item turn ins was was updated to support some safety netting and sounds to me from the players that item turn ins are completely failing altogether right now

Pretty safe to say any 'crashes' are taken care of and are identified, just a matter of time for Hunter to wake up and clean the rest of the mess up.

Like any and all turn-ins?  (CG/FG in Nexus and RoA in Surefall)
and Thanks for helping Akka.

edit: nvm should of read other threads first

seems all broken...FG/CG/Crafters Epics (all seem to get eatin) quests from SFG arent working either.....so assuming all are broken


Title: Re: ROLL BACK 11-28-12
Post by: 1flytrapp on November 29, 2012, 04:47:55 pm
FG quest was Working for me i finished CG on one of my rangers


Title: Re: ROLL BACK 11-28-12
Post by: Akkadius on November 29, 2012, 06:17:34 pm
So ultimately the issue was a null pointer in a Perl script that has to do with item hand in hand-lings, which has nothing to do with the updates. But the plugin for handling item turn ins was was updated to support some safety netting and sounds to me from the players that item turn ins are completely failing altogether right now

Pretty safe to say any 'crashes' are taken care of and are identified, just a matter of time for Hunter to wake up and clean the rest of the mess up.

Like any and all turn-ins?  (CG/FG in Nexus and RoA in Surefall)
and Thanks for helping Akka.

edit: nvm should of read other threads first

seems all broken...FG/CG/Crafters Epics (all seem to get eatin) quests from SFG arent working either.....so assuming all are broken


Not all turn in's because it has something to do with checking whether a client has a specific item, so wherever he has checks for that.


Title: Re: ROLL BACK 11-28-12
Post by: Hunter on November 30, 2012, 02:45:28 am
Akkadius is right.

Some NPC's only used a plugin to check if you have an item.

Some NPC's only used a different plugin to check the items you gave to the npc.

Some NPC's used both plugins.

With help from Akkadius and Secrets, we've figured out and should be safe to play again.

The original post for this topic was been edited with new details.

Enjoy!


Title: Re: ROLL BACK 11-28-12
Post by: Fanon_Emarr on November 30, 2012, 08:31:01 am
Big thumbs up to double loot. I missed out on Thanksgiving due to family stuff, so I'll gladly take the freebie.

Thanks for all the hard work as usual, Hunter.


Title: Re: ROLL BACK 11-28-12
Post by: Bladethorn on December 03, 2012, 08:00:49 am
Hunter, Thanks for the double loot weekend! Totally appreciate it, made grinding out some needed charm/armor upgrades less painful.