Server Maintainence

B

Billy

Hey guys, Billy here. If you noticed that the servers were offline, that was because they did a location change on the servers, causing the servers to go offline for a little bit. The servers are slowly coming back online! Enjoy Your day!
 
Server #1 is offline because the company we're going through is switching their provider. They have been having issues with hardware and connections for the past month.. and decided to switch from an LA-1 to an LA-2 datacenter located 2 miles from the original location. The expected downtime is 3 days, but i'm working diligently to get our stuff moved.

The backup build server is online again, to cover for now so we still have a place to build. Servers #2, #3, #4, #5, Murder, Deathrun & Prophunt had a router upgrade yesterday and they are configuring and working the bugs out of it which is why they were down today. We should have no further issues from this point on, aside from the Build Server fiasco.

NFO said:
Between 3am and 7am CDT on Wednesday, March 11, we plan to switch over to a new router in Chicago. The new router is already physically installed and ready to go, which means that we just need to move over the fiber connections to activate it. We expect for this process to take a few minutes, if everything goes smoothly, during which time your server will be unreachable.

After the cutover, we will be testing the new router to make sure that it works properly. This should generally be non-intrusive, but there is a chance that we may need to switch back to the old router, or that we run into a problem, leading to some further downtime during the maintenance window.

During this window, on-site staff will also be accessing a cabinet to install new servers. This should not cause an impact, but they will have to physically move a power strip while it is live, so there is a slim chance of a power disconnection for some machines. We will encourage them to be as careful as possible with this operation.

Update @ 4:37am CDT: We are still working on this. We have tried the new router twice and it has crashed twice (forcing us to revert back to the original router), but we believe that we may have isolated the cause in software and will be trying once more. If that fails, we will be doing yet more testing in the lab (on top of the dozens of hours done already, which, strangely, showed no problems!).

Update @ 6:06am CDT: The new router is in place and running. We are continuing to monitor, but the maintenance is finished.
Click to expand...
 
Server #1 has successfully been moved to a new location in California (LA-2) and is up and running again.
The old server is still running on (LA-1) for an undetermined amount of time.
Our hosting company has cancelled that network and the server will remain until that subscription period ends.


TBB Build (LA-2) - 173.234.253.22:27015 (NEW)
TBB Build (LA-1) - 192.210.148.245:27015 (OLD)

Server #1 Host said:
After a full day of downtime, the datacenter has brought the server back online.
They have chosen to not provide is with accurate information yet again, so once we have cloned all game servers to the other LA datacenter, we will be requesting SLA compensation.
Then once complete, cancelling the service with them as they do not know how to run a network, or even change out a stick of RAM, based on what they have told us in regards to the downtime.
We are sorry for the extreme amount of downtime, and will be compensating at a minimum of one month credit.
Everything within our power was done within 30 minutes of the server being detected as down, the rest has been due to ignorance in the LA datacenter which we have been with for 7 years, but apparently they want to no longer provide proper service.
Click to expand...
 
You must log in or register to reply here.
Back
Top Bottom