I have found the pattern: 503s occur daily at 4:00 PM PDT. In UTC (the time the server uses), this is 12:00 midnight. Because servers often have cron jobs that run at this time, I started moving vBulletin cron jobs like cleanup and birthday notices to 9:00 AM UTC (about the quietest part of the day). A week or so ago, FreezeWarp recommended I run all of them one by one to find out whether any of them caused a forum blackout. None of them caused outages and the errors continued promptly at midnight. Just today, I thought about server jobs and why only the forum goes down.

Then it hit me: the forum has a very large database. None of the other sites have one anywhere near its size, thus they only are down for about 10-15 seconds. The daily backups cause the downtime! Hopefully, we'll be able to move these to around 11:00 AM UTC, so they occur after the forum cron jobs and before the activity begins going back up.

So, the 503s were actually something good, just not running at the right time. With that, I should point out that the forum will be going down daily for scheduled maintenance at midnight UTC. (I know it's a terrible time. We're working on moving it!)

Good thing I'm not on at that time.

So what time is UTC in Texas... CST?

Midnight UTC = 6:00 PM Central. Note that that is only during non-DST. (I HATE DST!)

Yay no more getting logged off for no reason! I dont know why but the 503 messages make me log off.

Zorua, are you putting a check in the 'Keep Me Logged In' box? Just wondering.

And 6 PM sucks for me, because 6 is when I get on VR. O_O

Yes I am. But it still logs me out. I should try clearing cookies.

Well, I'm at least glad to see that it's not due to us leaking memory again.

Well I'm to lazy to check what that is my time, but, I'll check later. Good you figured it out!

Memory leaks were one of the causes, yes.

Yeah, I know that time sucks. That's why I want it moved.

If you are getting weird errors with staying logged in, definitely try clearing cookies.

Well, at least we know what's causing 'em. I did notice once that it happened around the same time two nights in a row, but I only thought of it as coincidence.

I believe I have no problem with the timing, since I think I'm in school during that time.

Both of the timings, the old and the new one are fine with me! I would be sleeping during the old one, and at school during the new one!

I haven't been getting 503s.

Some of them seem to be due to the memory overload, but look at what Cat said. The server probably shuts off access to the database during the backup to prevent data from changing during the backup.

No, servers don't normally do that. I remember that time Will, Alternate, and I were posting during the pre-move backup back in phpBB. It doesn't close.



Also, the memory thing is because the con jobs or whatever he called them must take up a lot of RAM based on what operation they do.

I haven't seen a 503 yet. I think it closes down the server to prevent memory leaks. It's irritating to the users but activates as a protection mechanism.

What I meant before was that ispCP Omega may be shutting off database and/or site access during its backups, not vBulletin.

Isn't the vb on the database tho?

This reminds me of how every Chatzy server always has to crash for 3 minutes at 12:32 PST. Well, at least we found the reason why it kept doing that.

By the way, if you didn't already know Cat, I experienced a "Database error" yesterday at night, around 9:30 PM. Does it have to do with those backups?

We're moving servers soon. Don't bother mentioning the last hour of database errors.

Fivebean sucks.

I'm guessing you'll put the message on huge letters in the index... And I'll still not notice it.

Yeah, I don't want a bunch of noobish mods (you remember who you are...) posting during that time, forcing me to lock the whole forum even from viewing.