Clan x86

Announcements => [x86] Announcements => Topic started by: iago on July 13, 2008, 08:26:54 pm

Title: Downtime tomorrow (July 14, 2008)
Post by: iago on July 13, 2008, 08:26:54 pm
Hey everybody,

I'm going to be swapping out the Web server tomorrow. I already have Apache and PHP installed on the new server, and I'm actually using it right now (hitting the same DB as the old server). So basically, I'm prepared to start the migration.

However, it's going to take some doing, and it's going to break stuff. I guarantee. So be prepared for a little downtime/instability tomorrow evening when I do it. I expect no more than ~30 mins of downtime, followed by a couple hours of instability (as I fix things I broke), and probably another week of minor instability as people report other broken things to me.

I'm hoping this goes smoothly. The biggest change is to security -- I will be using suphp on the new server, so everything will run in the context of its owner. That means that if somebody writes crappy code in their home directory (for example, if I upload something stupid to ~ron), it can't affect other sites on the server without a privilege escalation attack or similar. I'm also going to be making other changes that you probably won't notice.

The downside of using suphp is that PHP has to run as a CGI module instead fo an Apache module. That means it runs somewhat slower. When I first tried this, it was noticeably slower, but I upped the RAM dedicated to the Web server and now it's running the same as the old one. I guess it just likes having the extra RAM.

So yeah, expect downtime tomorrow, everything should be back to normal after that.
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: Camel on July 14, 2008, 12:28:04 am
If you look at the actual implementation in apache of how CGI interactions occur, you'll quickly understand why PHP became a module :)
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: iago on July 14, 2008, 08:20:10 am
Yeah, it's definitely understandable. But I'm willing to give up the performance gain for the added security of running scripts as their user rather than as apache.

It's odd that PHP doesn't have anything built in for that, yet...
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: Skywing on July 14, 2008, 11:50:17 am
Ouch.  Switching to CGI from a server module is painful.

I would make sure you have response time and CPU/memory usage graphs before and after so you'll have a baseline for how much of a performance degredation you are looking at.  (In my experience, it's been very severe.  I would not recommend it at all.)

In general, however, I would assume that any PHP code uploaded to the server can run native code, and thus simply not allow untrusted PHP code.  PHP is a mess; just look through bug reports (http://bugs.php.net/search.php?search_for=crash&boolean=1&limit=10&order_by=&direction=ASC&cmd=display&status=Open&php_os=&phpver=&assign=&author_email=&bug_age=0) with all the various heap corruption and other almost surely exploitable but not until somebody releases a proof of concept (for purposes of fixing them in a timely fashion, from the PHP team's perspective) problems.

I would stick with the apache module and not run untrusted PHP code, and let that be the end of it.
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: iago on July 14, 2008, 12:22:43 pm
I do have graphs I can check, so I'll know. For the amount of traffic/weight of the apps, I'm not too worried. I wonder what hosting providers do to prevent others from looking at their code, though?

After talking to you, I think I'll install the main stuff as a module, and when others want an account or want to use code, I'll let them do it in the context of themselves.
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: Camel on July 14, 2008, 12:58:46 pm
Facebook uses PHP; their index.php file was leaked after they failed to secure their application platform.
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: iago on July 14, 2008, 09:43:00 pm
All right, this is done. I went with mod_php for the forum, and cgi for everything else. If it causes serious issues, it's a quick fix to change it.

I'm aware that this broke themes, I'm fixing those right away.

I'm aware that this will also break a lot of other things, please let me know. I went with the, "if I don't remember it, don't allow it" strategy, and will fix things on a case-by-case basis. :)

Oh yeah, and I moved the forum to forum.x86labs.org. You'll be required to log in and to remember a new URL. Deal. :)
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: Ergot on July 14, 2008, 09:54:50 pm
You bitch. Should I update the RSS url as well?
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: iago on July 14, 2008, 09:56:41 pm
Yes, you most certainly should. :P

<edit> Also, you win the award for being the first person (besides me) to access the site by its new URL. Congratulations!

MetalMilitia was a close second (10 seconds after). :)
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: iago on July 14, 2008, 10:02:21 pm
Themes should be working now. Let me know if any aren't, or if you have other issues!
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: Ergot on July 14, 2008, 10:12:54 pm
Smilies are @ fail? I think the favicon is a bit too racy.

And really? I think I was just auto-redirected :O! What do I win?
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: iago on July 14, 2008, 10:24:40 pm
Smilies are @ fail? I think the favicon is a bit too racy.
Oops, should be fixed now.

Haha @ the favicon.. for some reason, that icon was always in the /forum folder, but obviously it was never used. That's awesome! Anyways, I deleted it.

And really? I think I was just auto-redirected :O! What do I win?
The task of making a new favicon!
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: Ergot on July 14, 2008, 10:45:49 pm
[tex]x86[/tex]

^--- Looking good?
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: iago on July 14, 2008, 10:53:17 pm
[tex]x86[/tex]

^--- Looking good?
Haha, no. :)

Blaze did one, though! http://forum.x86labs.org/favicon.ico
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: Ergot on July 15, 2008, 04:53:23 am
Isn't that the same racy one from before :O?
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: iago on July 15, 2008, 08:16:25 am
It's surprisingly hard to make Firefox update it. Refresh a bunch of times.:)
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: Camel on July 15, 2008, 02:45:48 pm
iago, [spoiler] tags still don't work. Is this related to the move? :)
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: iago on July 16, 2008, 08:27:28 am
Hmm? Did we ever have spoiiler tags?
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: Joe on July 16, 2008, 09:16:31 am
[spoiler]no[/spoiler]
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: Camel on July 16, 2008, 12:34:22 pm
I agree with Joe. You should add them.
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: Sidoh on July 16, 2008, 10:38:44 pm
I actually agree. I think it'd be a nice feature for a bunch of different forums.  The obvious ones are the media for movie reviews and such, as well as the math/puzzles forum when posting answers or revealing hints.
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: iago on July 16, 2008, 10:40:31 pm
I actually agree. I think it'd be a nice feature for a bunch of different forums.  The obvious ones are the media for movie reviews and such, as well as the math/puzzles forum when posting answers or revealing hints.
Feel free :)

I think Blaze found something earlier, but I was at work and wasn't really paying attention. :)
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: Camel on July 16, 2008, 11:01:08 pm
Something like this (http://custom.simplemachines.org/mods/index.php?mod=50).
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: iago on July 18, 2008, 07:12:44 pm
So does anybody care about the forum path change (x86labs.org/forum -> forum.x86labs.org)? Did anybody even notice? :P
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: deadly7 on July 18, 2008, 07:32:34 pm
So does anybody care about the forum path change (x86labs.org/forum -> forum.x86labs.org)? Did anybody even notice? :P
Me! I had to log in again. :P
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: rabbit on July 18, 2008, 10:18:54 pm
So does anybody care about the forum path change (x86labs.org/forum -> forum.x86labs.org)? Did anybody even notice? :P
Me! I had to log in again. :P
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: Joe on July 19, 2008, 12:00:15 am
So does anybody care about the forum path change (x86labs.org/forum -> forum.x86labs.org)? Did anybody even notice? :P
Me! I had to log in again. :P
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: iago on July 19, 2008, 08:51:31 am
So does anybody care about the forum path change (x86labs.org/forum -> forum.x86labs.org)? Did anybody even notice? :P
Me! I had to log in again. :P
Well, that would have happened anyways, so get over it. :P

Besides logging in, I meant the name itself.
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: deadly7 on July 19, 2008, 01:46:38 pm
Well, that would have happened anyways, so get over it. :P

Besides logging in, I meant the name itself.
Well, when I wasn't automatically logged in, I wondered what was up. I knew I didn't have my cookies deleted or anything. So I snuck a peek at the URL and noticed the change. :P
Title: Re: Downtime tomorrow (July 14, 2008)
Post by: Camel on July 20, 2008, 11:06:44 pm
Something like this (http://custom.simplemachines.org/mods/index.php?mod=50).

bump