Important Things 1999

IT1999: Having discuzed a lot of software, maybe it’s time to change some other topics.

To know the architecture of other websites, or basic opration, or maintanance, the following information may be useful for a webmaster.

IT1999: There is no new ideas of architecture, only reconsidering cache, SLB(Server Load Balance) and network storage with the most cheapest solution. All of concepts existed early before 2001, the difference is now we have more choices, especially with open source codes, and there is no need to explain the concept of SLB and CDN any more. Unlike Sina.com.cn, Sohu.com, none of them use Alteon(Nortel now), F5, Radware, … as SLB device. So does caching device, most of  web 2.0 sites use open source application to cache images.

======================================

QCon San Francisco 2008: http://qconsf.com/sf2008/

Web 2.0 Summit 2008: http://en.oreilly.com/web2008/public/content/home

MySQL Conference & Expo 2008: http://en.oreilly.com/mysql2008/

More: http://conferences.oreillynet.com/

Others:

Wikimedia Architecture: http://highscalability.com/wikimedia-architecture

Youtube Scalability:

http://kylecordes.com/2007/07/12/youtube-scalability/

The following content is copied from above URL.

YouTube Scalability Talk

Cuong Do of YouTube / Google recently gave a Google Tech Talk on scalability.

I found it interesting in light of my own comments on YouTube’s 45 TB a while back.

Here are my notes from his talk, a mix of what he said and my commentary:

In the summer of 2006, they grew from 30 million pages per day to 100 million pages per day, in a 4 month period. (Wow! In most organizations, it takes nearly 4 months to pick out, order, install, and set up a few servers.)

YouTube uses Apache for FastCGI serving. (I wonder if things would have been easier for them had they chosen nginx, which is apparently wonderful for FastCGI and less problematic than Lighttpd)

YouTube is coded mostly in Python. Why? “Development speed critical”.

They use psyco, Python -> C compiler, and also C extensions, for performance critical work.

They use Lighttpd for serving the video itself, for a big improvement over Apache.

Each video hosted by a “mini cluster”, which is a set of machine with the same content. This is a simple way to provide headroom (slack), so that a machine can be taken down for maintenance (or can fail) without affecting users. It also provides a form of backup.

The most popular videos are on a CDN (Content Distribution Network) – they use external CDNs and well as Google’s CDN. Requests to their own machines are therefore tail-heavy (in the “Long Tail” sense), because the head codes to the CDN instead.

… …

Read more please click here to visit : http://kylecordes.com/2007/07/12/youtube-scalability/