Web app scale and scalability
I'm borrowing from Om Malik's clever "Scale and Scalability" article and podcast (a must listen), which is turn borrows of course from Jane Austen. I've been consuming a lot of media lately on growing Web 2.0 sites, and handling the additional traffic concerns made possible by the programmable web. And I've been contrasting such growth headaches using the LAMP stack against more formal, closed frameworks.
I'll use Microsoft's .NET as an example and then discuss open source architectures.
Outside of best practice coding and performance tuning of the web applications themselves and aside from bandwidth considerations, most Microsoft apps can be co-hosted, with several sites per box. Despite the tidal wave of criticism against IIS, this is a credit to Microsoft's web server platform. During growth, the primary mentality of site managers overseeing ASP.NET sites with massive daily traffic is first to cleanly separate machines, with one box each functioning as web server(s) and database servers. Under larger loads, getting a dedicated box would be the typical next step. The next tier under heavy loads would be to build out the distributed processing, adding more servers (web farms) or more processors (web gardens). Features directly baked-in to the .NET Framework like garbage collection, managed code and ensure automated, if not better, memory management and object disposal.
The large concern for running such architectures is cost. Closed systems can be pretty expensive to run on your own, and private hosting services, while outstanding, largely aren't as cheap as their LAMP counterparts.
In contrast, since most Web 2.0 endeavors are born of open source, garage frameworks and have limited budgets, the need to organically grow a site's capacity to process requests and serve information can be daunting (outside of theory, I'm not a hands-on server guy). Excellent resources to consult on tools and techniques to utilize are the Danga/LiveJournal whitepaper on scaling web apps running LAMP, and the podcasts from Flickr's Cal Henderson and del.icio.us's Joshua Schachter from the 2006 Future of Web Apps conference. Topics discussed include things like using separate servers for HTML and multimedia, memcache, database indexing strategies and more.
Although I totally love Ruby on Rails, my one criticism of it is that we don't have enough projects akin to the size/scope of Amazon, eBay, del.icio.us or major-scale site running it yet. Lots of blogs, some really cool projects, but not much that gets Net Neutrality traffic yet. So in my mind, the jury's out on RoR.
But the one thing that all platforms have in common in terms of scalability management is the critical, basic need for proper planning, testing, performance monitoring, upgrade thresholds (at what baseline point does our traffic get such that we need to start buying more servers), intelligent use of caching, downtime notification for administrators (paging, e-mails, SMS), and clearly-defined escalation paths (who to contact when something goes awry).
Bottom line: don't skimp on scaling.
I'll use Microsoft's .NET as an example and then discuss open source architectures.
Outside of best practice coding and performance tuning of the web applications themselves and aside from bandwidth considerations, most Microsoft apps can be co-hosted, with several sites per box. Despite the tidal wave of criticism against IIS, this is a credit to Microsoft's web server platform. During growth, the primary mentality of site managers overseeing ASP.NET sites with massive daily traffic is first to cleanly separate machines, with one box each functioning as web server(s) and database servers. Under larger loads, getting a dedicated box would be the typical next step. The next tier under heavy loads would be to build out the distributed processing, adding more servers (web farms) or more processors (web gardens). Features directly baked-in to the .NET Framework like garbage collection, managed code and ensure automated, if not better, memory management and object disposal.
The large concern for running such architectures is cost. Closed systems can be pretty expensive to run on your own, and private hosting services, while outstanding, largely aren't as cheap as their LAMP counterparts.
In contrast, since most Web 2.0 endeavors are born of open source, garage frameworks and have limited budgets, the need to organically grow a site's capacity to process requests and serve information can be daunting (outside of theory, I'm not a hands-on server guy). Excellent resources to consult on tools and techniques to utilize are the Danga/LiveJournal whitepaper on scaling web apps running LAMP, and the podcasts from Flickr's Cal Henderson and del.icio.us's Joshua Schachter from the 2006 Future of Web Apps conference. Topics discussed include things like using separate servers for HTML and multimedia, memcache, database indexing strategies and more.
Although I totally love Ruby on Rails, my one criticism of it is that we don't have enough projects akin to the size/scope of Amazon, eBay, del.icio.us or major-scale site running it yet. Lots of blogs, some really cool projects, but not much that gets Net Neutrality traffic yet. So in my mind, the jury's out on RoR.
But the one thing that all platforms have in common in terms of scalability management is the critical, basic need for proper planning, testing, performance monitoring, upgrade thresholds (at what baseline point does our traffic get such that we need to start buying more servers), intelligent use of caching, downtime notification for administrators (paging, e-mails, SMS), and clearly-defined escalation paths (who to contact when something goes awry).
Bottom line: don't skimp on scaling.
posted by Jason Salas @ 8:37 AM
0 Comments:
Post a Comment
Links to this post:
Create a Link
<< Home