The Jason Salas Experience

Amazon-owned Alexa announced in grand fashion (at least its an impressive blog post) the Alexa Web Search Platform Beta, a series of developer tools that empower distant-end programmers with the ability to create innovative search services.

Now, [the following] infrastructure is yours to use via the Alexa Web Search Platform:

• Three online web snapshots of up to 100 terabytes each
• Powerful tools to sift through the content to create your own data set
• Upload, compile and run your own programs on a processing cluster across the data set
• Store your output on a storage cluster
• Integrate your data into a search index
• Access your new search via Amazon Web Services

Om Malik analyzes this commoditization of search, and John Battelle speculates on what this means for the future of search. Any developer managing a highly-trafficked site with diversified content in a variety of formats (including me) will tell you that search is always the hardest thing to build properly. Any aspiring hacker can drop a CGI script into a site or implement a FrontPage file-scan utility and get somewhat accurate results, but considering the required full-text searching, indexing, processing, relational concerns, archiving and other issues, this is a daunting task, technologically and economically, for anyone.

Alexa's trying to corner the market - largely by being first to do so with an entire suite of search tools, not just simple APIs that have been criticized like those offered by Yahoo! and Google - of being platform, not destination.