Saturday, February 28, 2009
Web Hooks & XMPP - right tool for the right job
Software developers and system architects, whether they know it or not, are approaching a rather intriguing crossroads of possibilities in terms of achieving efficient data exchange and creating compelling user real-time experiences. Two established platforms - the event-driven callbacks method of the aptly-named Web Hooks and the bandwidth-friendly federated communications of the eXtensible Messaging and Presence Protocol (XMPP) - are rapidly emerging as major players for managing proper communications across the Internet.
It's important to know the strengths and benefits of each technology and recognize which technique is best suited to achieve your objectives. So let's consider each platform's effectiveness.
Anytime a Web Hooks and XMPP discussion arises, scalability is typically the first criterion challenged. Politics notwithstanding, the general consensus seems to favor XMPP for rapid-fire, high-volume, push-based traffic. Web Hooks on the other hand stake a strong claim in scenarios seeing smaller, less frequent calls to public APIs. The time-tested web growth strategy to create server farms, server gardens, utilize edge computing or to go the cloud route all exist and are applicable for callbacks.
Yet opinions still vary on exactly how scalable XMPP-based stacks might truly be, noting performance and latency concerns in cases where servers have to manage massively large rosters in IM applications - Twitter's May 2008 removal of its XMPP firehose data stream being the canonical example. MetaJack suggests such cases as being candidate for componentized or S2S bot design, as opposed to the bot-as-client method so often used.
Not many, if any, case studies or formal patterns are in existence to guide us in designing well-tuned networks in cases of XMPP applications. Again, the decision tree on whether to scale up or scale out in Web Hooks scenarios, based on the number of requests against a server, is well documented. The most important thing to remember when comparing XMPP and HTTP stacks is that the traffic levels are different - the former being low-bandwidth push of XML fragments to users based on availability, the latter being the traditional request/response method.
With Web Hooks relying on a URL registering a server script to an event on a remote application, a web server with appropriate platform support/permissions is all that's needed to get up and running. Most people can put their own development web servers online, and free services like AppJet and GitHub exist for no-cost script authoring/storage, but are subject to downtime and other inconveniences common to third-party hosting. Conversely, free open source Jabber servers and clients are legion but are arguably more difficult to setup than Apache or IIS. And hosted services for instant messaging like Jabber.org and Google Talk have historically throttled clients incurring heavy loads (i.e., Gnip's main gripe). The Pownce team postulated that clients running their own web servers will greatly outnumber those maintaining XMPP servers.
XMPP's redeeming quality is its efficient use of network resources, sending small messages only to those clients who are capable of receiving them at that moment, as well as supporting out-of-band communications for larger payloads like voice and video. But the feature is largely limited to messaging. Callbacks can perform whatever functionality you program them to do, with the tradeoff being variable consumption levels of bandwidth, memory and processing cycles. So there's the rub.
Yahoo! has stated that a major architectural dealbreaker in choosing pure XMPP PubSub over Web Hooks for its Fire Eagle real-time geolocation service was a concern about a lack of presence data, not wanting to unnecessarily send messages to hosts that may be offline, as well as the taxation of enforcing SSL for each and every endpoint. (Comparatively, Google Latitude provides such geolocation via triangulation of Wi-Fi access points and radio towers.)
XMPP features a secure environment through enforced server dialbacks and inherent support for SSL and SASL. However, XMPP Standards Foundation executive director Peter St. Andre begrudgingly admitted that after a decade of using the protocol, the community hasn't been able to implement an official encryption format. From the Web Hooks perspective, SSL, authorization and other traditional security measures can be leveraged over existing HTTP techniques, as any of the major web frameworks supports their own cryptography libraries.
Generally speaking, web API calls can directly move through Port 80 without additional configuration. The XMPP community has made some very impressive strides with NAT traversal, facilitating messages seamlessly passing through the corporate firewall.
Google developers Brad Fitzpatrick and Bret Slatkin expressed their frustration at how XMPP development has fragmented due to the fact that XEP-0060 (the PubSub extension) is more convoluted than the Core spec, coupled with architects not following the protocol extension as written. (This clutter gave rise to the development of the Personal Eventing Protocol, a greatly reduced subset of PubSub.) Conversely, the web services model, operating over standards such as SOAP, HTTP, WSDL and other structures, is widely understood and adhered to in client apps. Web Hooks, being a software pattern not bound to a formal spec, also has a little bit more leeway than the rigidity of a full protocol.
XMPP over HTTP
While each technology hasn't quite hit the development mainstream just yet, cursory views on it vary. The big knock against XMPP today is twofold: people see XMPP merely as IM and not a flexible messaging framework, and that it's not HTTP. Pundits quickly point out HTTP's many shortcomings in cross-network communications, while others remind us not to forget its simplicity. The appropriately named "Web Hooks" has more of a natural grokability to it.
Any modern-day developer knows server-side coding in some language, so adopting Web Hooks makes for a very smooth transition. XMPP's use of long-lived TCP connections to stream XML fragments between servers might mean web devs have to delve into the unfamiliar world of socket programming; and perhaps going even deeper, having to get into GUIs, servers, and possibly bots. This makes for a sizable learning curve not everyone is going to want to undertake.
One area where Web Hooks have a distinct advantage over XMPP is their accessibility across developmental platforms. A handful of XMPP clients, libraries and SDKs exist for .NET and Mono, but they're still vastly outnumbered by those for Ruby, Python, Erlang, PHP, Java, Perl and the like. Ironically, the Jabber project, the IM platform driving XMPP, has been around nearly as long as server-side web development, but has really only seen a large interest uptake in recent history.
While advocates of both Web Hooks and XMPP passionately defend their preferred system of data exchange, friendly and at times pragmatically aggressive challenges are being lobbed across forums, blogs and IRC channels about HTTP's survivability as we move toward delivering real-time experiences. Some have even proposed abandoning the moniker "web services" altogether, suggesting we using "online services" by virtue of political correctness.
There's little argument against the fact that XMPP scales better. But the low barrier to entry of the Web Hooks model of registering callbacks is orders of magnitude more achievable than setting up a back-end architecture that to many IT shops will be foreign territory. The learning curve, supported platforms, infrastructural requirements, security issues and ease of deployment make this a very compelling discussion.
Thursday, February 26, 2009
7 Questions for Jeff Lindsay on Web Hooks
Sunday, February 22, 2009
Movie Review: Friday the 13th
Saturday, February 14, 2009
Tutorial: Twitter alerts for your Qik live mobile streams
The hit "55" to send out twitter notifications only apply to Nokia phones, on iPhone, there is a "send alert" button. Twitter alert is not yet available on Windows Mobile and BlackBerry, users have to set "Tweet all videos" under edit profile on their profile page.
Friday, February 13, 2009
7 Questions for Notifixious
Thursday, February 12, 2009
The arrows in my Twitter quill
- Mobile Web
- TwitterFeed, tapping KUAM's RSS feed
- Via FriendFeed
- TweetDeck (user agent: "XMPP Gateway")
Tuesday, February 10, 2009
An open letter to RIM about IM interop for BlackBerries
Sunday, February 08, 2009
25 Random Things About Me
- I'm distantly related to abolitionist author Harriet Beecher Stowe on my mom's side.
- I've never done any drugs. Ever. Not a one.
- Anything I've ever enjoyed success in, I've always failed miserably at the first time out.
- Although I've played in volleyball tournaments half my life, I never played a second in high school. Two weeks before the first game of my senior year, the only year I made the team, I broke my wrist and was out for the season.
- I'm really good with kids...I'm totally willing to embarrass myself publicly for their amusement.
- I've worn my Schwimmeresque hairstyle since 1998.
- I'm very subversive and non-conformist. Peer pressure's never really had an affect on me.
- Joe Montana is still my hero.
- The thing most people don't realize about my photographic memory is I can't control it. Some things just stick.
- I didn't drink my first beer until I was 20.
- At 22 I changed religions.
- My sense of humor is very referential, upbeat, observational, perverted, narcissistic and self-deprecating.
- I'm a total mama's boy. As an infant if you tried to carry me and you weren't either of my parents, I'd throw a major tantrum. And by that I mean I'd pee all over you.
- The only places I've ever lived have been Guam and Connecticut.
- I've been working every single day of my life since I was 19. Seriously, Monday through Sunday I'm doing something productive.
- I was the first kid in my neighborhood to flip Super Mario Bros.
- I'm lactose intolerant, which means I can't enjoy dairy like most of you...but I can drop 15 lbs. in two days if I wanted to. Hehehe.
- My lucky number is 4.
- I believe that staying home and enriching the mind can be as fulfilling as going out and killing brain cells.
- I used to have a pretty intense sleepwalking problem. I'd leave the house, go all the way up the street, turn around, and get back in bed.
- Everything I know about software development has been self-taught.
- The only thing I've been doing longer than playing guitar is breathing. I picked up the instrument at age 5.
- My father was president of the university I attended. I later put myself through graduate school online, cranking out my MBA while working full-time as a web developer/newscaster.
- I'd pick EPCOT Center over Disneyland in a heartbeat.
- The movies "Big" and "Wall Street" served as the motivating factors for me majoring in marketing.
Friday, February 06, 2009
Google Latitude - the first 48 hours
When I caught word Wednesday night (Guam time) that Google had released a real-time geolocation service for mobile/desktop, my jaw hit the floor. I've Twittered about it incessantly over the past couple of days as I've hacked away at it, trying to get it to work after it didn't immediately take. So here are some notes from the field when you get the geolocation bug.
Understand the Google Latitude operating environment. The one criticism I've got about Latitude is how surprisingly weak the initial documentation is about it. There's one YouTube clip, a couple of Google Blog posts and that's it for now. As such, a lot of people don't get it immediately. Here's the skinny: your phone is the base unit that allows your geolocation to be dynamically updated. As you move around, your phone reports these changes. You could do this with a laptop with a persistent Internet connection of some sort, but that would be cumbersome and annoying...not to mention the fact that at some point your computer would die. You can track your own progress and that of your GMail contacts over your phone after downloading/installing Google Maps 3.0.0, or on your desktop by way of an iGoogle portal web widget. That's it in a nutshell.
Don't just 'accept'. If you get an invite from a GMail user, merely accepting won't get it done...you've got a bit of work still to do. I've invited several of my social network friends to join in the fun, most of whom just merely accepted without properly setting up their profile. So I see them listed as having confirmed, but they're not reporting their geolocation.
Real-time works on your mobile, not your desktop. Many people I've introduced the app to who use it over the Web don't see me moving about as I do it get frustrated. On phones, you can see yourself moving in real-time (insanely cool!), but that's a feature that works on phones only. The desktop component refreshes itself every couple of minutes and redraws each user's updated geolocation with their avatar. It's not real-time and isn't intended to be.
Don't fear Google Gears. When you setup the desktop component to Google Latitude (enabling the widget within your personalized iGoogle portal), you'll be prompted to download a small plugin, Google Gears, and to allow Google Latitude access to your computer. This lets the service run certain operations in the background to update your position. Don't worry...it's harmless and sounds worse than it is.
Go outside and play. I'll admit I've never worked with GPS devices before, so getting over that learning curve helped greatly in understanding how Latitude works. After about a day of frustration in not getting my geolocation to be shown - I sometimes jump back and forth between Guam, South Dakota, Macedonia, Honduras and Vietnam - a buddy at DOCOMO suggested I re-synch my GPS settings with satellites. I told him I did, to which he replied, "No - stand OUTSIDE and synch it." I'll be damned, it worked. As such, I've also noticed dramatically more consistent GPS coverage and responsiveness by the app when I'm not within a building. So this might be an incentive to take a stroll and watch yourself move on your mobile.
GPS is better than cell towers. If you're offered a choice between being tracked by Global Positioning System, the Wi-Fi access points or the cell tower triangulation schema that Google uses to pinpoint your place in the world at any given moment, opt for GPS. A co-worker who used the cell tower method was accurate to within 200 feet of where she actually was. When using GPS, my geoposition was dead-on accurate to within 20 feet...at most. That's crazy!
Google Latitude is a battery killer. Because you're constantly sending data back and forth to determine your geolocation in real-time and also because the screen can be backlit for long periods of time as you track yourself and others in your social network, your phone WILL heat up because of all the data communication passing back and forth and redrawing of the screen graphics. Depending on your data plan with your wireless provider, this could also get pretty expensive. So while a neat app to play with, realize this will eat away at your battery's life.
OK, enough blabber. Get with the program! If you know me, add me and let's have some fun.
Thursday, February 05, 2009
Google Latitude on Guam
DON'T ride the waves
Book Review: What Would Google Do?
Tuesday, February 03, 2009
Disruptive technology: delivering real-time web experiences
Monday, February 02, 2009
No fiction. No drama.
It's all reality, all stuff that's actually happened or applications of things that really exist in the world.
So online services aside, here's a snapshot at how I get my media fix.
I've always been an avid reader, having consumed great works, but largely within a specific silo - humanities, academic essays, sportswriting, biographies, technology, accounts of the music industry, business tomes. I've always had an affinity for knowledge and at a very young age embraced literature, discovering the marks people have made in history and also the projections great thinkers have put forth about the future. I don't largely crossover into imagined stories.
That said, contrast my movie viewing habits. While I appreciate the classics, the theater's my wind-down time. I largely prefer my Silver Screen experience to be one that entertains, not teaches. So I normally frequent action, horror, sports and comedy. Typical guy stuff.
So now we arrive at my mid-range: the boob tube. Television for me has always been the great panacea as far as delivering content. I turn to TV for education, entertain and enlightenment, so I'll watch anything you put in front of me. On any given night you can find me taking in a cartoon reeking of post-modernism, a dumbass b-comedy, a documentary on Hitler, science fiction, a cooking show, and a Baroque production - all in the same session. Heck, I've been known to sit through infomercials, marveling at their deliberate scripting as a means of low-level mind control.
As someone who's been profoundly influence by mass media his entire life, the disparity in how I separate my information across platforms is interesting, upon review.
So if you ever see me around and want to have a chat about the finer points of Hamlet, feel free...I prefer Mel Gibson's version. :-)
Sunday, February 01, 2009
My magazine interview on Guam web tech
Subscribe to Posts [Atom]