March 2009 Archives

Tony Ryb is retiring from Nortel on Good Friday and I feel honored
that a person with his knowledge, dedication and experience chose TMC
publications and TMCnet as an outlet for his valuable thoughts.

I first met him at COMNET a show in DC around 1996 and asked him to
write for CTI Magazine when I realized what a valuable resource he
was. I always admired how he distilled complexity into simplicity to
the benefit of readers.

I am first here at his surprise farewell party and trust he won't read
this before he gets here. Tony will continue to blog on TMCnet on his
Hyperconnected Enterprise blog and I wish him all the best.

These cards were at all the table settings. Classy, very classy.

I have been in 30 minute meetings throughout the two days I have been at VoiceCon and will be doing the same at CTIA which I travel to tomorrow. In between meetings I scrambled to find nourishment and potentially a restroom. Why did I do this? Well dear readers, I do it for you... I am trying to stay on top of the entire market - meet with all the important companies and report back.

I have met with Alcatel-Lucent, Motorola, Tandberg, NEC, Digium, Calabrio, AVST, EasyRun and many others. I took lots of notes and hope to turn each meeting into an article and/or blog entry as time permits.

In the mean time I am off to a surprise party for someone - I can't tell you who yet but it should be a good one. More to come.

Recently, Apple applied for a patent which allows iPhones to integrate with biometrics allowing more precise authentication of the users of the fine touch-screen device. An eagle eyed reader of the patent noticed however that the iPhone used in the patent application had been jailbroken. What's that? Apple jailbroke an iPhone and used it to help with their patent filing?

So the obvious question is how can Apple tell its customers to do something they obviously can't help doing themselves? Moreover, will this new information help the Electronic Frontier Foundation in its ongoing struggle to make iPhone jailbreaking an exception to the Digital Millennium Copyright act?

More at engadget.

One final shot of New York from Rockefeller Center. Next stop for me
is Voicecon, CTIA and a Metaswitch customer event.

I love NY because concerts seem to spontaneously erupt.

Yes indeed, at the base of the lamp post was an honest to goodness concert taking place during the day. Good stuff.
 

A shot between my NYC meetings.

A nice day for meetings in the city.

The big news yesterday was that Skype is coming soon to an iPhone near you and while it seemed to take forever to make it happen I would be willing to wager that Skype will become one of the most popular iPhone apps overnight. I will be at the CTIA show next week and that is when the rumor is to become reality according to Om Malik. This event would be a good place for the launch and we can expect media frenzy when it happens. After all, the iPhone and Skype are both media darlings - put them together and you have even more reason to write that glowing article.

Many people in the communications space have asked me recently what TMC's secret is. After all, we are in the toughest media environment of our lifetimes and we produced our best show ever and have more paying customers online than at virtually any other time in our history.

The answer may lie in a bit of luck and some skill. The lucky part is we built our first online community for a customer about a decade ago. And since this time we have invested a small fortune in building our own proprietary technology which allows us to build highly-ranked, viral, news-driven communities for customers. Well over 100 of these sponsored communities live on TMCnet and generally consist of the tabs at the top and down the left of most of our pages.

Moreover TMCnet now houses millions of pages of content which gives the site tremendous prominence. And we have ranked very high on search engines for many years which has generated a tremendous amount of links to the 100+ articles we write a day and other content such as blog entries on the site.

The community product is called a GOC or "gock" and stands for Global Online Community. When we launched the program the term "organic search results" was probably not common but now, these communities help our customers rank extremely high for a variety of keywords which are important to them.

I know what you are going to say. Rich, that is what those click ads are for. Well to be honest the value of an organic search result is much higher to the searcher because it is not blatantly paid for and moreover it is not in a sea of other ads. Most importantly, research shows less that 20% of people even click on search ads. What about the other 80%?

TMC's communities answer the request we have been hearing -- How do you recreate the best part of tradeshows online?(shots from last ITEXPO East February 2009 in Miami)

Finally, unlike search ads, these communities help your own site(s) rank high organically by providing links. Moreover they help companies build their brand and thought leadership.

In addition, as a news-driven entity, GOCs draw traffic from other pages on TMCnet, newsletters, the TMCnet home page, news search engines and traditional search services. They are multimedia in nature, allowing companies to interface with customers via audio, video and of course text.

Example of an IP-PBX GOC (click to see full screen image)

Many of you have told me over the years that there needs to be a way to combine the best part of trade shows online. The GOC program is exactly this as it brings in your potential customers from around the world and gives them a reason to come back and see your message as the news is constantly updated. It is a very busy 24x7 community consisting of the most targeted people available on the web. And it is targeted by the news you find important.

In addition, it is measurable, and includes a wealth of metrics which can be used to analyze your spend and justify it up the chain of command.

For the reader the benefit is clear. They come to the GOC and bookmark it so they can keep up to date on the latest happenings in the space. How many people come? Well our record is over 650,000 pages viewed on a GOC in one month but typical results are between 250,000-500,000 per month. Generally, each GOC will average about 100,000 unique visitors per month - and they are targeted exclusively by content. In other words, you can use this program to build a community of people interested in subjects such as colocation, IP communications, HD voice, next generation communications, fixed mobile convergence or anything else in virtually any field. Click on any of these above links to see how the design is different and mirrors the look and feel of the sponsor.

If you are interested in learning more, here is an updated (4/14/2009) video which describes the program. Feel free to drop me an email for more.

One of the most challenging issues for companies which provide service over the net is keeping systems up and running 100% of the time. No company has been able to keep their services running constantly but certainly Google is one of the leaders in this area. Their virtualized architechture was actually developed with the idea that many of the servers in the company's data centers will fail. Still, outages happen and when they do, the company like all others has to react. Here's how and a quick excerpt.

Data Center Knowledge: Google has many data centers and distributed operations. How do Google's systems detect problems in a specific data center or portion of its network?

Urs Holzle: We have a number of best practices that we suggest to teams for detecting outages. One way is cross monitoring between different instances. Similarly, black-box monitoring can determine if the site is down, while white-box monitoring can help diagnose smaller problems (e.g. a 2-4% loss over several hours). Of course, it's also important to learn from your mistakes, and after an outage we always run a full postmortem to determine if existing monitoring was able to catch it, and if not, figure out how to catch it next time.

DCK: Is there a central Google network operations center (NOC) that tracks events and coordinates a response?

Urs Holzle: No, we use a distributed model with engineers in multiple time zones. Our various infrastructure teams serve as "problem coordinators" during outages, but this is slightly different than a traditional NOC, as the point of contact may vary based on the nature of the outage. On-call engineers are empowered to pull in additional resources as needed. We also have numerous automated monitoring systems built by various teams for their products, that directly alerts an on-call engineer if anomalous issues are detected.

DCK: How much of Google's ability to "route around" problems is automated, and what are the limits of automation?

Urs Holzle: There are several different layers of "routing around" problems - a failing Google File System (GFS) chunkserver can be routed around by the GFS client automatically, whereas a datacenter power loss may require some manual intervention. In general, we try to develop scalable solutions and build in the "route around" behavior into our software for problems with a clear solution. When the interactions are more complex and require sequenced steps or repeated feedback loops, we often prefer to put a human hand on the wheel.

DCK: How might a facility-level data center power outage present different
challenges than more localized types of reliability problems? How does
Google's architecture address this?

Urs Holzle: The Google within-datacenter infrastructure (GFS, machine scheduling, etc) is generally designed to manage machine specific outages transparently, and rack/machine group outages as long as the mortality is a fraction of the total pool of machines. For example, GFS prefers to store replicated copies of data on machines on different racks so that the loss of a rack may create a performance degradation but won't lose data.

Datacenter level and multi-region unplanned outages are infrequent enough that we use manual tools to handle them. Sometimes we need to build new tools when new classes of problems happen. Also, teams regularly practice failing out of or routing around specific datacenters as part of scheduled maintenance.

DCK: A "Murphy" question: Given all the measures Google has taken to prevent downtime in its many services, what are some of the types of problems that have actually caused service outages?

Urs Holzle: Configuration issues and rate of change play a pretty significant role in
many outages at Google. We're constantly building and re-building systems, so a trivial design decision six months or a year ago may combine with two or three new features to put unexpected load on a previously-reliable component. Growth is also a major issue - someone once likened the process of upgrading our core websearch infrastructure to "changing the tires on a car while you're going at 60 down the freeway." Very rarely, the systems designed to route outages actually cause outages themselves; fortunately, the only recent example is the February Gmail outage (Here's the postmortem in PDF format).

OK this article which discusses Twitter killing the economy is satirical but it does seem coincidental that the more people get on the service the worse the stock market does. Maybe if Twitter were to merely speed up its service, the world would gain a massive productivity boost which would increase our GDP? Are you a fan of Twitter like Jeff Pulver who wants to launch an event in the space or do you lean more towards the Jennifer Aniston side of the argument where Twitter was partly responsible for your breakup?

If you are interested in the stories and headlines I find useful, I invite you to bookmark my Google Reader feed which I update fairly regularly. I occasionally will add comments as well. In fact, since I started using this page, I have found myself blogging less. I hope you find this resource useful.