A client of mine just lost three webinar recordings from their webinar vendor's online server. I spent quite a while working with Cisco WebEx tech support to find out why three out of 56 recordings on their server could no longer be played back or downloaded.
The final word was that WebEx had encountered some database corruption in late July, losing the linkages between audio and video that created a properly encoded recording file. Sure enough, our three dead recordings were dated between July 15 and 30.
The tech engineer I spoke to said that while no public posting or online notice was available, the account holder of record for each affected customer had been emailed to inform them of the potential problem. I have been promised a copy of the email, but it hasn't come through yet. I don't think a single email to the named account holder is sufficient. Technical emails can be overlooked, ignored, or filtered as junk. The named account holder in a large organization might have nothing to do with the day-to-day management of the webinars being delivered by various departments and groups.
I am also a bit annoyed that first-line WebEx support knew nothing about the problems and took me through a long set of troubleshooting steps trying to prove that the problem was with my playback or my account. It took two or three levels of escalation before somebody found the internal tech alert about the database problem.
More troubling is the fact that when WebEx initially discovered the problem that resulted in lost customer data, they did not or could not restore the corrupted files from backups. When you advocate storing customer data on your servers, it is incumbent upon you to have a trusted backup and recovery mechanism for that data.
This is not the first time I have run into unrecoverable loss of webinar recordings. In 2011 I reported a similar loss of client archives with omNovia. omNovia added a comment to my post saying that they had upgraded their backup/recovery mechanism in response to the problem. But my client's recordings were still gone, never to be seen again.
And way back in 2010 I reported on how I lost a recording in Citrix GoToWebinar during the automated format conversion process. A GoToWebinar manager added a comment to that post saying they were working on a project to eliminate the conversion vulnerability, but three years later the process is exactly the same and exactly as vulnerable as ever.
As with most other things in life, if you want ultimate reliability, you need to take ultimate responsibility. For mission-critical recordings, try to find a way to use an external audio/video recorder to capture the webinar. Save it to your own locations that you can backup and recover if necessary. It's extra effort and extra inconvenience for you. But at least it gives you a fallback in the event of a catastrophic failure from your web conferencing vendor.
