Tuesday, February 19, 2008

Cool Technology of the Week

Storage backup and data recovery is at the top of my list of things that will keep me awake in 2008. In healthcare IT, we need short recovery times with minimal or no data loss. Accomplishing this with once-a-day tape backups is not possible. The Cool Technology of the Week, Data Domain de-duplication storage, solves this problem.

At BIDMC, we generate 28 terabytes of new file and email storage each year. Basic file stores have grown so large that we struggle to copy them within our 24 hour backup window. Additionally, our disaster recovery efforts now require us to replicate our data across two geographic locations.

Tape backup, which has been in use at BIDMC for decades, suffers from a variety of problems. Tape backups are time-consuming. Tapes are fragile and require physical security when transported. The time required to retrieve and recover from tape stresses our service availability objectives. In years past, we considered backup to disk, but the economics did not work. Data Domain de-duplication now makes disk an economical backup media. Here's how.

Instead of making full tape backups, we can backup changes on a sub-block level to disk then compress the result. There is an important distinction between a tape-based, incremental backup and de-duplication. With incremental backups, files that changed since the last backup are copied. A major problem with this approach is that each incremental backup must be recalled in sequence to recover files. This is a slow and complex process that does not detect if the same file was stored in several locations.

De-duplication, on the other hand, has sophisticated methods for identifying changes at the sub-block level. For example, if a spreadsheet has '&date' in the heading, each time you save it, the date in the title will change. An incremental backup will copy the whole document again. De-duplication at the sub-block level will only copy the date change. If multiple copies of the file are sent in email, it will only save one copy.

Over a two year period we examined many products from many companies. Most of them required proprietary hardware, specialized software, new management tools, training, and multiple staff to support the technology 24x7x365. We believe in information life cycle management/hierarchical storage management, but want one set of tools and compliance with the technology standards already in use in our data center. We chose Data Domain because:
  • The product de-duplicates at the sub-block level yielding better reduction ratios
  • The product looks like regular storage supporting NFS and CIFS file mounts
  • The product requires little training since it's completely managed by Data Domain
  • The product is an in-line appliance and does not require installation of server agents
  • The product works with all our existing backup software
  • The product is highly reliable, using RAID 6 SATA drives and built in hardware redundancy
In our pilot, Data Domain averaged 1:20 compression. It worked well with our existing backup software which made the transition from tape to disk easy. We're deploying 15 terabytes of Data Domain disk storage in our production and backup data centers, replicating data between the two over our a private Ethernet link. Through the use of de-duplication and compression, 15 terabytes will hold the equivalent of 300 terabytes in tape storage. Our goal over the next two years is to completely eliminate the need for tape in the data center.

We're so impressed with Data Domain's performance as a backup infrastructure that we're also planning to use it as an archival tool for less frequently accessed files. To do so, we'll first implement file virtualization technology such as Acopia or Rainfinity. This will enable us to move content from one storage medium to another without impacting file shares, our web-based file access tools, or our SSLVPN remote file access applications. The combination of file virtualization and Data Domain will enable us to support three tiers of storage.

Tier 1 - SAN storage with lower density, high performance drives.
Tier 2 - SAN or NAS storage with high density, low performance drives.
Tier 3 – NAS-based, archival storage with high density drives coupled with Data Domain de-duplication and compression.

With these 3 tiers of storage, we'll reduce our cost of information life cycle management while reducing complexity.


After a 2 year journey exploring backup, recovery, and archiving solutions, I feel we've finally found the answer that will let me sleep at night in 2008.

3 comments:

eminiman said...

Mr. Halamka,

I have recently come across your blog and enjoy your entries. I spent a considerable amount of my career with UnitedHealth Group as a CIO. It's a different side of the health care fence with some different challenges, but technology acts as a common denominator. I have since left to enjoy some consulting.

Anyway, I can confirm that life as a CIO is definitely easier with these new, smaller storage devices. I have been partial to EqualLogic, (now Dell storage).

Either way, it sounds like you made a good decision with your storage solution.

Again, great blog and comments.

Unknown said...

Very good post. Healthcare lean training is very important to ensure process improvement occurs continually.

Good read, keep up the good work!

Anonymous said...

花蓮,賞鯨,花蓮旅遊,租車,花蓮,花蓮旅遊網,花蓮,租車,花蓮租車,花蓮旅遊,花蓮旅遊,花蓮租車,花莲租车,租车,租車,花莲,租車公司,行易花蓮租車旅遊網,花蓮,租車,花蓮,花蓮旅遊景點,花蓮,一日遊,一日遊,溯溪,賞鯨,泛舟,花莲租车,租车,花莲,花莲旅游,花莲租车,租车,花莲,花莲旅游,租车,花莲,花莲旅游,租车,花莲租车,花莲,花莲旅游,花蓮,花東,租車,花蓮,花蓮,旅遊,花東,租車,花蓮,花蓮,租車,花東,花蓮,旅遊,花東,租車,花蓮,旅遊租車,花蓮旅遊,花蓮租車,花東旅遊,花蓮租車,花蓮租車,花蓮旅遊,租車,花蓮旅行社,花蓮旅遊景點,花蓮旅遊行程,花蓮旅遊地圖,花蓮一日遊,花蓮租車,花蓮租車旅遊網,花蓮租車,花蓮租車,花蓮租車,花東旅遊景點,租車,花蓮旅遊,花東旅遊行程,花東旅遊地圖,花蓮租車公司,花蓮租車,花蓮旅遊租車,花蓮租車,花蓮旅遊,花蓮賞鯨,花蓮旅遊,花蓮旅遊,租車,花蓮租車,花蓮租車 ,花蓮 租車,花蓮,花蓮旅遊網,花蓮租車網,花蓮,租車,花東 旅遊,花蓮 租車,花蓮,旅遊,租車公司,花蓮,花蓮旅遊,花東旅遊,花蓮地圖,包車,花蓮,旅遊租車,花蓮 租車,租車,花蓮租車資訊網,花蓮旅遊,租車,花東,花東地圖,租車公司,租車網,花蓮租車旅遊,租車,花蓮,賞鯨,花蓮旅遊租車,花東旅遊,租車網,花蓮海洋公園,租車 ,花蓮 租車,花蓮,花蓮旅遊,花蓮租車公司,租車花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮包車,花蓮租車網,花蓮旅遊,花蓮租車,花蓮旅行社,花東旅遊,花蓮包車,租車,花蓮旅遊,花蓮租車,花蓮一日遊,租車服務,花蓮租車公司,花蓮包車,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮包車,花蓮租車網,花蓮旅遊,花蓮租車,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮租車,租車網,花蓮租車公司,花蓮旅遊,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮租車公司,花蓮一日遊,租車,租車服務,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮旅遊,花蓮賞鯨,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮包車,花蓮租車網,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,租車花蓮,花蓮租車,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,租車花蓮,花蓮租車,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮包車,花蓮,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮包車,花蓮租車,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮包車,花蓮租車,花蓮旅遊,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮包車,花蓮租車網,租車公司,花蓮租車,花蓮租車公司,花蓮一日遊,花蓮旅遊,花蓮旅遊租車,花蓮租車網,花蓮租車,花蓮一日遊,租車花蓮,花蓮租車,花蓮旅遊租車,花蓮租車,花蓮租車旅遊,花蓮租車,花蓮旅遊,花蓮旅遊,花蓮包車,花蓮溯溪,花蓮泛舟,花蓮溯溪,花蓮旅遊,花蓮旅遊,花蓮租車,租車公司,花蓮旅遊租車,花蓮租車,租車,花蓮旅遊,花蓮租車,花東旅遊,花蓮賞鯨,花蓮旅遊,花蓮泛舟,花蓮賞鯨,花蓮溯溪,花蓮泛舟,花蓮泛舟,花蓮溯溪,花蓮旅遊,花蓮旅遊,花蓮租車,花東旅遊,花蓮,花東,花蓮旅遊,花東旅遊,花蓮租車,花蓮,花東,花蓮旅遊,花蓮租車,花東旅遊,花蓮旅遊,花蓮租車,租車,花蓮旅遊,花蓮租車,花蓮旅遊租車,花蓮旅遊,花蓮租車,花蓮,花東旅遊萬事通,花蓮旅遊,租車,花蓮旅遊,花蓮租車,花蓮旅遊,花蓮租車,花蓮租車,花蓮租車,花蓮旅遊,花蓮租車,花蓮旅遊,花蓮租車,花蓮旅遊,花蓮租車,花蓮旅遊,花蓮租車,花蓮租車,花蓮包車,花蓮旅遊,花蓮租車,花蓮太魯閣,花蓮包車,花東旅遊,花蓮旅遊行程,花蓮旅遊,花蓮 租車,花蓮租車,花蓮租車旅遊,花蓮旅遊租車,租車,花蓮旅遊推薦,花蓮旅遊包車,花蓮租車,花蓮,花蓮租車,花蓮地圖,花蓮旅遊,花蓮旅遊資訊網,花蓮旅遊景點,賞鯨,花蓮旅遊行程,花蓮旅遊,花蓮旅遊租車,花東旅遊景點,花東旅遊行程,花蓮旅遊,花蓮租車,租車,花東旅遊,花蓮旅遊,花蓮租車,花蓮,旅遊達人,旅遊達人blog,花蓮租車旅遊資訊網,花蓮,租車,花蓮,花東旅遊,地圖,租車,賞鯨泛舟溯溪,租車,[ 芝麻店家 ] 花蓮租車旅遊資訊網,花蓮租車-花蓮旅遊租車資訊網 ,旅遊網,旅遊景點,花蓮行程,花蓮,花東,旅遊租車,旅遊,花蓮,租車,花東旅遊,花蓮租車旅遊,行易旅遊民宿資訊網,民宿,推薦花蓮民宿,花蓮民宿推薦,花蓮民宿,花蓮民宿資訊網,花蓮民宿網,花蓮民宿資訊網,花蓮,旅遊,花蓮,一日遊,花蓮好玩的地方,花蓮,,一日遊,花東,租車,旅遊,花蓮旅遊,花東旅遊,花蓮租車,花蓮租車,花蓮旅遊-花東旅遊萬事通