Whitepaper: Healthcare Data Management For MEDITECH

Executive Summary

In terms of providing a comprehensive data management and protection solution for MEDITECH’s Health Care Information System (HCIS) environments, BridgeHead Software should be your vendor of choice. Early in its rich history, MEDITECH recognised there were some data protection limitations regarding their HCIS. Three key challenges were identified:

• How to overcome the problems in creating a consistent backup of the MEDITECH database
• How to reduce the stress of the backup load on general system performance
• How to reduce the time taken to back up the massive numbers of very small files generated by the EMR application (that take even longer to restore)

MEDITECH opted to work with BridgeHead Software to try and resolve these issues. This collaboration resulted in the release of the Integrated Serverless Backup (ISB) protocol in 2002. ISB enables hospitals to create a consistent ‘Point-in-Time’ copy of the MEDITECH data within the storage array, where it is then accessed directly by the BridgeHead server to commence the backup process (to tape, disk or other supported media). ISB ensures data consistency imposes no load on the MEDITECH production environment, and allows a daily backup to be performed (that can then be taken offsite to provide the critical protection required for the MEDITECH data). ISB continues to serve many hospitals in their data protection requirements for their MEDITECH HCIS.

ISB ensures data consistency imposes no load on the MEDITECH production environment, and allows a daily backup to be performed

However, some healthcare organisations required a protection methodology that reduced the amount of time taken to adequately recover their systems and data. The issue was that restoring data can take time – especially when there is an additional lag where the backup tapes need to be retrieved from their offsite location first. This led to further collaboration from which Integrated Disaster Recovery (IDR) was born.

IDR builds on the Point-in-Time technology of ISB, but instead of writing it to an alternative target, it retains the copy within the storage array itself. This makes an IDR copy very fast to create and provides an immediate resource for recovery. IDR can also leverage storage array snapshotting (that only needs to remember the changed parts of a volume) to produce an immediately available copy of the MEDITECH data without using large amounts of storage. BridgeHead’s MEDITECH data protection solutions offer the market’s widest range of recovery options. To determine the most appropriate solution for a hospital, it is necessary to establish the required Recovery Point Objective (RPO), i.e. the last backup point recorded to which a hospital can restore from; and the required Recovery Time Objective (RTO), i.e. the length of time it would take to restore a backup to an operational status.

Both RTO and RPOs can easily be enhanced when utilising BridgeHead’s IDR Backup Agent. BridgeHead supports the widest range of array technology, including all MEDITECH certified storage arrays, giving hospitals choices and flexibility over the storage decisions they make at each refresh cycle.

Further, BridgeHead is constantly looking to enhance its offerings, continuing to evolve its MEDITECH data management solutions, including:

• ISB and IDR
• Cascading Backup – where the IDR Agent continues to make frequent copies of the MEDITECH HCIS data, but BridgeHead uses the most recent IDR copy as the source for the backup, thereby reducing RPO and RTO
• Restore Utility – a dedicated ISB/IDR Restore Utility allows hospitals to easily view and restore all of their MEDITECH data from all of the available restore points
• MEDITECH Backup Facility (MBF) – where the MEDITECH HCIS is only responsible for preparing the data; BridgeHead’s ISB Agent interfaces with the storage array and does the rest
• BridgeHead’s Healthcare Data Management (HDM) Solution for MEDITECH SCA – where a combination of backup and archiving methods are used to efficiently manage and protect the large volume of data and number of files generated by MEDITECH’s SCA module
• (Coming soon) View Only Electronic Medical Records (VOEMR) – this service allows MEDITECH data to be accessed in View Only mode while MEDITECH application updates are applied that would otherwise require system downtime

However, the ultimate proof of BridgeHead’s pedigree in managing MEDITECH data is clearly illustrated by the fact that MEDITECH uses BridgeHead’s solutions for backups within its own organisation.

To determine the most appropriate solution for a hospital, it is necessary to establish the required Recovery Point Objective and the required Recovery Time Objective

Although BridgeHead Software is best known for its MEDITECH capabilities, its important to recognise that its Healthcare Data Management (HDM) Solution extends to manage all of a hospital’s data, clinical and administrative. Some of BridgeHead’s HDM Solution’s capabilities include:

• DICOM Archiving Agent (DICOMStoreTM) a vendor neutral archive (VNA) to help hospitals manage their DICOM-based data from PACS applications found in many disciplines such as cardiology, mammography, ophthalmology, but typically found in radiology
• VMware Backup Agent for creating VMware snapshots of virtual machines that are then accessed from the (physical) BridgeHead server, allowing the back-up process to occur with no load placed on the VMware server
• Other hospital data BridgeHead’s portfolio extends to a wide variety of Archiving and Backup Agents to manage and protect all manner of a hospital’s applications and data, from Oracle, Cache and SQL database backup through to Exchange email archiving. These Agents can be deployed over a wide range of servers, including Windows, UNIX, LINUX, NDMP and Netware

A Brief History Of MEDITECH Data Protection

Back in 2002, MEDITECH recognised there were challengesin relation to the protection data from their Health Care Information System (HCIS). These could be split into three distinct issues:

• The consistent backup of the MEDITECH database it is vital that the database application and the backup solution work in concert in order to avoid the synchronisation problems resulting in the recording of partial transactions or a half updated index – even where the database is not a Windows file system
• Reducing the stress of the backup load on system performance there was a concern around the impact on MEDITECH’s system performance when back-ups were running. This was primarily as backup, by nature, reads all of the data and, thereby, directly affects the I/O system, potentially slowing the application. Additionally, MEDITECH had no support for automated tape libraries or sharing drives between systems. Earlier solutions on MEDITECH’s MAGIC OSAL relied on a dedicated manual tape drive on each server writing a tape every night. With tape capacities approaching (and now exceeding) a terabyte, backing up one server per tape, and requiring a drive for each server, was uneconomical
• The successful backup of massive numbers of very small files generated by MEDITECH’s Client-Server EMR module the process of opening and closing a large volume of small files, using a traditional file-by-file backup, takes too long to ever achieve a fast data rate

After discussions with many vendors, MEDITECH decided to work with BridgeHead Software to try and resolve these issues.

In 2002, the fruits of the BridgeHead and MEDITECH collaboration were introduced in the form of the Integrated Serverless Backup (ISB) protocol. BridgeHead’s ISB (now known as BridgeHead’s ISB Backup Agent) works directly with the MEDITECH HCIS and the hospital’s storage array. In order to create a consistent ‘point in time’ copy of the MEDITECH data, ISB communicates with the MEDITECH application to briefly quiesce the data, flush the caches, and then produce a consistent copy within the storage array. This Point-in-Time copy is then accessed directly by the BridgeHead server to commence the backup. This methodology guarantees the data consistency of the back-up and imposes no load on the MEDITECH production servers during the backup process.

In BridgeHead’s experience, most recoveries today are likely to use an IDR copy that can either be instantly restored or that can be provided to MEDITECH to enable them to repair the hospital’s live system

BridgeHead’s MEDITECH ISB Backup Agent allows backup to be made directly to tape or to disk (or any of the other technologies supported by the BridgeHead Healthcare Data Management (HDM) Platform). In order to fully protect the backups that are created, they must be taken offsite and/or copied to a secondary, secure location.

The proof of BridgeHead’s ISB Backup Agent’s benefits is clearly illustrated by the fact that MEDITECH uses the solution for back-ups within its own organisation1. In order to access the data from an ISB, the back-up must first be restored (this is the case even if the backup is written to a disk type device – this is because a backup is packaged for media). Backup tapes are normally taken to offsite locations to increase the level of data protection through the avoidance of onsite disasters. Consequently, these tapes would need to be retrieved before the data recovery process can commence.

Both BridgeHead and MEDITECH understood that the combined time of tape retrieval and restoring of the data may not be optimum for many hospitals. Consequently, they once again collaborated to devise an alternative solution – one that would reduce this overhead and provide healthcare facilities with quicker data recovery options. As a result of their collaborative efforts, BridgeHead’s MEDITECH IDR Backup Agent was born.

In contrast to ISB, IDR makes a copy of the MEDITECH data within the storage array itself as opposed to writing it to tape. This IDR copy is created extremely quickly, designed to leverage the storage array features of the device in situ (where applicable). In this instance, the IDR Agent makes the copies using space efficient storage array snapshots. These snapshots only need to remember the changed parts of a volume – they get any unchanged data from the original source. Consequently, this allows multiple copies of the MEDITECH data to be maintained without consuming lots of storage space2. Furthermore, the IDR copies are directly accessible and immediately available from within the storage array, thereby providing a resource for instant recovery. In BridgeHead’s experience, most recoveries today are likely to use an IDR copy that can either be instantly restored or that can be provided to MEDITECH to enable them to repair the hospital’s live system.

A Choice Of Technology

BridgeHead’s MEDITECH data protection solutions offer the widest range of recovery options, from basic ISB, through to synchronous replication at a disaster recovery (DR) site with frequent IDR checkpoints.

The entry level (and lowest cost) option is basic ISB backup (which is the minimum requirement for a copy of last resort, usually consisting of a tape backup that is sentn to a secure offsite storage facility). However, ISB on its own only provides the most basic disaster protection. This is easily illustrated when examining the two key metrics of ‘RPO’ and ‘RTO’:

RPO (Recovery Point Objective)

RPO (Recovery Point Objective) is defined by how far back from the time of disaster a hospital IT department would have to go to restore a backup, i.e. how long ago was the backup taken. For a once a day tape backup, this could be as much as 24 hours. If the most recent tapes have not yet reached a secure store when a disaster occurs, a site might have to even use the previous day’s backup

RTO (Recovery Time Objective)

Scalable Options Offer Hospitals Flexibility

The flexibility and versatility of BridgeHead’s MEDITECH data solutions ensure that hospitals can improve and scale the extent of their protection as required to improve their RPOs and RTOs.

RPO can easily be enhanced by making more frequent backups – a feature that BridgeHead’s MEDITECH IDR Backup Agent offers by providing the ability to make more regular checkpoints of consistent data. Any technology employed by a hospital that does not directly co-operate with the MEDITECH HCIS risks delivering an inconsistent database and data corruption may occur that might not be uncovered until much later.

BridgeHead Software supports the widest range of storage array technology, always including the option to restore to another volume to allow MEDITECH extended access to repair the live data

RTO can also be easily improved. Using BridgeHead’s IDR Backup Agent, the MEDITECH HCIS, and its data, can be directly accessed (depending on the technology used) or immediately copied/restored without the need for tapes (that would largely come from the secure offsite location). In this case, the choice of array technology used provides some interesting ‘trade-offs’. More space efficient snapshot-type technologies, available within certain storage facilities, reduce the storage cost. They are also easy to access, although not generally suitable for production use. The mechanisms to restore a snapshot, or promote it for live use, depend on the storage array technology.

BridgeHead Software supports the widest range of storage array technology, always including the option to restore to another volume to allow MEDITECH extended access to repair the live data. Many hospitals require multi-site protection. In the simplest cases, this is probably a DR space at a shared, rented facility, protecting the MEDITECH data and enabling recovery if a datacentre is lost. However, this solution isn’t quick. In this scenario, hospitals have the option to utilise BridgeHead’s ‘out of frame’ IDR technology.

BridgeHead’s ‘out of frame’ IDR offers a range of choices from simply making a copy of the MEDITECH HCIS to another array (via direct fibre channel writes or via another server using TCP/IP), through to leveraging the power of array-based remote mirroring, with frequent, pre-defined IDR checkpoints at the secondary site, for a near instant recovery.

The Perfect Endorsement

Having worked with MEDITECH closely for so long, BridgeHead’s success has grown as MEDITECH’s share of the HIS market has increased. Today, over 1,200 hospitals around the world utilise BridgeHead’s data management products. However, the perfect endorsement really comes from the fact that MEDITECH use BridgeHead’s technology for their own data protection, in-house.

Staying In Front

BridgeHead’s relationship with MEDITECH is such that it continues to invest time and effort in designing and developing new, innovative data management and protection solutions for hospitals utilising the MEDITECH HCIS. Among these are:

Cascading Backup

The MEDITECH HCIS can only make one backup at a time. Recognising this, BridgeHead introduced the concept of a ‘Cascading Backup’ as an option with the IDR solution. The difference with a ‘Cascading Backup’ is that the IDR Agent continues to make regular copies of the MEDITECH HCIS data, but directly uses the most recent IDR version as the source for the backup, eliminating the need to interact with the MEDITECH application. This works in a very similar way to a ‘disk-to-disk-to-tape’ type backup, but designed specifically for the MEDITECH environment.

Restore Utility

In 2011, BridgeHead introduced a dedicated ISB/IDR Restore Utility. This application allows hospitals to easily see all the restore points for each MEDITECH server. In addition, it provides an easy to use graphical restore wizard to request a restore

MEDITECH Backup Facility (MBF)

In 2012, MEDITECH, in collaboration with BridgeHead Software, refined the ISB process to re-align the workflow. With MBF, BridgeHead takes full responsibility for the array manipulation, while MEDITECH is now only responsible for preparing the databases (briefly quiescing the system and making the data on disk transactionally consistent). In adopting this method, the skills of both partners are better utilised: MEDITECH prepares the data; BridgeHead interfaces with the storage array.

BridgeHead enthusiastically endorses this approach as it allows the ISB solution to support new array facilities without imposing on MEDITECH’s release cycle. MBF also enables BridgeHead to take advantage of array ‘consistency groups’ – this lessens the time that the MEDITECH system has to pause while the storage device creates a frozen consistent ‘point in time’ base for the backup (or IDR copy). MBF also allows BridgeHead to introduce space efficient ‘out of frame’ IDR options with pre-defined restore points on alternative arrays that act as secondary datacentres for DR purposes.

Deployment is also simplified because there is no need to configure storage array utilities on every MEDITECH file or database server.

BridgeHead’s Healthcare Data Management (HDM) Solution for MEDITECH SCA

BridgeHead Software remains the only vendor to support all MEDITECH certified arrays. Why would a hospital choose a product restricted to one vendor when they can use the market leader that supports ALL MEDITECH certified devices?

When MEDITECH’s SCA module was introduced, BridgeHead quickly realised that it created new problems for hospitals due to the massive number of files generated and the resulting huge storage requirements. The files produced by MEDITECH SCA are static, i.e. highly unlikely to change and very rarely accessed. Consequently, they are a perfect target for a data management method combining archiving and backup. BridgeHead’s ISB provides a fast whole disk restore. However, if the disk space required grows to many terabytes, then even the ISB process will take a significant amount of time.

BridgeHead did a detailed analysis on the MEDITECH SCA data and realised that the older files (generally representing patients long since discharged) were extremely unlikely to be used again and so could be migrated to secondary storage. By utilising archiving, instead of trying to save all data within a backup cycle, hospitals can make multiple copies of individual files onto different media, in different locations, thereby removing the data from the primary storage and reducing the size of the backup. BridgeHead’s File Archiving Agent (also known as ‘FileStore’) has become the de-facto standard for SCA data management and protection.

Coming in 2013… View Only mode to minimise MEDITECH downtime during updates

BridgeHead’s engineering team continues to work closely with MEDITECH and has plans in 2013 to introduce a new View Only Electronic Medical Records (VOEMR) service. VOEMR seeks to allow access to MEDITECH data in View Only mode while the MEDITECH service would otherwise be down to apply updates. VOEMR builds on the rich ISB/IDR heritage using the same array technology and protocols to deliver a Point-in-Time copy of the data to MEDITECH so that they can use that in View Only mode while they update the live database.

Clearly such a process requires a lot of co-ordination between the MEDITECH applications and the backup application manipulating the store arrays. In 2012,MEDITECH approached BridgeHead to help them develop the necessary protocols and commands. This collaboration features the same teams from both MEDITECH and BridgeHead that created ISB, continuing to work together as previously – a true testament, not just to the great work they do, but also the long standing stability of both groups.

In relation to MEDITECH, BridgeHead Software is best known for its data protection solutions. However, the readth of BridgeHead’s offering extends beyond this remit in relation to both MEDITECH and non-MEDITECH applications typically found within a hospital environment.

DICOM Archiving

BridgeHead’s HDM Solution doesn’t stop at MEDITECH specific data – its strategic vision is to protect ALL of a hospital’s data, clinical and administrative. Currently, one of the most sought after requirements by hospitals is a solution to help manage their DICOM-based data from PACS applications – typically, but not restricted to, radiology. BridgeHead’s DICOM Archiving Agent (also known as DICOMStore) works in conjunction with BridgeHead’s HDM Platform to provide a Vendor Neutral Archive (VNA) for the efficient and effective management of DICOM data. In addition, BridgeHead also offers Professional Services covering, for example, the importing of DICOM data into the BridgeHead VNA and a Tag Morphing service to translate and standarise DICOM tags

VMware

BridgeHead recognises the key importance of virtualisation in today’s healthcare environments. VMware is the global market leader in virtualisation technologies and, as might be expected, its solutions are often found within hospitals. As a result, BridgeHead has developed its VMware Backup Agent to support capabilities for VMware’s advanced VADP backup facilities, which offer agent-free backups (in most cases). The VMware Backup Agent utilises VMware APIs to create VMware snapshots of virtual machines that are then accessed from the (physical) BridgeHead server. This allows the backup process to occur with no load placed on the VMware server. Effectively, this process could be described as ISB for VMware.

Other Hospital Data

BridgeHead’s portfolio extends beyond its capabilities with MEDITECH. It has both Archiving and Backup Agents to manage and protect all manner of a hospital’s applications and data, from Oracle, Cache and SQL database backup through to Exchange email archiving. These Agents can be deployed over a wide range of servers, including Windows, UNIX, LINUX, NDMP and Netware.

Managed Services

In addition to its software solutions, BridgeHead also offers Managed Services for Backup and Recovery for hospitals utilising MEDITECH’s HCIS. This offering was designed to help relieve the day-to-day pressure on healthcare IT by providing an outsourced service for the creation of ‘clean’ backups, thereby providing peace of mind around the protection of their HCIS data.

For those customers who require a more rigorous, comprehensive outsourced offering, BridgeHead has created a premium version of the service. This service effectively makes BridgeHead a part of a customer’s disaster recovery team and includes an onsite presence, as necessary, to support general restores, actual data recovery events or tests. The service also expands to include 24x7 coverage of failure notifications.

Support

At BridgeHead, customer support is of paramount importance. Hospitals utilising BridgeHead’s support function will not be subjected to remote call centres with staff reading from a script or explaining the same problem over and over again to numerous levels of support.

BridgeHead’s support personnel are highly trained Healthcare Data Management specialists, each experienced in installations and implementations of BridgeHead solutions. This has been regularly acknowledged within the BridgeHead customer base with hospitals lauding the work carried out by their support representatives in the speedy resolution of technical issues to conclusion.

BridgeHead prides itself on being big enough to provide a professional service while small enough to care about every customer.

All MEDITECH Components, All MEDITECH Supported Arrays

BridgeHead is proud of its record of being the first to support ISB on MAGIC OSAL, Client-Server 5.x, EMR, Client-Server 6.x and SCA. BridgeHead was also the…

• First to support EMC Symmetrix
• First to support EMC Clariion
• First to support EMC RecoverPoint

BridgeHead is also....

And the investment of supporting future MEDITECH related platforms and technologies will continue. BridgeHead Software remains the only vendor to support all MEDITECH certified arrays. Why would a hospital choose a product restricted to one vendor when they can use the market leader that supports ALL MEDITECH certified devices?

Conclusion

BridgeHead’s HDM Solution doesn’t stop at MEDITECH specific data – its strategic vision is to protect ALL of a hospital’s data, clinical and administrative.

BridgeHead Software remains the only vendor to support all MEDITECH certified arrays. Why would a hospital choose a product restricted to one vendor when they can use the market leader that supports ALL MEDITECH certified devices?

BridgeHead has unique capabilities in the protection of MEDITECH’s HCIS with its Integrated Serverless Backup (ISB) and Integrated Disaster Recovery (IDR) Agents. BridgeHead also has data management solutions for MEDITECH’s SCA module adding further strength to their offering. These are mature technologies that have been around for a long time, used by over 1,200 hospitals around the world, and so are considered ‘tried and tested’ – albeit they are constantly evolving to suit the needs of the market. Yet the best endorsement is purely the fact that MEDITECH utilise the products for their own in-house data protection.

In addition to its pure MEDITECH offerings, BridgeHead’s HDM Solution also extends to assist in the data and storage management of other areas within a hospital – all managed from the same environment. These include DICOM Archiving (a VNA for medical image management such as radiology, cardiology and the like), VMware backup, as well as file and system protection of other healthcare applications and data such as databases, operating systems, email, PDFs and office productivity documents. Wrap these offerings in a rich services portfolio, including a managed service for MEDITECH backup (and archiving), plus first class support from a dedicated team, and BridgeHead provides a great recipe for healthcare data management for hospitals who have invested in the MEDITECH HCIS.

Reference

1. MEDITECH and Bridgehead Software Celebrate 10 Years of Back-up Success. Click here

2. MEDITECH ISB and IDR Back-up Agent Information. Click here