Customer Testimonial
HealthNow New York Inc.

Customer Testimonial from HealthNow New York Inc:

HealthNow is a leading provider of healthcare insurance and related services.

HealthNow uses Sybase ASE 12.0 servers to maintain a terabyte-sized data production database as part of our infrastructure. Some weeks ago, we started receiving reports of application errors and data corruption ("695" errors) on our production Sybase ASE 12.0 server.

To investigate, we used DBCCs and discovered that several large, critical tables appeared to have corruption. We opened a case with Sybase Technical Support, who ended up having us use some more DBCC commands to try to "zap" the offending locations in the database. We also tried manually copying out rows to a backup table and then removing those rows in an attempt to isolate the corrupted area.

These attempts met with limited success; we were able to recover most of the critical data, but were unable to fix the continuing corruption errors being seen by users. Even worse we were not sure of where the corruption would show up next , how it started , and how to fix it

At this point, we contacted White Sands Technology about their Disaster Recovery Toolset and related services. Within minutes of negotiating a contract, White Sands's technicians, using their remote Lend-a-Hand system, began analyzing the database.

It turned out that, through some hardware or software error, an allocation map page in the database had gotten overwritten by another page from somewhere else, thus causing the original corruption as well as other side-effects in the database.

Using one of their recovery utilities, the White Sands engineer rebuilt the allocation map page and loaded the fixed page back into the database, thus fixing the original corruption. Since by this point we had already recovered the data we needed from the affected tables, all we had to do now was simply truncate and drop those tables, and then the database was in good working order.

We tried the fix first on our development server, and it worked fine and then came up clean after running a new consistency check using the Disaster Recovery Toolset. So we then applied the fix to our production server and it worked fine there as well.

The White Sands tools really saved us a lot of down time -- reloading a database in the hundreds of gigabytes is not something you do lightly, even if it is possible given the fact that corruption like this can lie dormant for several days or even weeks. And miraculously, it was able to restore our damaged production server while it remained on-line, so users were not affected at all!

I can’t say enough about the true professionalism shown by the folks at White Sands. They truly partnered with us during this whole process even working at night and on the weekend to minimize the effect to our users. We feel much more comfortable about the state of our database and our ability to tackle problems of this nature in the future.

Regards,

Jeffrey Kawa
Manager of Technical Services