Our project represents an unprecedented venture into the realm of "healthcare super data," as our client calls it. The intricacies of this highly complex system required meticulous attention, considering the absence of comparable products in the current market.
- For instance, the data is ingested in various formats, including CSV, HL7, and different versions of FHIR, like FHIR STU3 and FHIR R4. While some sources provided completed, enriched resources, others offered less detailed data, and sometimes, this volume varied significantly, although the resource IDs in the database seemed the same. So, we had to check and validate everything case by case.
- Also, prioritizing data accuracy, we devised a strategic plan to promptly trace and rectify discrepancies. This involved establishing robust rules on how exactly the system is supposed to handle one error or another or how to modify resources to ensure the highest quality of healthcare data.
- A daring migration from AWS to GCP FHIR storage was a pivotal moment in our journey. We moved from the database which provides a closed database solution as a FHIR server. Instead, we chose GCP FHIR storage as the main database on which we are aimed to build our solutions, so to say, the heart of the system. The decision was made that all related services should also move to GCP for convenience and consistency. The migration process is now completed. The number of GCP services is constantly expanding and will continue to do so.
- With information derived from seven different vendors, our task is to obtain, validate, and process diverse datasets. In general, we can leverage existing resources with sufficient information to generate entirely new data entities.
As of now, the system manages a staggering 20 million resources in production, terabytes of historical data, and handles diverse imports weekly, including real-time hospital data and monthly/quarterly data from open national databases.
The development process avoids ready-made solutions or frameworks, employing entirely custom code to meet unique requirements. Our transformative journey continues, shaping the landscape of healthcare data management.