About the Customer
A large healthcare provider network operating across multiple facilities aimed to modernize its data and analytics environment to improve clinical, operational, and financial insights while maintaining strong controls over sensitive healthcare information.
Business Challenge
· Data silos across EHR, claims, laboratory, radiology, and other clinical and operational systems, limiting unified data access and insights.
· Slow analytics and delayed reporting caused by legacy and fragile ETL pipelines, impacting timely decision-making.
· Limited data governance and inconsistent access controls, creating challenges in securely managing sensitive PHI datasets.
· Challenges in enabling ML and AI use cases such as readmission risk prediction, capacity forecasting, and revenue leakage detection due to unreliable and duplicated data sources.
Solution Delivered: Databricks Lakehouse Architecture
Adroitent implemented Databricks Lakehouse architecture using the Medallion pattern (Bronze, Silver, and Gold) to create a unified, governed, and AI-ready data platform.
Key Solution Components
Data Ingestion (Bronze):
- Data ingestion from EHR records, claims data, HL7/FHIR messages, and lab data
- Enabling raw data into Delta live tables to support data reliability
Data Transformation (Silver):
- Standardization, deduplication, patient/provider entity resolution, code normalization (ICD/CPT), and data quality rules
- Pipeline orchestration via Databricks-native workflows and structured monitoring
Data Curated (Gold):
- Curated data for quality measures, revenue cycle dashboards, clinical ops, and population health
- BI enablement and model-ready feature tables for ML model orchestration
Project Highlights
- Data reliability and performance: Databricks Lakehouse reference patterns helped ingest and transform data. To improve trust and auditability, the platform was standardized on Delta Live tables, leveraging capabilities such as ACID reliability, schema enforcement, and historical versioning.
- Governance, Security & HIPAA-Aligned Controls: Governance was implemented using Unity Catalog to enable centralized access control, auditing, and lineage across workspaces and data assets. Various Security controls included:
- Role-based and attribute-based access for PHI vs non-PHI datasets (row/column-level controls where required)
- Audit logging and lineage visibility for compliance and investigations
- HIPAA-aligned configuration approach based on Databricks HIPAA guidance (PHI handling, security posture, and operational controls).
- BI + ML Model enablement: The MLFLow was used on a single platform with curated marts and feature-ready datasets. The ML models were fine-tuned based on the business use case and implemented.
Technology Stack
Databricks Platform, Delta live tables, MLFlow
Business Outcome
- Faster patient data onboarding by using reusable data ingestion patterns
- Improved patient reports (Streaming and real-time analytics)
- Secure, compliant access to sensitive healthcare data thru effective governance
- Accelerated reporting through comprehensive dashboards, enabling faster and more informed decision-making.
- Improved trust in analytics through standardized, curated datasets and governed data access.
- Accelerated the customer’s ML initiatives with feature-ready data products
