Data Engineer - Healthcare
Data Engineer
About Sand
Sand Technologies is a global Physical AI company using data and AI to make critical industries work better. We partner with governments, cities and enterprises to improve how essential systems operate across healthcare, water, energy, telecommunications and infrastructure.
Our work delivers proven real-world impact. We have built AI systems that help manage London’s water supply, supported telecom network planning across hundreds of cities, and developed digital healthcare platforms serving tens of millions of people across Africa. From intelligent command centers to AI-powered infrastructure platforms, we help organizations sense, analyze and act in complex environments.
Our people are ambitious, curious and relentlessly practical. Our teams work alongside clients in the field, solving hard problems and deploying solutions that last. With colleagues across Africa, Europe, the UK and the US, we operate across the full stack - from research and engineering to deployment and capability building.
Our mission is simple: to harness AI to solve humanity’s most pressing challenges.
About the role
Delaware’s designated health information exchange and Health Data Utility is bringing its clinical data repository, its All Payer Claims Database and its internal warehouse onto one governed, longitudinal record. Sand is delivering that platform on Symmetri, our data platform, running inside the client’s own AWS account.
The scale is real. The exchange has moved clinical data since 2007 and claims since 2013. It handles roughly 4.7 million HL7 messages and 220,000 CCDs a month from about 120 data senders, holds claims covering about 87% of Delawareans, and maintains around 5.5 million patient identities across 611 sources.
None of it can stop while we move it. Results delivery, admission alerts and state claims reporting run throughout. We migrate source by source, in parallel, with reconciliation the client can see and a rehearsed rollback on every cutover.
You will own the pipelines and the migration waves that make this happen: building the ingestion and transformation layer for HL7 v2, C-CDA FHIR R4 and claims flat files, mapping them onto one health data ontology and proving - record by record - that what landed in the new platform matches what left the old one.
We operate with a strong code-first, “data as a product” mindset, where testing, reliability, observability and performance are non-negotiables. In a regulated health environment with fixed federal funding deadlines, that discipline is the job, not a nice-to-have.
What you’ll do
- Build the ingestion layer. Design and operate pipelines for HL7 v2 (ADT, ORU, ORM, MDM, VXU), C-CDA, FHIR R4 and claims flat files, parsing documents and messages into discrete, coded data elements rather than stored blobs.
- Land everything raw and replayable. Every message and file is stored unchanged, time-stamped and re-runnable, with element-level provenance: source system, sender, received time, original message, pipeline and mapping version.
- Map sources to one ontology. Transform records into shared objects — Person, Encounter, Condition, Result, Medication, Claim — aligned to FHIR R4 and US Core on the clinical side and the APCD Common Data Layout on the claims side. Map local codes to ICD-10, CPT, LOINC, RxNorm and SNOMED CT.
- Run the migration waves. Shadow load, parallel run, cutover with rollback, retire after acceptance. Four waves across claims history, clinical history, live feeds and outbound consumers, each one signed off by the client before anything is switched off.
- Prove the data moved correctly. Build the row-level and record-level reconciliation that compares counts, totals, identities and sampled clinical content between old and new. These reports are reviewed by the client's data quality staff and gate every cutover.
- Make data quality measurable per sender. Validate structure, required fields, code validity, duplicates and volumes against each sender's baseline, and produce six-dimension quality scorecards that go back to the source system owners.
- Engineer for failure. Every pipeline ships with retries, a dead-letter queue and a circuit breaker. A record that fails a quality check is never silently dropped — it goes to a steward, who fixes and replays it or discards it with an audit record.
- Optimize what exists. Profile current pipelines for latency, failure rate, rejected messages and data quality, establish baselines, and rebuild them with validation at intake.
- Write the data contracts. Every migrated dataset carries its schema, quality targets, owner, refresh schedule and consumers.
Who you are
Core engineering
- 5+ years designing and operating large-scale data platforms in cloud or hybrid environments, with production ownership rather than project handover.
- Strong proficiency in Python and SQL, ideally with dbt, and a track record of clean, tested, production-quality code.
- Deep hands-on expertise with distributed processing (Spark/PySpark) and streaming or event-driven systems (Kafka or equivalent).
- Experience building lakehouse architectures on open table formats, with transactions, schema evolution and time travel.
- Experience deploying and operating on AWS, including CI/CD and infrastructure-as-code. Kubernetes experience is valuable.
- Demonstrated ability to design highly available, observable, production-grade data systems with defined SLAs and SLOs.
Migration and data quality
- Proven experience migrating production data between platforms without interrupting live services — shadow loads, parallel runs, reconciliation and rehearsed rollback.
- Fluency in reconciliation technique: row counts, checksums, record-level comparison, and the judgement to know when a mismatch is a real defect rather than a modelling difference.
- Experience building automated data quality frameworks, lineage capture and alerting as part of the pipeline, not bolted on afterwards.
Health data and regulated environments
- Working knowledge of health interoperability standards — HL7 v2, C-CDA, FHIR R4 and US Core. Experience with IHE profiles (XDS.b, XCA, PIX, PDQ) is a strong advantage.
- Familiarity with clinical and claims terminology: ICD-10, CPT, LOINC, RxNorm, SNOMED CT, and value set management.
- Experience with healthcare claims data, ideally All Payer Claims Database work or the APCD Common Data Layout.
- Experience building secure, governed data environments in regulated settings, with working knowledge of HIPAA and HITECH. Exposure to 42 CFR Part 2, Data Segmentation for Privacy (DS4P) or consent enforcement is highly valued.
- Experience working with an enterprise master patient index and identity resolution. Verato, Mirth Connect or comparable integration engines are a plus.
How you operate
- Comfortable in multi-stakeholder environments where a public agency, incumbent vendors, payers and more than a hundred data senders all have a say.
- Systems-thinking mindset, focused on scalability, modularity and long-term platform evolution.
- Able to work to hard external deadlines and acceptance gates without trading away test coverage or auditability.
We would rather hear from you than not. If you are strong on data engineering and migration but have not yet worked with health standards, say so and apply. Health interoperability is learnable by a good engineer; the migration discipline and the regulated-environment instincts are much harder to teach.
Location
This role is not a remote position. You must be based in the United States and able to travel to the client site in Dover, Delaware as the project requires, including for design workshops and cutover windows. Travel is heaviest during Discovery and around production cutovers rather than on a fixed weekly pattern.
How we work
Due to the highly collaborative and internationally distributed nature of our work, successful candidates must be comfortable operating in small teams while contributing to larger, globally coordinated efforts. A strong sense of ownership, self-motivation and discipline in maintaining clear and consistent communication through virtual collaboration tools and video conferencing is essential.
Apply for this job
*
indicates a required field
