Lead Data Engineer (healthcare solution)

Intellias is looking for a Lead Healthcare Data Engineer to join a large-scale digital healthcare initiative focused on building a FHIR-based health data platform and transforming fragmented healthcare data into clean, standardized, and trustworthy information. The role requires both deep hands-on engineering expertise and broad technical influence — establishing reusable frameworks and standards, mentoring experienced engineers, and raising the engineering bar across the team. Project Overview: We're helping a US digital-health platform get its patients' medical data in order: lab results, diagnoses, medications, visits and more. The data comes from many different clinics and labs, each with its own format, so right now it's pretty messy. Our job is to clean up and standardize that data close to the source, bring scattered logic into one place, and see the real data quality every day instead of guessing. Data engineers will work mostly in Databricks/Spark, building pipelines, reference data and quality checks together with the dev team and clinical experts. Requirements: - 8+ years of professional experience in data engineering, healthcare data platforms, HealthTech, or healthcare interoperability. - Proven Lead-level technical ownership of complex production systems, including the ability to set technical direction and solve problems spanning multiple components and engineering teams. - Strong hands-on expertise in Python and production-grade data engineering. - Deep experience with Apache Spark/PySpark at scale, including partitioning, performance optimization, memory tuning, and distributed processing. - Experience with modern lakehouse technologies such as Databricks and Delta Lake, or comparable platforms. - Strong experience designing and operating large-scale ETL/ELT pipelines, including CDC, incremental/delta processing, idempotency, schema evolution, and batch/streaming workloads. - Proven experience diagnosing and eliminating complex performance and scalability bottlenecks in distributed data-processing systems. - Deep hands-on experience with FHIR R4, including resources, profiles, Bundles, validation, and large-scale healthcare data transformation. - Strong knowledge of US Core, USCDI, HL7 v2.x, and C-CDA-to-FHIR transformation patterns. - Strong understanding of healthcare terminology standards including SNOMED CT, LOINC, and RxNorm. - Experience working with clinical, claims, coverage, eligibility, or comparable healthcare data models. - Strong experience with data quality, deduplication, record/patient linking, terminology normalization, and validation. - Strong SQL skills and understanding of data modeling, schema evolution, data quality, and production data architecture. - Experience with Git, CI/CD, automated testing, observability, and production operations. - Strong understanding of healthcare security, privacy, and regulatory requirements, including HIPAA and PHI-safe engineering practices. - Proven ability to establish reusable engineering frameworks, architectural patterns, standards, and data-quality practices adopted by other engineers. - Demonstrated technical leadership through architecture/design reviews, mentoring, pairing, and cross-team engineering influence. - Practical experience using AI coding agents such as Claude Code or comparable agentic engineering tools as an integral part of the development lifecycle. - Strong engineering judgment around when to use AI/LLM-based approaches versus deterministic rule-based solutions, particularly where correctness, cost, performance, or PHI safety are critical. - Professional English sufficient for direct collaboration with U.S.-based technical and business stakeholders. Nice to Have - Deep experience with large-scale FHIR data modeling and healthcare interoperability platforms. - Experience with Master Patient Index (MPI), patient matching, identity resolution, or large-scale record linkage. - Experience with payer data exchange, including CARIN Blue Button, Da Vinci implementation guides, or similar standards. - Experience working with pharmacy, laboratory, or other specialized healthcare datasets. - Knowledge of ICD-10, CPT, CVX, and additional clinical terminology/code systems. - Experience with 21st Century Cures Act, ONC certification, CMS Interoperability Rule, or related U.S. healthcare regulatory frameworks. - Experience building LLM/agent-based capabilities for clinical data extraction, normalization, terminology mapping, or healthcare data quality. - Experience designing reusable AI skills or agentic workflows adopted across engineering teams. - Contributions to open-source healthcare, FHIR/HL7 communities, or relevant industry working groups. Responsibilities: - Set the technical direction for healthcare data engineering across ingestion, transformation, validation, normalization, and delivery. - Own the end-to-end architecture of large-scale distributed data pipelines transforming heterogeneous source data into clean, standards-conformant FHIR. - Design and evolve scalable solutions using Python, Spark/PySpark, Databricks/Delta Lake, and modern data-processing technologies. - Diagnose and eliminate complex performance and scalability bottlenecks across distributed data workloads. - Establish standards and reusable frameworks for data quality, deduplication, patient/record linking, terminology normalization, and reliable idempotent processing. - Define architectural patterns for FHIR transformation, validation, schema evolution, and production delivery. - Build and evolve automated testing and validation capabilities for data quality and FHIR conformance. - Establish engineering patterns and reusable tooling that allow the broader team to deliver new healthcare data capabilities faster and more consistently. - Champion PHI-safe engineering practices and ensure healthcare data solutions meet applicable security, privacy, and compliance requirements. - Make AI-native engineering a practical part of the team's development model, using coding agents and reusable AI capabilities to automate repetitive work and accelerate complex engineering tasks. - Establish engineering guardrails for AI-assisted development, ensuring AI-generated solutions remain correct, tested, secure, observable, and PHI-safe. - Determine where deterministic processing provides better reliability than AI/LLM-based approaches and establish appropriate architectural patterns for both. - Mentor Senior and Middle engineers through design reviews, pairing, technical guidance, and knowledge sharing. - Drive complex, ambiguous engineering problems from initial investigation through architecture, implementation, and production operation. - Collaborate with engineering, product, security, data, and healthcare-domain stakeholders to translate complex interoperability requirements into scalable technical solutions. - Represent strong healthcare data engineering practices in relevant architecture, standards, and technical discussions. Why this position: - This role offers the opportunity to set the technical direction for healthcare data engineering on a big platform operating at scale. - As a Lead-level Individual Contributor, your impact extends well beyond individual pipelines or services. You will own some of the platform's hardest engineering problems, establish reusable frameworks and standards, and influence how an entire engineering team builds, validates, and operates healthcare data solutions. - The technical scope combines large-scale distributed data engineering, FHIR/HL7 interoperability, healthcare terminology, data quality, cloud data platforms, and AI-native engineering. - AI is a core part of the engineering model rather than an experimental add-on. You will help determine how coding agents and reusable AI capabilities can increase engineering leverage while maintaining the rigor required for production healthcare systems and PHI-sensitive data. - This is a hands-on Staff IC position: you will set direction and mentor engineers, but your technical authority comes from remaining deeply involved in architecture, implementation, performance optimization, and production problem solving.