Copy of Senior DataOps Engineer

N-iX is looking for Senior DataOps Engineer to join the team Client Overview: Our client is an Azerbaijani telecommunications company, the largest mobile network operator in Azerbaijan. The main products are: Fixed telephony, Mobile telephony, Internet services, Wireless broadband, and Value-added services. Project Objectives: The primary goal is to accelerate the client’s Data & AI initiatives via a secure, hybrid cloud foundation on AWS while systematically modernizing the IT estate as part of the cloud migration. Key Project Objectives include: - Cloud Foundation & Landing Zone: Deploy target hybrid network architectures, establishing a secure Landing Zone and hybrid Data/AI platforms on AWS. - Security, Compliance & Governance: Operationalize on-prem tokenization (achieving zero raw PII in the cloud), resolve policy blockers to include AWS in the ISMS, and establish a Cloud Center of Excellence (CCoE) to govern Cloud adoption. - AI Chatbot & Voicebot Design & Implementation: Develop and operationalize a flagship Customer Care Chatbot and Voicebot as the first hybrid-setup consumer. Key Responsibilities: - Design, implement, and maintain the AWS Data Platform foundation, including Amazon S3 lake layout, Apache Iceberg table format standardization, and AWS Glue Data Catalog integration. - Build and optimize scalable batch and streaming data pipelines using Amazon EMR (Serverless and EMR-on-EKS), Apache Spark/PySpark, Apache Flink, and Amazon Athena with workgroup cost caps. - Implement data tokenization and de-identification pipelines on the data side (PA-T) using Protegrity/Spark UDFs for batch, Spark Streaming for micro-batch, and Kafka Connect SMT for streaming PII masking prior to cloud transit. - Build and manage real-time streaming architectures with Amazon MSK (Managed Streaming for Apache Kafka), MSK Replicator/MirrorMaker2, and Schema Registry integration. - Implement Medallion Architecture (Bronze, Silver, Gold layers) for lakehouse data modeling, automating Iceberg table registration and backfill frameworks. - Execute data migration waves across non-PII and PII datasets using AWS DataSync, automated register steps, and streaming migration configs. - Configure data access control and governance models using AWS Lake Formation, cross-engine authorization, AWS Macie for PII detection, and Informatica Data Catalog (Axon, EDC, BDQ) integration. - Build automated schema-drift identification components, GitLab CI/CD pipeline integration for dataset synchronization, and automated data contracts/circuit breakers. - Implement DataOps observability, centralizing logging via Amazon CloudWatch, configuring FinOps cost & anomaly monitoring, and setting up automated alerts. - Author technical documentation, operational runbooks, disaster recovery (DR) procedures, and cutover/rollback playbooks for data platform hardening. Requirements: Mandatory Technical Skills: - 4+ years of hands-on experience as a Data Engineer or DataOps Engineer building enterprise-grade data platforms and pipelines. - Strong expertise with AWS Data Analytics stack: Amazon S3, AWS Glue Data Catalog, AWS Lake Formation, Amazon EMR (EMR-on-EKS / Serverless), Amazon Athena, AWS DataSync, and Amazon MSK. - Deep experience with Apache Iceberg table format, cataloging, compaction, and schema evolution. - Proficient in Apache Spark / PySpark and Spark Streaming for batch, micro-batch, and real-time data processing. - Strong experience in Medallion Lakehouse Architecture design and implementation (Bronze, Silver, Gold layers). - Practical experience in implementing data tokenization and encryption at scale (e.g., Protegrity, Thales, FPE, or Spark UDF-based de-identification pipelines). - Solid knowledge of event-driven architectures & streaming: Apache Kafka / Amazon MSK, Kafka Connect (SMT), Schema Registry, and MirrorMaker2 / MSK Replicator. - Expertise with Relational Databases (Amazon RDS, PostgreSQL, Oracle) and data synchronization techniques. - Hands-on experience with DataOps CI/CD & Automation: GitLab CI/CD, Infrastructure-as-Code (Terraform / AWS CDK), schema-drift detection, and data contract validation. - Familiarity with data governance tools and enterprise data catalogs (e.g., Informatica Axon/EDC, AWS Lake Formation). Strong Plus (Nice-to-Have Skills): - AWS Certified Data Analytics – Specialty or AWS Certified Data Engineer – Associate. - Experience in telecom domain data models, CDR processing, and high-throughput real-time telemetry. - Experience with cloud-side tokenization/detokenization via Athena UDFs / AWS Lambda. - Familiarity with containerization (Docker, EKS, Kubernetes) for big data runtimes. - Experience with AWS Macie and FinOps cost-allocation/anomaly-detection frameworks. Soft Skills & Team Fit: - Strong critical thinking, problem-solving, and analytical skills. - Excellent communication and collaboration skills to work closely with cross-functional teams (Data Science, Cloud/Platform, Security, Governance). - Results-oriented, proactive mindset with strong ownership of deliverables within an Agile / Scrum framework. - Upper-Intermediate+ English level (written and spoken). We offer*: - Flexible working format - remote, office-based or flexible - A competitive salary and good compensation package - Personalized career growth - Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more) - Active tech communities with regular knowledge sharing - Education reimbursement - Memorable anniversary presents - Corporate events and team buildings - Other location-specific benefits *not applicable for freelancers