Senior Data Engineer

Senior Data Engineer Location: Sheffield or Birmingham A leading global financial services organisation is seeking a Senior Data Engineer to join its Cybersecurity Science & Analytics function. The successful consultant will join a multi-skilled Platform & Data Engineering team responsible for delivering engineering capabilities across a strategic cybersecurity data lake and analytics platform. The role combines data engineering, cloud infrastructure, automation, platform operations and production support responsibilities. Responsibilities • Design, develop and optimise cloud-based data pipelines and ETL workflows. • Ingest data from internal and external sources into enterprise data platforms. • Create and maintain curated datasets and reusable data assets. • Improve reliability, automation and operational resilience of data services. • Support Azure cloud infrastructure and data platform operations. • Work closely with engineering teams to onboard new data sources and define schemas. • Contribute to exploratory data analysis and platform enhancement initiatives. • Support real-time analytics and data availability improvements. Essential Skills • Strong Data Engineering experience. • Azure cloud platform expertise. • Enterprise ETL and data pipeline development. • Python or equivalent scripting experience. • SQL and/or Kusto Query Language (KQL). • GitHub and CI/CD practices. • Linux administration and scripting. • Infrastructure as Code, ideally Terraform. • Azure DevOps. • SRE / production support experience. Desirable Skills • Azure Data Factory • Databricks • Unity Catalog • Synapse Analytics • Kubernetes• Azure Functions • Logic Apps • Kafka or Azure Event Hubs • Spark Streaming • Prometheus • Grafana • Elasticsearch • Cosmos DB • Power BI • SIEM and SOAR platforms Experience • Large enterprise environments • Cloud-based data platforms • Data lake architecture and delivery • Regulated industries advantageous • Cybersecurity experience beneficial but not essential Working Environment • Collaborative global engineering team • Multi-disciplinary technology environment • Strong learning and development culture • Exposure to cybersecurity, cloud platform engineering and advanced analytics