Senior Data Engineer

Pay

On application

Type

Contract

Sector

IT & Internet

Closes

in 19 days

Recruiter

Mediro ICT · All jobs at Mediro ICT

About the role

Our client in Johannesburg is recruiting for a Senior Data Engineer to join their team. The Senior Data Engineer will provide hands-on support for migrating reporting workloads from Microsoft Fabric and Power BI to Databricks AI/BI. The role will focus on data pipeline development and optimisation, dependency analysis, validation, performance and cost optimisation, while also enabling and upskilling the client’s internal data team.

• Senior-level engineer with strong analytical, problem-solving and Databricks skills. Able to work independently within an existing team, manage changing priorities, optimise performance and costs, communicate risks clearly, and support documentation, knowledge transfer and stakeholder collaboration.
• The Senior Data Engineer will provide technical delivery support across a data and reporting migration workstream. Responsibilities include reviewing current Microsoft Fabric and Power BI reporting workloads; analysing existing reporting requirements; identifying source data, transformations and technical/reporting dependencies; reviewing the existing Databricks architecture; and assessing the utilisation of Silver, Gold and Metrics layers.
• The resource will support the migration of priority reporting workloads into Databricks AI/BI and develop or optimise the associated data pipelines and data-layer components where required. The role will also assist in establishing Databricks consumption baselines, reviewing compute and storage utilisation, assessing Genie/token consumption, identifying redundant or duplicated data assets, identifying legacy technology for retirement, and highlighting inefficient pipelines or processing.
• As migration progresses, the engineer will develop and optimise required pipelines, complete Gold and Metrics-layer development, validate migrated reporting outputs, support UAT and remediation, monitor workload efficiency, and support performance and stability improvements.
• During the final stages of the engagement, the resource will assist with final report validation, data accuracy checks, technical issue resolution, documentation, knowledge transfer, legacy retirement activities, and the establishment of any remaining optimisation backlog for the internal data and analytics team.

What they're looking for

• Databricks, Azure, Microsoft Fabric & Power BI, Medallion architecture, Cost and Performance Optimisation.

More IT and developer jobs · More IT and developer jobs in Johannesburg · All jobs at Mediro ICT