About the role
Our client is seeking a highly skilled Senior Data Engineer to lead their data migration project from on-premises data Database to Azure Cloud. The ideal candidate will have deep expertise in Azure Data Factory and SQL database migrations, coupled with a robust background in data architecture and ETL processes.
Responsibilities
- Lead the migration of data from Oracle Databases and other on-premises SQL databases to Azure Data Factory (ADF).
- Design, implement, and manage ETL processes including batch jobs and Change Data Capture (CDC) in Azure.
- Utilize SFTP for secure data transfer within ODI or ADF environments.
- Performance Tuning: Regularly monitor and optimize the performance of Azure databases and ETL processes to ensure they meet SLA requirements for speed and uptime.
- Architect scalable data solutions on Azure, ensuring performance, reliability, and security.
- Collaborate with cross-functional teams to define data needs, storage, and processing requirements.
- Optimize and troubleshoot data pipelines for efficiency and scalability.
- Document data processes and maintain data models for future reference and maintenance.
- Automation of Data Operations: Develop automation scripts or leverage Azure Automation for routine tasks such as database backups, job scheduling, and health checks to improve operational efficiency.
- Cross-Platform Integration: Ensure seamless integration of Azure data services with on-premises systems or other cloud platforms where applicable, enhancing interoperability.
- Scalability Planning: Anticipate future data growth and plan for scalability, ensuring that the data infrastructure can scale up or out as needed without significant redesign.
About You
- At least 2 years of experience with Azure Data Factory (ADF).
- Proven, recent experience in migrating SQL databases to Azure, specifically:
- Preference for Oracle Databases and Oracle Data Integrator (ODI) to ADF migration, but open to experience with any on-prem SQL (Microsoft SQL, MySQL, etc.) to ADF.
- Strong experience in database ETL, management of batch jobs, and implementation of CDC.
- Experience with SFTP in either ODI or ADF.
- Knowledge of Oracle Service Bus (OSB) and handling SOAP APIs.
- Experience in data architecture, particularly within Azure environments.
- Bachelor’s or Master’s degree in Computer Science, Information Systems, or related field.
- Minimum of 5 years of experience in data engineering or a related discipline, with at least 2 years focused on Azure technologies.
- Excellent problem-solving skills and ability to work under tight deadlines.
- Strong communication skills to articulate complex data concepts to non-technical stakeholders.
Desirable Skills
- Experience with real-time data streaming tools like Striim or Qlik.
- Familiarity with Azure Cloud integration technologies such as Azure Logic Apps, Function Apps, and Azure Service Bus.
- Experience with Azure DevOps for CI/CD pipelines related to data engineering tasks.
- Knowledge of GraphQL and Hasura for API management.
The Benefits
- Gain hands-on experience with emerging technology
- Initial 6-month contract with possible extensions
- Utilise your Azure expertise in the education sector
Please apply by upload your current resume in Microsoft Word format only (.doc or .docx). If you would like to have a confidential discussion, please contact Angeline Richards on +61 0404629321, quoting reference JN -012025-39264. Want to know more about Davidson? Visit us at www.davidsonwp.com