Data Engineer (Spark Scala DevOps) at Unison Consulting Pte Ltd
Sydney, Australia
Job Descrption
Implement data transformation, aggregation, and enrichment processes to support various data analytics and machine learning initiatives
Collaborate with cross-functional teams to understand data requirements and translate them into effective data engineering solutions
Ensure data quality and integrity throughout the data processing lifecycle
Design and deploy data engineering solutions on OpenShift Container Platform (OCP) using containerization and orchestration techniques
Optimize data engineering workflows for containerized deployment and efficient resource utilization
Collaborate with DevOps teams to streamline deployment processes, implement CI/CD pipelines, and ensure platform stability
Implement data governance practices, data lineage, and metadata management to ensure data accuracy, traceability, and compliance
Monitor and optimize data pipeline performance, troubleshoot issues, and implement necessary enhancements
Implement monitoring and logging mechanisms to ensure the health, availability, and performance of the data infrastructure
Document data engineering processes, workflows, and infrastructure configurations for knowledge sharing and reference
Stay updated with emerging technologies, industry trends, and best practices in data engineering and DevOps
Provide technical leadership, mentorship, and guidance to junior team members to foster a culture of continuous learning and innovation to the continuous improvement of the analytics capabilities within the bank
Requirements
5+ Years of experience as Data Engineer(Spark, Scala and DevOps)
Design and deploy data engineering solutions on OpenShift Container Platform (OCP) using containerization and orchestration techniques
Document data engineering processes, workflows, and infrastructure configurations for knowledge sharing and reference
Provide technical leadership, mentorship, and guidance to junior team members to foster a culture of continuous learning and innovation to the continuous improvement of the analytics capabilities within the bank
Your CV has been submitted successfully.
Complete form below to directly Send your CV / Linkedin Profile to Data Engineer (Spark Scala DevOps) at Unison Consulting Pte Ltd.
@
You will receive all responses from employer on this email
Example: Application for the post of 'Accountant'
Example: Introduce your self and give purpose of your application
*All fields are mandatory.
Loading...
UNISON CONSULTING PTE LTD 0 job found
No jobs found for this company. Try other companies.