Data Engineer – Azure Cloud - AI

Main responsibilities:

• Design and implement our Data and AI central data platform as well as related tools/systems for advanced business analytics and enterprise data governance
• Build and maintain enterprise cloud data warehouse
• Migrate existing in-house systems into cloud.
• Manage data modeling design, writing, and optimizing ETL jobs
• Deliver and manage in-house and cloud-native data solutions to meet business requirements across firm-wide business units
• Assist in creating and monitoring analytics dashboards, for different business functions
• Ensure quality, integrity and accuracy of datasets through tracked, secured and auditable controls
• Work with stakeholders to assist with data-related technical issues and support their data needs.

Requirements:

• Core Python/ PySpark skills, including: PySpark for Databricks, Spark context understanding and pipeline implementation in PySpark Python scripting. Django/Flask framework knowledge., Python-based API, and Azure function writing.,  JSON & Parquet/Delta format schema understanding; Databricks SQL
• Azure Databricks knowledge: workspace, notebook, code repos management (Bitbucket), Azure DevOps experience
• Great coding skills on Python and SQL, with solid understanding of object-oriented analysis and design.
• Working knowledge of common algorithms and data structures, with strong analytical and problem-solving skills
• Basic understanding and experience with ML/AI concepts (e.g., deep learning, deep reinforcement learning, deep Bayesian learning), workflows, and toolsets (Jupyter Notebook, etc.), and libraries (Numpy, Pandas, Scikit Learn, PyTorch, etc.) preferably in both cloud-native and desktop deployments.
• Demonstrated ability to understand, work with and deliver robust solutions in more than one programming language, framework, technology stack, runtime environment, etc.

If you are keen please send in your application to Komal at komal@kstalentsolutions.com