AWS Data Engineer
Module lead(Depends on the project) Qualification:
Bachelors in science , engineering or equivalent
Responsibility:
Requirement Understanding and Analysis:
- Review the raw requirements to understand the key outcomes expected by the client.
- Identify data source and conduct data profiling to understand data and its quality.
- Create a logic that needs to be incorporated in the architecture to ensure the data meets the requirements.
- Create POC to conduct the feasibility of the requirement, as required.
Data Discovery & Profiling:
- Post signoff, design basic queries to analyze the existing data.
- Share sample data with the client to give a snapshot of the expected outcome.
- Create a low level design document capturing SQL queries to achieve the target stage.
- Share the document with the ETL team.
- Conduct system integration testing during data migration from the source to the target state to ensure the requirements are met.
- Conduct the root cause analysis of the issue to identify the solutions to resolve it.
- Conduct testing to ensure the issue has been resolved.
- In case of data issues, report it to the source team to fix the data.
- If data is corrupted during migration, identify ways to cleanup the data.
- Generate reports to fix the issues as required.
- Understand the impact on data based on the release schedule.
- Based on the understanding of the legacy system, provide inputs to the data modeler to map the attributes to the fields.
- Share the data mapping documents with the data modeling team.
- Support data modeling team to understand the data / data type in the legacy system.
- Support data modeling team in defining table name and table data type as per the standards and best practices defined within the project.
- In case of any new table in Data Warehouse, identify the source values to be uploaded in the table.
- Provide inputs to data modeling regarding the source data for the same.
Knowledge Management:
- Create and update knowledge articles (industry Point of View, Case studies, lessons learnt document etc) in the knowledge management repository.
- Maintain knowledge management database.
- Conduct training for team members on data profiling tools and techniques.
Sales Support:
- Based on the inputs shared by the stakeholders.
- share the inputs on RFPs on the data profiling and system analysis aspect.
- Evaluate/ identify the tools best fit for the customer requirement.
Must Have Skills
