Senior Data Engineer

Job Description

• Develop data ingestion, summarization, processing, and cleansing scripts

• Maintenance of data pipeline

• Development of Web APIs

• Identify, mine, and assemble large, complex data sets from different data sources that meet functional/non-functional business requirements

• Maintain and optimize the performance of the data warehouse (schemas, indexes, etc.)

• Design, implement and maintain ETL scripts

• Optimize SQL queries, tables, views, and procedures

• Collaborate with data analysts, data scientists, DevOps, and other teams

• Write documentation

• Coordinate with the IT team to maintain the data systems

Requirements:

• University degree in Computer Science or related discipline

• 3-5 years of experience working in data warehousing, data architecture, and/or data engineering at an enterprise scale

• Must have experience in ETL design, implementation, and maintenance

• Experience writing, maintaining, and optimizing complex SQL queries, tables, views, and procedures

• Knowledge in data warehousing, including mapping and transformations, data dependencies, data source analysis and profiling, OLTP, OLAP, and the different schema.

• Comfortable in using CLI and working in a Linux environment

• Must have experience in Python, Databrick and AWS

• Knowledgeable on web security and performance

• Can handle semi-structured data formats – JSON, XML, HTML, etc.

• Experience in using Git

• Knowledgeable on statistics and machine learning is a plus

• Experience working with AWS services is a plus

• Natural problem solver and critical thinker

• Detail oriented and highly adaptive

• Good communication skills

• Fast learner