 Design, build, and maintain cloud-based data pipelines and workflows that support analytics and operational systems.

 Integrate data from various sources using APIs and cloud services.

 Develop clean, efficient, and test-driven code in Python for data ingestion and processing.

 Optimize data storage and retrieval using big data formats like Apache Parquet and ORC.

Requirements

 At least 3 years in a data engineering role working on data integration, processing, and transformation use cases with open-source languages (i.e. Python) and cloud technologies.

 Strong programming skills in Python specifically for API integration and data libraries, with emphasis on quality and test-driven development.

 Demonstrated proficiency with big data storage formats (Apache Parquet, ORC) and practical knowledge of pitfalls and optimization strategies.

 Demonstrated proficiency with SQL

 Experience with data modeling:

o Relational modeling

o Dimensional modeling

o NoSQL modeling

 Working knowledge of IaC on AWS (CloudFormation or CDK)

 Working knowledge of AWS Services:

o Required: Glue, IAM, Lambda, DynamoDb, Step Functions, S3, CloudFormation or CDK

o Nice-to-have: Athena, Kinesis, MSK, MWAA, SQS

 Experience with orchestration of data flows/pipelines: Apache Airflow or Dagster

 Nice-to-have:

o Experience with data streaming (Kinesis, Kafka)

o Experience with Apache Spark

o Client-facing experience, multi-cultural team experience, technical leadership, team leadership