ETL Developer

Job Description :
● Support the operation of the Enterprise Data Warehouse and Data Lake.
● Develop and maintain automated ETL pipelines using scripting languages such as Python, Spark, SQL and AWS services such as S3,Redshift, Lambda.
● Develop and optimise data warehouse and data lake tables using best practice for DDL, physical and logical tables, data partitioning, compression, and parallelization.
● Develop and maintain data warehouse and data lake metadata, data catalogue, and user documentation for internal business customers.
● Support business customers develop, troubleshoot, and optimise complex SQL and ETL solutions to solve reporting, metrics, and analytics problems.
● Work with internal business customers and software development teams to gather and document requirements for data publishing and data consumption via data warehouse, Data Lake, and analytics solutions.
● Develop, test, and deploy code using internal software development toolsets. This includes the code for deploying infrastructure and solutions for secure data storage, ETL pipelines, data catalogue, and data query.
● Perform necessary system maintenance tasks, debugging, troubleshooting, and performance tuning.

Requirements :
● Bachelor’s Degree in Computer Science or any equivalent degree
● 4 + years experienced in software industry preferably in a startup environment
● Expert in SQL query writing for various scenarios and solutions
● Good knowledge on Airflow
● Knowledge on different databases and usage
● Experience in working with databases like AWS Redshift/Postgre SQL
● Experience in creating tables, processing data from various type of source systems to
tables
● Experience in troubleshooting of data issues, performance improvements on SQL
queries
● Analytical and problem-solving ability. Self-motivated