Required Qualifications & Skills – Senior Data Engineer
We are seeking an experienced Senior Data Engineer to design, build, and maintain scalable data pipelines and distributed data processing solutions.
Mandatory / Primary Skills:
Python
PySpark
AWS
SQL
Experience
- 4 to 7 years of hands-on experience in Data Engineering and building scalable data solutions.
- Strong experience working with AWS-based data engineering services and cloud data platforms.
Technical Skills
- Strong hands-on expertise in Python and PySpark for large-scale data processing.
- Advanced proficiency in SQL, including writing complex queries, query optimization, and data analysis.
- Strong experience with AWS services relevant to data engineering and cloud-based data processing.
- Strong understanding of Apache Spark and distributed data processing systems.
- Experience designing, developing, and maintaining scalable and high-performance ETL/ELT data pipelines.
- Experience in data ingestion, transformation, integration, and processing from multiple data sources.
- Experience working with structured, semi-structured, and unstructured data such as JSON, logs, PDFs, and other file formats.
- Strong knowledge of relational and non-relational databases.
- Experience in data quality checks, validation, performance tuning, and optimization.
Analytical & Problem-Solving Skills
- Strong ability to perform deep-dive data analysis and troubleshoot complex data-related issues.
- Proven experience in building scalable data solutions to address complex business requirements.
- Strong problem-solving and analytical skills.
Soft Skills
- Strong verbal and written communication skills.
- Good organisational and stakeholder-management skills.
- Proactive self-starter with the ability to work independently and take ownership.
Technical Agility
- Ability to adapt quickly to new technologies and data platforms.
- Passion for staying current with emerging trends and best practices in Data Engineering, AWS, and Big Data technologies.