Experience: 5 years
Employment Type: Full-time
Job Summary:
We are looking for a Data Engineer with 5 years of experience to design, develop, and optimize data pipelines and infrastructure. The ideal candidate will have strong expertise in Big Data technologies, ETL processes, and cloud platforms. You will work closely with data scientists, analysts, and business teams to build scalable and high-performance data solutions.
Key Responsibilities:
Data Pipeline Development:
Design, implement, and maintain scalable ETL and ELT pipelines for efficient data ingestion, transformation, and loading.
Data Architecture & Modeling:
Develop data models and storage solutions that align with business requirements.
Optimize database performance and data retrieval processes.
Big Data Processing:
Work with distributed computing frameworks such as Apache Spark, Hadoop, or Kafka to process large datasets efficiently.
Cloud Data Engineering:
Deploy and manage data solutions on AWS, GCP, or Azure (e.g., Redshift, BigQuery, Snowflake).
Data Quality & Governance:
Implement data validation, quality checks, and monitoring to ensure data integrity and compliance with industry standards.
Collaboration & Optimization:
Work closely with cross-functional teams to understand data needs and provide optimized solutions.
Continuously improve and optimize existing data infrastructure for performance and cost-effectiveness.
Required Skills & Qualifications
Education
Bachelor s or Master s degree in Computer Science, Data Engineering, Information Technology, or a related field.
Technical Skills
Programming Languages: Proficiency in Python, SQL, and Scala/Java.
Big Data Technologies: Experience with Spark, Hadoop, Hive, or Kafka.
Cloud Platforms: Hands-on experience with AWS (Glue, Redshift, S3, Lambda), GCP (BigQuery, Dataflow), or Azure (Data Factory, Synapse Analytics).
ETL & Data Warehousing: Strong knowledge of ETL processes, data warehousing concepts, and pipeline automation.
Databases: Expertise in PostgreSQL, MySQL, NoSQL (MongoDB, Cassandra), and Data Lakes.
DevOps & CI/CD: Familiarity with Docker, Kubernetes, Terraform, and CI/CD pipelines for data deployments.
Level of Expertise
ETL(Extract, Transform, Load) - 4 years
SQL - 4 years
Database development - 4 years
Python - 3 years
Java (All Versions) - 2 years
Power BI - 3 years
Tableau - 2 years
AWS - 2 years
Snowflake - 2 years
Data Warehousing - 3 years
NoSQL - 1 year
