Data Engineer
Job Id: DE001
Eligibility:
4 to 6 years of experience in building data processing applications using Hadoop, Spark and NoSQL DB and Hadoop streaming
4 to 6 years of experience in building data processing applications using Hadoop, Spark and NoSQL DB and Hadoop streaming
As a data engineer at Saras Analytics, you will be responsible for building and maintaining large-scale data pipelines as well as create and data pipelines that deal with large volumes of data.
Primary Responsibilities:
- Experience in Database programming using multiple flavor of SQL and Python
- Understand and translate data, analytic requirements and functional needs into technical requirements
- Build and maintain data pipelines to support large scale data management projects
- Ensure alignment with data strategy and standards of data processing
- Deploy scalable data pipelines for analytical needs
- Experience in Big Data ecosystem – on-prem (Hortonworks/MapR) or Cloud (Dataproc/EMR/HDInsight)
- Experience in Hadoop, Pig, SQL, Hive, Sqoop and SparkSQL
- Experience in any orchestration/workflow tool such as Airflow/Oozie for scheduling pipelines
- Exposure to latest cloud ETL tools such as Glue/ADF/Dataflow
- Understand and execute IN memory distributed computing frameworks like Spark (and/or DataBricks) and its parameter tuning, writing optimized queries in Spark
- Hands-on experience in using Spark Streaming, Kafka and Hbase
- BE/BS/MTech/MS in computer science or equivalent work experience.
- 4 to 6 years of experience in building data processing applications using Hadoop, Spark and NoSQL DB and Hadoop streaming
Additional Responsibilities:
- Exposure to latest cloud ETL tools such as Glue/ADF/Dataflow is a plus
- Expertise in data structures, distributed computing, manipulating and analyzing complex high-volume data from variety of internal and external sources
- Experience in building structured and unstructured data pipelines
- Proficient in programming language such as Python/Scala
- Good understanding of data analysis techniques
- Solid hands-on working knowledge of SQL and scripting
- Good understanding of in relational/dimensional modelling and ETL concepts
- Understanding of any reporting tools such as Looker, Tableau, Qlikview or PowerBI
Educational qualifications preferred
- Category: Bachelor’s Degree, Master’s Degree
- Field specialization: Computer Science
- Degree: Bachelor of Engineering – BE, Bachelor of Science – BS, Master of Engineering – MEng, Master of Science – MS