About the Role:
We are looking for a Big Data Engineer that will work on the collecting, storing, processing, and
analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for
these purposes, then maintaining, implementing, and monitoring them. You will also be
responsible for integrating them with the architecture used across the company.
Key Responsibilities -
>> Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
>> Implementing ETL process
>> Monitoring performance and advising any necessary infra structure changes
>> Defining data retention policies
Desired Experience -
>> B.Tech in Computer Science or related fields with 3-5 years of professional experience
Technical Skills -
>> Proficient understanding of Java / Scala
>> Proficient understanding of distributed computing principles
>> Proficiency with Map Reduce, Hadoop, Spark.
>> Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
>> Good knowledge of Big Data querying tools, such as Pig, Hive, Cassandra and Impala
>> Experience with Flink, Kafka Streams
>> Experience in developing ETL pipelines using EMR or other schedulers like Oozie, Airflow etc.
>> Experience with NoSQL databases, such as Aerospike, HBase, MongoDB
>> Good understanding of Lambda Architecture, along with its advantages and drawbacks
>> Experience with Big data stores such as Redshift, ElasticSearch, Druid.
>> Worked closely with the data science team
>> Experience with OLTP and OLAP
>> Experience with AWS / GCP