Senior Big Data Engineer

Please note - This is a 100% remote opportunity and you can work from any location.

About the team:

You will be a part of Cactus Labs which is the R&D Cell of Cactus Communications. Cactus Labs is a high impact cell that works to solve complex technical and business problems that help keep us strategically competitive in the industry. We are a multi-cultural team spread across multiple countries. We work in the domain of AI/ML especially with Text (NLP - Natural Language Processing), Language Understanding, Explainable AI, Big Data, AR/VR etc.

The opportunity: Within Cactus Labs you will work with the Big Data team. This team manages Terabytes of data coming from different sources. We are re-orchestrating data pipelines to handle this data at scale and improve visibility and robustness. We operate across all the three Cloud Platforms and leverage the best of them.

In this role, you will get to own a component end to end. You will also get to work on could platform and learn to design distributed data processing systems to operate at scale.

Responsibilities:

• Build and maintain robust data processing pipelines at scale
• Collaborate with a team of Big Data Engineers, Big Data and Cloud Architects and Domain SMEs to drive the product ahead
• Follow best practices in building and optimize existing processes
• Stay up to date with the progress in the domain since we work on cutting-edge technologies and are constantly trying new things out
• Build solutions for massive scale. This requires extensive benchmarking to pick the right approach
• Understand the data in and out and make sense of it. You will at times need to draw conclusions and present it to the business users
• Be independent, self-driven and highly motivated. While you will have the best people to learn from and access to various courses or training materials, we expect you to take charge of your growth and learning.

Expectations from you:

• 4-7 Years of relevant experience in Big Data preferable with Java
• Highly proficient in distributed computing and Big Data Ecosystem - Hadoop, HDFS, Apache Spark
• Good understanding of data lake and their importance in a Big Data Ecosystem
• Being able to mentor junior team members and review their code
• Experience in working in a Cloud Environment (AWS, Azure or GCP)
• You like to work without a lot of supervision or micromanagement.
• Above all, you get excited by data. You like to dive deep, mine patterns and draw conclusions. You believe in making data driven decisions and helping the team look for the pattern as well.

Preferred skills:

• Familiarity with search engines like Elasticsearch and Bigdata warehouses systems like AWS Athena, Google Big Query etc
• Building data pipelines using Airflow
• Experience of working in AWS Cloud Environment
• Knowledge of NLP and ML

Job Perks

We seek professionals who see differently; who find opportunity where others don't; and who look within themselves and know that with the right support and team, they can impact the world!

Join us-

• If you like working remotely. We're Remote-first organisation.
• If you are keen on getting a global exposure
• If you like the freedom to innovate & build products
• If you want to be part of the team that works in the domain of AI/ML especially with Text (NLP - Natural Language Processing), Language Understanding, Explainable AI, Big Data, AR/VR etc.
• If you're keen on being part of a culture that values people for their talent, personality, competency, and the ability to learn and grow