Software Engineer - Big Data

About project:

The Octoflow project is a tool for mapping and certifying network flows. Octoflow collects information via APIs from data sources (IPAM, DNS, etc.) to perform ingestion and mapping of flows. It will also be responsible for adding business-related features around the data.

JD:

• Develop code and test case scenarios by applying relevant software craftsmanship principles and meet the acceptance criteria.

• Complete the assigned learning path and contribute to daily meetings.

• Deliver on all aspects of Software Development Lifecyle (SDLC) in-line with Agile and IT craftsmanship principles.

• Take part in team ceremonies be it agile practices or chapter meetings.

• Deliver high-quality clean code and design that can be re-used.

• Actively, work with other Development teams to define and implement API's and rules for data access.

• Perform bug-free release validations and produce test and defect reports.

• Contribute to developing scripts, configuring quality and automating framework usage.

• Run and maintain test suites with the guidance of seniors.

• Support existing data models, data dictionary, data pipeline standards, storage of source, process and consumer metadata.

Technical Skills

• Languages: Scala, Shell

• Data Processing: Spark, SQL

• Databases: PostgreSQL, Hive, Starburst (query tools)

• Big Data: Hadoop, Kafka

• CI/CD: Jenkins, Ansible, Control-M

• Methodology: Agile, backlog management, collaborative work

Soft Skills

• Rigor and adherence to group standards (security, quality, compliance)

• Ability to work in an international environment

• Proactive in optimizing development

Hands on experience 2-3 years is required.