Data & AI Engineer Porto

Data & AI Engineer
Porto
Porto, Porto District, Portugal

Key Responsibilities

1. Develop Scala/Spark programs, scripts, and macros for data extraction, transformation, and analysis.
2. Design and implement solutions to meet business requirements.
3. Support and maintain existing Hadoop applications and related technologies.
4. Develop and maintain metadata, user access, and security controls.
5. Develop and maintain technical documentation, including data models, process flows, and system diagrams.

Requirements

1. Minimum 3-5 years of experience from Scala/Spark related projects and/or engagements.
2. Create Scala/Spark jobs for data transformation and aggregation as per the complex business requirements.
3. Should be able to work in a challenging and agile environment with quick turnaround times and strict deadlines.
4. Perform unit tests of the Scala code.
5. Raise PR, trigger build, and release JAR versions for deployment via Jenkins pipeline.
6. Should be familiar with CI/CD concepts and processes.
7. Peer review the code.
8. Perform RCA of the bugs raised.
9. Should have an excellent understanding of the Hadoop ecosystem.
10. Should be well versed with below technologies.

Preferred

1. Relevant certifications (e. g. , Scala, Spark, Hadoop, Performance).
2. Knowledge of other programming languages (e. g. , Python, R).
3. Insight into
   - based solutions such as Snowflake.
4. Experience in Financial Services, preferably in the Credit risk domain.