Data Engineer Lisboa

Data Engineer
Lisboa
Lisboa, Lisboa, Portugal

Hi! We are DATA Group and we are searching for the best talent! Our goal is to simplify our clients' lives with innovative IT solutions. We operate at a global scale and we are expanding to Portugal!

If you are passionate and have the desire to make a difference, we want to get to know you! Join us to be part of this incredible adventure!

Who are we looking for? YOU!

Location: Lisbon

Working model:
- site/hybrid

Candidates must be living in Portugal.

Requirements:

Role and Responsibilities

• Uploading data to the Big Data Platform
• Targeted preparation of data for specific use cases and creation of data profiles
• Development of
  -
  - use data pipelines
• Coordination of the service provider's activities (only complex)

Skills

• Proficient in the programming languages Apache Spark, Python, R, Scala, Java, and Kotlin
• Understanding of AWS Cloud Ingestion solutions
• Understanding of efficient data processing and attitude in S3 and databases (on-prem and cloud)
• Knowledge of relevant source systems and their data types and structures
• Knowledge in the implementation of batch and
  - time interfaces
• Extended expertise in the field of software engineering for the series production of productive data products (incl. testing, deployment, code quality, logging, monitoring, operation)
• Experience in the development of distributed systems
• Experience in Docker-based deployment and Resource Manager
• Experience in CI/CD pipelines and tools
• Knowledge and practical experience with Workflow Managers
• Detailed knowledge of agile software development according to Scrum
• Technical knowledge of the automotive industry