Principal Data Engineer Castelo Branco

Principal Data Engineer
Castelo Branco
Castelo Branco, Castelo Branco, Portugal

We are seeking an experienced Principal Data Engineer to lead a team in developing and maintaining robust, scalable data pipelines, bridging
- premises and cloud environments, and delivering
- time analytics systems. This role requires deep expertise in data engineering and streaming technologies, combined with strong leadership skills to drive the team towards achieving business objectives. The manager will collaborate with
- functional teams including architecture, product, and software engineering to ensure the delivery of
- quality data solutions aligned with company goals. Required Skills:5+ years of
- on experience in data engineering, including expertise in Python, Scala, or Java. Deep understanding of Apache Kafka for stream processing workflows (required)Proficiency in managing and optimizing databases such as Postgre
SQL, My
SQL, MSSQL, and Oracle. Familiarity with analytical databases like Click
House or similar. Familiarity with both cloud solutions (AWS, Azure, Google Cloud) and
- premises environments as part of
- optimization efforts. Knowledge of additional data tools and frameworks such as Redis, Rabbit
MQ, Superset, Cube. js, Minio, and Grafana (optional but beneficial). Strong leadership and mentoring skills, with the ability to guide a team and provide technical direction. Experience ensuring system reliability, scalability, and data integrity through best practices. Familiarity with i
Gaming industry terminology and challenges is highly preferred. Responsibilities:Provide technical leadership, including making key decisions on solution design, architecture, and implementation strategies. Lead and mentor a team of data engineers, serving as the primary point of contact for technical guidance. Design and oversee the implementation of scalable, efficient data pipelines and architectures, with a strong focus on stream processing. Develop and maintain robust data storage and processing solutions, leveraging tools like Apache Kafka, Redis, and Click
House. Guide the migration of selected
- based solutions to
- premises tools, optimizing costs while maintaining performance and reliability. Collaborate with stakeholders to gather requirements, propose designs, and align data strategies with business objectives. Ensure system reliability and scalability, with a focus on high availability and robust data transfer mechanisms (e. g. , "at least once" delivery). Stay
-
- date with emerging technologies and evaluate their potential application to improve the overall data ecosystem.