DATA ENGINEER - ETL / AWS Lisboa

DATA ENGINEER - ETL / AWS
Lisboa
Lisboa, Portugal

Social network you want to login/join with:

Location: Lisbon / Portugal

Languages: Portuguese (C1/C2) and English (B2)

Working model: Hybrid

Instructions: Please send your CVs in English and make sure to include all skills and experience that match the requirements of the opportunity. This will significantly increase your chances of success.

We are looking for a skilled Data Engineer to join our team. The ideal candidate will be responsible for designing and implementing scalable data solutions while ensuring efficiency and reliability.

Responsibilities:

1. Design and implement scalable data pipelines using AWS services such as Glue, Lambda, Step Functions, and Kinesis.
2. Develop and optimize ETL workflows to process large datasets efficiently.
3. Build and maintain data lakes and data warehouses using S3, Redshift, Athena, and Lake Formation.
4. Ensure data integrity, governance, and security through proper IAM policies, encryption, and compliance frameworks.
5. Work with structured and unstructured data to enable analytics, AI/ML, and business intelligence use cases.
6. Optimize SQL and No
   SQL databases (Redshift, Dynamo
   DB, RDS, Open
   Search) for performance and cost efficiency.
7. Automate infrastructure deployment using Infrastructure as Code (Ia
   C) tools like Terraform or AWS Cloud Formation.
8. Implement
   - time and batch data processing using AWS Glue.
9. Collaborate with Data Scientists, AI/ML Engineers, and Dev
   Ops teams to support
   - driven applications.
10. Monitor and troubleshoot data pipelines with Cloud
    Watch, Datadog, or ELK Stack.

Requirements:

1. Strong experience as a Data Engineer with expertise in AWS cloud environments.
2. Proficiency in Python, SQL, and Spark for data processing and transformation.
3. Hands-on experience with AWS Glue, Redshift, Athena, EMR, S3, and Lambda.
4. Experience with ETL orchestration tools (Step Functions, Airflow, Prefect, Dagster).
5. Familiarity with containerization (Docker, Kubernetes, ECS) and CI/CD pipelines.
6. Understanding of data security, IAM policies, and encryption best practices.

Nice to Have:

1. AWS certifications such as AWS Certified Data Analytics – Specialty or AWS Certified Solutions Architect.
2. Experience with Machine Learning and AI/ML data pipelines in AWS.
3. Knowledge of serverless data engineering with Lambda and API Gateway.
4. Hands-on experience with No
   SQL databases (Dynamo
   DB, Mongo
   DB, Open
   Search).