Principal Software Engineer, AI Quality and Testing

Location: Lisbon, Portugal / Krakow, Poland

Working Pattern: Fully Remote (Portugal); Hybrid (Poland)

Join our AI Delivery & Engineering team and provide principal-level technical leadership for AI quality, testing and evaluation. You'll help ensure reusable AI assets and enterprise solutions are assessed through credible, risk-based and reproducible evidence.

Impact You'll Own

• Define quality and evaluation standards for AI assets and end-to-end solutions.

• Strengthen supplier quality through independent reviews of test plans, code, methods and results.

• Guide decisions on test strategy, coverage, repeatability, cost and residual limitations.

• Improve representative evaluations, evidence traceability and use-case-specific assurance.

• Advance regression, automation, drift and adversarial testing standards.

What You'll Do

• Define risk-based standards for software testing and probabilistic AI behavior.

• Review supplier test plans, designs, test artifacts, evaluation methods and results.

• Assess evidence at component, integration and end-to-end business use-case levels.

• Guide partners on evaluation trade-offs, asset reuse and justified exceptions.

• Review dataset fitness, provenance, baselines, groundedness and reproducibility.

• Set standards for CI/CD quality gates, regression, drift and failure scenarios.

• Document readiness recommendations, residual limitations and required remediation.

Skills and Capabilities

Must-haves

• Degree in Computer Science, IT, Software Engineering or a related discipline.

• Enterprise software engineering experience in cloud-native and distributed systems.

• Hands-on enterprise AI evaluation and integrated-solution testing experience.

• Knowledge of deterministic testing and probabilistic AI or agent assessment.

• Experience with risk-based strategy, representative datasets and reproducible evaluations.

• Ability to assess groundedness, factual correctness, task completion and tool actions.

• Experience with CI/CD quality gates, evaluation harnesses and regression evidence.

• Knowledge of dataset provenance, baselines, drift and change-impact assessment.

• Proven technical leadership through standards, independent review and supplier assurance.

• Strong communication, cross-team influence, mentoring and documentation skills.

Nice-to-have

• Advanced study in AI/ML, distributed systems or cloud/platform engineering.

Rewards and Benefits

Permanent local contract with a competitive salary plus Tabaqueira employee benefits including:

• Life and Health insurance

• Employee Pension Plan

• 26 holiday days

• National and international growth opportunities

• Diverse and international work environment

For Poland: In this position, you will earn no less than 327660 PLN gross per year.

Equal Opportunities Statement

We believe that diverse perspectives and experiences make us stronger. As an equal opportunity employer, we are committed to creating a workplace where every individual feels welcomed, respected, valued, and empowered to reach their full potential. We encourage applications from people of all backgrounds and identities, and provide reasonable accommodations to ensure an accessible recruitment process. If you need support or adjustments during your application, let us know. We're here to help you succeed.