Job Information
EPAM Systems, Inc. Senior Data Engineer (Databricks, Scala) in Portugal
DESCRIPTION
Currently, we are looking for Senior Data Engineer with Databricks and Scala expertise to make our team even stronger.
Our client is a publicly traded German online fashion and beauty retailer active across Europe.
RESPONSIBILITIES
Develop, monitor, and operate the most used and most critical curated data pipeline - Sales Order Data (incl. Post-order information, e.g. shipment, return, payment). This pipeline is processing hundreds of millions of records to provide high-quality datasets for analytical and machine learning use-cases
Consulting with analysts, data scientists, and product managers to build and continuously improve "Single Source of Truth" KPI for business steering such as the central Profit Contribution measurement
Leverage and improve a cloud-based tech stack that includes AWS, Databricks, Kubernetes, Spark, Airflow, Python, and Scala
REQUIREMENTS
Expertise in Apache Spark along with Spark streaming & Spark SQL
Experience with Databricks and delta-lake
Fluency in Scala programming language
Good understanding & experience with CI/CD
Rich working experience with Github
Fluency working with AWS landscape
Ability to build Apache Airflow pipelines
Upper-Intermediate level of English, both spoken and written (B2+)
NICE TO HAVE
Presto
Exasol
Starburst
Superset
WE OFFER
Competitive compensation depending on experience and skills
Variety of projects within one company
Being a part of a project following engineering excellence standards
Individual career path and professional growth opportunities
Internal events and communities
Flexible work hours
EPAM Systems, Inc. is an equal opportunity employer. We recognize the value of diversity and inclusion in creating success for our customers, business partners, shareholders, employees and communities. We are committed to recruiting, hiring, developing and promoting employees without discrimination. As a global employer, this commitment includes complying with all laws in the countries in which we operate. Nevertheless, we believe equal employment practices should not be limited to what the law requires. Equal opportunity and inclusion are essential to motivate, empower and recognize the best in everyone.
At EPAM, employment actions are based on individual qualifications, without regard to race, color, religion, creed, gender, pregnancy status, sexual orientation, gender identity, gender expression, marital or familial status, national origin, ancestry, genetics, age, disability status, veteran status, citizenship status when otherwise legally able to work, or any other characteristic protected by law.