Job Description

• Work within a delivery team to ensure Product Stories acceptance criteria is met by writing automated tests with Pytest
• Develop test scenarios to analyze and compare data sources and targets in data pipelines
• Test data pipeline workflows end to end: data loads, data structures, data transformations, and overall data quality
• Ensure that data is delivered with the expected format, structure, and data transformations
• Work with QE team to enhance and streamline code. Share knowledge and experience with other QE members
• Continually strive to improve DDLC, code, CI/CD, and engineering best practices

• BS in engineering, mathematics, computer science, software development or other technical field, or relevant experience
• Experience with ETL and data pipeline testing
• Strong SQL skills
• Experience with PostgreSQL, Parquet, Delta Lake
• Ability to work with members of a team in different disciplines.
• Ability to proactively ask questions, and seek answers
• Ability to work with remote staff
• Willingness to be flexible in technologies and flexible in team assignments
• Excellent coding skills, preferably in Python.
• Experience working in the Agile Development Framework

• Spark SQL, PySpark
• Databricks, AWS Glue, Ascend
• Great Expectations (GX)
• Familiarity with Git, Jenkins, Docker, AWS S3, IntelliJ, PyCharm, Jira

Job Tags

Similar Jobs