Job Description
Responsibilities: - Work within a delivery team to ensure Product Stories acceptance criteria is met by writing automated tests with Pytest
- Develop test scenarios to analyze and compare data sources and targets in data pipelines
- Test data pipeline workflows end to end: data loads, data structures, data transformations, and overall data quality
- Ensure that data is delivered with the expected format, structure, and data transformations
- Work with QE team to enhance and streamline code. Share knowledge and experience with other QE members
- Continually strive to improve DDLC, code, CI/CD, and engineering best practices
Qualifications: - BS in engineering, mathematics, computer science, software development or other technical field, or relevant experience
- Experience with ETL and data pipeline testing
- Strong SQL skills
- Experience with PostgreSQL, Parquet, Delta Lake
- Ability to work with members of a team in different disciplines.
- Ability to proactively ask questions, and seek answers
- Ability to work with remote staff
- Willingness to be flexible in technologies and flexible in team assignments
- Excellent coding skills, preferably in Python.
- Experience working in the Agile Development Framework
Nice to Have: - Spark SQL, PySpark
- Databricks, AWS Glue, Ascend
- Great Expectations (GX)
- Familiarity with Git, Jenkins, Docker, AWS S3, IntelliJ, PyCharm, Jira
Job Tags
Remote job, Flexible hours,