Minimum Qualifications:
- Experience: Minimum 4 years
- University or advanced degree in engineering, computer science, mathematics, or a related field
- Strong hands-on experience in Databricks using PySpark and Spark SQL (Unity Catalog, workflows, Optimization techniques)
- Experience with at least one cloud provider solution (GCP preferred)
- Strong experience working with relational SQL databases.
- Strong experience with object-oriented/object function scripting language: Python.
- Working knowledge in any transformation tools, DBT preferred.
- Ability to work with Linux platform.
- Strong knowledge of data pipeline and workflow management tools (Airflow)
- Working knowledge of Git hub /Git Toolkit
- Expertise in standard software engineering methodology, e.g. unit testing, code reviews, design documentation
- Experience creating Data pipelines that prepare data for ingestion & consumption appropriately.
- Experience in maintaining and optimizing databases/filesystems for production usage in reporting, analytics.
- Working in a collaborative environment and interacting effectively with technical and non-technical team members equally well. Good verbal and written communication skills.
- Preferred experience working with big data tools: Big Data tech stack (Hadoop, Spark, Kafka etc.)