Senior Data Engineer
Viafoura
This job is no longer accepting applications
See open jobs at Viafoura.See open jobs similar to "Senior Data Engineer" Work In Tech.About Viafoura
About the Role
Key Responsibilities
- Design and implement scalable data pipelines using modern tools like Apache Spark, Kafka, or Airflow
- Build and maintain data warehouses and data lakes using cloud platforms (AWS, Snowflake)
- Optimize data storage solutions and implement effective data partitioning strategies
- Ensure data architecture follows best practices for security, reliability, and performance
- Develop ETL/ELT processes to ingest data from various sources including APIs, databases, and streaming platforms
- Monitor and troubleshoot data pipeline performance, implementing automated alerting systems
- Collaborate with software engineers to integrate data systems with production applications
- Maintain data quality through validation frameworks and testing protocols
- Provide technical guidance on complex projects
- Evaluate and recommend new technologies and tools for the data stack
- Lead architecture discussions and contribute to technical decision-making processes
- Document data systems and establish engineering best practices
Required Qualifications
- Bachelor's degree in Computer Science, Engineering, or related technical field
- 5+ years of experience in data engineering or related roles
- Strong programming skills in Python, Scala, or Java
- Extensive experience with SQL and database technologies (PostgreSQL, MySQL, etc.)
- Proficiency with cloud platforms and services (AWS, Snowflake)
- Experience with distributed computing frameworks (Spark) and workflow orchestration tools (Airflow, Dagster)
- Knowledge of data modeling concepts and dimensional modeling techniques
Preferred Qualifications
- Experience with streaming technologies (Kafka, Kinesis, Pub/Sub)
- Familiarity with containerization and orchestration (Docker, Kubernetes)
- Background in machine learning operations (MLOps) and feature engineering
- Experience with data governance and compliance frameworks
- Knowledge of infrastructure as code (Terraform, CloudFormation)
- Experience with Apache Iceberg
This job is no longer accepting applications
See open jobs at Viafoura.See open jobs similar to "Senior Data Engineer" Work In Tech.