Лига Цифровой Экономики 7 дней назад
Data Engineer
- Москва
- Удалённо
- Полная занятость
PythonSQLKafkaETLData EngineeringDWHApache SparkHadoopS3Apache AirflowApache IcebergParquetORC
Описание
Responsibilities
- Implement data replication
- Develop ETL processes for integration tasks and test them
- Decompose tasks, estimate duration and complexity of each stage
- Analyze and formalize customer requirements
- Build a solution prototype
- Analyze data quality
- Write project documentation (FSD, BRD, regulation, Admin Guide, User Guide)
- Test functionality and hand over to the customer
- Perform ad-hoc data extractions based on customer requirements
Requirements
- Confident SQL skills
- Experience with Python
- Experience with Apache Spark, Hadoop/S3/S3-like, Airflow
- Skills in decomposing complex tasks and managing priorities
- Ability to create clear and structured technical documentation
Nice to have
- Experience with Parquet/ORC, Apache Iceberg
- Experience with Kafka
- Knowledge of Data Vault architecture
We offer
- Internal seminars, internal meetups, we love learning new things
- Voluntary medical insurance with dentistry for employees and a discount on purchasing voluntary medical insurance for immediate family members
- Equipment for comfortable work
- Professional development sessions for personnel, resulting in an individual development plan for each employee
Hiring process
- Apply