Data Engineer

  • Москва
  • Удалённо
  • Полная занятость
PythonSQLKafkaETLData EngineeringDWHApache SparkHadoopS3Apache AirflowApache IcebergParquetORC

Описание

Responsibilities

  • Implement data replication
  • Develop ETL processes for integration tasks and test them
  • Decompose tasks, estimate duration and complexity of each stage
  • Analyze and formalize customer requirements
  • Build a solution prototype
  • Analyze data quality
  • Write project documentation (FSD, BRD, regulation, Admin Guide, User Guide)
  • Test functionality and hand over to the customer
  • Perform ad-hoc data extractions based on customer requirements

Requirements

  • Confident SQL skills
  • Experience with Python
  • Experience with Apache Spark, Hadoop/S3/S3-like, Airflow
  • Skills in decomposing complex tasks and managing priorities
  • Ability to create clear and structured technical documentation

Nice to have

  • Experience with Parquet/ORC, Apache Iceberg
  • Experience with Kafka
  • Knowledge of Data Vault architecture

We offer

  • Internal seminars, internal meetups, we love learning new things
  • Voluntary medical insurance with dentistry for employees and a discount on purchasing voluntary medical insurance for immediate family members
  • Equipment for comfortable work
  • Professional development sessions for personnel, resulting in an individual development plan for each employee

Hiring process

  • Apply