Kitrum company
Required skills
— 5+ years of experience in data engineering, with a focus on big data technologies.
— Advanced skills in Scala and/or Python.
— Proficiency in Apache Spark/PySpark/Spark SQL DSL
— Strong experience with AWS (EC2, IAM, S3, Glue, EMR)
— Extensive knowledge of CI/CD processes and tools (Jenkins, GitHub Actions).
— Proficiency in data processing pipelines/database design and architecture (Lambda, Kappa, Datalake/Lakehouse architecture)
— Familiarity with Databricks (Jobs, SQL Warehouse, OverWatch, Unity Catalog).
— Strong understanding of data streaming and real-time data processing.
— Experience with message brokers (Kafka) and Change Data Capture (CDC) techniques.
— Proficiency in SQL and NoSQL databases.
— Level of English: Upper-Intermediate.
As a plus
— Experience with Apache Airflow, including writing plugins and custom operators.
— Experience with Terraform
— Strong grasp of AWS data platform services and their strengths/weaknesses
— Strong experience using Jira, Slack, JetBrains IDEs, Git, GitLab, GitHub, Docker, Jenkins
Responsibilities
— Lead the design, development, and maintenance of scalable and efficient data pipelines.
Conduct code reviews and create design documents for new features.
— Provide technical support and mentorship to other teams within the organization.
— Participate in management meetings to discuss priorities, scopes, deadlines, and cross-team dependencies.
— Develop, optimize, and tune Apache Spark jobs (Scala, PySpark, Spark SQL).
— Implement CI/CD processes using Jenkins and GitHub Actions.
— Design and implement data processing pipelines using AWS and GCP services.
— Work with Apache Airflow to manage and automate workflows, including custom plugin development.
— Utilize Databricks for job scheduling, SQL warehousing, and data visualization.
We offer
— High compensation according to your technical skills;
— Interesting projects with great Customers;
— 5-day working week, 8-hour working day, flexible schedule;
— Democratic management style & friendly environment;
— Full remote;
— Annual Paid vacation — 30 b/days + unpaid vacation;
— Paid sick leaves — 6 b/days per year;
— Corporate Perks (external training, English courses, corporate events/team buildings);
— Professional and personal growth.
