- Company Name
- Carbonsoft
- Job Title
- Big Data Consultant
- Job Description
-
**Job Title:**
Big Data Consultant
**Role Summary:**
Design, develop, and implement end‑to‑end big data solutions that enable data‑driven decision making. Work with data analytics, ETL processes, and data warehousing to store, process, and analyze large volumes of structured and unstructured data. Collaborate with cross‑functional teams to optimize data pipelines and deliver actionable insights.
**Expectations:**
- Deliver scalable, reliable big data architectures that meet business objectives.
- Maintain high data quality and performance across all pipelines.
- Communicate complex findings clearly to technical and non‑technical stakeholders.
- Work collaboratively in an on‑site environment, consistently meeting project milestones.
**Key Responsibilities:**
- Design and architect big data solutions using Hadoop, Spark, Hive, or equivalent frameworks.
- Build, schedule, and monitor ETL workflows and data pipelines.
- Implement and maintain data warehousing solutions, including schema design and data modeling.
- Validate data integrity and quality through automated checks and manual review.
- Analyze data sets, generate reports, and provide insights that support business strategy.
- Troubleshoot performance bottlenecks and data-related issues.
- Document architecture, processes, and best practices for internal use.
- Collaborate with data scientists, analysts, and business stakeholders to define requirements.
**Required Skills:**
- Proficiency in big data technologies (Hadoop ecosystem, Spark, Hive, etc.).
- Strong experience with ETL principles and tools (Talend, Informatica, Airflow, etc.).
- Data warehousing knowledge (Snowflake, Redshift, BigQuery, etc.) and data modeling.
- SQL and scripting skills (Python, Scala, Bash).
- Familiarity with cloud platforms (AWS, GCP, Azure) and their data services.
- Solid analytical and problem‑solving abilities.
- Excellent communication and teamwork skills.
**Required Education & Certifications:**
- Bachelor’s or Master’s degree in Computer Science, Data Science, or related technical field.
- Certifications in big data or cloud data engineering (e.g., AWS Certified Big Data – Specialty, Google Professional Data Engineer, Microsoft Certified: Azure Data Engineer Associate) are desirable.