- Company Name
- UniSysTech Consulting
- Job Title
- Kafka Developer
- Job Description
-
Job title: Kafka Developer
Role Summary:
Design, develop, and maintain high‑performance Kafka‑based data streaming pipelines, integrating Kafka Streams, Connect, and producer/consumer components with AWS cloud services to support scalable, real‑time event‑driven applications.
Expactations:
- Deliver robust, high‑availability streaming solutions on schedule.
- Collaborate across engineering, architecture, and operations in an Agile environment.
- Troubleshoot complex distributed system issues, ensuring minimal downtime.
- Continuously optimize Kafka broker, topic, and cluster performance.
- Leverage AWS services (Lambda, ECS, ECR, EC2, etc.) to embed streaming pipelines into cloud‑native architectures.
Key Responsibilities:
1. Architect and implement Kafka producers, consumers, and connectors.
2. Configure and tune Kafka broker settings, topics, partitions, replication, and retention policies.
3. Develop real‑time processing jobs using Kafka Streams (KStream).
4. Manage Kafka Connect jobs for data ingestion and outbound integration.
5. Utilize Confluent Platform tools (Schema Registry, Control Center, KSQL) to administer pipelines.
6. Monitor Kafka performance metrics and log data; proactively detect and resolve bottlenecks or anomalies.
7. Implement security, ACLs, and compliance controls for data streams.
8. Integrate Kafka pipelines with AWS services (Lambda, ECS/ECR, EC2) for deployment and scaling.
9. Document architecture, data flows, and operational procedures.
10. Participate in code reviews, unit testing, and continuous integration processes.
Required Skills:
- Deep hands‑on experience with Apache Kafka broker, KStreams, Kafka Connect, and producer/consumer development.
- Proficiency with Confluent Platform components and ecosystem.
- Strong knowledge of AWS cloud services (Lambda, ECS, ECR, EC2, IAM, CloudWatch, VPC).
- Expertise in scaling, tuning, and monitoring distributed streaming systems.
- Solid debugging, troubleshooting, and performance‑analysis capabilities.
- Effective collaboration and communication in cross‑functional teams.
- Familiarity with Agile practices and CI/CD pipelines.
Required Education & Certifications:
- Bachelor’s degree in Computer Science, Software Engineering, Information Technology, or related field.
- Confluent Certified Developer – Kafka (CCDK) or equivalent, and/or AWS Certified Developer – Associate or AWS Certified Big Data – Specialty are strongly preferred.