Senior Spark Developer- (Hybrid)

We are seeking a highly skilled and experienced Senior Spark Developer to join our team. This role demands a deep understanding of Apache Spark and its ecosystem, along with a strong foundation in Java/J2EE application development. You will be responsible for designing, developing, and optimizing Spark-based solutions to process large volumes of data, collaborating with various teams to deliver high-performance and scalable data applications. This is a challenging and rewarding opportunity to contribute to a cutting-edge data engineering team.

Responsibilities:

• Design, develop, and implement complex Spark applications for processing large datasets, leveraging your expertise in distributed computing and data manipulation techniques.
• Partner with management teams to integrate Spark-based solutions into existing data infrastructure, ensuring seamless data flow and optimal performance.
• Resolve critical data processing challenges by evaluating complex business processes, system processes, and industry standards, applying your deep understanding of Spark's capabilities.
• Provide technical leadership and mentorship to other engineers, guiding them on Spark best practices and promoting efficient development strategies.
• Develop and maintain coding standards, testing procedures, and debugging techniques specifically for Spark applications.
• Contribute to the overall data architecture, ensuring that Spark solutions align with business goals and integrate effectively with other data technologies.
• Appropriately assess risk when making technical decisions, considering factors such as data security, performance, and scalability.
• Drive compliance with applicable laws, rules, and regulations, adhering to company policies and demonstrating sound ethical judgment.

Qualifications:

• 6+ years of application development experience, with a significant focus on Java/J2EE and Apache Spark.
• Strong command of Object-Oriented Programming (OOP) principles and design patterns in Java.
• Excellent understanding of data structures and algorithms, and their application in Spark development.
• Proven experience developing complex Java/J2EE applications, encompassing all areas of the technology stack.
• Thorough knowledge and hands-on experience with Hadoop, MapReduce, YARN, Sqoop, Pig, Hue, Unix, Impala, and Cassandra on Mesos.
• Extensive experience designing, developing, and deploying Spark applications in a production environment, handling large volumes of data.
• Deep understanding of Spark architecture, including Spark SQL, Spark Streaming, RDDs, and DataFrames.
• Experience with ETL and BI tools is a plus.
• Experience working in an agile environment using Scrum methodologies.
• Expertise in designing and optimizing software solutions for performance and stability, particularly in the context of Spark.
• Expertise in troubleshooting and problem-solving complex technical issues.
• Experience with test-driven development (TDD).
• Cloudera certification (CCDH) is an added advantage.

Preferred Qualifications:

• Experience with cloud-based Spark platforms (e.g., Databricks, AWS EMR, Azure HDInsight).
• Experience with other big data technologies (e.g., Kafka, Hive, HBase).
• Experience with data visualization tools and techniques.

Education:

• Bachelor's degree in Computer Science, Engineering, or a related field.

This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.

------------------------------------------------------

Job Family Group:

------------------------------------------------------

Job Family:

------------------------------------------------------

Time Type:

------------------------------------------------------

Most Relevant Skills

------------------------------------------------------

Other Relevant Skills

------------------------------------------------------

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.

If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.

View Citi’s EEO Policy Statement and the Know Your Rights poster.