Sr. Big Data Engineer

  • Competitive
  • New York, NY, USA New York NY US
  • Permanent, Full time
  • S&P Global
  • 23 Jun 18 2018-06-23

Sr. Big Data Engineer

JobDescription :
Job Description:
S&P Global Ratings is looking for an experienced Big Data Engineer to join Data Engineering team within Chief Data Office, a team of data and technology professionals who define and execute the strategic data roadmap for S&P Global Ratings. The successful candidate will participate in the design and build of S&P Ratings cloud based analytics platform to help develop and deploy advanced analytics/machine learning solutions.

The Team
You will be an expert contributor and part of the Rating Organization's Data Services Team. This team, who has a broad and expert knowledge on Ratings organization's critical data domains, technology stacks and architectural patterns, fosters knowledge sharing and collaboration that results in a unified strategy. All Data Services team members provide leadership, innovation, timely delivery, and the ability to articulate business value. Be a part of a unique opportunity to build and evolve S&P Ratings next gen analytics platform.

Our Hiring Manager Says
If you are an individual that brings demonstrated experience of delivering big data projects as a data engineer, this is an excellent opportunity. I am looking for someone with sound technical knowledge, can be hands-on, worked on transformational initiatives, and can drive results.

Responsibilities:

  • Design and develop efficient and scalable data pipelines between enterprise systems and analytics platform
  • Work closely with Data Science team and participate in development of feature engineering pipelines
  • Provide technical expertise in the areas of design and implementation of Ratings Integrated Data Facility with modern AWS cloud technologies such as S3, Redshift, EMR, Hive, Presto and Spark
  • Build and maintain a data environment for speed, accuracy, consistency and 'up' time
  • Support analytics by building a world-class data lake environment that empowers analysts to determine insights into revenue and power products across the organization
  • Work with the machine learning engineering team to build a data eco system that supports AI products at scale
  • Ensure data governance principles adopted, data quality checks and data lineage implemented in each hop of the data
  • Partner with the chief data office, enterprise architecture organization to ensure best use of standards for the key data domains and use cases
  • Be in tune with emerging trends Big data and cloud technologies and participate in evaluation of new technologies
  • Ensure compliance through the adoption of enterprise standards and promotion of best practice / guiding principles aligned with organization standards

    Experience & Qualifications:
  • BS or MS degree in Computer Science or Information Technology
  • 8+ years of experience as data engineer at an innovative organization
  • 4+ years of hands-on experience in implementing data lake systems using AWS cloud technologies such as S3, Redshift, EMR, Hive, Presto and Spark
  • Expert managing AWS services (EC2, S3, Route 53, ELB, VPC, cloudwatch, Lambda) in a multi account production environment
  • Experience With Machine Learning Libraries and Frameworks (TensorFlow , MLlib) is an added advantage
  • Exposure to R , SparklyR , and Other R packages is a Plus
  • Experience with development frameworks as well as data and integration technologies such as Informatica, Python, Scala
  • Expert knowledge of Agile approaches to software development and able to put key Agile principles into practice to deliver solutions incrementally.
  • Monitors industry trends and directions; develops and presents substantive technical recommendations to senior management
  • Excellent analytical thinking, interpersonal, oral and written communication skills with strong ability to influence both IT and business partners
  • Ability to prioritize and manage work to critical project timelines in a fast-paced environment
  • Financial services industry experience