Data Platforms and DataOps Engineer | Washington, DC
Job Title: Data Platforms and DataOps Engineer
Location: Washington, DC
Date Posted: 6/10/2024
Responsibilities:
Lead and provide technical support for the existing ETL infrastructure and ensure it is running at optimal performance.
Collaborate with cross-functional teams to understand data integration needs and provide infrastructure solutions.
Manage security configurations and protocols to safeguard ETL infrastructure.
Work closely with the security team to address any security vulnerabilities, and compliance validation/remediation.
Install, configure, upgrade, and apply EBF’s in Informatica Suite – PC, DQ, IDMC,MDM, Metadata Manager, Web Services, Proactive Monitoring, DPM and Power Exchange.
Implement, manage, configure, and customize Informatica EDC to ensure organization’s effective data cataloging and data governance requirements.
Support application teams and business stakeholders to maintain ELT related data pipelines, data integration workflows using Azure Synapse, Azure Data Factory, Azure Databricks, and other Azure services. Automated deployments of Informatica products using Azure DevOps CICD pipelines.
Support application teams with additional data platforms such as Databricks, Data Factory and Microsoft Fabric products using Azure DevOps CICD pipelines.
Integrate MDM components such as IDD, ActiveVOS, Elastic Search, ESB, and Active MQ for Business processes.
Manage projects and operations using Azure boards and ServiceNow.
Lead ETL infrastructure service Business Continuity exercises.
Adopt Site Reliability Engineer (SRE) practices into ETL operations.
Adopt Azure board to manage projects and to track work programs deliverables.
Develop and foster a collaborative working relationship with the Lines of Business Data Integration teams, consultants, and project managers to develop processes, documentation, and solutions related to deployments.
Stays abreast with current trends in ETL stack on cloud along with flexibility & adaptability to extend beyond Informatica stack into other data platforms as needed.
Analyze current application process and recommend improvements.
Maintain Informatica processes running on Linux OnPrem and Cloud Servers, take necessary proactive measures for better ETL infrastructure performance, including server configuration, resources usage optimization and tuning.
Provide input on architectural design considering changes in technology within the context of ITS’ current implementation and future roadmap.
Advise on new ETL features, Technologies, and components implementation.
Stays abreast of new technology trends in project management area, and the broader IT industry.
Provide regular status updates to Supervisor and Manger ensuring timely completion of work programs execution and responsibilities.
Guide team members and oversee operational tasks.
Qualifications:
Master's degree with 5 years relevant experience or Bachelor's degree with at least 7 years' relevant experience.
Experience in the Informatica ETL platforms - PC, MDM, IDQ and IDMC, DPM is a large enterprise is a must.
Experience using ADO CICD pipeline and cloud services is a must.
Experience using integration platforms with ERP systems and cloud services is a must.
Experience with integration of MDM components such as IDD, ActiveVOS, Elastic Search, ESB, and Active MQ, is a must.
Familiarity with performance analysis using performance monitoring and tuning tools is a must.
Experience in data integration workflows using Azure Synapse, Azure Data Factory, Azure Databricks, and other Azure services, is a plus.
Knowledge and/or experience of SAP BODS software is a plus
Understanding of cloud native deployment concepts such as DataOps, and cost optimization techniques such as FinOps is a plus
ETL code migration, workflow automation using Rundeck, is a plus.
Strong development experience with Java, Python, JavaScript.
Strong troubleshooting, analytical, and problem-solving skills.
Experience in Agile Methodology, Proficient knowledge of current industry practices, standards, and procedures.
Excellent problem solving skills including troubleshooting and reviewing deployment pipelines, scripts, configuration etc.
Team player who works well with technical and non-technical resources, results-oriented track record and the maturity to escalate issues, as needed.
Excellent interpersonal skills, including an ability to work effectively in a team/task force as a participant or team leader.
Excellent oral and written communications skills and experience interacting with business, application, and infrastructure teams.
About Scalar Solutions:
Scalar Solutions is an IT Staffing and Digital Consulting firm that specializes in designing and implementing solutions related to business intelligence, data science, and cloud. We are located in Pinebrook, New Jersey, with offices in New York City, North Carolina, and Gurgaon, India. Our core team consists of Subject Matter Experts (SMEs) and Senior Technical Consultants who have decades of hands-on experience handling major high-visibility enterprise-level IT projects involving technology mergers, integrating platforms, revamping existing infrastructure and systems, migrating legacy systems and processes, customizing solutions to meet specific business challenges and goals.