Job Information
PJM Interconnection LLC Sr. (Lead) Data Engineer I (II) - REQ-2024-3821 in Audubon, Pennsylvania
Sr. (Lead) Data Engineer I (II) - REQ-2024-3821
DESCRIPTION/RESPONSIBILITIES: Flexible Work Arrangement - Hybrid
The Sr. (Lead) Data Engineer is responsible for designing and implementing methods to improve data: reliability, security and quality across the enterprise. They combine raw information from different sources to create consistent and machine-readable formats. They also develop, test and build algorithms, architectures and infrastructure that enable data extraction and transformation for various modeling needs for example predictive or prescriptive modeling. Other functions can include organizing large datasets.
Essential Functions: * Analyze and organize raw data, cataloging fields and entities * Building data architectures, and data pipelines to collectively bring together data * Analyzing, planning, and defining data architecture framework(s), including security, reference data, metadata, and master data * Participation in analyzing business needs and objectives with analysts, data Architects and engineers, developers, report writers and report consumers * Inform and support the organizations data strategy * Conduct complex data analysis and collaboration with analysts, developers and * Prepare data for prescriptive and predictive modeling * Build algorithms and prototypes to support data integration and other data related functions * Identify opportunities for data acquisition and determine data update velocity * Collaborating with other teams within the organization to devise and implement data strategies, build models, and assess customers needs and goals * Work with Data Architects in developing and supporting patterns that data engineers, developers and others can follow * Support and inform data governance practices * Creating and implementing data management processes and procedure
Characteristics & Qualifications: Required: * BS degree in Computer Engineering, Information Systems, Statistics, Informatics or at least 5 years of experience Data Engineer or Architect role * At least 5 years of experience Strong knowledge of SQL * Experience with data pipeline and workflow management tools * Experience in Azure data engineering and cloud data processing services * Knowledge of Programming e.g. SAS, Python, PySpark, R, Oracle PL/ SQL * Experience in one or more of the following: C, C++, Java, Python, Go, Perl or Ruby. * Experience with algorithms, data structures, complexity analysis and software design. * Experience with managing database infrastructure (particular emphasis on Oracle and Sql Server) * Experience with configuration management tools like Puppet, Chef * Experience with Unix, AIX, Debian, Windows operating systems, containers, container orchestrators * Experience with git based code repositories. * Experience using and administering JEE application servers including JBoss, WebSphere, or WebLogic
Preferred: * MS degree in Computer Science, Information Systems, Statistics or Informatics * At least 5 years of work experience * Experience with SAS development tools and datasets (e.g. SAS Grid, SAS Display manager, SAS Studio ) * Hands on experience in Data Warehousing and ETL tools * Experience in Big Data technologies (e.g. Spark, Databricks,) Experience building and optimizing 'big data' data pipelines, architectures, and data sets * Performance/security-oriented data design, including partitions, different types of indexes, views, and most effective data management practices. * Experience with Azure Synapse Analytics, Azure DevOps * Experience with Purview, PowerBI, Alteryx, Qlik, or Tableau Experience with NoSQL * Experience with PJM operations, markets, and planning functions
Equal Opportunity Employer-minorities/females/veterans/individuals with disabilities/sexual orientation/gender identity