Principal Software Engineer - Infrastructure Automation (Terraform)

Job description

At Early Warning, we’ve powered and protected the U.S. financial system for over thirty years with cutting-edge solutions like Zelle®, Paze℠, and so much more. As a trusted name in payments, we partner with thousands of institutions to increase access to financial services and protect transactions for hundreds of millions of consumers and small businesses. Positions located in Scottsdale, San Francisco, Chicago, or New York follow a hybrid work model to allow for a more collaborative working environment. Candidates responding to this posting must independently possess the eligibility to work in the United States, for any employer, at the date of hire. This position is ineligible for employment Visa sponsorship. Principal Software Engineer - Infrastructure Automation Overall Purpose The Principal Software Engineer - Infrastructure Automation is an enterprise technical leader responsible for the architecture, strategy, and engineering direction of the infrastructure-as-code platform used to provision and manage infrastructure across Early Warning. This position owns the technical model for Terraform, Ansible, policy-as-code guardrails, and the shared modules and services consumed by engineering teams. This is a hands-on software engineering and systems design role that operates with very little guidance. The Principal Engineer proactively identifies material problems and opportunities, frames the solution, resolves ambiguous cross-domain decisions, and drives execution through adoption. The role establishes enterprise standards and governance and delivers secure, maintainable, and well-tested platform capabilities that automate regulatory and operational controls across cloud and on-premises environments. Essential Functions Operate with very little guidance to identify enterprise infrastructure automation problems and opportunities, define the target solution and execution path, resolve cross-domain tradeoffs, and drive delivery through measurable adoption and outcomes. Set the enterprise technical strategy and reference architecture for infrastructure automation, including architecture decision records, platform boundaries, operating models, and consequential build-versus-buy decisions. Establish software engineering standards for infrastructure modules, policy, and tooling, including versioned interfaces, testing, code review, release management, deprecation, and automated regression coverage. Own Terraform as the enterprise default for infrastructure provisioning across cloud and on-premises estates, including module taxonomy, ownership, private registry strategy, and adoption standards. Define and deliver opinionated Terraform modules for AWS services and common service patterns that provide secure, compliant, observable defaults for encryption, logging, tagging, networking, and IAM. Establish the enterprise override and exception model so legitimate deviations remain visible, reviewable, policy-checked, time-bound, and auditable. Architect the Terraform module test and release system, including native Terraform tests, Terratest, tflint, static analysis, semantic versioning, signed artifacts, and policy gates on plan output. Own the HCP Terraform or Terraform Enterprise operating model, including workspaces, run tasks, policy sets, protected state, drift detection, and plan/apply governance independent of the CI system that initiates a run. Own the enterprise Ansible platform architecture, including collections, roles, execution environments, Molecule and lint standards, and Ansible Automation Platform/AWX operating practices. Define the configuration-as-code architecture for network infrastructure, including structured configuration as source of truth, safe rollback, virtual topology and reachability validation, post-change verification, compliance, and drift detection. Own the enterprise policy-as-code architecture and Rego policy library for Terraform plans, Ansible constraints, and Kubernetes admission; ensure policy is tested, versioned, released, and distributed as a governed product. Design programmatic controls for segregation of duties, protected state, change review, and break-glass access with automatic evidence capture and time-boxed expiration. Eliminate static credentials from infrastructure change workflows through OIDC-federated, short-lived IAM roles and dynamic secrets for cloud, database, and network credentials. Define the infrastructure automation control framework and map enforcement points to PCI DSS, SOX ITGC, NYDFS Part 500, FFIEC, and other applicable requirements; convert manual controls into continuously evidenced automated controls. Serve as the technical authority for infrastructure-as-code controls in internal audits, external audits, and regulatory examinations, and author enterprise engineering standards and control narratives. Architect self-service capabilities for module discovery, scaffolding, exception workflows, compliance dashboards, and APIs that expose module, policy, and compliance state to internal platforms. Prototype and de-risk the most complex infrastructure automation and guardrail problems; mentor senior engineers and raise the architecture, design, and code-review bar across engineering. Partner with CI/CD, Security, Network Engineering, AIOps, and observability leaders to integrate infrastructure modules, controls, telemetry, and governance across shared engineering platforms. Define and report enterprise measures of platform effectiveness, including module adoption, policy pass rate, plan/apply success, control coverage, and time to remediate drift and compliance violations. Support the company's commitment to risk management and protecting the integrity and confidentiality of systems and data. Minimum Qualifications Education and/or experience typically obtained through a bachelor's degree in computer science, engineering, or a related technical field. Typically ten or more years of related experience in software, platform, cloud, or infrastructure engineering, including significant experience leading enterprise-scale technical architecture. Demonstrated excellence in software engineering and distributed-systems design, including rigorous reasoning about interfaces, state, failure modes, idempotency, and blast radius. Demonstrated ability to operate with very little guidance, proactively identify unstructured technical problems or opportunities, define the solution and execution strategy, and drive complex work from concept through implementation and organizational adoption. Demonstrated customer-first leadership at enterprise scale, balancing customer and developer outcomes with risk, reliability, and control requirements. Demonstrated ability to model disagree and commit: challenge assumptions with evidence, drive timely decisions, and align the organization behind the final direction. Strong production programming experience in Python or Go building tools, controllers, services, APIs, or custom scanners - not solely HCL and YAML. Demonstrated experience architecting and operating platform capabilities used by multiple engineering organizations, including testing, release management, reliability, and operational ownership. Deep Terraform expertise at enterprise scale, including module composition, testing, private registries, state governance, version strategy, and plan/apply controls. Deep experience with Ansible beyond playbook development, including collections, execution environments, Molecule testing, linting, and Ansible Automation Platform/AWX operations. Fluency with OPA/Rego or a comparable policy language, including authoring, testing, versioning, and operating policy-as-code libraries. Deep knowledge of AWS infrastructure and security, particularly IAM, federated identity, short-lived credentials, networking, encryption, and secure configuration patterns. Working knowledge of Kubernetes sufficient to design and evaluate infrastructure, identity, and admission-control policies. Demonstrated experience embedding security, regulatory, and change-management controls into automated infrastructure workflows in financial services or a comparably regulated industry. Demonstrated ability to influence enterprise architecture and engineering standards across organizational boundaries without direct authority. Exceptional communication skills, including architecture decisions, engineering standards, executive communication, audit narratives, and regulatory control documentation. Background and drug screen. Preferred Qualifications Experience owning HCP Terraform or Terraform Enterprise at scale, including workspace strategy, run tasks, policy sets, state governance, drift management, and platform operations. Experience architecting internal developer platforms, infrastructure self-service products, developer portals, or platform APIs. Experience independently identifying an enterprise platform gap, establishing the technical and operating model, and leading implementation and adoption across multiple engineering organizations. Experience building full-stack internal engineering products using modern front-end frameworks such as React, Next.js, or Angular and backend/API technologies such as TypeScript, Python, or Go. Experience with network automation architecture, including structured configuration, virtual topology testing, reachability or ACL analysis, safe rollback, and golden-configuration compliance. Experience designing dynamic secrets systems and OIDC-based workload identity for cloud, database, and network infrastructure. Experience serving as a technical authority in audits or regulatory examinations and mapping automated controls to PCI DSS, SOX ITGC, NYDFS Part 500, or FFIEC guidance. Experience in FinTech, banking, payments, or another highly regulated industry. Current advanced AWS, Terraform, Kubernetes, security, or related technical certification. The above job description is not intended to be an all-inclusive list of duties and standards of the position. Incumbents will follow instructions and perform other related duties as assigned by their supervisor. The base pay scale for this position in: Phoenix, AZ/ Chicago, IL in USD per year is: $173,000 - $230,000. Additionally, candidates are eligible for a discretionary incentive plan and benefits. This pay scale is subject to change and is not necessarily reflective of actual compensation that may be earned, nor a promise of any specific pay for any specific candidate, which is always dependent on legitimate factors considered at the time of job offer. Early Warning Services takes into consideration a variety of factors when determining a competitive salary offer, including, but not limited to, the job scope, market rates and geographic location of a position, candidate’s education, experience, training, and specialized skills or certification(s) in relation to the job requirements and compared with internal equity (peers). The business actively supports and reviews wage equity to ensure that pay decisions are not based on gender, race, national origin, or any other protected classes. Some of the Ways We Prioritize Your Health and Happiness Healthcare Coverage – Competitive medical (PPO/HDHP), dental, and vision plans as well as company contributions to your Health Savings Account (HSA) or pre-tax savings through flexible spending accounts (FSA) for commuting, health & dependent care expenses. 401(k) Retirement Plan – Featuring a 100% Company Safe Harbor Match on your first 6% deferral immediately upon eligibility. Paid Time Off – Flexible Time Off for Exempt (salaried) employees, as well as generous PTO for Non-Exempt (hourly) employees, plus 11 paid company holidays and a paid volunteer day. 12 weeks of Paid Parental Leave Maven Family Planning – provides support through your Parenting journey including egg freezing, fertility, adoption, surrogacy, pregnancy, postpartum, early pediatrics, and returning to work. And SO much more! We continue to enhance our program, so be sure to check our Benefits page here for the latest. Our team can share more during the interview process! Early Warning Services, LLC (“Early Warning”) considers for employment, hires, retains and promotes qualified candidates on the basis of ability, potential, and valid qualifications without regard to race, religious creed, religion, color, sex, sexual orientation, genetic information, gender, gender identity, gender expression, age, national origin, ancestry, citizenship, protected veteran or disability status or any factor prohibited by law, and as such affirms in policy and practice to support and promote equal employment opportunity and affirmative action, in accordance with all applicable federal, state, and municipal laws. The company also prohibits discrimination on other bases such as medical condition, marital status or any other factor that is irrelevant to the performance of our employees. Early Warning Services LLC is a proud participant in E-Verify, a federal program to help ensure a legal and authorized workforce. As part of our hiring process, we electronically verify the employment eligibility of all new hires through E-Verify. For more information on your rights and responsibilities under E-Verify please visit Home | E-Verify .