Senior
Lead Software Engineer - Python/PySpark/Databricks/AWS
We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible.
As a Lead Software Engineer - Python/PySpark/Databricks/AWS at JPMorganChase within the Corporate Technology team, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. As a core technical contributor, you are responsible for conducting critical technology solutions across multiple technical areas within various business functions in support of the firm’s business objectives.
Job responsibilities
-
Executes creative software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problems
-
Develops secure high-quality production code, and reviews and debugs code written by others
-
Identifies opportunities to eliminate or automate remediation of recurring issues to improve overall operational stability of software applications and systems
-
Leads evaluation sessions with external vendors, startups, and internal teams to drive outcomes-oriented probing of architectural designs, technical credentials, and applicability for use within existing systems and information architecture
-
Leads communities of practice across Software Engineering to drive awareness and use of new and leading-edge technologies
-
Drives team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test strategy acceleration, incident/root-cause analysis support), while establishing consistent validation standards (secure coding, peer review, automated testing) and promoting reuse of effective patterns across the team.
-
Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation.
-
Drives team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test strategy acceleration, incident/root-cause analysis support), while establishing consistent validation standards (secure coding, peer review, automated testing) and promoting reuse of effective patterns across the team.
-
Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation.
Required qualifications, capabilities, and skills
-
Formal training or certification on software engineering concepts and 5+ years applied experience
-
Demonstrated experience leading effective use of approved AI-assisted software development tools (e.g., for coding, code review, test acceleration, troubleshooting) with the ability to set team expectations for validating AI outputs for correctness, performance, and security.
-
Strong understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; experience coaching engineers on safe, compliant adoption within delivery practices
-
Hands-on practical experience delivering system design, application development, testing, and operational stability
-
Experience building and operating Databricks Lakehouse solutions hosted on Amazon Web Services (AWS), including Amazon S3, Identity and Access Management (IAM), Key Management Service (KMS), basic networking concepts (VPC/security groups), and logging/auditing; Experience using Delta Lake (ACID-compliant tables, partitioning strategies, schema evolution) and Apache Spark on Databricks, including performance optimization (cluster sizing, skew mitigation, join strategies, caching, and file sizing/compaction); Experience delivering batch and streaming data pipelines (Structured Streaming, incremental processing, backfills, late-arriving data handling) and implementing governance/security controls in Databricks (e.g., Unity Catalog, table/column-level permissions, credential passthrough where applicable), with operational ownership including monitoring/alerting, incident response, root-cause analysis (RCA), and service level objective/service level agreement (SLO/SLA) management.
-
Advanced in one or more programming language(s) including Python, PySpark
-
Advanced understanding of agile methodologies such as CI/CD, Application Resiliency, and Security
-
Architect Databricks Lakehouse solutions, including bronze/silver/gold (or equivalent) layering and domain-oriented data products; implement resilient, scalable ingestion from AWS sources into Databricks using batch and streaming patterns (including CDC where required).
-
Build maintainable pipelines using Delta Live Tables (DLT) and/or Databricks Jobs/Workflows with modular design, documentation, and runbooks; ensure production readiness through retries, checkpointing, idempotency, safe re-runs, and defined replay/backfill procedures; implement testing practices including unit/integration tests, data quality checks, and contract testing; Apply governance-by-design controls (least privilege, PII classification, auditing, lineage/metadata, controlled sharing/consumption); optimize Spark/Delta performance and cost (cluster right-sizing, storage layout, job/warehouse spend); lead design/code reviews and mentor engineers; partner cross-functionally with stakeholders and security/platform teams; deliver CI/CD and infrastructure-as-code for Databricks + AWS with promotion across environments and strong version control/code review discipline.
-
Demonstrated experience leading effective use of approved AI-assisted software development tools (e.g., for coding, code review, test acceleration, troubleshooting) with the ability to set team expectations for validating AI outputs for correctness, performance, and security.
-
Strong understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; experience coaching engineers on safe, compliant adoption within delivery practices
Preferred qualifications, capabilities, and skills
- AI experience
Sign up to apply and find out right away if you're a fit.
Is this role right for you?
Your agent will tell you — in seconds.
Sign up and I'll tell you right away how well JPMorgan Chase matches you — what you already have, and what's missing. Then I stay on it: I search for you and only write when I find something worth your time.