Data Engineer (AWS, Spark)

Postgres
UNITED STATES
September 22, 2026
Full Time
Apply Now

Job Description

Company: Peregrine Advisors
Location: Washington, DC
Workplace Type: Hybrid
Employment Type: Permanent
Pay / Compensation: $103,000 to $140,000 per year

About the Role

We are a data and technology innovation hub and a Benefit Corporation working at the center of the federal government’s mission to deliver for client stakeholders and the US public. At Peregrine Advisors, you will build the pipelines that move a federal agency’s data from source to platform. But we hire people, not seats. As the work evolves, you will learn new systems and tools, take on greater responsibility, and help develop the firm’s capabilities, tools, and lines of business. We move our best to where the hardest problems are.

This is a full-time W-2 position with a salary of $103,000 to $140,000 per year. It is a hybrid work arrangement based in the Washington, DC metropolitan area, and the commuting cadence varies by assignment. The initial engagement requires United States citizenship and the ability to obtain a Public Trust determination.

You are a data engineer who wants to get better at it, and you know which parts you have not mastered yet. You care as much about whether the data is trustworthy as whether it arrives, and you do your best work alongside people who push you. You experiment, fail, learn, and repeat quickly. You would rather own an outcome than be handed a task.

What You’ll Do / Key Responsibilities

  • Build Spark-based extract, transform, and load (ETL) pipelines with Glue, Amazon EMR, Lambda, and Step Functions.
  • Write the processing in Python and PySpark.
  • Design the S3 layer, including Parquet, partitioning, and lifecycle, feeding Apache Iceberg tables.
  • Connect PostgreSQL on Amazon Aurora and DynamoDB, with Trino for federated Structured Query Language (SQL) across them.
  • Deliver pipelines that keep data clean and trustworthy at the volumes a federal platform runs at.
  • Automate ingestion that used to be handled case by case.
  • Build the storage design everything downstream depends on.
  • Adapt your development workflow as AI tools evolve, using them to help implement, test, and improve the components you own (giving tools clear context, reviewing/testing output, and remaining accountable for delivered code).
  • Collaborate alongside developers, engineers, data scientists, architects, and strategists on work ranging from strategy to implementation.

Requirements & Qualifications

  • 4+ years of data engineering experience (Associate, 4+ years level).
  • Bachelor’s degree.
  • Hands-on data engineering on AWS: Spark ETL (Glue, Amazon EMR) in Python and PySpark.
  • S3 data-lake design (Parquet, partitioning, lifecycle) feeding Apache Iceberg tables, Amazon Aurora PostgreSQL, and DynamoDB.
  • Event orchestration (Lambda, Step Functions, SQS/SNS) with secrets management and monitoring.
  • Proven understanding of data quality, validation, and lineage.
  • Infrastructure-as-code (CloudFormation or Terraform).
  • The judgment to build in a regulated environment where accuracy and auditability are not optional.
  • Basic proficiency in writing, PowerPoint, and Excel.
  • United States citizenship and the ability to obtain a Public Trust determination.

Preferred / Bonus Qualifications (if applicable)

  • Master’s degree in a relevant field.
  • Trino or comparable federated SQL across the lake and relational stores.
  • Apache Ranger-governed access.
  • Legacy ETL migration (for example DataStage).
  • Federal information technology or high-volume data experience.
  • Familiarity with AI-assisted developer tooling.

Benefits & Perks

  • Medical, dental, and vision insurance with the employee premium fully paid and half of dependent premiums.
  • Employer-paid life, accidental death, and short-term and long-term disability insurance.
  • 401(k) matched 100% up to 4% of salary, vesting immediately.
  • Unlimited paid time off.
  • Sponsored professional certifications and continuing education (e.g., Data Management Capability Assessment Model (DCAM) and AWS technical certifications).
  • Extensive onboarding and rotation across functions to expand your skills and perspective.

When we talk: Be prepared to discuss a difficult problem you worked through, the decisions you made, what happened, and what you learned.

What we commit to: As a Benefit Corporation, our commitment runs three ways: real, measurable value for our clients; government that works better for the public; and a team that makes everyone in it better. We hire people who want to help build the firm, not just work at it. If that is you, apply.

Peregrine Advisors is an equal opportunity employer.

Peregrine exclusively works with OPEN Data Jobs to recruit our team. Register with OPEN Data Jobs to be considered for this and future openings.

Related Jobs