Staff Architect & Data Engineer Greater Boston

Hi, I'm Chandra Shekar Reddy

Over 16 years of experience building high-performance stream processing pipelines, big data infrastructure, and microservices using Scala, Python, Rust, Spark, Flink, Dataflow/Beam, GCP, AWS, Java.

Chandra Shekar Reddy Gadipally

About Me

I am a senior engineer specialized in stream processing, data operations, and big data architecture. My expertise covers the full lifecycle of data-intensive systems, from designing real-time analytics pipelines to architecting resilient microservices.

My technical focus centers on Scala, Python, and Rust, functional programming, stream engines (Beam/Dataflow, Flink, Spark Streaming), cloud infrastructure (AWS & Google Cloud), and robust devops practices (Docker, Kubernetes, Terraform).

During my free time, I paraglide, watch birds soar in thermal updrafts, read, hike, play tennis, and tinker with Arduino.

Core Skills

Languages

Scala (7+ years) Python Rust Haskell Java (7+ years) SQL Elm Javascript / TypeScript R

Data & ML

Apache Flink Apache Spark Kafka AWS Kinesis Databricks Hadoop Snowflake ElasticSearch Airflow MongoDB Pandas / Scikit-learn

Web, Cloud & DevOps

Play Framework Http4s / Cats Sbt REST APIs AWS Google Cloud Docker Kubernetes Terraform GitHub Actions / Jenkins Neovim / Tmux Gemini CLI / Antigravity

Work Experience

Staff Architect, Data Engineering

Jack Henry & Associates
Sep 2024 - Present
  • Hands-on Architecture: Actively designing, writing, and deploying production-grade code for batch and real-time streaming data pipelines.
  • Streaming real-time CDC (Change Data Capture) events from credit union backend databases into Google Cloud Platform (GCP) using Debezium.
  • Leveraging Google Cloud services including Cloud Dataflow, Cloud Pub/Sub, BigQuery, and Cloud Storage (GCS) for high-throughput batch and stream processing.
  • Orchestrating data workflows using Google Managed Airflow (Cloud Composer), defining infrastructure with Terraform, containerizing services with Docker, and automating with GitHub Actions.
Tech Stack:
GCP (Dataflow, Pub/Sub, BigQuery, GCS, Cloud Composer) Debezium Terraform Docker GitHub Actions Gemini CLI / Antigravity Python Rust Stream Processing

Contract Software Engineer (Scala)

Disney (Streaming)
Jun 2023 - Mar 2024
  • Enhanced Disney’s adaptive traffic routing application (Panama), distributing HULU, Disney+, ESPN, and STAR+ user traffic to Disney & third-party CDNs to optimize cost and performance.
  • Designed advanced targeting features for routing users via Private Network Interfaces (PNI) using partner ASNs.
  • Aggregated real-time telemetry and usage data using Apache Flink, AWS Kinesis, Lambda, and Timestream.
  • Optimized user playback session summarization and anonymization pipelines using Apache Spark, Databricks, Delta Lake, and S3.
Tech Stack:
Scala Cats Http4s Doobie Flink AWS Kinesis Spark Databricks Terraform Timestream

Data Engineer IV

Stash Invest
Jun 2022 - Jun 2023
  • Created and maintained robust ETL pipelines ingesting fintech data from Zendesk, Alloy, Apex, Stride bank, Mixpanel, and third-party APIs.
  • Implemented customer audio transcription, anonymization, and summarization using OpenAI Whisper, Pyannote, and Microsoft Presidio.
  • Built interactive Crypto advice dashboards for risk management teams in Looker.
Tech Stack:
AWS Spark Scala Python Airflow OpenAI Whisper Redshift Looker

Senior Software Engineer (Data Operations)

Datasembly
Jul 2021 - Jun 2022
  • Streamlined retail product data collection and built new ETL pipelines using Spark, Kafka, and Airflow (Astronomer).
  • Developed computer vision and OCR solutions to extract nutrition facts and ingredients from product label images.
  • Refactored handling of 2-prefix UPC (GTIN12) barcodes across the core platform.
Tech Stack:
Scala Python Tesseract Airflow Spark GCP Kubernetes Snowflake

Contractor - NLP & Data Engineer

Verizon
Aug 2020 - Jul 2021
  • Enhanced and optimized Verizon FiosTV voice-search Natural Language Understanding (NLU) service.
  • Re-architected and built the next-generation NLU models using the RASA framework, validating performance using Spark.
Tech Stack:
Scala Play Framework Python RASA ElasticSearch MongoDB Spark

Contractor - Sr Software & Data Engineer

Edward Jones Investments
Mar 2012 - Aug 2019
  • Designed Spark-Scala batch analytics pipelines detecting account trends, conversions, commission reductions, and asset flows.
  • Calculated Z-Scores on aggregated financial events, cross-referencing with rule engines for compliance reporting.
  • Built a web application for calculating Required Minimum Distributions (RMD) and compiling advisory dashboards.
  • Consolidated fee-structure processing and dividend reinvestment platforms into unified modern web tools.
Tech Stack:
Scala Spark Java Spring Oracle DB Highcharts Hibernate

Education & Credentials

Education

M.S. in Computer Science

Illinois Institute of Technology

Chicago, IL | 2009 - 2010

Focus: Algorithms, Science of Programming, Advanced Database Systems

B.Tech. in Electronics & Computer Engineering

Jawaharlal Nehru Technological University

Hyderabad, India | 2004 - 2008

Professional Certifications

Lightbend Academy

Scala, Akka & Reactive Systems

Astronomer

Apache Airflow Orchestration

Coursera

Data Science, ML & Programming