Python Developer

About the company

• ProArch is seeking an experienced AWS Glue Python Developer for cloud-based data integration solutions

Responsibilities

• Design and develop inbound and outbound file processing using AWS Glue, Python, and PySpark
• Develop reusable Python modules, utilities, and frameworks for data ingestion, validation, transformation, and error handling
• Build batch file processing using Amazon S3 and AWS Glue
• Configure Amazon EventBridge rules for scheduling and event-based processing
• Develop AWS Lambda functions for lightweight processing, automation, validation, and integration
• Implement messaging and notification solutions using Amazon SNS and Amazon SQS
• Configure Amazon CloudWatch logs, metrics, dashboards, and alarms
• Develop workflows using AWS Glue Workflows, AWS Step Functions, or equivalent orchestration services
• Integrate AWS services with relational databases, APIs, file systems, and external applications
• Implement data ingestion from CSV, XML, JSON, Excel, fixed-length, variable-length, and delimited files
• Develop data validation, reconciliation, audit, retry, and exception-handling processes
• Optimize AWS Glue jobs for performance, scalability, and cost efficiency
• Implement secure access using AWS IAM roles, policies, encryption, and secrets management
• Support production deployments, incident resolution, root-cause analysis, and performance tuning
• Collaborate with DevOps teams to implement CI/CD pipelines
• Prepare technical documentation, data flow diagrams, operational procedures

Requirements

• 8+ years of experience in Python development and data engineering
• Strong hands-on experience with AWS Glue and PySpark
• Excellent knowledge of Python, OOP, and exception handling
• Experience designing ETL pipelines and data processing frameworks
• Strong experience with Amazon S3: bucket structures, lifecycle policies, partitioning, file formats
• Experience with Amazon EventBridge for scheduled and event-driven execution
• Experience developing and deploying AWS Lambda functions using Python
• Experience with Amazon SNS and SQS for notifications, messaging, and asynchronous processing
• Experience with Amazon CloudWatch for logging, monitoring, alerting, and troubleshooting
• Understanding of AWS IAM, security policies, encryption, and least-privilege access
• Experience with relational databases: Oracle, PostgreSQL, MySQL, or Amazon RDS
• Strong knowledge of SQL and database connectivity from Python or AWS Glue
• Experience with REST/SOAP APIs and external system integrations
• Understanding of JSON, XML, CSV, Excel, and fixed-width file formats
• Experience with Git and software development best practices

Nice to have

• Experience with AWS Step Functions or AWS Glue Workflows
• Experience with CI/CD tools: Jenkins, GitHub Actions, GitLab, or AWS CodePipeline
• Knowledge of data quality, reconciliation, lineage, and audit requirements

Conditions

• On-site work in Chennai