Security

Balancing Cost Optimization and Security Boundaries A while back, I was working with a team that wanted to consolidate their EKS clusters to save money. FinOps loved it, engineering approved it - made total sense from a cost perspective. But as we started mapping services to clusters, someone asked: “Can we run our payment API and our internal dashboard on the same nodes?” That question led me down a rabbit hole that ended with me building a lab to see what actually happens when you mix sensitive workloads with debug tooling. Turns out, the answer is “it depends” - but it depends on things most teams don’t think about. ...

In the final part of this series on column-level encryption, I explain how to work with encrypted columns in Amazon Redshift Serverless and decrypt specific fields on the fly using an AWS Lambda function written in Go. By combining the scalability of Redshift with the flexibility of Lambda, you can securely access sensitive data without compromising performance or security best practices. Why Redshift and Lambda? Amazon Redshift is an excellent choice for analytics on sensitive data because of its performance and scalability for large datasets. When working with sensitive information, such as SSNs or credit card numbers, encryption is essential for compliance and security. AWS Lambda makes it possible to decrypt this data dynamically for specific use cases, like table joins. Writing the decryption logic in Go ensures fast and efficient execution. ...

This post describes how you can build an AWS Glue ingestion job with PySpark aes_encrypt() function to encrypt sensitive columns. It is part of a series that shows how column-level encryption can be deployed at scale using AWS Glue, AWS KMS and Amazon Athena or Amazon Redshift. Introduction In this post, I demonstrate setting up an AWS Glue ingestion job to encrypt sensitive columns using AWS KMS. We will also explore key management approaches and their impact on organizational security practices. ...

This post describes how you can use PySpark aes_encrypt() function to encrypt sensitive columns when ingesting data. It is part of a series that shows how column-level encryption can be deployed at scale using AWS Glue, AWS KMS and Amazon Athena or Amazon Redshift. Introduction In an era where data breaches are increasingly common, securing sensitive data is not just a best practice but a necessity. As Werner Vogels, Amazon’s CTO, wisely put it: “Dance Like Nobody’s Watching. Encrypt Like Everyone Is.” In this series, we explore how to deploy column-level encryption at scale using an integration of PySpark, AWS Glue, AWS KMS, and Amazon Redshift or Amazon Athena. ...