AWS

Amazon Redshift now allows direct querying of S3 data lake tables located in different AWS Regions, eliminating the need for data replication. Enhanced VPC routing ensures that query traffic between Redshift and S3 remains within the customer's VPC, addressing security and compliance requirements. This capability is powered by the integrated data lake query engine running on both provisioned and serverless cluster compute.

• Query cross-Region S3 tables directly without copying or replicating data first.
• VPC routing keeps data lake query traffic within the customer's private network.
• Supports both provisioned and serverless Redshift cluster configurations.
• Enables global analytics and consolidated reporting across distributed regions.

AWS has launched a preview of the Well-Architected Agent, an AI-driven service that supersedes Trusted Advisor and the Well-Architected Tool. It correlates metrics and application topology against best practices to deliver prioritized recommendations based on business goals. The agent also analyzes Terraform, CDK, and CloudFormation templates to generate automation-ready fixes.

• Replaces legacy Trusted Advisor and Well-Architected Tool with an AI agent.
• Prioritizes recommendations by business impact and implementation effort.
• Supports IaC analysis across Terraform, CDK, and CloudFormation.
• Generates automation-ready fixes for identified architectural issues.
• Correlates metrics with topology for contextualized optimization advice.

AWS Glue Data Catalog now natively supports table optimization, statistics generation, and crawler discovery for Apache Iceberg Version 3 tables. Users can auto-maintain tables using binpack, sort, or z-order compaction strategies to enhance query performance and reduce storage costs by cleaning up expired snapshots. The update also includes NDV statistics for better query planning and support for V3-specific data types like variant, geospatial, and nanosecond timestamps.

• Glue crawlers can now discover and register Iceberg V3 tables stored in S3 automatically.
• Support for binpack, sort, and z-order compaction strategies to optimize V3 table performance.
• Generate NDV statistics to help analytics engines plan queries more efficiently.
• Native support for Iceberg V3 data types including variant, geospatial, and nanosecond precision.
• Automated maintenance helps remove orphan files and expired snapshots to lower storage costs.

Amazon DynamoDB adds filtered export capabilities, allowing users to export specific subsets of table data to Amazon S3. This feature leverages key condition, filter, and projection expressions to select exactly which items and attributes are included in the output. The resulting dataset is optimized for offline analytics, granular recovery, or cross-account data movement.

• Use key condition, filter, and projection expressions to precisely control exported data subsets
• Reduces S3 storage and processing costs by exporting only relevant items and attributes
• Enables granular data recovery and efficient cross-account data migration slices
• Supports analytics use cases by generating focused datasets without full table dumps

Amazon EMR Serverless has increased its per-job shuffle storage limit from 200 GB to 1 TB. This change allows data engineers to run production-scale Apache Spark jobs involving large joins, aggregations, and sorts without hitting storage constraints. The update removes a key bottleneck for migrating enterprise workloads that process massive datasets in a serverless environment.

• Shuffle storage limit for EMR Serverless jobs increased to 1 TB from 200 GB
• Enables production Spark workloads with large joins, aggregations, and sorting
• Removes storage constraints previously blocking terabyte-scale dataset processing
• Simplifies migration of enterprise big data workloads to serverless infrastructure

AWS Glue now supports system-managed materialized views for Apache Iceberg, restricting write access to the engine itself. This prevents users with S3 or catalog write permissions from altering precomputed results, ensuring data integrity. Users define the view via SQL and optionally schedule refreshes, with Glue handling the storage as standard Iceberg tables in S3.

• Only AWS Glue can write to the materialized view, blocking external S3 or catalog writes.
• Precomputed query results remain immutable, preventing accidental or malicious data corruption.
• Define views with SQL and optional refresh schedules for automated maintenance.
• Results are stored as standard Apache Iceberg tables in Amazon S3 for compatibility.