Amazon Web Services Launches S3 Files, Letting Developers Access Cloud Storage Like a Local Hard Drive

The technical challenge of making this work was significant and, according to AWS, took years to get right. Andy Warfield, a vice president and distinguished engineer who leads S3 engineering at AWS, published an unusually candid account of the development process alongside the announcement. The core problem, Warfield explained, is that files and objects are fundamentally different by design. Files can be edited in place and shared across applications in real time. Objects in S3 are stored and retrieved as complete units, and millions of applications are built around that assumption. Changing either model risked breaking existing systems at massive scale.

The team’s first approach failed. “We did the only sensible thing you can do when you are faced with a really difficult technical design problem: we locked a bunch of our most senior engineers in a room and not let them out till they had a plan that they all liked,” Warfield wrote. “Passionate and contentious discussions ensued,” he added. “And then finally we gave up.”

The solution the team eventually landed on stopped trying to hide the boundary between files and objects and instead made it a central part of the design. S3 Files uses a stage and commit model borrowed from version control systems like Git, where changes accumulate on the file system side and are pushed back to S3 as complete objects, preserving the guarantees that existing S3 applications depend on.

Google and Microsoft both offer tools for accessing cloud object storage through file system interfaces, but AWS is positioning S3 Files as a deeper integration backed by a fully managed file system rather than a simple adapter. The feature is built on Amazon’s Elastic File System and is available today across AWS regions worldwide. The company said it has been in customer testing for approximately nine months.