Showing posts with label S3. Show all posts
Showing posts with label S3. Show all posts

Thursday, January 14, 2016

Bash script to email new S3 bucket files as compressed attachments (UDPATED)

I've written a simple bash script that checks for new files in an AWS S3 bucket and emails any that it finds as a compress (tar.gz) attachment - you can find it at my Github account under the name "S3-Filer-Mailer". I built it as a supplement for a contact form that relies on S3 as a back-end, rather than a php mailer or database. Using S3 for contact forms is attractive because it is so unattractive to spammers. There is no way to corrupt this sort of setup for spamming or to get hands on a database through the form, because it isn't connected to one.

Why not use Amazon's Simple Notification Service (SNS)? For one, AWS charges more for SNS than it does for S3 queries and downloads. For another, if this sort of functionality is available through SNS it is not clearly documented.

Getting back to the topic of security, the script establishes two network connections - one a connection to S3 to retrieve the files, the other sending the email. The S3 connection is encrypted using TLS; I'm going to add an extra pipe in here to gpg2 as time permits to encrypt the attachments themselves to close the loop - or you can do it yourself by adding a line with gpg -e -r Name foo.txt, where Name is the name you used while generating the public key you wish to use to encrypt the file. Adding encryption support as a command line operator is easy, but I want to add it as part of more general sanity-checking input.

The script was built and tested on RHEL, but it should work in any Linux that supports bash. This is pre-pre-alpha version, so no complaining. The obvious and immediate functionality problem ATM is that the script assumes that only files that contain a string with today's date in their filename were created today (plus the string has to be in format YYYYMMDD). When my copious spare time allows I will get to adding an option to filter results via regex; for now users can do this fairly simply by piping an additional grep command between grep ${TODAY} and > ${FILE} on line 16 of

The script includes two files, an executable file ( and a configuration file (S3-Filer-Mailer.conf). To get things working, move both files to a computer running Linux and modify the S3-Filer-Mailer.conf file settings; that is where you will specify your email address and your S3 bucket. You can also limit the script to a subdirectory of your bucket in the conf file. The script is recursive, so if you specify the root directory of your bucket it will check every subdirectory. For the time being, that is the only way to specify multiple subdirectories; similarly disabling recursiveness requires modifying the executable.

Also, dependencies. There are some. Only one of them should take more than 5 seconds to install, the AWS Command Line Interface. You will need Python for that if you don't already have it. On the bright side, if you want to do cool stuff with AWS and you are using linux you should be happy to drag more crap to a CLI, right? The only other dependency is mailx.

UPDATE: I've moved this from a gist to a full-fledged Github repo, and I've made a few updates that make this script significantly less lame.

The earliest version of this required sharutils to uuencode attachments, but that is no longer necessary. Relying entirely on mailx encoding also resolved an ongoing issue in which Mozilla Thunderbird did not properly recognize attachments.

Variables that need to be changed in order for the script to function have been placed into a separate .CONF file.

Sunday, February 1, 2015

Uploading HTML forms to Amazon S3 using PHP

Dynamically uploading information to S3 can be a bit challenging to do initially, particularly in PHP where a lot of the documentation is either really new or really old.

Amazon has a PHP SDK, which is available as either a .phar file or can be installed using Composer. That's cool for building a new project, but what if you have a pre-existing project or form and just want to be able to dump the text output to S3?

I've put together some code at Github that will take care of that issue. The only requirement is PHP and an Amazon S3 account.

Download or clone the files here:

Your Amazon access keys and other configuration are stored in a single configuration file. Just fill out your login info into the configuration file and include the php form where you need it as outlined in the file and you should be all set!

NSA Leak Bust Points to State Surveillance Deal with Printing Firms

Earlier this week a young government contractor named Reality Winner was accused by police of leaking an internal NSA document to news outle...