Showing posts with label bash. Show all posts
Showing posts with label bash. Show all posts

Thursday, January 14, 2016

Bash script to email new S3 bucket files as compressed attachments (UDPATED)

I've written a simple bash script that checks for new files in an AWS S3 bucket and emails any that it finds as a compress (tar.gz) attachment - you can find it at my Github account under the name "S3-Filer-Mailer". I built it as a supplement for a contact form that relies on S3 as a back-end, rather than a php mailer or database. Using S3 for contact forms is attractive because it is so unattractive to spammers. There is no way to corrupt this sort of setup for spamming or to get hands on a database through the form, because it isn't connected to one.

Why not use Amazon's Simple Notification Service (SNS)? For one, AWS charges more for SNS than it does for S3 queries and downloads. For another, if this sort of functionality is available through SNS it is not clearly documented.

Getting back to the topic of security, the script establishes two network connections - one a connection to S3 to retrieve the files, the other sending the email. The S3 connection is encrypted using TLS; I'm going to add an extra pipe in here to gpg2 as time permits to encrypt the attachments themselves to close the loop - or you can do it yourself by adding a line with gpg -e -r Name foo.txt, where Name is the name you used while generating the public key you wish to use to encrypt the file. Adding encryption support as a command line operator is easy, but I want to add it as part of more general sanity-checking input.

The script was built and tested on RHEL, but it should work in any Linux that supports bash. This is pre-pre-alpha version, so no complaining. The obvious and immediate functionality problem ATM is that the script assumes that only files that contain a string with today's date in their filename were created today (plus the string has to be in format YYYYMMDD). When my copious spare time allows I will get to adding an option to filter results via regex; for now users can do this fairly simply by piping an additional grep command between grep ${TODAY} and > ${FILE} on line 16 of

The script includes two files, an executable file ( and a configuration file (S3-Filer-Mailer.conf). To get things working, move both files to a computer running Linux and modify the S3-Filer-Mailer.conf file settings; that is where you will specify your email address and your S3 bucket. You can also limit the script to a subdirectory of your bucket in the conf file. The script is recursive, so if you specify the root directory of your bucket it will check every subdirectory. For the time being, that is the only way to specify multiple subdirectories; similarly disabling recursiveness requires modifying the executable.

Also, dependencies. There are some. Only one of them should take more than 5 seconds to install, the AWS Command Line Interface. You will need Python for that if you don't already have it. On the bright side, if you want to do cool stuff with AWS and you are using linux you should be happy to drag more crap to a CLI, right? The only other dependency is mailx.

UPDATE: I've moved this from a gist to a full-fledged Github repo, and I've made a few updates that make this script significantly less lame.

The earliest version of this required sharutils to uuencode attachments, but that is no longer necessary. Relying entirely on mailx encoding also resolved an ongoing issue in which Mozilla Thunderbird did not properly recognize attachments.

Variables that need to be changed in order for the script to function have been placed into a separate .CONF file.

Tuesday, November 25, 2014

How To Find Files Over a Certain Size Using Redhat/CentOS/Fedora Linux

Here is a quick tip for all of those Redhat/CentOS/Fedora users out there. Do you need to find all files over a certain size, either in a specific directory, your current directory, or in your entire computer/server?

No problem, just execute the following:

find / -type f -size +500000k -exec ls -lh {} \; | awk '{ print $9 ": " $5 }'

In the example above, I am looking for all files over 500MB in size (500000k, where k = kilobytes). The place where I have typed "/" in the above command indicates the path to search in. By selecting "/" I am searching in the entire filesystem; I could easily indicate a specific directory by changing my command as follows:

find /path/to/my/directory -type f -size +500000k -exec ls -lh {} \; | awk '{ print $9 ": " $5 }'

Alternatively, I could search in my current directory by replacing "/" with "." like so:

find . -type f -size +500000k -exec ls -lh {} \; | awk '{ print $9 ": " $5 }'


Friday, September 26, 2014

Patching Your Redhat Server for the Shellshock Vulnerability


Alright guys, this is a biggie. Shellshock allows remote code execution and file creation for any server relying on bash v3.4 through v1.1. If you are using Redhat or CentOS and the default shell, your server is vulnerable.

The patching history was sketchy, as well. If you patched immediately when the bug came out using CVE-2014-6271, you are still likely vulnerable (as of right now, 9/26/2013 12:50PM EST). Run the following to apply the patch:

#yum update bash

You need CVE-2014-7169 if you are using Red Hat Enterprise Linux 5, 6, and 7. Note that 2014-7169 DOES NOT address the following operating systems, which as of right now are still not fully patched: Shift_JIS, Red Hat Enterprise Linux 4 Extended Life Cycle Support, Red Hat Enterprise Linux 5.6 Long Life, Red Hat Enterprise Linux 5.9 Extended Update Support, Red Hat Enterprise Linux 6.2 Advanced Update Support, and Red Hat Enterprise Linux 6.4 Extended Update Support

If you applied CVE-2014-6271 and need the rest of the patch, reference RHSA-2014-1306

Diagnosis / Am I Vulnerable?

Copy, paste and run the following command from your shell prompt:

env 'x=() { :;}; echo vulnerable' 'BASH_FUNC_x()=() { :;}; echo vulnerable' bash -c "echo test"

If the output of the above command contains a line with only the word "vulnerable" you are still vulnerable. Depending on what version you are using and what patches you have applied, the command output will be different.

A completely vulnerable system will do this:

$ env 'x=() { :;}; echo vulnerable' 'BASH_FUNC_x()=() { :;}; echo vulnerable' bash -c "echo test" vulnerable 
bash: BASH_FUNC_x(): line 0: syntax error near unexpected token `)' 
bash: BASH_FUNC_x(): line 0: `BASH_FUNC_x() () { :;}; echo vulnerable' 
bash: error importing function definition for `BASH_FUNC_x' 

Systems patched with CVE-2014-6271 but not CVE-2014-7169 will do this:

$ env 'x=() { :;}; echo vulnerable' 'BASH_FUNC_x()=() { :;}; echo vulnerable' bash -c "echo test" 
bash: warning: x: ignoring function definition attempt 
bash: error importing function definition for `x' 
bash: error importing function definition for `BASH_FUNC_x()' 

Systems that used the RHSA-2014-1306  patch do this:

$ env 'x=() { :;}; echo vulnerable' 'BASH_FUNC_x()=() { :;}; echo vulnerable' bash -c "echo test" 
bash: warning: x: ignoring function definition attempt 
bash: error importing function definition for `BASH_FUNC_x' 

Next we have to test the file creation aspect of the Shellshock vulnerability. Execute the following command, in its entirety, from your shell:

cd /tmp; rm -f /tmp/echo; env 'x=() { (a)=>\' bash -c "echo date"; cat /tmp/echo

This is what a non-vulnerable system will provide:

$ cd /tmp; rm -f /tmp/echo; env 'x=() { (a)=>\' bash -c "echo date"; cat /tmp/echo 
cat: /tmp/echo: No such file or directory

If you're extra paranoid like me you may just want to double check that there is no file "echo" in your /tmp directory. A system that is still vulnerable will respond to the command by providing the data and time according to your system clock and creating the file. The initial output will look similar to this:

$ cd /tmp; rm -f /tmp/echo; env 'x=() { (a)=>\' bash -c "echo date"; cat /tmp/echo 
bash: x: line 1: syntax error near unexpected token `=' 
bash: x: line 1: `' bash: error importing function definition for `x' 
Fri Sep 26 11:49:58 GMT 2014

Please guys, check your servers and get this wrapped up as quickly as possible. I can't stress enough how dangerous this vulnerability is, particularly given how many administrators allow direct access to their servers through one port or another. Feel free to contact me if you have any additional questions or concerns. I am happy to help.

Billing systems development now available

Good news for current and future clients of Josh Wieder Technical Consulting : customers can now retain a variety of unique services related...