Saturday, December 3, 2016

Assigning default ownership to all new files in a directory

Getting the hang of Linux file-system permissions can be tricky for beginners. I still have problems every now and again translating symbolic permission notation to octal permission notation and back again. One common scenario which can be complicated to enact in practice is the creation of default permissions for files inside of given directories. Although not a direct translation, in Windows this sort of functionality is usually implemented by selecting the "Allow propagation on child objects" setting when viewing Security Properties for a directory. But how to get this done in Linux?

The preferred approach is the use of Access Control Lists using setfacl. Since Linux kernel 2.6, the acl flag is enabled by default with most standard filesystems. There's already several solid explanations for how to use Linux ACLs. But, there are scenarios in which this can be difficult or impossible to implement; using exotic filesystems or older kernels, etc. Or you just might find ACL syntax confusing and try to avoid it unless absolutely needed. Whatever the reason, if all you need is to have all files in a given directory owned by a specific group and/or assigned a specific set of permissions, here's a down and dirty way to do it using a set-group-id bit.

umask 002 /somedirectory/              ### With this mask default subdirectory permissions are 775
                                                         ###  and default file permissions are 664 (-rw-rw-r--)
chgrp somegroup /somedirectory/   ### here we assign our preferred group
chmod g+s /somedirectory/             ### here we assign a set-group-id bit to the directory

Keep in mind that the umask in this example is pretty wide open. If you're not familiar with umask, take a look at one of the many guides floating around. I am much more likely to use a umask of 022 or 077 in production than the above example.

Tuesday, November 8, 2016

A nasty pair of MySQL exploits grant attackers system root from any database user

Four days ago I received an email from Dawid Golunski through the list illustrating one of the more brutal pair of security vulnerabilities I have seen recently. Here's how it works.
    The exploit uses a vulnerability within MariaDB, PerconaDB (and/or XtraDB Cluster) and MySQL to, first, gain access to the 'mysql' system user using any mysql user that has CREATE / INSERT / UPDATE permissions. The first part revolves around a race condition when sql generates temporary files as part of the `REPAIR table` command. Then using the mysql system user the second vulnerability grants the attacker root access to the server using a clever hack that takes advantage of mysql_safe's approach to writing to file based error logs. Below I've provided a list of vulnerable server versions. Just about any server using the more recent (unpatched) stable releases of MySQL or MariaDB through CentOS is vulnerable (Percona isn't part of the standard CentOS repositories), with a few of caveats.
    The first caveat is that an unpatched vulnerable server can prevent at least the 2nd exploit by disabling symlinks through /etc/my.cnf using skip-symbolic-links or symbolic-links=0
    The next caveat is that the 2nd exploit also depends on using file-based mysql logging. Using syslog will avoid trouble.
    The third caveat is that for the 1st exploit to work an attacker needs a mysql user and password.
    There is some good news here. The latest stable versions of MariaDB at least disable symbolic links in my.cnf by default (its been a while since I installed MySQL through the repo but I'm fairly sure its disabled here as well). And how would an attacker get a MySQL user anyway?
    Consider that because *any* MySQL user to be used, an un-patched shared server used by a hosting company would depend on the security competency of every one of that c customers to securely handle database authentication. Not only are there a variety of exploits available for obtaining a standard database user, but its depressingly common for web designers to place their connection strings with un-encrypted database username and password into world-readable files. There are a variety of feeds and sites that scan the internet for and compile such files.
    And even without the use of the 2nd exploit, an attacker can still do an enormous amount of damage without server root with only the mysql system user. The attacker will have full access to the MySQL system files. An attacker could easily delete an entire database instance, for example.
    Of course the best part is that this is a vulnerability in MySQL itself. Upgrading MySQL is the scariest, riskiest upgrade there is among standard repo software. A lot of admins compile it from source or install it from a direct RPM (in which cases symlinks are enabled by default). And applications are closely linked with the database version. Even successful upgrades can easily break applications that run on that database as calls used by the application become deprecated. Upgrading applications has substantial costs, whether you develop the application itself or license it. A patch was already in circulation before these exploits were posted, but for all of the reasons listed above, vulnerable databases will be active for years.

Here are the impacted DB versions:

 < 5.5.52
 < 10.1.18
        < 10.0.28

 <= 5.5.51
 <= 5.6.32
 <= 5.7.14

Percona Server
 < 5.5.51-38.2
 < 5.6.32-78-1
 < 5.7.14-8

Percona XtraDB Cluster
 < 5.6.32-25.17
 < 5.7.14-26.17
 < 5.5.41-37.0

Here the first two links below contain a comprehensive breakdown of both exploits with example scripts that you can run to test.

This link includes a video illustrating how a compromise takes place using the example scripts:

Friday, August 26, 2016

Dell, how I hate thee (let me count the ways)

A Dell "feature" that appears to be designed to force customers to use only Dell parts reduced the speed of a set of SSDs one of my customers installed on their rack-mountable R900 server by a factor of 1000.

Before I get into this, there are some provisos. This server was using Linux kernel 2.6.32. The SSDs involved are Samsung 850 Pro SATA-style solid state disks. SSD is not quite ready for prime time in the 2.6.32 kernel; NVMe support was first added in 3.3, TRIM wasn't available at all until 2.6.33,
and a ton of other things we all take for granted like the device mapper are part of the 4.* kernel.

Consumer-level Samsung drivers bring their own issues. Despite what the knuckle-heads on Reddit have to say about the topic, the Linux kernel still blacklists queued TRIM functions from every Samsung SSD in the 8** series. As of the latest Github commit as of this writing for kernel 4.8 queued TRIM still doesn't work for these devices.

More importantly, the R900 isn't a new server. This is an 8 year old box. There is a SAS backplane involved which, although having a theoretical max data transfer of 3.0 Gbps, was designed before SSDs were widely available, and introduces a bunch of contacts, wiring and complexity that is likely all screwed up and almost certainly not optimized for fat-guy Peta Belly Flops of computing power.

Initial benchmarking with fio and ioping in addition to monitoring CPU iowait times with top and checking out iostat had this server's SSDs performing *slower* than a similar server with 7500 RPM sata disks in a ZFS pool.

I did a bunch of stuff to this box hoping to shake a few extra IOPS out of it. I installed Dell's dsu to get my hands on the latest drivers & firmware (under the mistaken belief that an update on either front had been released in the last decade). I had never physically seen this server; so there was a lot of lspci-ing and modprobe-ing.

Luckily, I stayed focused on the controller & backplane a SAS 6/iR (FW and LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 08) (FW 1.06), respecticely. Eventually I stumbled upon this post that described - accurately - how the Dell was automatically stepping down the SATA port speed on non-Dell-certified disks from SATA II (3gbps) to SATA I (1.5gbps).

Many moons ago, Dell's RAID cards would simply not allow users to install non-Dell disks. My experience with the R900 using BIOS version 1.2.0 would indicate that - although I am able to use non-certified disks without fatal errors, the backplane deliberately slows these disks down without reason, and in a way that is almost always transparent to the end user. I will hold off on making accusations here until I get my hands on the source code for this firmware, but the evidence up to this point is fairly damning. If anyone from Dell has an explanation for this sort of behavior, I would be happy to publish your feedback here.

There is a workaround for this issue, albeit its incredibly hack-y. It involves the use of the (now defunct) lsiutil application (available here or direct mirror here or you can now find the Linux packages on my GitHub page).  This application allows us to make calls directly to the backplane. In this case, the fix involves resetting the minimum link speed on the back plane from 1.5Gbps to 3.0Gbps. 

Heres a step-by-step:

   - download the zip file
   - unzip the file in a directory of your choice; # unzip -d /home/joshw/lsiutil/
   - navigate to the directory referencing your OS; # cd /home/joshw/lsiutil/Linux/
   - identify the version of the application matching your processor/OS bit type. For linux, there is a 32 bit, AMD64 and x86_64 version. I selected the x86_64 and applied an executable bit: # chmod +x lsiutil.x86_64
   - make sure youre root: # sudo su
   - run the application: # ./lsiutil.x86_64

You should see something like this:

# ./lsiutil.x86_64

LSI Logic MPT Configuration Utility, Version 1.62, January 14, 2009

1 MPT Port found

     Port Name         Chip Vendor/Type/Rev    MPT Rev  Firmware Rev  IOC
 1.  /proc/mpt/ioc0    LSI Logic SAS1068E B3     105      00192f00     0

Select a device:  [1-1 or 0 to quit]

 Most likely you will only have one option here, so select it by pressing 1:

Select a device:  [1-1 or 0 to quit] 1

 1.  Identify firmware, BIOS, and/or FCode
 2.  Download firmware (update the FLASH)
 4.  Download/erase BIOS and/or FCode (update the FLASH)
 8.  Scan for devices
10.  Change IOC settings (interrupt coalescing)
13.  Change SAS IO Unit settings
16.  Display attached devices
20.  Diagnostics
21.  RAID actions
22.  Reset bus
23.  Reset target
42.  Display operating system names for devices
45.  Concatenate SAS firmware and NVDATA files
59.  Dump PCI config space
60.  Show non-default settings
61.  Restore default settings
66.  Show SAS discovery errors
69.  Show board manufacturing information
97.  Reset SAS link, HARD RESET
98.  Reset SAS link
99.  Reset port
 e   Enable expert mode in menus
 p   Enable paged mode
 w   Enable logging

From here select option 13

Main menu, select an option:  [1-99 or e/p/w or 0 to quit] 13

You will be immediately prompted with some configuration questions. Just press RETURN to keep the current / default values:
SATA Maximum Queue Depth:  [0 to 255, default is 8]
Device Missing Report Delay:  [0 to 2047, default is 0]
Device Missing I/O Delay:  [0 to 255, default is 0]

Eventually you will be dumped out here:

PhyNum  Link      MinRate  MaxRate  Initiator  Target    Port
   0    Enabled     1.5      3.0    Enabled    Disabled  Auto
   1    Enabled     1.5      3.0    Enabled    Disabled  Auto
   2    Enabled     1.5      3.0    Enabled    Disabled  Auto
   3    Enabled     1.5      3.0    Enabled    Disabled  Auto
   4    Enabled     1.5      3.0    Enabled    Disabled  Auto
   5    Enabled     1.5      3.0    Enabled    Disabled  Auto
   6    Enabled     1.5      3.0    Enabled    Disabled  Auto
   7    Enabled     1.5      3.0    Enabled    Disabled  Auto

Select a Phy:  [0-7, 8=AllPhys, RETURN to quit]

Select 8 to make changes to all of the available ports simultaneously

Select a Phy:  [0-7, 8=AllPhys, RETURN to quit] 8

Again, you will be prompted for several values. You want to be very careful here as we only want to change one value - MinRate (this should be the second value you are prompted to modify. Every other value should remain default by pressing RETURN.

Link:  [0=Disabled, 1=Enabled, or RETURN to not change]
MinRate:  [0=1.5 Gbps, 1=3.0 Gbps, or RETURN to not change] 1
MaxRate:  [0=1.5 Gbps, 1=3.0 Gbps, or RETURN to not change]
Initiator:  [0=Disabled, 1=Enabled, or RETURN to not change]
Target:  [0=Disabled, 1=Enabled, or RETURN to not change]
Port configuration:  [1=Auto, 2=Narrow, 3=Wide, or RETURN to not change]

Once you've finished you will be dumped back to the port menu:

PhyNum  Link      MinRate  MaxRate  Initiator  Target    Port
   0    Enabled     3.0      3.0    Enabled    Disabled  Auto
   1    Enabled     3.0      3.0    Enabled    Disabled  Auto
   2    Enabled     3.0      3.0    Enabled    Disabled  Auto
   3    Enabled     3.0      3.0    Enabled    Disabled  Auto
   4    Enabled     3.0      3.0    Enabled    Disabled  Auto
   5    Enabled     3.0      3.0    Enabled    Disabled  Auto
   6    Enabled     3.0      3.0    Enabled    Disabled  Auto
   7    Enabled     3.0      3.0    Enabled    Disabled  Auto

Press RETURN from here to save your changes.

Select a Phy:  [0-7, 8=AllPhys, RETURN to quit]

You'll be prompted again for some other values; again keep the defaults or current values by pressing RETURN:

Persistence:  [0=Disabled, 1=Enabled, default is 1]
Physical mapping:  [0=None, 1=DirectAttach, 2=EnclosureSlot, default is 2]
Number of Target IDs to reserve:  [0 to 32, default is 8]

This will take you back to the main menu. Select 0 from here to save & quit:

Main menu, select an option:  [1-99 or e/p/w or 0 to quit] 0

... Which takes you back to the device menu. Hit 0 again and you are finally done:

     Port Name         Chip Vendor/Type/Rev    MPT Rev  Firmware Rev  IOC
 1.  /proc/mpt/ioc0    LSI Logic SAS1068E B3     105      00192f00     0

Select a device:  [1-1 or 0 to quit] 0
root at someServer in /home/joshw/Linux

Here are some real-world benchmarks using fio showing before & after metrics.



fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 4096MB)
Jobs: 1 (f=1): [m(1)] [100.0% done] [21804KB/7072KB/0KB /s] [5451/1768/0 iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=490: Fri Aug 26 13:37:42 2016
  read : io=3071.7MB, bw=20940KB/s, iops=5235, runt=150207msec
  write: io=1024.4MB, bw=6983.2KB/s, iops=1745, runt=150207msec
  cpu          : usr=1.79%, sys=11.75%, ctx=786417, majf=0, minf=24
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=786347/w=262229/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64


fio --name=randwrite --ioengine=libaio --iodepth=1 --rw=randwrite --bs=4k --direct=0 --size=64M --numjobs=32 --runtime=60 --group_reporting --iodepth=16
randwrite: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
Starting 32 processes
Jobs: 28 (f=28): [_(2),w(16),_(1),w(7),E(1),w(5)] [89.8% done] [0KB/311.7MB/0KB /s] [0/79.8K/0 iops] [eta 00m:05s]
randwrite: (groupid=0, jobs=32): err= 0: pid=4447: Fri Aug 26 13:05:05 2016
  write: io=2048.0MB, bw=47437KB/s, iops=11859, runt= 44209msec
    slat (usec): min=19, max=13715K, avg=2573.62, stdev=166998.51
    clat (usec): min=5, max=13728K, avg=38699.79, stdev=646546.36
     lat (usec): min=26, max=13729K, avg=41273.42, stdev=667725.13
    clat percentiles (usec):
     |  1.00th=[  454],  5.00th=[  540], 10.00th=[  580], 20.00th=[  636],
     | 30.00th=[  692], 40.00th=[  756], 50.00th=[  868], 60.00th=[ 1160],
     | 70.00th=[ 5536], 80.00th=[11328], 90.00th=[17536], 95.00th=[24704],
     | 99.00th=[41216], 99.50th=[52480], 99.90th=[12779520], 99.95th=[13697024],
     | 99.99th=[13697024]
    lat (usec) : 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%, 250=0.02%
    lat (usec) : 500=2.57%, 750=36.65%, 1000=17.06%
    lat (msec) : 2=8.94%, 4=3.35%, 10=8.24%, 20=15.22%, 50=7.37%
    lat (msec) : 100=0.29%, 250=0.01%, >=2000=0.26%
  cpu          : usr=0.13%, sys=2.11%, ctx=67366, majf=0, minf=982
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=99.9%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=524288/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=16



test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 4096MB)
Jobs: 1 (f=1): [m] [100.0% done] [74324K/24284K/0K /s] [18.6K/6071 /0  iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=59026: Fri Aug 26 19:10:52 2016
  read : io=3070.6MB, bw=74180KB/s, iops=18545 , runt= 42386msec
  write: io=1025.5MB, bw=24775KB/s, iops=6193 , runt= 42386msec
  cpu          : usr=8.79%, sys=55.13%, ctx=796212, majf=0, minf=20
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=786053/w=262523/d=0, short=r=0/w=0/d=0

Jobs: 32 (f=32)
randwrite: (groupid=0, jobs=32): err= 0: pid=36168: Fri Aug 26 17:09:26 2016
  write: io=2048.0MB, bw=1168.1MB/s, iops=299251 , runt=  1752msec
    slat (usec): min=7 , max=21074 , avg=91.01, stdev=468.57
    clat (usec): min=7 , max=21729 , avg=697.25, stdev=1302.17
     lat (usec): min=15 , max=21799 , avg=789.95, stdev=1385.85
    clat percentiles (usec):
     |  1.00th=[  118],  5.00th=[  390], 10.00th=[  482], 20.00th=[  506],
     | 30.00th=[  524], 40.00th=[  540], 50.00th=[  556], 60.00th=[  580],
     | 70.00th=[  588], 80.00th=[  604], 90.00th=[  620], 95.00th=[  636],
     | 99.00th=[10688], 99.50th=[10688], 99.90th=[14656], 99.95th=[20608],
     | 99.99th=[20864]
    bw (KB/s)  : min=24175, max=50576, per=3.10%, avg=37155.51, stdev=7603.34
    lat (usec) : 10=0.01%, 20=0.01%, 50=0.01%, 100=0.27%, 250=2.72%
    lat (usec) : 500=13.04%, 750=82.28%, 1000=0.11%
    lat (msec) : 2=0.04%, 4=0.05%, 10=0.20%, 20=1.20%, 50=0.07%
  cpu          : usr=7.61%, sys=70.17%, ctx=2186, majf=0, minf=913
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=524288/d=0, short=r=0/w=0/d=0

Monday, August 15, 2016

Building a new gaming PC (with some digs about NZXT)

Never let it be said that I do not support the aspirations of today's young people. My contribution to the next generation was helping a local teen build out a high powered gaming PC. It was my first time installing a closed-loop liquid cooling system (the Kraken x61). 

Building a gaming rig. Notice how decades of IT work has resulted in a Quasimodo hunch
The highlights of the PC included the following:

    - Intel i7 6700K CPU
    - NZXT Kraken x61 liquid cooling system
    - Nvidia GeForce GTX 1080 graphics card
    - ASUS Z170-E motherboard
    - NZXT S340 case
    - Corsair Vengeance LPX DDR4 RAM
    - Samsung EVO 850 SSD
    - EVGA Fully Modular GQ 650W power supply

During build-out I encountered two issues that weren't the result of my own fumbling, shaky hands. One of these issues I think is forgivable and the other is not.

The ASUS 7170-E motherboard and its associated BIOS is specifically designed with gaming and overclocking in mind. I'll save the overclocking for another time; what I had problems with is the BIOS' "QFan" monitoring system, which was unable to recognize the Kraken x61 CPU fan. Both the BIOS and motherboard appears to have more of a focus on open loop cooling systems - the 7170-E has a dedicated on-board four port connector labeled "WPUMP" for water pumps, and the QFan monitoring within the BIOS treats CPU fans and pumps as separate entities.

Booting with default BIOS settings continuously failed with an error message complaining of an invalid RPM lower limit assigned to the CPU Fan, and that I should resolve this by either disabling that lower limit within the BIOS or confirming that the CPU Fan was attached to the on-board CPU Fan Header. Of course, neither of these resolved the problem. I tried a ton of alternatives - plugging the CPU fan into the WPUMP header, changing the fan type from "auto" to "PWM", etc - without any success. Eventually I was forced to resort to Google, where I found a post on a message board suggesting to completely disable QFan's CPU Fan monitoring functionality (you can do this by pressing F9 to go the Advanced Setup menu, then goto Monitor, scroll down to CPU Fan monitoring and press space to disable). Doing this resolved the issue.

Once I had Windows installed and I had installed NZXT's CAM application (through which I was able to manage and monitor CPU Fan speed and other metrics without any problems), I decided to update the BIOS in one more attempt to resolve this whole QFan business. I can't remember the BIOS versions involved off hand other than that I know it was a major release - something similar to a leap from 0230 to 1205. I was pleasantly surprised to see that immediately after upgrading the BIOS the x61 CPU fan was recognized. With this confirmed, I powered off the system one last time to do some last minute-cable management (I'll post a pic at some point). However, when I rebooted a second time, the error reappeared even though the BIOS upgrade was successful - strangely, the BIOS screen became heavily pixelated At that point I gave up and disabled QFan again which once again got be through POST without issues (the pixelation disappeared).

This obstacle didn't bother me; I'm not sure whether the issue with the Kraken or with the ASUS BIOS, but the fix was simple enough (even if it took me a while to figure out). The second obstacle I ran into, also related to an NZXT part, was more frustrating.

We went with the Kraken x61 fan for two reasons. First - because its claimed stats for air circulation along with its claimed decibel rating were among the best in its price range. And second - the case preferred by my gamer customer was an NZXT S340, so I rubbed both of my brain cells together and reasoned that NZXT coolers are bound to fit more easily into NZXT cases than other comparable coolers. Of course, I didn't reach this conclusion on my own. NZXT's documentation for their mid-size S340 case clearly says "Full 280mm radiator support for the latest Kraken cooler".

So let me be the first to say that, no, a 280mm radiator will not fit within the NZXT S340. There is, in fact, 280mm of fan space. You can easily install two separate 140mm fans. And you can also install a 280mm radiator in addition to those fans (after physically modifying the case). What will not fit as the case is designed is the two thick rubber tubes that are attached to every cooling radiator on the market. Compounding the bullsh*t nature of the claim is that NZXT only manufactures closed-loop coolers, and the x61 is the only 280mm cooler that NZXT makes.

I did get it to work (that's why I make the big bucks), but the installation should have been much easier. There are conceivably a few different ways to do this, but here is what I did:

     - I removed the front panel and used a pair of wire cutters to remove the series of small,
        completely pointless plastic tabs inside the panel that prevent the case from closing with the
        Kraken radiator inside.

     - There are two metal holes in the front of the S340 case where the fans are designed to be
        mounted. I threaded the CPU cooler and tubes through the top-most hole. This, in turn, makes it         impossible to mount the fans where they are designed.

     - I then proceeded to install the top fan into the top hole, leaving enough space for the tubes. This
        means that only two screws can be used to secure the fans instead of 8. These two screws will             connect the bottom of the top fan where the top of the bottom fan was designed to be screwed in.

     - This leaves *just* enough clearance for the bottom fan to be squeezed into the remaining space.
        To avoid shaking, I installed a pair of adhesive bumper strips to the bottom of this fan. Instead of         using screws, a series of zip ties can be used to make sure the fans stay in place. This, in addition         to gravity and pressure, will keep the fans and radiator in place.

This is very much a hack. There are other options for getting this to work, but the list is narrowed by the very small amount of clearance in the front of the server and the radiator itself.

Is it possible that I overlooked something significant in the install of the cooler with this case? Absolutely. If that is in fact the case, and I made a mistake, I'm still going to blame NZXT - because the case provided no documentation for how to install a 280mm cooler, and the documentation for the cooler only included instructions for larger cases (where the cooler is installed on top).

Anyway, I hope this post helps save readers some time and prevents a few headaches.

Sunday, August 14, 2016

Pandora account compromise warning message

Here is a copy of the email I was sent by Pandora to inform me that my account was compromised kindof but not really and it was totally not their fault.

Pandora account compromise confirmation

This is somewhat old news (I received this email July 6th) but the more copies of this online the better, IMO.

There are a number of things about this email that irritate me. First of all, the email is so incredibly vague that I have absolutely no idea what happened. Someone, somewhere posted my Pandora username (email address?) on the internet along with, presumably, one of the bazillion passwords associated with it. Who posted this information? Why? Where was it taken from? Was it stolen from one of Pandora's infrastructure providers?

If what Pandora implies in the email is true - that the compromise is completely unrelated to Pandora in any way - why are they sending me this email? Does Pandora scour the internet for the email addresses and account names of its many users? If Pandora had no responsibility for this breach and they sent me this message in order to be proactive to protect me - which is great - then why couldn't they be more forthcoming with detailed information? I get that many of Pandora's users are going to be non-technical, but you can include a link to a website with a comprehensive explanation of what happened or simply format the email to begin with a "tl;dr" version, followed by an exhaustive version for nerds.

There are no hard and fast rules for dealing with a compromise, but Pandora's message left me with many more questions than answers.

Thursday, August 4, 2016

Stay classy, Microsoft

Someone more cynical than myself might think that Microsoft's sudden 66% decrease of OneDrive storage space is a bait & switch - give away the space for free until users become dependent, than take it away, threaten to delete it, forcing those who have become accustomed to the free service to pony up and pay.

NSA Leak Bust Points to State Surveillance Deal with Printing Firms

Earlier this week a young government contractor named Reality Winner was accused by police of leaking an internal NSA document to news outle...