tag:blogger.com,1999:blog-44117205046085053632024-02-20T10:51:43.801-05:00Josh Wiederadmin, dev, curmudgeon.Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comBlogger296125tag:blogger.com,1999:blog-4411720504608505363.post-79742776002305114702023-03-19T18:44:00.001-05:002023-03-19T18:44:11.092-05:00Facebooks IP block is scanning home networks<p> <table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjhPCynLY-d_ZoXpB-R4sdbKUs5nf6WRacULr2YsTh2hGQpc1rn1V4kB5Hd3OjJMEuG2km6u9iwx5nX0cJpDVjCgKogEaWRyf8kxAmS9VLSKK5VCTh1jMx3pxdFmG9Ui-tXjSsPiVW4HyA7h4_pfGjEZaWc8X9uZ4nsBe63sB3_OR_6VMvyZ5c_-aXA/s1040/Screenshot%202023-03-19%20at%207.38.07%20PM.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="362" data-original-width="1040" height="172" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjhPCynLY-d_ZoXpB-R4sdbKUs5nf6WRacULr2YsTh2hGQpc1rn1V4kB5Hd3OjJMEuG2km6u9iwx5nX0cJpDVjCgKogEaWRyf8kxAmS9VLSKK5VCTh1jMx3pxdFmG9Ui-tXjSsPiVW4HyA7h4_pfGjEZaWc8X9uZ4nsBe63sB3_OR_6VMvyZ5c_-aXA/w496-h172/Screenshot%202023-03-19%20at%207.38.07%20PM.png" width="496" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Residential ISP log data from the east coast US<br /><br /></td></tr></tbody></table>Mark: if you're reading this I need you to get right on this issue please. (Thanks!)</p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;"># whois.ripe.net</span></p><p class="p2" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 13px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;"></span><br /></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">inetnum:<span class="Apple-converted-space"> </span>31.13.64.0 - 31.13.127.255</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">netname:<span class="Apple-converted-space"> </span>IE-FACEBOOK-20110418</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">country:<span class="Apple-converted-space"> </span>IE</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">org:<span class="Apple-converted-space"> </span>ORG-FIL7-RIPE</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">admin-c:<span class="Apple-converted-space"> </span>NE1880-RIPE</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">tech-c: <span class="Apple-converted-space"> </span>NE1880-RIPE</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">status: <span class="Apple-converted-space"> </span>ALLOCATED PA</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">mnt-by: <span class="Apple-converted-space"> </span>RIPE-NCC-HM-MNT</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">mnt-by: <span class="Apple-converted-space"> </span>meta-mnt</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">mnt-routes: <span class="Apple-converted-space"> </span>fb-neteng</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">created:<span class="Apple-converted-space"> </span>2011-04-18T12:00:34Z</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">last-modified:<span class="Apple-converted-space"> </span>2022-10-29T00:51:39Z</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">source: <span class="Apple-converted-space"> </span>RIPE # Filtered</span></p><p class="p2" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 13px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;"></span><br /></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">organisation: <span class="Apple-converted-space"> </span>ORG-FIL7-RIPE</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">org-name: <span class="Apple-converted-space"> </span>META PLATFORMS IRELAND LIMITED</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">country:<span class="Apple-converted-space"> </span>IE</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">org-type: <span class="Apple-converted-space"> </span>LIR</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">address:<span class="Apple-converted-space"> </span>4 GRAND CANAL SQUARE, GRAND CANAL HARBOUR</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">address:<span class="Apple-converted-space"> </span>462129</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">address:<span class="Apple-converted-space"> </span>Dublin</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">address:<span class="Apple-converted-space"> </span>IRELAND</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">phone:<span class="Apple-converted-space"> </span>+0016505434800</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">fax-no: <span class="Apple-converted-space"> </span>+0016505435325</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">admin-c:<span class="Apple-converted-space"> </span>PH4972-RIPE</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">mnt-ref:<span class="Apple-converted-space"> </span>RIPE-NCC-HM-MNT</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">mnt-ref:<span class="Apple-converted-space"> </span>meta-mnt</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">mnt-by: <span class="Apple-converted-space"> </span>RIPE-NCC-HM-MNT</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">mnt-by: <span class="Apple-converted-space"> </span>meta-mnt</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">abuse-c:<span class="Apple-converted-space"> </span>RD4299-RIPE</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">created:<span class="Apple-converted-space"> </span>2011-04-07T13:16:29Z</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">last-modified:<span class="Apple-converted-space"> </span>2022-10-29T00:51:40Z</span></p><p class="p1" style="font-family: Menlo; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size: 11px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-variant-ligatures: no-common-ligatures;">source: <span class="Apple-converted-space"> </span>RIPE # Filtered</span></p>Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-64797794099179126932021-10-10T23:35:00.024-05:002021-10-11T16:15:19.428-05:00EC2 swap device management & fixing "swapoff failed: Cannot allocate memory"<p> One of the sillier things I've done as an AWS/linux admin is provision an EBS disk as swap to an EC2 instance. I kept getting max allocate errors for a script I needed to run to execute a series of database queries. Reprovisioning to a new EC2 instance class with more RAM wasn't feasible at the time for some long-forgotten reason. </p><p>I would never do this if I owned the disks - provisioning swap to SSD will greatly reduce the lifetime of the disk, among many reasons why this is less than ideal. But Amazon has plenty of money. I figured I could cheaply provision an EBS volume & buy myself enough swap to complete the query. Then, in some point in the future, I could create a more beautimous solution.</p><p>Well, if you're a sysadmin you know how this story ends. I moved onto other fires/projects, quickly forgot about the swap situation, and here I am years later, deprovisioning the server, in all its swappy glory.</p><p>This wouldn't warrant a blog post, except for the fact that I received an error when trying to disable swap using "swapoff -a":</p>
<code>swapoff failed: Cannot allocate memory</code>
<p>In this case, the swap had about 750MB of swap in use, and this tiny little EC2 Nano instance only had about 5MB of free RAM. In order for me to detach the EBS swap device, I needed a temporary place to store the swap that is currently in use <i>assuming that the server must stay online. </i>Another option would have been to have edited my /etc/fstab file to comment out the line binding the EBS UUID to /dev/swap and rebooting. </p><p>Another method of resolving the issue is to shift the used swap space to a temporary swap file. This does not require a reboot and allows me to reclaim the EBS device immediately. That's what I opted to do, and here is how I did that:</p><p>First, you need to find the path to the swap device or file. Because I am a dummy and used an EBS device for this purpose, I could easily find that path using the blkid command (note I am censoring command output, so your output will look different than mine):</p>
<code>
$ sudo blkid<br />
/dev/nvme1n1: UUID="****" TYPE="swap"<br />
[...]
</code><br /><br />
<div>I also need to determine how much of the swap device is being consumed. The swap file I create must be at least that size. NOTE: It is <b><i>not</i></b> necessary for the new swap file to match the size of the prior swap device, or to use the same size blocks as the swap device. Because this is the case I can easily find the size using the "free" command:</div>
<br />
<div><code><div>$ free -h</div><div> total used free shared buff/cache available</div><div>Mem: 464M 202M 5.5M 15M 260M 233M</div><div>Swap: 8.0G 816M 7.2G</div></code></div>
<br />
<div>I then proceed to create the temporary swap file. I went a bit bigger than I needed to, creating a 1GB swap file. Its generally a good idea to give a bit of wiggle room here to be on the safe side. Note how in the dd command below, I am specifying a count of 1,024,000 blocks, each 1024 bytes in size. Multiple those numbers and you get 1048576000 bytes or 1GB:</div><div><div></div>
<br />
<code><div>$ sudo dd if=/dev/zero of=/home/swap bs=1024 count=1024000</div><div>1024000+0 records in</div><div>1024000+0 records out</div><div>1048576000 bytes (1.0 GB) copied, 6.8052 s, 154 MB/s</div></code></div>
<br />
<div>Just to confirm blocksize isn't required to match, my old EBS swap device used blocks half that size (there are other situations where blocksize is very relevant, just not here):</div><div><br /></div>
<div><code><div>$ sudo fdisk -l /dev/nvme1n1</div><div>Disk /dev/nvme1n1: 8589 MB, 8589934592 bytes, 16777216 sectors</div><div>Units = sectors of 1 * 512 = 512 bytes</div><div>Sector size (logical/physical): 512 bytes / 512 bytes</div><div>I/O size (minimum/optimal): 512 bytes / 512 bytes</div><div><br /></div></code><div></div></div><div>Lock down permissions for the new swap file:</div><div><br /></div><div><code>$ sudo chmod 0600 /home/swap</code><p>Use 'mkswap' to properly configure the file for swapping:</p><p></p><code><p>$ sudo mkswap /home/swap</p><p>Setting up swapspace version 1, size = 1023996 KiB</p><p>no label, UUID=1f3f78ed-8e5d-4672-bbd1-59e6f29c8b08</p></code><p> And tell the host to use the new file with 'swapon':</p><p></p><code>$ sudo swapon /home/swap</code><p>Unless you use the -v flag, a successful swapon will not return output to the command line. Its still a good idea to check the "free" command to make sure that the new swap has been applied:</p><p></p><p></p><code><p>$ free -h</p><p> total used free shared buff/cache available</p><p>Mem: 464M 198M 5.5M 15M 260M 237M</p><p>Swap: 9.0G 734M 8.3G</p></code><p></p><p></p><p>Notice the total swap amount: 9.0G. This number includes the 1GB file we just created as well as the 8GB swap device that already existed. Linux does not show two separate swap devices in this context, the same way it would not show two different sticks of RAM in this context.</p><p>Finally, we are ready to disable the old swap device. Here I am using the -v flag to look out for additional information in the event of an error. But the command was successful, so swapoff simply prints back my command when it completes: </p><p></p><code>$ sudo swapoff -v /dev/nvme1n1</code><code>swapoff /dev/nvme1n1</code><p>Running the free command again, we can see that the total Swap value has been reduced by about 8GB</p><code>$ free -h</code><code> total used free shared buff/cache available</code></div><div><code>Mem: 464M 390M 5.9M 27M 67M 34M</code><p></p><code>Swap: 999M 524M 475M</code><p>Looks good. I'm not quite done yet, though. Even though I don't plan to reboot immediately, I still need to remove the /etc/fstab entry for the old swap device. I used a text editor to add a hash symbol (#) to the line relevant to the swap device:</p><div><div><code>#/dev/nvme1n1 none swap sw 0 0</code></div></div><div><code><br /></code></div><div>Before taking the final step of running "swapoff" and deleting the new swap file, take a moment to double-check the settings for RAM-hungry applications and their memory settings.</div><div><br /></div><div>Running MySQL? Check out your innodb_buffer_pool_size & innodb_log_file_size settings in my.cnf (or my.cnf.d/server.cnf). </div><div><br /></div><div>Running PHP CGIs? Check out your memory_limit settings. </div><div><br /></div><div>In my case, removing a few unused databases and resetting innodb_buffer_pool_size to reflect the current available memory made an enormous difference. Here is the output of "free" immediately after those changes; those changes freed up over 500MB of memory ... not bad for a system with, well, 500MB of RAM.</div><div><br /></div><div><div></div><code><div>$ free -h</div><div> total used free shared buff/cache available</div><div>Mem: 464M 238M 77M 14M 149M 199M</div><div>Swap: 999M 84M 915M</div><div><br /></div></code><div></div></div><div><div>But there are memory-relevant settings in the kernel itself, too. Check out the kernel's swappiness value as shown below:</div><div><br /></div><div><div></div><code><div>$ sysctl vm.swappiness</div><div>vm.swappiness = 10</div><div><br /></div></code><div></div></div><div>The higher the swappiness value, the more inclined the kernel is to swap memory out of RAM. You want a low value here if you plan to disable swap.</div><div><br /></div><div>You will also want to check out the vm.vfs_cache_pressure kernel setting. This value determines how the kernel reclaims memory from swap. From the kernel documentation:</div><div><br /></div>
<code>At the default value of vfs_cache_pressure=100 the kernel will attempt to
reclaim dentries and inodes at a "fair" rate with respect to pagecache and
swapcache reclaim. Decreasing vfs_cache_pressure causes the kernel to prefer
to retain dentry and inode caches. When vfs_cache_pressure=0, the kernel will
never reclaim dentries and inodes due to memory pressure and this can easily
lead to out-of-memory conditions. Increasing vfs_cache_pressure beyond 100
causes the kernel to prefer to reclaim dentries and inodes.
Increasing vfs_cache_pressure significantly beyond 100 may have negative
performance impact. Reclaim code needs to take various locks to find freeable
directory and inode objects. With vfs_cache_pressure=1000, it will look for
ten times more freeable objects than there are.</code></div><div><span style="font-family: monospace;"><br /></span>
<div>As it turns out, I was unable to "swapoff" the temporary swap file until I modified these values, which had been customized as part of MariaDB performance tuning that long ago stopped being relevant:</div>
<br />
<div><code><div># swapoff /home/swap</div><div>swapoff: /home/swap: swapoff failed: Cannot allocate memory</div><div><br /></div></code><div></div></div><div>Typically, permanent changes to kernel parameters are made in /etc/sysctl.conf. I modified my own values as follows:</div><div><br /></div><div><div></div><code><div>vm.swappiness=1 ### changed from a value of 10</div><div>vm.vfs_cache_pressure=100 ### changed from a value of 200</div></code></div></div>
<br />
<div>So a quick review: at this point, all swaps have been disabled, and thanks to the tweaks above, the host no longer requires swap to meet its memory requirements, as can be seen in free:</div><div><br /></div><div>
<code># free -h<div> total used free shared buff/cache available</div><div>Mem: 464M 282M 66M 46M 116M 123M</div><div>Swap: 0B 0B 0B</div><div><br /></div>
</code><div></div></div>
<div>However, I am still paying Amazon for that silly EBS device. If we just absent-mindedly follow the Amazon documentation for detaching EBS devices, we would find the device name for the swap volume and umount it.</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJKd_5GhIA3Raut3SxJ8jQu5r5fsMuoFIAVYFsz9bz-vlvP4fA9MohjorV8iO5SyTYDNrF2K-vACgvmt2GGSkBKbGms4vacu5Km9kBTFBToq-b3FBX7Ffv_Jv_rQTAfe8rBnI3lEPuYWs/s1142/block-devs.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="298" data-original-width="1142" height="168" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJKd_5GhIA3Raut3SxJ8jQu5r5fsMuoFIAVYFsz9bz-vlvP4fA9MohjorV8iO5SyTYDNrF2K-vACgvmt2GGSkBKbGms4vacu5Km9kBTFBToq-b3FBX7Ffv_Jv_rQTAfe8rBnI3lEPuYWs/w640-h168/block-devs.jpg" width="640" /></a></div><br /><div>We can see in the screenshot that the smaller 8GB volume is /dev/sdf - that is the swap device. But if we try to umount that device, Linux cannot find it:</div><div><br /></div><div><div></div><code><div>$ sudo umount -d /dev/sdf</div><div>umount: /dev/sdf: mountpoint not found</div><div><br /></div></code><div></div></div><div>This is because Amazon never planned for users to do something as silly as mount an EBS as swap. Our earlier commands reference /dev/nvme1n1. If we try to umount that, we can see it is no longer mounted:</div><div><br /></div><div><div></div><code><div># umount -d /dev/nvme1n1</div><div>umount: /dev/nvme1n1: not mounted</div></code><p>The point is, we already took care of unmounting the device when we eliminated the swap partition. So we can skip directly to detaching the volume using the AWS console. Be careful to select the correct Volume ID (its a good idea to use a Name tag to avoid mistakes).</p><p>And that's it. What could be easier? :/ </p><div></div></div><p></p></div>Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-24836390800098134892021-10-10T21:22:00.008-05:002021-10-11T20:06:41.091-05:00S3 static webhosting, DKIM signature size errors & why DNS prefers UDP<p>This weekend I spent some time migrating a few low-traffic websites from Nginx to AWS S3's static web hosting service.</p><p>In theory, this is a straightforward process: move content from the old webroot to an S3 bucket that shares the name of the domain, enable static web hosting for the bucket & set a security policy that enables anonymous web users to see that content.</p><p>In practice, there's a bit more involved:</p><p>1. S3 bucket resource paths can change, which will result in DNS failures unless you use a Route 53 hosted zone. You don't need to buy a domain from Amazon to do this, but you do need to use their nameservers. This isn't free, and there is an extra fee for DNSSEC.</p><p>2. Want an SSL/TLS certificate? Of course you do. This means generating a certificate within Amazon Certificate Manager. In most circumstances (without "legacy" client support for example), there is no charge for the certificate. But to serve traffic using that certificate requires provisioning a web distribution to Amazon's CDN, CloudFront. </p><p>3. If you are a <b><i>power user</i></b> (ha) you might even have some honest-to-god web applications, like with a CGI. S3 can't handle this on its own, but there are tools that can address this while hosting with S3. Some folks like to host, say, their Wordpress site within S3 by running the installation process in a "normal" webserver, than migrating the files to S3 (the database can stay where it is or move to RDS). But that is a hack, and not a great one. S3 still will not run CGIs, and its hard to see the value of Wordpress without being able to use any of its functionality or run a template that cannot run PHP. For my use case, I was dealing with small business card-type websites - there is no authentication, but there might be some simple script functionality, like a contact form. Fortunately, AWS' serverless architecture tools enable me to fairly quickly work around this using a combination of AWS Lambda, Amazon API Gateway & Amazon SES. There is a great blog post detailing <a href="https://aws.amazon.com/blogs/architecture/create-dynamic-contact-forms-for-s3-static-websites-using-aws-lambda-amazon-api-gateway-and-amazon-ses/">this process here</a>, The workflow looks like this:</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOGVQCGrUyrTKV4NYpenOajIvsXhEcfEmlmVX8TX5_neICRcXHtw3s8JwkyXOu3Xqn8VgP0T5_ezO8kXqTp3TGxmTuxlmq6hz2LilkGPr_F5BQ2ml35GIKxtw_19bH67nQfnCPzm2KyUw/" style="margin-left: 1em; margin-right: 1em;"><img data-original-height="220" data-original-width="820" height="173" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOGVQCGrUyrTKV4NYpenOajIvsXhEcfEmlmVX8TX5_neICRcXHtw3s8JwkyXOu3Xqn8VgP0T5_ezO8kXqTp3TGxmTuxlmq6hz2LilkGPr_F5BQ2ml35GIKxtw_19bH67nQfnCPzm2KyUw/w640-h173/image.png" width="640" /></a></div><br />So, all of this is well and good and didn't provide me with too many obstacles (except for the one domain I missed the DNSSEC DS record deletion for).<p></p><p>The only error I encountered during this process was a bit of a surprise: I was unable to directly transfer my pre-existing DKIM signatures to Route 53. When I did so, I was greeted with the error "CharacterStringTooLong (Value is too long) encountered with {Value}". An example of this error is pictured here:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8Fk_3KWbMnJY6nh_GUEoZGP2jDRh9mEkosZWsye0XyvDur576pUp2OqIY56iwKVWiEF18TbnSmq-izP0WEiw-QICUZaKfPXzPWH1qW4Q1P9GZwpPPRSTYbXEW_l4AII7k2TZV2ONXCmE/s1807/dkim-toolong.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="866" data-original-width="1807" height="306" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8Fk_3KWbMnJY6nh_GUEoZGP2jDRh9mEkosZWsye0XyvDur576pUp2OqIY56iwKVWiEF18TbnSmq-izP0WEiw-QICUZaKfPXzPWH1qW4Q1P9GZwpPPRSTYbXEW_l4AII7k2TZV2ONXCmE/w640-h306/dkim-toolong.jpg" width="640" /></a></div><p>The problem is that there is an RFC-mandated 255 character maximum for TXT record values, and a 2048 bit DKIM signature requires more characters than that. This is not a limitation that is unique to Route 53 or Amazon. The 255 character convention was established in <a href="https://datatracker.ietf.org/doc/html/rfc1035">RFC1035</a> ... in 1987. This isn't an Amazon thing. So why do so many people first encounter this error in Route 53?</p><p>In most circumstances, users do not modify DNS zone files directly. Instead, they use some sort of application or interface created by the domain registrar to modify those zones. Those interfaces tend to reconfigure user input into an RFC-compliant zone on the backend. There are positives to this (DNS confuses the hell out of a lot of people) and negatives (its a crutch that can lead to fundamental misunderstandings re: zone format). IMO, its fine that different registrars cater to different customer skill levels, but I would prefer if registrars that seamlessly modify user zone input were more forth-coming about what is occurring. For example, it wouldn't be to hard to have some sort of javascript break up TXT record user input into smaller blocks after a user has completed entering the record. This would be trivial to implement, not require users to know TXT length rules but would also help to educate users.</p><p>Anyway, that's all very interesting. How do we get a working DKIM signature in Route 53, though?</p><p>One option is to use DKIM with a 1024 bit key. Is this as good as a 2048 bit key? No. But it is <i>much</i> better than nothing: a bit like wearing a cloth mask vs an N95 mask.</p><p>The optimal workaround, however, is to take advantage of a provision for extending TXT values established in <a href="https://www.ietf.org/rfc/rfc4408.txt">RFC4408</a> (Amazon <a href="https://aws.amazon.com/premiumsupport/knowledge-center/route53-resolve-dkim-text-record-error/">has a blog post</a> describing how to do this):</p>
<code>3.1.3. Multiple Strings in a Single DNS record
As defined in [RFC1035] sections 3.3.14 and 3.3, a single text DNS
record (either TXT or SPF RR types) can be composed of more than one
string. If a published record contains multiple strings, then the
record MUST be treated as if those strings are concatenated together
without adding spaces. For example:
IN TXT "v=spf1 .... first" "second string..."
MUST be treated as equivalent to
IN TXT "v=spf1 .... firstsecond string..."
SPF or TXT records containing multiple strings are useful in
constructing records that would exceed the 255-byte maximum length of
a string within a single TXT or SPF RR record.</code>
<p>Notice how the RFC doesn't mention DKIM. SPF was invented seven years prior to DKIM, in the heady days of 1997 (although it would be many more years before they both became an industry standard).</p><p>But <i>why </i>is the size of DNS records such a problem? Are those smarty pants at the IETF just inventing arbitrary rules to, like, keep us down, man? It turns out that the size issue doesn't follow directly from DNS at all - the size limit is the result of the amount of information that can be included within a UDP packet. Again, quoting RFC4408:</p>
<code> The published SPF record for a given domain name SHOULD remain small
enough that the results of a query for it will fit within 512 octets.
This will keep even older DNS implementations from falling over to
TCP. Since the answer size is dependent on many things outside the
scope of this document, it is only possible to give this guideline:
If the combined length of the DNS name and the text of all the
records of a given type (TXT or SPF) is under 450 characters, then
DNS answers should fit in UDP packets. Note that when computing the
sizes for queries of the TXT format, one must take into account any
other TXT records published at the domain name. Records that are too
long to fit in a single UDP packet MAY be silently ignored by SPF
clients.</code>
<p>This merely regresses the question: instead of establishing an arbitrary size for DNS records, we now have an arbitrary selection of a communication protocol, UDP, with arbitrary size requirements. Why does DNS use UDP, when we could use TCP and stuff the collected works of Shakespeare into a TXT record? <i>Computer Networking - A Top-Down Approach</i> by Kurose & Ross provides us with several reasons (h/t to <a href="https://stackoverflow.com/a/47708388/15531179">alhelal</a> for the quote):</p><p></p><code><p>1. <i>No connection establishment - </i>that triple-handshake that makes TCP so reliable is slow. DNS doesn't need a reliable, ongoing connection. All that is needed is the answer to a single question: what IP belongs to this hostname. Kurose & Ross state this is the main reason why DNS uses UDP. It isn't particularly mysterious, but it makes sense: DNS precedes a panoply of other transactions. If DNS is slow, all of those other transactions will be slow, as well. </p><p>2. <i>No connection state - </i>saving state information introduces substantial overhead. A UDP-based DNS resolver can handle many more connections than a TCP-based DNS resolver. The advantages of maintaining state - identifying packet transmission order, congestion control - do not make a lot of sense to address within the DNS session itself.</p><p>3. <i>Smaller packet header overhead</i> - TCP packet headers contain 20 bytes vs 8 bytes per UDP packet header. Those bytes add up fast.</p></code><p></p><p><br /></p>Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-15527040978304214502021-09-28T10:30:00.004-05:002021-09-28T10:30:45.298-05:00The tetraquarks are coming. Or are they?<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8y4rjePHKd_fX9KkmyXNDP1GmubHiUTbeHg0wvHPrPPbjG91MqB7O6WxKpo3vtk0g3nZY8XpZ2ERHc1nAthwZCe0-waBylBk_wymrbb0aNRu539fWTlY8P0kE_52cUCgTezDyJv5QWg0/s272/quarkish.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="Like this, but four of them, compressed into a tight ball." border="0" data-original-height="186" data-original-width="272" height="274" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8y4rjePHKd_fX9KkmyXNDP1GmubHiUTbeHg0wvHPrPPbjG91MqB7O6WxKpo3vtk0g3nZY8XpZ2ERHc1nAthwZCe0-waBylBk_wymrbb0aNRu539fWTlY8P0kE_52cUCgTezDyJv5QWg0/w400-h274/quarkish.jpg" title="Like this, but four of them, compressed into a tight ball." width="400" /></a></div><br /><p></p><p>There has been grumblings since July that some of the folks over at LHC may have discovered a new fundamental particle: the <a href="https://www.quantamagazine.org/impossible-particle-discovery-adds-key-piece-to-the-strong-force-puzzle-20210927/">dicharm tetraquark</a>.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAWXgSNHGRqDmhbS4kuhZuggbIVPndo-RWDVW0oFIeET0Am-N8-BVCEGMg3ew5NgCL5PJFhz1HpJnsiSWqcTS-KX5BhlxNAV0RSyWBu8MTh-J0sUStR53v_zehsPPk-MD307rIt4-nHxQ/" style="margin-left: 1em; margin-right: 1em;"><img alt="OK its really more like this. But don't you want to live in a world where they look like Star Trek Quark? I do." data-original-height="663" data-original-width="635" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAWXgSNHGRqDmhbS4kuhZuggbIVPndo-RWDVW0oFIeET0Am-N8-BVCEGMg3ew5NgCL5PJFhz1HpJnsiSWqcTS-KX5BhlxNAV0RSyWBu8MTh-J0sUStR53v_zehsPPk-MD307rIt4-nHxQ/w383-h400/image.png" title="OK its really more like this. But don't you want to live in a world where they look like Star Trek Quark? I do." width="383" /></a></div><br />From Quanta Magazine:<p></p><p></p><blockquote>[Igor] <span style="background-color: white; color: #1a1a1a; font-family: Merriweather, Georgia, serif; font-size: 16px;">Polyakov went away and double-checked his analysis of data from the Large Hadron Collider beauty (LHCb) experiment, which the Syracuse group is part of. The evidence held. It showed that a particular set of four fundamental particles called quarks can form a tight clique, contrary to the belief of most theorists. The LHCb collaboration reported the discovery of the composite particle, dubbed the double-charm tetraquark, at a conference in July and in </span><a href="https://arxiv.org/abs/2109.01038" style="background-color: white; box-shadow: rgba(0, 0, 0, 0) 0px 0px 0px inset, rgb(26, 26, 26) 0px 1px 0px; box-sizing: border-box; font-family: Merriweather, Georgia, serif; font-size: 16px; text-decoration-line: none; transition-duration: 0.2s; transition-property: color, box-shadow, -webkit-box-shadow; transition-timing-function: ease-in-out;" target="_blank">two</a><span style="background-color: white; color: #1a1a1a; font-family: Merriweather, Georgia, serif; font-size: 16px;"> </span><a href="https://arxiv.org/abs/2109.01056" style="background-color: white; box-shadow: rgba(0, 0, 0, 0) 0px 0px 0px inset, rgb(26, 26, 26) 0px 1px 0px; box-sizing: border-box; font-family: Merriweather, Georgia, serif; font-size: 16px; text-decoration-line: none; transition-duration: 0.2s; transition-property: color, box-shadow, -webkit-box-shadow; transition-timing-function: ease-in-out;" target="_blank">papers</a><span style="background-color: white; color: #1a1a1a; font-family: Merriweather, Georgia, serif; font-size: 16px;"> posted earlier this month that are now undergoing peer review.</span></blockquote><span style="background-color: white; color: #1a1a1a; font-family: Merriweather, Georgia, serif; font-size: 16px;"></span><p></p><p>Everybody loves a new particle. But early results from the LHC have jumped the gun before. And there is a debate about what exactly the LHC results mean. The leading alternative explanation at this point is the observation detected not a new composite particle but a rare Triangle Singularity:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEijpoBp_MarkeOGM0NlZBWBj61QMduhIOscevxc7QdTETcokJjoJqk21bH0ztFWxfs3d5mM3XpZ9FO3uEIIfQmdjA9iigfebLmvhPMdAy1xFryaJSaAmC2ne71oVzKn1IDEX_HQn4Mt-vM/s765/notaquark.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="755" data-original-width="765" height="395" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEijpoBp_MarkeOGM0NlZBWBj61QMduhIOscevxc7QdTETcokJjoJqk21bH0ztFWxfs3d5mM3XpZ9FO3uEIIfQmdjA9iigfebLmvhPMdAy1xFryaJSaAmC2ne71oVzKn1IDEX_HQn4Mt-vM/w400-h395/notaquark.jpg" width="400" /></a></div><p>Quarks have maintained an aura of mystery since their discovery. The leading theoretical description of their behavior, chromodynamics, is at best a very rough approximation of their behavior and at worst impractical for use because of the complexity of the math involved.</p><p>Stay tuned.</p>Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-85240656616997917462021-09-13T10:42:00.008-05:002021-09-14T23:02:21.953-05:00Here is how to mitigate CVE-2021-40444<p><b>UPDATE: Microsoft has released a patch for CVE-2021-40444 as of 9-14 ... but that doesn't mean its been installed on your systems yet, so check! The KB varies by distro, but it should be around KB5005565-KB5005568 for recent Windows 10 x64 versions.</b></p><p>CVE-2021-40444 is a new remote code execution vulnerability in Windows that involves embedded ActiveX controls in Office document files (.doc, .docx, .docm, .dochtml). All versions of Windows, including Server distros, are impacted.</p><p>Exploits of this vulnerability are in the wild now. The Windows preview pane plays a role in the vulnerability; I haven't seen an example of the exploit, but <a href="https://msrc.microsoft.com/update-guide/vulnerability/CVE-2021-40444">Microsoft's recommended steps</a> for mitigation involve disabling the preview pane for relevant file types.</p><p>No security patch is available yet, but it is possible to mitigate the threat. Below, I've embedded code for a registry key that you can use to automatically patch your Windows 10 PC. The registry key simply automates Microsoft's recommended mitigation steps I described a moment ago.</p><p>Copy and paste the code below into a text file. Save that file with a ".reg" file extension (<a href="https://helpdeskgeek.com/windows-10/how-to-change-file-type-in-windows-10/">here</a> is how to do that if you aren't sure how to do that).</p><p><b><i>Remember to backup your registry before installing the .reg file!</i></b></p><p><br /></p>
<script src="https://gist.github.com/jwieder/a321cccf9b900cf7cd72a965b6e0d76a.js"></script>Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-38169115956969510412021-08-31T11:22:00.001-05:002021-08-31T11:22:09.897-05:00This is a PPTP VPN intervention.<div><b>Six years ago (sigh), I wrote but never published this blog post begging users to find an alternative to the PPTP VPN protocol. They were already years out of date at that time. Even today, however, well-known companies like ExpressVPN are still providing PPTP to clients despite the fundamentally insecure nature of these types of tunnels. Consider this an intervention.</b></div><div><br /></div>For years, the Virtual Private Network (VPN) has been a mainstay of those trying to keep snoopers away from their online activities. Its important to keep in mind that a VPN is one part of a secure and private online presence - without complementing the use of a VPN with additional tools and habits, the security offered is more narrow than many users believe.<br />
<br />
There are two main reasons to use a VPN. <div><br /></div><div>First and foremost, a VPN is a means of encapsulating your network traffic within an encrypted "tunnel". This makes it extremely difficult to see or manipulate that network traffic. This is typically the type of VPN you would use to connect to a corporate intranet, for example to "login to the office from home". It is possible to purchase this type of VPN service, for example from an internet service, data center or virtual hosting provider. Alternatively, a business could configure certain types of routing devices (whether a router as such or a type of dedicated security appliance like a firewall) to provide VPN connectivity.<br />
<br />
The second purpose of a VPN is anonymity. Under certain circumstances, a VPN can conceal a user's IP address. Anonymity of this kind if typically not possible when using a corporate intranet, unless it is used maliciously. Corporate VPNs are typically designed specifically to be audit-able, to be able to identify malicious network behavior and who was responsible after the fact. In order to anonymize traffic, VPN providers must carefully configure the equipment providing the VPN tunnel to make it very difficult to fully audit connections to those tunnels.</div><div><br /></div><div>This has obvious appeal for criminals, who might use VPNs to hide their behavior on the internet. The situation isn't as black and white as it sounds, though. Some of the "criminals" using VPNs are simply trying to read a newspaper online that has been banned by their government, or to publish unpopular speech.</div><div><br /></div><div>Users should exercise caution when using a third-party VPN provider, though. Whatever promises a VPN provider makes are just that: promises. It is practically impossible for most users to verify just how secure a VPN provider is. Users must still trust that the VPN provider lives up to their word about server or network architecture, logging practices, etc.</div><div><br /></div><div>Despite that, users still have some control over just how good or bad their VPN security experience will be, and it has to do with protocol selection. Identify a secure VPN requires information about how the VPN is setup that we might not have access to. But identifying a <i>bad </i>VPN is often straightforward. Any VPN connection using an insecure VPN protocol will itself be insecure, and the user will always have access to why VPN protocol you are using.</div><div><br /></div><div>There is one protocol in particular that has lingered well past its utility, and should be avoided at just about all costs: PPTP.</div><div><br />For many, many years, Microsoft's Point-to-Point Tunneling Protocol<span face="Helvetica, Arial, sans-serif" style="background-color: white; color: #222222; font-size: 15px; line-height: 21px;"> (</span>PPTP) was the default VPN solution for Windows. I mean this in the literal sense - PPTP capability was installed as part of Windows starting all the way back in NT 4.0 and remains a component of Microsoft's OS networking suite as of Windows 8.1.<br />
<br />
Microsoft's PPTP relies on a proprietary version of the Challenge-Handshake Authentication Protocol called MS-CHAP, for which there are two versions - MS-CHAPv1 and MS-CHAPv2 - which, as the name implies, handles authentication for initiating a VPN tunnel (MS-CHAP has commonly been used in a variety of other scenarios as well, like RADIUS). Meanwhile, PPTP relies on Microsoft Point-to-Point Encryption (MPPE) to actually encrypt the data transferred through the tunnel.<br />
<br />
From its inception to this day, PPTP and the technology that underlies it has been riddled with significant security issues. The <a href="https://www.schneier.com/paper-pptp.html">definitive cryptanalysis of the original PPTP platform using MS-CHAPv1</a> was published by Bruce Shneier and Peiter Zatko ("Mudge" of l0pht) in 1998. Their own summary of their findings demonstrates how ugly the truth was:<br />
<span style="font-size: x-small;"><br /></span>
<li style="background-color: white; border: 0px; color: #222222; font-family: Helvetica, Arial, sans-serif; line-height: 21px; margin: 0px 0px 0.35em; padding: 0px;"><span style="font-size: x-small;">password hashing -- weak algorithms allow eavesdroppers to learn the user's password</span></li>
<li style="background-color: white; border: 0px; color: #222222; font-family: Helvetica, Arial, sans-serif; line-height: 21px; margin: 0px 0px 0.35em; padding: 0px;"><span style="font-size: x-small;">Challenge/Reply Authentication Protocol -- a design flaw allows an attacker to masquerade as the server</span></li>
<li style="background-color: white; border: 0px; color: #222222; font-family: Helvetica, Arial, sans-serif; line-height: 21px; margin: 0px 0px 0.35em; padding: 0px;"><span style="font-size: x-small;">encryption -- implementation mistakes allow encrypted data to be recovered</span></li>
<li style="background-color: white; border: 0px; color: #222222; font-family: Helvetica, Arial, sans-serif; line-height: 21px; margin: 0px 0px 0.35em; padding: 0px;"><span style="font-size: x-small;">encryption key -- common passwords yield breakable keys, even for 128-bit encryption</span></li>
<li style="background-color: white; border: 0px; color: #222222; font-family: Helvetica, Arial, sans-serif; line-height: 21px; margin: 0px 0px 0.35em; padding: 0px;"><span style="font-size: x-small;">control channel -- unauthenticated messages let attackers crash PPTP servers</span></li>
PPTP VPN servers were susceptible to simple spoofing DOS attacks, passwords could be recovered, data could be decrypted and the protocol relied entirely on user passwords as an encryption key (compare such an archaic method with PSK encryption schemes which were available at the same time). As early as the late 90's software suites capitalizing on the insecurity of the Microsoft platform became widely circulated (like l0phtcrack).<br />
<br />
To their credit, Microsoft reacted quickly with an update - MS-CHAPv2 was introduced, and a variety of issues were resolved. The number of sessions was no longer leaked by VPN servers, spoofing became more difficult, bi-directional keys were added to prevent decryption using XOR and packet structure was updated to remove a mechanism that allowed a remote attackers to spoof password failure transactions to DOS a VPN server.<br />
<br />
With that said, Microsoft continued to offer support for MS-CHAPv1 for years, including in new products released after Microsoft was well aware of just how insecure their product was. The old PPTP implementation <a href="https://support.microsoft.com/en-us/kb/926170">would not be deprecated until the release of Windows Vista</a> in January 2007, <i>eight years after</i> the Shneir/Zatko cryptanalysis paper was published. Given the ease with which a transition could be accomplished from MS-CHAPv1 to MS-CHAPv2, it is impossible to reconcile a delay of that length with a genuine concern for user security. Those who continue to entrust mission critical security infrastructure to Microsoft products would do well to think long and hard about this episode in the company's history.<br />
<br />
That is not to say that the revised PPTP using MS-CHAPv2 is a good protocol. PPTP is not a good protocol and should never be completely abandoned.<br />
<br />
Many of the problems with the original protocol remain in the new revision. A year after their first publication, <a href="https://www.schneier.com/paper-pptpv2.html">Schneir and Zatko released a second cryptanalysis of the new PPTP platform</a>. As in the first version, the protocol's entropy remained completely dependent upon a user's password and vulnerable to passive brute-force techniques. Therein lies the crux of the problem.<br />
<br />
Fundamentally, <a href="https://www.cloudcracker.com/blog/2012/07/29/cracking-ms-chap-v2/">a PPTP tunnel relies on a RC4 hash that in turn is used to encrypt three DES ciphers</a>. In 1998, the Electronic Frontier Foundation's $250K "Deep Crack" platform won the <a href="http://gilchrist.ca/jeff/distrib-des2-2.html">DES Challenge II-2 by decrypting a DES-encrypted message in a mere 56 hours</a>. "Deep Crack" was a single computer. Before the turn of the millennium, brute-forcing a relatively strong PPTP password would be a tall order with commonly available computational resources. Few hackers could afford a quarter million dollar box. That said, the claim that PPTP could ever provide security against a state-level attacker was always fanciful. VPN providers like Pirate Bay's <a href="https://www.ipredator.se/">iPredator</a> continue to provide PPTP VPN service<a href="http://arstechnica.com/tech-policy/2009/03/the-pirate-bay-to-roll-out-secure-vpn-service/"> marketed as protection against government surveillance</a>. Such a claim is among the worst kind of IT service malpractice imaginable, given the possible consequences to an activist or whistleblower who incorrectly entrusts their safety to such a service. It is not hyperbole to say that lives are literally at stake.<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuL-MJTumOvO_QfFpilZYO9IyvsLvIJ3y_V6G_kkSySBQXK10iSKa9zTLoLt1EoSFRy8SZTdIqsgv0xM9eY5s6hOjYHszFl5lPI4MQssn-VuVZ_5bQr6ov4DKM4dseEow2NeGBnd_fwBA/s1600/ipredator.PNG" style="margin-left: auto; margin-right: auto;"><img alt="iPredator VPN advertisement" border="0" height="403" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuL-MJTumOvO_QfFpilZYO9IyvsLvIJ3y_V6G_kkSySBQXK10iSKa9zTLoLt1EoSFRy8SZTdIqsgv0xM9eY5s6hOjYHszFl5lPI4MQssn-VuVZ_5bQr6ov4DKM4dseEow2NeGBnd_fwBA/s640/ipredator.PNG" title="iPredator VPN advertisement" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">iPredator's VPN service landing page <span style="font-size: 12.8px;">as of 2015.</span><span style="font-size: 12.8px;"> iPredator continues to offer PPTP VPN</span></td></tr>
</tbody></table><br /><div>
Fifteen years later, brute-forcing a PPTP tunnel RC4 hash can be accomplished within 24 hours by uploading the data to a service called <a href="https://www.cloudcracker.com/">CloudCracker</a>. At the time of its release in 2012, the service relied on a 40-core, 48 FPGA single system built by <a href="http://www.picocomputing.com/">Pico Computing</a>. Furthermore, an opensource application called <a href="https://github.com/moxie0/chapcrack">chapcrack</a> can be used on its own or in conjunction with CloudCracker to fully decrypt sessions and tunnel password information. There are a number of other effective auditing tools as well - <a href="https://code.google.com/p/mschapv2acc/">mschapv2acc</a> comes to mind immediately (I apologize if I have left your own project out).<br />
<br />
At this point, anyone with even the most basic technical acumen and a $300 laptop can decrypt a PPTP session. The idea that a VPN like this would inhibit surveillance from a state actor, as iPredator did, is laughable.<br />
<br />
I have gone after PPTP not because it is the only insecure VPN platform, but because it is very likely the most commonly used insecure VPN platform. PPTP remains in wide circulation. It is long past due for PPTP to go the way of acoustic couplers and glide gracefully into the great beyond.<br /></div></div>Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-14943440339141195782021-08-31T10:46:00.003-05:002021-08-31T10:46:26.417-05:00Electromagnetic eavesdropping is cheap & easy - so why doesn't anyone believe it exists?<div><i>Below, I've included what would have been the first post in a series of posts I wrote about the </i><i>badBIOS </i><i>controversy in October 2013. I found the evidence in support of badBIOS to be unconvincing and I was concerned by how popular badBIOS became despite those obvious shortcomings. This wasn't a situation where an overexcitable press ran with a story that turned out to be inaccurate; the most early and adamant believers in </i><i>badBIOS </i><i>weren't reporters, they were ITSEC professionals. How were so many of us publicly duped by what was essentially a conspiracy theory?</i></div><div><i><br /></i></div><div><i>This post doesn't address badBIOS directly. However, badBIOS was presumed to somehow involve the manipulation of computers using acoustic transmissions. This post provides some historical context behind a strain of computer science research in this field and shows how commonly held beliefs about the feasibility of these attacks were generally inaccurate at the time of writing. In future posts I would have explored how these misunderstandings could have made it more likely for members of the community to distrust early criticism of badBIOS</i></div><div><br /></div><b>Electronic surveillance relying on electromagnetism, radio and acoustics have been widely understood for over 60 years. Why do some in the IT security community dismiss such techniques as "<a href="http://arstechnica.com/security/2013/10/meet-badbios-the-mysterious-mac-and-pc-malware-that-jumps-airgaps/">equivalent of a Bigfoot sighting</a> </b><b>[sic]</b><b>" while others are convinced such techniques are widely used and highly aggressive?</b><br />
<br /><span style="font-size: large;"><b>A brief, bad history of emissions security exploitation (and why it's cheaper than you think)</b></span><br />
<br />
In 1985, a Dutch researcher named Wim van Eck published a proof of concept for a simple and inexpensive system that could reproduce the visual data of a remote video display unit (Electromagnetic Radiation from Video Display Units: An Eavesdropping Risk? PTT Dr. Neher Laboratories, St. Paulusstroat 4. 2264 XZ Leidschendam, The Netherlands. <a href="http://cryptome.org/emr.pdf">Download PDF from Cryptome</a>.) Using this method, it became trivial to retrieve visual information from, for example, computer monitors using only a standard television receiver. Using a directional antennae and amplifier, van Eck's method proved effective at several hundred meters.<br />
<br />
Many incorrectly believe that van Eck essentially created the technique of remote receiver surveillance. As van Eck explains:<br />
<blockquote class="tr_bq">
"<i>It is possible in some cases to obtain information on the signals used inside the equipment when the radiation is picked up and the received signals are decoded. [...] This problems is not a new one; defense specialists have been aware of it for over 20 years.</i>" </blockquote>
Despite the understanding that such techniques were possible, the van Eck paper still proved to be somewhat of a bomb thrown into the security industry. Van Eck again explains why:<br />
<blockquote class="tr_bq">
"<i>Until recently it was considered very difficult to reconstruct the data hidden in the radiated field, and it was therefore believed that eavesdropping on digital equipment could only be performed by professionals with access to very sophisticated detection and decoding equipment. As a result, digital equipment for processing information requiring medium or low level protection, such as private or business information, is not protected against eavesdropping of this kind.</i>" </blockquote>
Consequently, when, for example, Markus G. Kuhn of Cambridge University claimed in his (equally ground-breaking) 2004 paper Electromagnetic Eavesdroping Risks of Flat-Panel Displays that "<i>Electromagnetic eavesdropping of computer displays [was] first demonstrated to the general public by van Eck in 1985</i>" we are forced to correct his assertion by maintaining that van Eck was the first to demonstrate <i>cheap</i> and <i>widely available</i> electromagnetic eavesdropping of computer displays to the general public. Those suspicious of my assertion here are welcome to consider <span style="background-color: white;">R.L. Dennis' August 1966 paper, </span>"<span style="background-color: white;">Security and Privacy in Computer Systems", a brief summary of which is provided in a <a href="http://www.amazon.com/Security-Privacy-Computer-Systems-Information/dp/0471406112">text-book of the same title from 1973 edited by Lance Hoffman</a>: </span><br />
<span style="background-color: white;"><br /></span><blockquote class="tr_bq"><i><span style="background-color: white;">"Passive infiltration may be accomplished by wiretapping or by </span>electromagnetic <span style="background-color: white;">pickup of the traffic at any point in the system. Although considerable </span><span style="background-color: white;">effort has been applied to counter such threats to defense communications, </span></i><i><span style="background-color: white;">nongovernmental approaches to information privacy usually assume that </span><span style="background-color: white;">communication lines are secure, when in fact they are one of the most </span></i><span style="background-color: white;"><i>vulnerable parts of the system. </i>[p. 77]"</span></blockquote>
<blockquote class="tr_bq">
<span style="background-color: white;">"</span><i><span style="background-color: white;">In addition to the spectrum of threats arising from wiretapping, electro</span><span style="background-color: white;">magnetic radiation from terminals must be considered.[12] Electromagnetic </span><span style="background-color: white;">radiation characteristics will depend heavily on the type of terminal, </span><span style="background-color: white;">and may in some cases pose serious shielding and electrical-filtering </span><span style="background-color: white;">problems. More advanced terminals using cathode ray tube for information </span><span style="background-color: white;">display may create even greater problems in trying to prevent what has been </span></i><span style="background-color: white;"><i>called 'tuning in the terminal on Channel 4.'</i> [p. 84]" </span></blockquote>
<div>
<span style="background-color: white;">Note that review of unclassified documents concerning these techniques, as well as cryptography, must be considered with the regulatory burden on the period. Until very recently, those attempting to publicize cryptography and penetration testing techniques regularly found themselves running afoul of the US Federal Government. As late as 1999 Federal prosecutors sough a conviction against Israeli citizen </span><span style="background-color: white;">Shalom Shaphyr for the alleged</span><span style="background-color: white;"> transport of TEMPEST equipment to a foreign country. Incredibly, but perhaps not surprising to readers now familiar with modern anti-terrorism arrest techniques, Shaphyr was not caught with equipment - he was sold the equipment by an FBI agent as part of a sting operation. For a brief survey of the dangers that security innovators ran in the 1990's, please see my own "</span><a href="http://www.joshwieder.net/2014/10/is-encryption-becoming-illegal-again.html">Is Encryption Becoming Illegal Again</a><span style="background-color: white;">?"</span></div>
<div>
<br />
For at least as long as techniques for video display unit reproduction have existed, the US military has invested in countermeasures to foil those techniques. It was as early as the mid-1950's that the US established the <span style="background-color: white;">Transient Electromagnetic Pulse Emanation Standard</span> with the publication of <span style="background-color: white;">NAG-1A, in the earliest attempt to shield military and diplomatic communication from electromagnetic eavesdropping. The standard, and the research projects supporting the standard, would collectively become known as TEMPEST. The Cold War would spawn a series of innovations in both surveillance and countermeasures of this kind. Unfortunately, a great deal of that history remains highly contested, classified and often both. One milestone that can provide readers with an idea of just how long this technology has been in practical usage was <a href="http://www2.warwick.ac.uk/fac/soc/pais/people/aldrich/vigilant/lectures/gchq/rafter/">Operation Rafter, in which British intelligence agents located KGB agent radio transceivers using radiation from oscillators</a> - even when the radios were not transmitting.</span><br />
<br />
A complete history of US TEMPEST research could easily fill a book, as such a history would necessarily cover over 60 years of research and innovation. The brief overview here is meant to help clarify a few important points about electromagnetic eavesdropping. </div><div><br /></div><div>Throughout the development of these surveillance techniques, attackers have sought and succeeded in using electromagnetic radiation to reproduce visual data, as retrieved from emanations from cathode ray tubes for example, as well as to reproduce non-visual data, as retrieved from the sounds of a teletype machine in operation. These early methods have been almost completely unaddressed in modern computer development. Keylogging from acoustic surveillance is not a technique limited to teletypes, as IBM employees Dmitri Asonov and Rakesh Agrawal demonstrated in their 2004 paper <i>Keyboard Acoustic Emanations </i>(non-acoustic typing behavior - so-called typing dynamics - is increasingly used as a biometric identifier and monitoring of this behavior has long been a default setting for Windows 10 devices).</div><div><br /></div><div>More recently, in 2011 and again this year, it was demonstrated by researchers from Tel Aviv Univeristy and Weizmann Institute of Science (along with <a href="http://www.tau.ac.il/~tromer/acoustic/#acks">numerous other contributors</a>) that there are a variety of surveillance techniques that can be used to <a href="http://www.tau.ac.il/~tromer/acoustic/">reproduce CPU operations</a> across an air gap in order to steal private keys from widely used encryption software. The most recent of the two Tel Aviv University attacks have accomplished something quite similar to what van Eck did decades ago: illustrating how this manner of surveillance can be accomplished on the cheap. <a href="https://www.tau.ac.il/~tromer/radioexp/">The new attack requires a simple software-controlled radio dongle or commercial radio available for somewhere between $20-$50</a> (the 2011 attack monitored acoustics from the CPU rather than electromagnetism), something any hobbyist could afford.</div><div><br /></div><div>The misunderstanding of the requirements of a successful EMG eavesdropping attack isn't limited to those outside the security field. When Joe Loughry of Lockheed Martin Space Systems and David A. Umphress of Auburn University published their seminal "Information Leakage from Optical Emanations", which as early as 2002 publicly exposed that NIC and router LED status lights could be used to determine data transferred over the relevant interface, they still felt it necessary to repeat the seemingly irreconcilable bungle that only some James Bond-esque group of super-powered spies could retrieve data using the methods previously outlined:<br />
<blockquote class="tr_bq">
"<i>Because of the high cost of equipment and the difficulty of intercepting and exploiting RF emanations, reports of successful attacks against emanations have been limited primarily to high-value sources of information such as military targets and cryptologic systems.</i>"</blockquote><p>This isn't to say that there are not serious practical difficulties with this approach: there are. Electromagnetic radiation exists on a spectrum that includes frequencies with very different types of behavior: X-Rays can go through things, while visible light typically can't. Both are "electromagnetic radiation". Typically, though, PC's don't express X-rays. Furthermore, data centers and similar facilities tend not to have windows. Furthermore, it is unclear how many of these types of attacks could translate to a virtualized environment. This isn't a problem that should keep people in charge of IT resources up at night. But it is worth noting that the reason these attacks are rare is likely the result of these practical limitations, <i>not </i>the general unavailability of the attack vector itself. </p></div>Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-48835142676188582792021-08-31T10:09:00.001-05:002021-08-31T10:09:54.701-05:00PHP empty() and the trouble of passing new zero values in formsLets say we have a form. The form is straight-forward: HTML radio buttons that posts to a PHP processing script that saves the results to a database. The radio buttons correspond to bindary responses to questions - Yes/No, True/False. These truth values are mapped to the integers '1' and '0', which are then stored in the database.<br />
<div>
<br /></div>
<div>
When such a form is creating new records, this is about as brainless as web-development gets. But things get more complicated when the form is used to update existing records. A useful update form will typically have a few basic features; among them, the option to update all of the data in the form or just some of the data. To do this, we have to check whether a variable is being updated or not.</div>
<div>
<br /></div>
<div>
One approach to checking on variable updates is to create an array with all of the variables to be considered, like this: </div>
<br />
<pre>$stuff = array( 'fee' => $_POST['fee'],
'fi' => $_POST['fi'],
'fo' => $_POST['fo'],
'fum' => $_POST['fum']
)
</pre>
<br />
<div>
Then you can parse the assignments of these variables, determine their value, and decide whether to discard the variable or update your database based on it. In this example, lets imagine that the variable 'fee' also serves as a key for the database table to be updated by this script:</div>
<br />
<pre>foreach($stuff as $variable => $value)
{
if ( (empty($value)) && ($variable='fee') ) {
echo "You need to provide a value for FEE to identify which record to update!";
break;
}
</pre>
<pre> elseif ( (!empty($value)) && ($variable!='fee') ) {
$query = ("UPDATE events SET ".$variable." = '".$value."' WHERE fee = ".$_POST['fee']."");
print $query . "<br>";
$link->exec($query);
echo "we updated the database with: " . $query;
continue;
}
</pre>
<pre> elseif (empty($value)) {
print $variable . " value was not passed to database";
continue;
}
}
</pre>
<br />
<div>
You will notice that in two of these conditional statements, <a href="https://secure.php.net/manual/en/function.empty.php">empty()</a> is used in order to identify whether a variable has been modified or not. In some circumstances this would work - for example, if this form was not multiple choice, but contained text fields, it would do a very good job at determining whether content was placed in those text fields.<br />
<br />
The trouble comes when a value contains a Yes/True/'1' value, and we want to use the update form to change that to a No/False/'0' value. Why? Because empty() does not check whether or not a variable assignment is in fact empty. Or more specifically, that's not <i>all</i> it does. empty() will return a TRUE value if any of the following conditions are met:<br />
<br />
<ul class="simplelist" style="background-color: #f2f2f2; color: #333333; font-family: "Fira Sans", "Source Sans Pro", Helvetica, Arial, sans-serif; font-size: 16px; line-height: 24px; margin: 0px 0px 1.5rem 1.5rem; padding: 0px;">
<li class="member"><em style="text-rendering: optimizelegibility;">""</em> (an empty string)</li>
<li class="member"><em style="text-rendering: optimizelegibility;">0</em> (0 as an integer)</li>
<li class="member"><em style="text-rendering: optimizelegibility;">0.0</em> (0 as a float)</li>
<li class="member"><em style="text-rendering: optimizelegibility;">"0"</em> (0 as a string)</li>
<li class="member"><span style="text-rendering: optimizelegibility;"><code style="font-family: "Fira Mono", "Source Code Pro", monospace; font-size: 0.875rem; font-stretch: normal; font-weight: 700; line-height: 1.375rem; overflow-wrap: break-word; word-wrap: break-word;">NULL</code></span></li>
<li class="member"><span style="text-rendering: optimizelegibility;"><code style="font-family: "Fira Mono", "Source Code Pro", monospace; font-size: 0.875rem; font-stretch: normal; font-weight: 700; line-height: 1.375rem; overflow-wrap: break-word; word-wrap: break-word;">FALSE</code></span></li>
<li class="member"><em style="text-rendering: optimizelegibility;">array()</em> (an empty array)</li>
<li class="member"><em style="text-rendering: optimizelegibility;">$var;</em> (a variable declared, but without a value)</li>
</ul>
What this means is that if we use empty(), and then attempt to change a value from '1' to '0' using our form, that change will be discarded.<br />
<br />
So what's the solution? I tend to use more specific conditionals. Instead of empty(), we can tell our script to discard variables that are actually empty in the human sense of the term (<i>x==</i>"") and tell our script to handle variables containing a zero value (<i>x</i>=='0') normally.<br />
<br />
<br />
<pre>foreach($stuff as $variable => $value)
{
if ( (empty($value)) && ($variable='fee') ) {
echo "You need to provide a value for FEE to identify which record to update!";
break;
}
</pre>
<pre> elseif ( (!empty($value)) && ($variable!='fee') ) {
$query = ("UPDATE events SET ".$variable." = '".$value."' WHERE fee = ".$_POST['fee']."");
print $query . "<br>";
$link->exec($query);
echo "we updated the database with: " . $query;
continue;
}</pre>
<pre></pre>
<pre><pre> elseif ( ($value=='0') && ($variable!='fee') ) {
$query = ("UPDATE events SET ".$variable." = '".$value."' WHERE fee = ".$_POST['fee']."");
print $query . "<br>";
$link->exec($query);
echo "we updated the database with: " . $query;
continue;
}</pre>
</pre>
<pre> elseif ($value=="") {
print $variable . " value was not passed to database";
continue;
}
}
</pre>
<div>
<br /></div>
Hope this helps! (<b>DISCLAIMER</b>: always sanitize input!)</div>
Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-80251210429982014872021-07-22T12:52:00.003-05:002021-07-22T13:09:21.023-05:00KMS client activation keys - Windows Server 2022, 2019 and 2016<div class="heading-anchor" id="windows-server-semi-annual-channel-versions" style="text-align: left;">This is the latest update of the list of <a href="https://docs.microsoft.com/en-us/windows-server/get-started/kmsclientkeys ">windows license keys for key management service activation</a> I publish every few years. Reference the <a href="http://joshwieder.blogspot.com/2012/11/windows-server-2012-activation-problems.html"><b>KMS activation post I wrote for Windows 2012</b></a><b> </b>for help installing one of the keys (`<b>slmgr /ipk</b> <i>yourkeyhere` </i>from a command prompt as an administrator)<br /> <br /></div><h2 class="heading-anchor" id="windows-server-semi-annual-channel-versions">Windows Server Semi-Annual Channel versions</h2>
<h3 class="heading-anchor" id="windows-server-version-1909-version-1903-and-version-1809">Windows Server, version 1909, version 1903, and version 1809</h3>
<div class="table-scroll-wrapper"><table class="table"><caption class="visually-hidden">Windows Server, version 1909, version 1903, and version 1809</caption>
<thead>
<tr>
<th>Operating system edition</th>
<th>KMS Client Setup Key</th>
</tr>
</thead>
<tbody>
<tr>
<td>Windows Server Datacenter</td>
<td>6NMRW-2C8FM-D24W7-TQWMY-CWH2D</td>
</tr>
<tr>
<td>Windows Server Standard</td>
<td>N2KJX-J94YW-TQVFB-DG9YT-724CC</td>
</tr>
</tbody>
</table></div>
<h2 class="heading-anchor" id="windows-server-ltscltsb-versions">Windows Server LTSC/LTSB versions</h2>
<h3 class="heading-anchor" id="windows-server-2022">Windows Server 2022</h3>
<div class="table-scroll-wrapper"><table class="table"><caption class="visually-hidden">Windows Server 2022</caption>
<thead>
<tr>
<th>Operating system edition</th>
<th>KMS Client Setup Key</th>
</tr>
</thead>
<tbody>
<tr>
<td>Windows Server 2022 Datacenter</td>
<td>WX4NM-KYWYW-QJJR4-XV3QB-6VM33</td>
</tr>
<tr>
<td>Windows Server 2022 Standard</td>
<td>VDYBN-27WPP-V4HQT-9VMD4-VMK7H</td>
</tr>
</tbody>
</table></div>
<h3 class="heading-anchor" id="windows-server-2019">Windows Server 2019</h3>
<div class="table-scroll-wrapper"><table class="table"><caption class="visually-hidden">Windows Server 2019</caption>
<thead>
<tr>
<th>Operating system edition</th>
<th>KMS Client Setup Key</th>
</tr>
</thead>
<tbody>
<tr>
<td>Windows Server 2019 Datacenter</td>
<td>WMDGN-G9PQG-XVVXX-R3X43-63DFG</td>
</tr>
<tr>
<td>Windows Server 2019 Standard</td>
<td>N69G4-B89J2-4G8F4-WWYCC-J464C</td>
</tr>
<tr>
<td>Windows Server 2019 Essentials</td>
<td>WVDHN-86M7X-466P6-VHXV7-YY726</td>
</tr>
</tbody>
</table></div>
<h3 class="heading-anchor" id="windows-server-2016">Windows Server 2016</h3>
<div class="table-scroll-wrapper"><table class="table"><caption class="visually-hidden">Windows Server 2016</caption>
<thead>
<tr>
<th>Operating system edition</th>
<th>KMS Client Setup Key</th>
</tr>
</thead>
<tbody>
<tr>
<td>Windows Server 2016 Datacenter</td>
<td>CB7KF-BWN84-R7R2Y-793K2-8XDDG</td>
</tr>
<tr>
<td>Windows Server 2016 Standard</td>
<td>WC2BQ-8NRM3-FDDYY-2BFGV-KHKQY</td>
</tr>
<tr>
<td>Windows Server 2016 Essentials</td>
<td>JCKRF-N37P4-C2D82-9YXRT-4M63B</td>
</tr>
</tbody>
</table></div>
<h2 class="heading-anchor" id="windows-10-all-supported-semi-annual-channel-versions">Windows 10, all supported Semi-Annual Channel versions</h2>
<p>See the <a data-linktype="external" href="https://support.microsoft.com/help/13853/windows-lifecycle-fact-sheet">Windows lifecycle fact sheet</a> for information about supported versions and end of service dates.</p>
<div class="table-scroll-wrapper"><table class="table"><caption class="visually-hidden">Windows 10, all supported Semi-Annual Channel versions</caption>
<thead>
<tr>
<th>Operating system edition</th>
<th>KMS Client Setup Key</th>
</tr>
</thead>
<tbody>
<tr>
<td>Windows 10 Pro</td>
<td>W269N-WFGWX-YVC9B-4J6C9-T83GX</td>
</tr>
<tr>
<td>Windows 10 Pro N</td>
<td>MH37W-N47XK-V7XM9-C7227-GCQG9</td>
</tr>
<tr>
<td>Windows 10 Pro for Workstations</td>
<td>NRG8B-VKK3Q-CXVCJ-9G2XF-6Q84J</td>
</tr>
<tr>
<td>Windows 10 Pro for Workstations N</td>
<td>9FNHH-K3HBT-3W4TD-6383H-6XYWF</td>
</tr>
<tr>
<td>Windows 10 Pro Education</td>
<td>6TP4R-GNPTD-KYYHQ-7B7DP-J447Y</td>
</tr>
<tr>
<td>Windows 10 Pro Education N</td>
<td>YVWGF-BXNMC-HTQYQ-CPQ99-66QFC</td>
</tr>
<tr>
<td>Windows 10 Education</td>
<td>NW6C2-QMPVW-D7KKK-3GKT6-VCFB2</td>
</tr>
<tr>
<td>Windows 10 Education N</td>
<td>2WH4N-8QGBV-H22JP-CT43Q-MDWWJ</td>
</tr>
<tr>
<td>Windows 10 Enterprise</td>
<td>NPPR9-FWDCX-D2C8J-H872K-2YT43</td>
</tr>
<tr>
<td>Windows 10 Enterprise N</td>
<td>DPH2V-TTNVB-4X9Q3-TJR4H-KHJW4</td>
</tr>
<tr>
<td>Windows 10 Enterprise G</td>
<td>YYVX9-NTFWV-6MDM3-9PT4T-4M68B</td>
</tr>
<tr>
<td>Windows 10 Enterprise G N</td>
<td>44RPN-FTY23-9VTTB-MP9BX-T84FV</td>
</tr>
</tbody>
</table></div>
<h2 class="heading-anchor" id="windows-10-ltscltsb-versions">Windows 10 LTSC/LTSB versions</h2>
<h3 class="heading-anchor" id="windows-10-ltsc-2019">Windows 10 LTSC 2019</h3>
<div class="table-scroll-wrapper"><table class="table"><caption class="visually-hidden">Windows 10 LTSC 2019</caption>
<thead>
<tr>
<th>Operating system edition</th>
<th>KMS Client Setup Key</th>
</tr>
</thead>
<tbody>
<tr>
<td>Windows 10 Enterprise LTSC 2019</td>
<td>M7XTQ-FN8P6-TTKYV-9D4CC-J462D</td>
</tr>
<tr>
<td>Windows 10 Enterprise N LTSC 2019</td>
<td>92NFX-8DJQP-P6BBQ-THF9C-7CG2H</td>
</tr>
</tbody>
</table></div>
<h3 class="heading-anchor" id="windows-10-ltsb-2016">Windows 10 LTSB 2016</h3>
<div class="table-scroll-wrapper"><table class="table"><caption class="visually-hidden">Windows 10 LTSB 2016</caption>
<thead>
<tr>
<th>Operating system edition</th>
<th>KMS Client Setup Key</th>
</tr>
</thead>
<tbody>
<tr>
<td>Windows 10 Enterprise LTSB 2016</td>
<td>DCPHK-NFMTC-H88MJ-PFHPY-QJ4BJ</td>
</tr>
<tr>
<td>Windows 10 Enterprise N LTSB 2016</td>
<td>QFFDN-GRT3P-VKWWX-X7T3R-8B639</td>
</tr>
</tbody>
</table></div>
<h3 class="heading-anchor" id="windows-10-ltsb-2015">Windows 10 LTSB 2015</h3>
<div class="table-scroll-wrapper"><table class="table"><caption class="visually-hidden">Windows 10 LTSB 2015</caption>
<thead>
<tr>
<th>Operating system edition</th>
<th>KMS Client Setup Key</th>
</tr>
</thead>
<tbody>
<tr>
<td>Windows 10 Enterprise 2015 LTSB</td>
<td>WNMTR-4C88C-JK8YV-HQ7T2-76DF9</td>
</tr>
<tr>
<td>Windows 10 Enterprise 2015 LTSB N</td>
<td>2F77B-TNFGY-69QQF-B8YKP-D69TJ</td></tr></tbody></table></div>.http://www.blogger.com/profile/16490250863425476101noreply@blogger.comIndia20.4949149 79.026558-11.654222530696547 43.870307999999994 52.644052330696553 114.182808tag:blogger.com,1999:blog-4411720504608505363.post-90354969250416790952021-07-22T12:04:00.004-05:002021-08-31T10:02:13.980-05:00If E.T. phones home, he won't use entagled qubits<p>I can recall listening to a radio program some 10-15 years ago. The host of the show believed that it would soon be possible to build a faster-than-light communications system using quantum entangled particles, and interviewed several people from a company who were seeking funding to somehow make that happen. <br /></p><p>But why not? </p><p>There would be tremendous value in some sort of "quantum phone" of
entangled particles that allowed for transferring messages faster than
the speed of light. </p><p>Quantum computers are a real thing now. Quantum key distribution could very well revolutionize public key cryptography. Yet if anything, quantum computing is a misnomer because it understates how fundamental quantum mechanics has been on recent technological innovation. Quantum mechanics has been around for a century now and all modern computers rely to some extent on the principles of quantum mechanics to function.</p><p></p><p>But there will be no quantum phone.<br /></p><p>Let's start by explaining how the quantum phone is usually pitched. <a href="https://www.forbes.com/sites/chadorzel/2016/05/04/the-real-reasons-quantum-entanglement-doesnt-allow-faster-than-light-communication/#42dc05953a1e">Forbes magazine has a 2016 article</a> that explains the idea in plain language:</p><p></p><blockquote>[...] could we use this property — quantum entanglement — to <em>communicate</em>
from a distant star system to our own? [...] You could, for example, keep
an entangled particle in an indeterminate state, send it aboard a
spacecraft bound for the nearest star, and tell it to look for signs of a
rocky planet in that star’s habitable zone. If you see one, make a
measurement that forces the particle you have to be in the +1 state, and
if you don’t see one, make a measurement that forces the particle you
have to be in the -1 state.</blockquote><p></p><p>You have a particle with a binary state being flicked off and on to communicate information. Just like a transistor inside of a conventional computer processor, right? This doesn't sound all that outrageous.<br /></p><p>The problem is with this bit: <i>make a
measurement that forces the particle you have to be in the +1 state. </i>There is a conceptual mismatch here: you can either make a measurement <i>or</i> you can force a particle into a different state, but you can't do both at the same time. Either you can interfere with the measurement, which breaks the the
relationship between that measurement of your entangled particles, or
you can not interfere with the measurement, in which case you will not
be able to send any meaningful information this way. Whatever specific
process you devise to measure the process, that process will be governed
by well-known statistical processes that make it impossible to encode information <i>without sending additional information alongside the entangled bits. </i><br /></p><p></p><p></p><p>Even if information is transferred between quantum entangled particles at the moment of their measurement - and that information transfer occurs faster than the speed of light - the relevance of this transfer is limited by the extremely strict limit on what kind of information is traveling: whatever statistical distribution governs the particular quantum process being measured.</p><p>All of this has been well established for many, many years - the last major break-though in quantum entanglement was arguably John Bell's theorem, which demonstrated Einstein/Polsky/Rosen's hidden variables interpretation of quantum entanglement was wrong. Bell first published that work in 1963. So why was I hearing a pitch for a quantum phone decades after that? Why does <a href="https://quantumxc.com/is-quantum-communication-faster-than-the-speed-of-light/" rel="nofollow">one company</a> claim:</p><p></p><blockquote>When we conquer communication through quantum entanglement, faster-than-light communications will become a real possibility.</blockquote><p></p><p>The fact remains: a mountain of theoretical and experimental work demonstrates the speed of light is a hard speed limit in the observable universe. An immense amount of knowledge about the universe can be deduced from that simple fact, but these facts are often obscured by being out of sync with our experience of the world. It is much easier to imagine what life might be like without such a barrier in place (awesome)</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7FkyRnlSKbfbDb_cz6jOvFOlmp57cHCef2H5dBDltwYriRZQta2okp-8PblsVmTw7vW663_mzesvZSHZXX4s4u6a_gonl8ECnIKePG4ua-8SPYJ-o-WI1QEHxmAViKECH4qCR7Edj878/" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="930" data-original-width="1655" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7FkyRnlSKbfbDb_cz6jOvFOlmp57cHCef2H5dBDltwYriRZQta2okp-8PblsVmTw7vW663_mzesvZSHZXX4s4u6a_gonl8ECnIKePG4ua-8SPYJ-o-WI1QEHxmAViKECH4qCR7Edj878/" width="320" /></a></div><p>The truth is, though, that the lightspeed barrier gets us a lot more than it asks of us. Causality, for one. Luminal particles can travel very fast, but they cannot have mass. This means they cannot experience the flow of time. Without time, it is impossible to say that one thing happened before another thing. This means you cannot say that one thing <i>caused</i> another thing. </p><p></p><p>Paradoxes are a clue that we have veered off the right logical track. Remain skeptical of any claims that depend on disregarding special relativity. Even if he got hidden variables wrong, Einstein remains a very safe bet in 2021.</p><p><i>Appendix 8-31-2021 </i>String theorist Joseph Polchinski provided some reasons to believe that superluminal communication via quantum entanglement could in fact be possible - provided that the Everett interpretation of quantum mechanics is correct. <a href="https://www.npl.washington.edu/av/altvw48.html">John Cramer describes the idea</a>:</p><p><span style="text-indent: 50px; widows: 1;"><i></i></span></p><blockquote><span style="text-indent: 50px; widows: 1;"><i>He </i>[</span>Polchinski] <i><span style="text-indent: 50px; widows: 1;"> goes on to describe an "Everett-Wheeler telephone". In standard QM in the Many Worlds scenario in which the wave function does not collapse, a measurement performed in one MW universe can have no effect on a measurement made in another. Polchinski demonstrates that in non-linear QM such measurements "talk" and can be used for</span><span style="text-indent: 50px; widows: 1;"><span class="Apple-converted-space"> </span>transmission of information from one MW branch universe to another</span><span style="text-indent: 50px; widows: 1;">. With Polchinski's non-linear quantum telephones you could talk to yourself at an earlier time or to your alter ego in an alternate universe.</span></i></blockquote><span style="text-indent: 50px; widows: 1;"></span><p></p><p>Polchinski's idea for a quantum phone depends upon an idea proposed by Nobel laureate Steven Weinberg. Today it is assumed - and has been assumed for the last century - that the Schrodinger equation is linear. <a href="https://ocw.mit.edu/courses/physics/8-04-quantum-physics-i-spring-2016/video-lectures/part-1/quantum-mechanics-as-a-framework.-defining-linearity/">Barton Zwiebach described it this way in his Quantum Physics I course at MIT</a>: "We have two solutions. We can add them. We have a single solution. You can scale it by a number." This is a very basic assumption about how, for example, electromagnetic waves interfere with one another to create a new wave. </p><p>Weinberg noticed that in areas of physics other than quantum physics, there are circumstances in which behavior can become non-linear. Weinberg proposed a methodology for testing for non-linearity, and asked why there could not also be circumstances where non-linearity appears in quantum mechanics, also. Polchinski built on Weinberg's work, demonstrating that the prohibition against superluminal communication is removed when linearity is removed. Polchinski went on to develop a specific non-linear modification to the Schrodinger equation that managed to remove the superluminal behavior of other approaches. The result, though, appears to make it possible to send information across quantum multiverses; <a href="https://youtu.be/IEDSAheh8Os?t=535">Polchinski even provided a possible methodology for doing this that he dubbed the Everett-Wheeler phone</a>.</p><p>This really seems a pretty strong argument against everything I said before the appendix. WTF?</p><p>The Everett-Wheeler quantum phone isn't as sexy as it sounds. Weinberg provided ideas for how to test for non-linearity in quantum physics, and people have looked for non-linearity. So far, evidence of quantum nonlinearity hasn't been found in the experiments. </p><p>Fans of Polchinski reply that the testing needs to be performed at a higher energy level (essentially, using a bigger particle collider). There is always a bigger collider, there is always a higher energy level (see for example Lost in Math by <a href="http://sabinehossenfelder.com/">Sabine Hossenfelder</a>). It might be worth it to test for non-linearity at the LHC, I don't know, but if such a test did happen and no evidence of non-linearity was found, the same complaint could conceivably be made: that we aren't looking in the correct range.</p><p>But hey, let's make a giant leap and assume that non-linearity does exist. That doesn't solve all the problems. </p><p>What is the absolute smallest possible amount of information that could be considered a <i>communication</i>? You can probably imagine getting an email with a single integer: 1. You'd probably call this a communication, even though it only includes a single bit of information (1). But really, the email includes an additional layer of information that is transmitted implicitly, by virtue of the fact that the bit was an email. An email implies several additional features that include information that is greatly in excess of a single bit: each email has certain fields called headers that must be included in order to be an email at all. These fields are things like "To:" (the recipient address), "From:" (the sender address), the subject line, etc..</p><p>The importance of that secondary data is revealed with a simple thought experiment. Imagine if we removed that implicit wrapper of information from the email. Let's say you turn on your computer, and a single integer - 1 - is displayed on your screen for 30 seconds. The computer then proceeds to boot up normally. Would you consider <i>that</i> to be a message? Most people with some idea of how a computer works would be more likely to assume that the integer was some sort of error in the computer (not an error message, though, because error messages at least indicate that they are an error e.g. by saying "ERROR", which requires more than a single bit of information)</p><p>Without the tell-tale signs of an email, the single bit of information lacks any of the characteristics of communication. There is more information contained implicitly in an email that is completely blank then there is in the single bit contained on its own.</p><p>And a single bit of information is all that can be sent using an Everett-Wheeler quantum phone. This size limit fundamentally makes bi-directional communication impossible. But bi-directionality would be impossible even without a size limit. This is because it is impossible to select a specific multiverse to send a message to. </p><p>In fact, that somewhat under-estimates how weird the situation would be. If we assume that the Everett or Many Worlds interpretation of quantum mechanics is correct, sending a single-bit message using an Everett-Wheeler phone would in fact <i>create</i> the parallel world that receives the "message". Let that sink in for a minute. We are really asking quite a bit of the word phone here. Even with the most robust data plan available using the most extravagant cellphone available, calling a person on the phone typically will not instantaneously create the person on the other end.</p>.http://www.blogger.com/profile/16490250863425476101noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-63715465775620703842021-07-19T19:07:00.002-05:002021-07-19T19:07:31.174-05:00Your spreadsheet is probably wrong<p>I watched <a href="https://www.youtube.com/watch?v=-_-Vg_B3nzM">Rob Eastaway's 2019 for the Royal Institute</a> today. Everything from RI is great and worth checking out, but Eastaway delivered a statistic I hadn't come across before: 90% of all spreadsheets contain errors. Mr Eastaway himself had only come across the statistic from another source, the European Spreadsheet Risks Interest Group (or EuSpRIG for short).</p><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="399" src="https://www.youtube.com/embed/-_-Vg_B3nzM" width="480" youtube-src-id="-_-Vg_B3nzM"></iframe></div><br /><p>This is not a trivial issue. EuSpRIG's website has a <a href="http://www.eusprig.org/horror-stories.htm">"horror stories" section</a> that demonstrates the gravity of errors in the wrong type of spreadsheet. Even if we discard the few stories involving malware embedded in spreadsheets like <a href="https://blog.didierstevens.com/2016/01/07/blackenergy-xls-dropper/">the BlackEnergy power plant shutdown</a> - for many reasons it makes sense to count and study malware separately from unintentional human and formulaic errors - the EuSpRIG lists dozens of separate incidents that involve <a href="http://catless.ncl.ac.uk/Risks/16.72.html#subj1">massive financial losses</a>. Taxes, criminal and medical records are all stored on spreadsheets. Single digit error rates have major repercussions.</p><p>Claims putting error rates for spreadsheets in the 84% to 90%+ percent range have been around for many years and in a wide variety of circumstances. Hawaii prof Roy Panko's work in this area is particularly compelling, and <a href="http://panko.com/ssr/index.html">his website is worth reviewing</a> even if you find the eye-catching statistic risible.</p><p>Absent further investigation we can assume that spreadsheet error rates are substantial because many spreadsheets involve an activity that researchers have long known are error-prone in humans: repetitive simple tasks, or what the literature describes as "simple nontrivial cognitive tasks". When spreadsheets are created by human input, they often involve repeatedly typing in small strings of text and digits. Rates if error for these types of task tend to be less than 5%. </p><p>But spreadsheets also have properties that make them particularly sensitive to small errors. A spreadsheet divides input into discrete cells, and allows users to perform calculations on that input by applying formula to those cells. This means that a single cell can effect any other cell in a spreadsheet. A human user might only mistake one cell's worth of input, but that mistake could impact every other cell.</p><p>The ratio of correct vs incorrect cells is not the only (or even preferable) way to determine the efficacy of a spreadsheet. The researcher might prefer to use a pre-established technique for auditing data errors and categorizing the number of errors identified by that technique in a given spreadsheet. </p><p>What does all of this mean, exactly? It never hurts to view data with a skeptical eye. We can create more accurately predictive models using spreadsheets if we account for the sort of inaccuracies we tend to find in them <i>in addition to</i> other sources of error in the data, such as collection methodology, et al. In this sense, spreadsheets are just like any other representation of information that human beings can create: useful in many ways, but slightly imperfect.</p>Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-9503185342881865252021-06-24T13:24:00.007-05:002021-08-31T11:50:11.721-05:00Annoying Error in Cassandra Quickstart using Docker for Windows<p><span style="font-size: small; font-family: courier;">I needed to setup a quick Cassandra environment in Windows today, but I ran into a problem when executing the <a href="https://cassandra.apache.org/quickstart/">quick start guide from Cassandra's (excellent) website</a>.</span></p>
<p><span style="font-size: small; font-family: courier;">The quick start assumes a working Docker environment. Because this is in Windows, WSL 2 has been configured, and the Docker for Windows binary has been properly installed:</span></p>
<div class="highlighter-rouge"><code>docker pull cassandra:latest <br />
docker network create cassandra<br />
docker run --rm -d --name cassandra --hostname cassandra --network cassandra cassandra</code></div>
<div class="highlighter-rouge"><p style="text-align: left;"><span style="font-size: small;"><span style="font-family: courier;"><span><code><span>From here you can either </span>load data into Cassandra from a file or start a prompt. A prompt is supposed to be opened like this per the documentation:</code></span></span></span></p></div>
<div class="highlighter-rouge"><span style="font-family: inherit;"><code>docker run --rm -it --network cassandra nuvo/docker-cqlsh cqlsh cassandra 9042 --cqlversion='3.4.4'</code></span></div>
<div class="highlighter-rouge"><span style="font-family: inherit;">This produces an error:<br /> </span></div>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEheid5XzEi0BTbrtpWe-1AB1FzsRtuxbFVIyaotlXRFBbzT9S9yEjOzFs2gBOouS4rhmcT7oqakS_ALftVwgbtkG818ng1R177LHDGNUgTujNNfnC6yre4KRc7dVu2funrfSCCx2BQ9kMk/s666/docker+error.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="172" data-original-width="666" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEheid5XzEi0BTbrtpWe-1AB1FzsRtuxbFVIyaotlXRFBbzT9S9yEjOzFs2gBOouS4rhmcT7oqakS_ALftVwgbtkG818ng1R177LHDGNUgTujNNfnC6yre4KRc7dVu2funrfSCCx2BQ9kMk/s16000/docker+error.jpg" /></a></div>
<br/>
<p>Error messages like this can be a little intimidating. There's a hint, though, in the last line. ValueError and 'invalid literal for int()' indicates that <a href="https://www.w3schools.com/python/python_casting.asp">a variable has been miscast</a>. We aren't told what the name of the variable is, but we are told that the variable should have been an integer. From this alone we could tell that there is likely a problem with one of the numbers I provided in the command line statement. I'm just trying to run docker from a Windows command line here, and I didn't feel like dealing with Powershell. This alone is a really good reasons to start throwing different types of quotes on the numeric variables I'm providing to Docker, 9042 and '3.4.4.'.</p><p>Helpfully, the last few characters of the error message let me know which one of these two numbers I should be focused on: "''3" is only part of '3.4.4'. Sure enough, replacing the single quotes with double quotes fixed the command:</p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgP052v_4EZQfDwIz4l8aTpOWJ0ktcCqJBvdFBFWQj9wQknsOM6EzKREYAaCA5gl0kBfZnoBrx2pnfiS2yEUPp-4Ll6UNACMYkDyY-NjuzbmxS_sv4tITyD3qZIYKx4Cg7Uuxet3NljfZA/s660/docker+win.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="146" data-original-width="660" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgP052v_4EZQfDwIz4l8aTpOWJ0ktcCqJBvdFBFWQj9wQknsOM6EzKREYAaCA5gl0kBfZnoBrx2pnfiS2yEUPp-4Ll6UNACMYkDyY-NjuzbmxS_sv4tITyD3qZIYKx4Cg7Uuxet3NljfZA/s16000/docker+win.jpg" /></a></div>
<br />
<p>This is pretty basic stuff, but I wasted more time than I would have liked to on it (its funny this comes a day after I post about bias from authority)</p>.http://www.blogger.com/profile/16490250863425476101noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-9203411921749614132021-06-23T23:00:00.000-05:002021-06-23T23:00:00.738-05:00Accounting for bias when analyzing public data<p style="text-align: left;"><span style="font-size: 18.72px; font-weight: normal;">We tend to overestimate the reliability of authority figures, and this impacts how we should analyze data for public policy.</span></p><h3 style="text-align: left;">Public data is an intrinsic appeal to authority</h3><p style="text-align: left;"><a href="https://wonder.cdc.gov/Deaths-by-Underlying-Cause.html">The CDC's WONDER database keeps track of causes of death within the United States</a>. When a death certificate is created for a person in the United States, the certificate includes a special code indicating the cause of death. Through a lengthy process, that information makes its way from the funeral home or hospital to a state registry to the National Vital Statistics System and finally to the CDC. CDC tracks that information in WONDER, which can be partially queried by the public.</p><p style="text-align: left;">WONDER is used by scientists, researchers and journalists for all sorts of reasons. It was data from WONDER that largely provided the justification for the claim the the United States has been undergoing an epidemic of heroin addiction. And by any measure, the US has a serious problem with heroin and abuse of other opiate drugs. But WONDER can only provide us with a rough indication of what the problem looks like in reality.</p><p style="text-align: left;">Lets say you want to count the number of people who have overdosed from heroin specifically - you want to make sure that overdoses from oxycontin and fentanyl are counted separately, because the chemicals have vastly different chains of custody and responsibility. Heroin in the US in 2021 is provided largely by drug cartels in Mexico. Oxycontin is created by legal drugmakers in the US and diverted or stolen. Fentanyl is somewhat of a combination of these two pictures, with the added wrinkle that the primary source of precursors for illegal fentanyl manufacture are provided by China. Let's say you want to determine the damages a legal drugmaker is responsible for in <a href="https://www.americanbar.org/news/abanews/aba-news-archives/2019/09/opioid-lawsuits-generate-payouts-controversy/">one of the recent lawsuits</a> surrounding that issue: there are many possible situations where knowing the difference between these numbers has profound consequences. </p><p style="text-align: left;">When we search WONDER for cause of death information, researchers tend to make several assumptions. </p><p style="text-align: left;"><span> 1. There is a consistent criteria for establishing cause of death</span><br /></p><p style="text-align: left;"><span><span> 2. The cause of death criteria must reliably distinguish between various causes of death</span><br /></span></p><p style="text-align: left;"><span><span><span> 3. The rate of processing errors must be minimized.</span><br /></span></span></p><p style="text-align: left;">All three of those assumptions are wrong as it relates to our test query for heroin-only deaths within WONDER. Of course, that's quite a claim from someone who doesn't have a medical degree. What am I talking about? Let's review each complaint above in more detail:<br /></p><h4 style="text-align: left;">There is a consistent criteria for establishing cause of death</h4><p>The information within WONDER is the result of thousands upon thousands of judgements by individuals across the country. Ostensibly, the <a href="https://www.cdc.gov/nchs/icd/icd10cm_pcs_background.htm">International Classification of Diseases</a> (ICD-10) provides just that. The ICD is a labeling system that allows clinicians to use the same "language" to specify diagnosis. That should ensure that diagnosis are specific and consistent.</p><p>Although ICD-10 codes themselves <i>can be</i> specific, there is a substantial degree of ambiguity around . There is no consistent way of resolving that ambiguity, and so we lack a consistent methodology for establishing cause of death. </p><p>A related, but distinct, concern here is not only how the criteria is applied but <i>who </i>is applying the criteria. There tends to be an assumption that cause of death is ascertained by a physician. That is true for most cases, but in some States the Coroner is an elected position with no formal medical requirement. Trained doctors are capable of doing bad work, too. The <a href="https://www.wsj.com/articles/the-cadaver-king-and-the-country-dentist-review-justice-miscarried-1519862849">consequences of incompetence</a> among US coroners is tragic, ongoing and not widely understood, but we should assume that its impacts could be statistically significant.</p><h4 style="text-align: left;">The cause of death criteria must reliably distinguish between various causes of death</h4><div>Let's assume for a moment that the problem with ICD-10 doesn't exist: the criteria are identical and applied the same way across the country. But what if the criteria don't work?</div><div><br /></div><div><a href="https://www.merckmanuals.com/home/drugs/administration-and-kinetics-of-drugs/drug-metabolism">The Merck Manual describes drug metabolism</a> as follows:</div><div><br /></div><div><span style="background-color: white; letter-spacing: 0.14px;"><span style="font-family: inherit;"></span></span><blockquote><span style="font-family: inherit;"><span style="background-color: white; letter-spacing: 0.14px;">Some drugs are chemically altered by the body (metabolized). The substances that result from metabolism (metabolites) may be inactive, or they may be similar to or different from the original drug in therapeutic activity or toxicity. Some drugs, called prodrugs, are administered in an inactive form, which is metabolized into an active form. The resulting active metabolites produce the desired therapeutic effects. Metabolites may be metabolized further instead of being excreted from the body. The subsequent metabolites are then excreted. Excretion involves </span>elimination of the drug<span style="background-color: white; letter-spacing: 0.14px;"> from the body, for example, in the urine or bile.</span></span></blockquote><span style="background-color: white; font-family: "Open Sans", Fallback, sans-serif; letter-spacing: 0.14px;"></span></div><div>How does this relate to how we detect overdoses in corpses? If drugs change when humans take them, how can we determine the sort of subtle measurements in concentration and causation to determine, for example, whether a patient with lung cancer stopped breathing because of their tumor or their medication? The <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2561112/">abstract from Ferner's 2008 article</a> in the British Journal of Clinical Pharmacology tells us we cannot:</div><blockquote><div><span style="background-color: white; font-size: 15.9991px;"><span style="font-family: inherit;">Clinical pharmacology assumes that deductions can be made about the concentrations of drugs from a knowledge of the pharmacokinetic parameters in an individual; and that the effects are related to the measured concentration. Post-mortem changes render the assumptions of clinical pharmacology largely invalid, and make the interpretation of concentrations measured in post-mortem samples difficult or impossible. Qualitative tests can show the presence of substances that were not present in life, and can fail to detect substances that led to death. Quantitative analysis is subject to error in itself, and because post-mortem concentrations vary in largely unpredictable ways with the site and time of sampling, as a result of the phenomenon of post-mortem redistribution. Consequently, compilations of ‘lethal concentrations’ are misleading. There is a lack of adequate studies of the true relationship between fatal events and the concentrations that can be measured subsequently, but without such studies, clinical pharmacologists and others should be wary of interpreting post-mortem measurements.</span></span></div><div></div></blockquote><div>Counting opiate-related morbidity is particularly difficult, because <a href="https://watermark.silverchair.com/32-4-319.pdf?token=AQECAHi208BE49Ooan9kkhW_Ercy7Dm3ZL_9Cf3qfKAc485ysgAAApswggKXBgkqhkiG9w0BBwagggKIMIIChAIBADCCAn0GCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMTBxQPtADzRT7RMWyAgEQgIICTnJc4kOxsFp_VZ_6lvAxHUdIwNjAK_W3VdyznHNjNrFiAbGziEUNxJwxJ7tm1NiCOv0zJQN7dLnW12Rk8VtyogsnOrJQbJy5JAPzCwjm0ToSA3O2fhmvaa9gkRWGtAPTaMpsKFwPOPN-R6RPMlJ0WeiN-Me-LY3riWcJaen6UAw-YCnCs8SkiFxBprJmSxisgXmVf2z0dqXYylVWfaIyWutcNP2GXZa6uNnH9FoM5ZrGFQr3cejx9ew-WZqJF4N8OqCTH5rt3SpzQ8QuaWTcmOTCpmdapoNsOJlQjya3Osd7t-xPRsil0O8fmE0HW1GXYlD3L7ZsDTsg2Jhonv0zYY66FfB3gX9H4ZjaxRoBTqX_3CSRiVvTtNJSX8mOm5NNVBc4NYmxAUtnrKAXy6Gg7ywfeEUXv6CedvN_LTS9XkqGj7qUuM44ECJ2IaKo39Rz772ID_giufhXBVHPTowSL0hwTG1itnpOT35AGsd31P4Gl2LogH1DN2INho9ckv6_8db72CvGo3sF7SY855TUaRY3V0Mq341GRdwOz4IqP6ZGbB2BB4hJTOR9kz4N-ZbwBW_Yn1jVQNB9ZI-m2Pzot8W1kJ9FtaTnKB4xS1mFDaw0vU7VHfEK1so7AvqzMeWhQbo1tDco6kgemOESH8RqO-iKPOuHvSte4va8vU0hvu7Jg0LJHwFGrdmrZ7iGfAk14DOwBJaMsoyAmQ-Phq_E-8ZZJNXpzMTvsB7J6UVSmUC7ztRErZFnooxMvvoj7AhIQ0Ua4Ao1cd4oVG6iJxj5">opiates tend to metabolize into other opiates</a>. How can you count the number of people who die from a specific opiate if users of different drugs will possess metabolites for the drug you're looking for?</div><div><br /></div><div>This particular error compounds with the previous error as doctors make their own determinations on how to resolve this particular ambiguity.</div><h4 style="text-align: left;">The rate of processing errors must be minimized.</h4><p style="text-align: left;">This concern is somewhat different than the last two. The prior two concerns were about how the data within WONDER is collected. Solving those problems would involve changing the sampling process of WONDER. When we talk about processing errors, we ware talking about problems that are involved with the administration and storage of the data rather than its initial collection.</p><p style="text-align: left;">Individual states have different rules for determining cause of death and who can make that determination. They also have very different ways of collecting and storing that data at the state level. I have only reviewed data from this level of aggregation from a few states. In one state, I identified a miscounting in death data for a single county. Apparently there had been a typo that added a digit to the number of deaths for that county.</p><p style="text-align: left;">This example stuck with me years after the fact because of how easily this type of error could be detected with simple programming tools. The state data contained several tables with overlapping values. This created a situation where the value from the typo wasn't just unusual but inconsistent with the other data. I didn't need to write some sophisticated model of this information - the value wasn't obviously wrong in terms of mean/median/mode sort of stuff, either. But the value didn't match other instances in the same dataset, and when that value was assumed to be true it invalidated pre-calculated column sums. Looking for this sort of error is the bare minimum in terms of minimizing errors in process.</p><h4>Ad verecundiam</h4><p style="text-align: left;">Appeals to authority are a fundamental logical fallacy: ad verecundiam. It gets all of us from time to time. We tend to assume that information provided by a trusted authority is more reliable than the same information provided by another source. Government is the classic example of such an authority.</p><p style="text-align: left;">None of the individual problems I have listed here are new to statisticians. These are very old problems. </p><p style="text-align: left;">Because researchers tend to overestimate the reliability of information provided from authority figures, we tend to underestimate errors in public data. When confronted with evidence of substantial process errors in public data, we tend to assume that these errors have already been "accounted for" in some way.</p><h4 style="text-align: left;"><p style="font-weight: 400;">Statistics provides us with the tools to account for bias introduced through both sampling and non-sampling errors. However, to accurately adjust for that bias it is necessary to accurately gauge the extent of that bias. </p></h4><h4 style="text-align: left;">This isn't about the CDC</h4><p>I've picked on a database that is administrated the CDC here. The CDC is just an example whose data I've worked with. The problem I am trying to illustrate isn't with the CDC but with the people who rely on information provided by the government: to some extent, that includes everyone.</p><p>None of the concerns here assume any sort of malevolence on the part of anyone in authority. Whether or not people in government are immoral people isn't germane. Bias must be accounted for any time we deal with statistics. My argument is precisely that sources of government information are <i>not</i> special in this respect. There would be bias problems whether the data was created by butchers or bakers or candle stick makers. </p><p>It is possible to underestimate reliability, as well. Even taking into account the issues I outlined above, the WONDER database is by far the most reliable source for national cause of death information. Weighting its reliability alongside sources with even less reliable processes does not get us closer to a clear representation of reality. The analysis of data compiled by the government can be a force for enormous social utility, but that capacity for utility is contingent upon our ability to truly listen to the numbers.</p>Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-27158164409444604512021-06-20T17:37:00.002-05:002021-06-20T17:37:30.336-05:00I've got my eye on you, Windows S-Mode<p>Over the last month or so I've noticed that all of the ads and retail descriptions of low-end PCs I have been seeing have updated their description of the included version of Windows. Instead of promising that purchasing a new PC includes "Windows 10", "Windows 10 Home" or "Windows 10 Pro", the operating system was listed as "Windows 10 S", as shown in the screenshot below:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgwni4m7tlkLOx1ZKkXX8v1JLw38XEIIejsWDNgvraUlCaKdZh7Q3cE45rxrruoSvI1DojUuq4Bd3mZ6qhp4DOXLi4ZXRSNzIqeQyUWxvHgA5UA0gc-TDaKGMCTY5KJjRhwMggmaDibM7E/s1891/windows+10+s+edition+example.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="Retail desktop PC ads on Amazon" border="0" data-original-height="932" data-original-width="1891" height="316" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgwni4m7tlkLOx1ZKkXX8v1JLw38XEIIejsWDNgvraUlCaKdZh7Q3cE45rxrruoSvI1DojUuq4Bd3mZ6qhp4DOXLi4ZXRSNzIqeQyUWxvHgA5UA0gc-TDaKGMCTY5KJjRhwMggmaDibM7E/w640-h316/windows+10+s+edition+example.jpg" title="Retail desktop PC ads on Amazon" width="640" /></a></div><br /><p>Windows 10 "S-Mode" isn't new: it was released as part of the Windows update released in April 2018. But using the term in a sales context - promising new PCs will include Windows S-Mode, while not clarifying what actual version of Windows will be included, is very new and suggests there is a tectonic shift in thinking about the Windows operating system and its relationship to consumers.</p><p>What is "S-Mode"?</p><p>According to Microsoft's FAQ, S-Mode is not a version or edition of the Windows 10 license, in the way that Home or Pro edition is.</p><p>Instead, S-Mode restricts certain basic Windows functionality. A computer would need to be have S-Mode enabled at the time of Windows 10 installation, whether that occurs as part of a pre-packaged install when a customer purchases a new PC or as part of an organization's volume license deployment process.</p><p>What functionality are we talking about, exactly? The use of browser is restricted to Microsoft Edge and only applications from the Windows Store can be installed. Comparisons have been made to ChromeOS, but they don't quite add up - there aren't explicit prohibitions like this.</p><p>Microsoft has stressed how these restrictions will make PCs running S-Mode faster and more secure. While true, this is true in a sense that is almost flippant. An S-Mode PC will run faster on average because it is doing less. By this metric, the fastest PC is the one that doesn't do anything at all.</p><p>Earlier this year, <a href="https://www.techrepublic.com/article/windows-10-s-mode-pros-and-cons/">an article covering the shift to S-Mode in TechRepublic</a> trumpeted the utility of S-Mode in education environments and S-Mode's ability to run on ARM processors. These reasons, too, strike me as rationalizations rather than the purpose of S-Mode. Native ARM compatibility through <a href="https://docs.microsoft.com/en-us/windows/uwp/porting/apps-on-arm">Windows 10 for ARM</a> may have been released at the same time as S-Mode, but S-Mode involves the much larger x86 desktop market. It is similarly unclear why a new "Mode" that will involve PCs that are not used for educational purposes is neccessary to help the education market: Microsoft has a variety of programs available, including unique licensing options for education, that would have allowed Microsoft to deliver unique products to educators. </p><p>It makes more sense to think of S-Mode as the latest iteration of a long-term goal at Microsoft: transforming Windows from a series of discrete (and unique) operating systems into a kind of grand, unified application framework. Providing developers with a single framework allowing for application execution in as diverse an array of devices as possible is the implicit goal of projects like the <a href="https://docs.microsoft.com/en-us/windows/uwp/get-started/universal-application-platform-guide">Universal Windows Platform (UWP)</a> and the Microsoft Store itself.</p><p>There is nothing wrong with that goal, but the Microsoft Store requirement makes it clear that a unified platform will be a curated platform. </p><p>For now, Microsoft continues to promise users the ability to <a href="https://support.microsoft.com/en-us/windows/switching-out-of-s-mode-in-windows-10-4f56d9be-99ec-6983-119f-031bfb28a307">deactivate S-Mode</a> without additional charge. Deactivating S-Mode should leave users with Windows 10 Home Edition. The S-Mode conversion process is one-way only: users would need to reinstall Windows to activate S-Mode again once it has been deactivated. This one-way conversion process is hard to understand with Windows 10 licensing in its current form. There are substantial differences in how applications function when executed within the Windows 10 Store framework vs the MSI framework. Application files are stored in different places, for example, but "disabling" S-Mode currently doesn't radically change the Windows that is installed on the device, yet.</p><p>It is hard to see how any short or medium term future could spell a default version of Windows 10 that is entirely curated. But it is not outside the realm of possibility that new Windows users will be charged to disable S-Mode at some point in the future. Apart from the additional cost, a curated Windows environment creates a situation in which there is <b><i>no </i></b>non-curated application environment available for non-technical users. The relative merits of a single company (Microsoft) curating its applications are dwarfed by the larger implications of creating a world where your application will have to be explicitly approved by one of three companies (Microsoft, Google, Apple) to have any practical chance of being installed on a user device.</p>Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-84781050127776370202021-03-16T07:14:00.003-05:002021-03-16T07:21:50.679-05:00An Anatidae Odyssey<p>During the pandemic I've been spending my time in a somewhat rural area. There's a lake. There's bugs. There's lots of flora and fauna of all shapes and sizes. Every once in a while I spot something incredible. About a year ago I was enjoying my morning coffee when I happened upon this in my backyard:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiF-1BS0xQ8JKZA4_dqL__h6bzvLI2eoGi63XbaOCl-fZ2ReL7_w2uMWnF12mzlk_FViXyq3FidNiMZWAfMFw8wJaiABHpO7I8UltzrtSVhzIw-6b5_6r_9NALmJWAPyu02G7lL4O6kTTg/s2048/IMG_20200314_114425.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="That's an alligator in my backyard, y'all" border="0" data-original-height="1536" data-original-width="2048" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiF-1BS0xQ8JKZA4_dqL__h6bzvLI2eoGi63XbaOCl-fZ2ReL7_w2uMWnF12mzlk_FViXyq3FidNiMZWAfMFw8wJaiABHpO7I8UltzrtSVhzIw-6b5_6r_9NALmJWAPyu02G7lL4O6kTTg/w640-h480/IMG_20200314_114425.jpg" title="That's an alligator in my backyard, y'all" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilpFSOzmTCrLc9blUnQbSwYZkcHKXQXwFeM4om_Mb2SKb1PCqfBcpMSw1eemz6B6VPyV19mx6OpmtpMD73HSrywTpAhtqYPsEwp2vZ4pO4Flemlep49vUCJhA3FfdouxXDK7vOR5_ypK0/s2048/IMG_20200314_114457+%25281%2529.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="Honey gator don't care" border="0" data-original-height="1536" data-original-width="2048" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilpFSOzmTCrLc9blUnQbSwYZkcHKXQXwFeM4om_Mb2SKb1PCqfBcpMSw1eemz6B6VPyV19mx6OpmtpMD73HSrywTpAhtqYPsEwp2vZ4pO4Flemlep49vUCJhA3FfdouxXDK7vOR5_ypK0/w640-h480/IMG_20200314_114457+%25281%2529.jpg" title="Honey gator don't care" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3REaTbsQwTuCuwiElFwvVxa6Pu6JDXtt8lzVqMJiXh-F6FP0uR2XuKakui4Hj4fLqBYzyQghQcZm9t_WhpnSULwWTCowCOMjyEcYyHlLiPxI3O9OiJqJHNwqJfikRIYnP_WMnKnO827A/s2048/IMG_20200314_114410+%25281%2529.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1536" data-original-width="2048" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3REaTbsQwTuCuwiElFwvVxa6Pu6JDXtt8lzVqMJiXh-F6FP0uR2XuKakui4Hj4fLqBYzyQghQcZm9t_WhpnSULwWTCowCOMjyEcYyHlLiPxI3O9OiJqJHNwqJfikRIYnP_WMnKnO827A/w640-h480/IMG_20200314_114410+%25281%2529.jpg" width="640" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><span style="text-align: left;"><br /></span></div><div class="separator" style="clear: both; text-align: center;"><span style="text-align: left;">Just FYI - I'm not using some giant </span><a href="https://www.borrowlenses.com/blog/what-is-a-telephoto-lens/" style="text-align: left;">telephoto lens</a> and zooming in from a mile away in those photos<span style="text-align: left;">. This gator was very close to my house and very comfortable there. Clearly, he lacks respect for private property.</span></div><p>I'm pretty confident that despite an APB from Fish & Wildlife, this gator is still at large. So when a poor duck hobbled into my backyard with a horrific mangled leg last week, the gator was my main suspect.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTQrbYqIuArdJz6OGdk91YG7I_81EJCSE3X3IvtggDnXSLwTqC-tccpcWpjTMLp_C_wafmAn5nqAG2gDVHR5mYQ5VphgYXRHRsgoxEpEQ7jukdJooK6kUgjjOTobCkGw_v64YypvYbS6U/s2048/IMG_20210216_091944.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="Please, sir, quack?" border="0" data-original-height="2048" data-original-width="1536" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTQrbYqIuArdJz6OGdk91YG7I_81EJCSE3X3IvtggDnXSLwTqC-tccpcWpjTMLp_C_wafmAn5nqAG2gDVHR5mYQ5VphgYXRHRsgoxEpEQ7jukdJooK6kUgjjOTobCkGw_v64YypvYbS6U/w480-h640/IMG_20210216_091944.jpg" title="Please, sir, quack?" width="480" /></a></div><p>The injury was shocking to look at & made it very difficult for the duck to move around. Making matters even worse for my feathered friend, the injured duck is considered invasive to my area. The excellent staff at the wildlife rescue operations near me would be legally mandated to euthanize the duck if I brought it to them. </p><p>I'll spare you a debate about the rightness or wrongness of euthanizing invasive species - its a complex issue on which reasonable people can disagree. But in this case I wanted to make sure euthanasia was necessary. </p><p>So, after a long chat with a vet friend of mine and the aforementioned wildlife rescue folks, I've been taking some measures to help the duck. I've been feeding him a mix of poultry crumbles with shredded lettuce, giving him lots of fresh water, and keeping the part of my yard where he has set up his HQ clear of twigs and debris. At feeding times, I check the leg for signs of infection, make sure he's eating and drinking, etc.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrsYaqCMOmYQBWu8LiOQCnfASC1jTMx-JMZgHhQCRkmi2wUl4TXc42JzyQx1bzm9tPJHoanrA5SIuE9JNBJQliBVzkSf8C1faPeDEqoc0A9Za4jrt2jdjVGqMnhd7HcaKWtB7LZ6REoyM/s2048/IMG_20210315_172208_5.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="Duck delights in delectable dinner" border="0" data-original-height="1536" data-original-width="2048" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrsYaqCMOmYQBWu8LiOQCnfASC1jTMx-JMZgHhQCRkmi2wUl4TXc42JzyQx1bzm9tPJHoanrA5SIuE9JNBJQliBVzkSf8C1faPeDEqoc0A9Za4jrt2jdjVGqMnhd7HcaKWtB7LZ6REoyM/w640-h480/IMG_20210315_172208_5.jpg" title="Duck delights in delectable dinner" width="640" /></a></div><p>So far, so good. Not only has the little dude started putting some weight back on, he is learning to walk again with just the one leg. He can fly & swim again, too. Not out of the woods yet, of course, but for just going through a duck version of the plot from <a href="https://www.rottentomatoes.com/m/127_hours">127 Hours</a> he is doing pretty good.</p><p>Speaking of movies, I'm convinced there is a Pixar script in this story somewhere.</p>.http://www.blogger.com/profile/16490250863425476101noreply@blogger.comUnited States37.09024 -95.7128916.749910824619004 -130.869141 67.430569175381 -60.556641tag:blogger.com,1999:blog-4411720504608505363.post-62484491567930332192021-03-11T07:34:00.002-05:002021-03-11T07:34:29.797-05:00Winning web design, from Amazon<p> Oooo boy, I can't wait to browse discounted "Dog Supplies" on Amazon. <br /></p><p><br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglVCoUNxLg-GyKHsrW_T_W72JPzSKSb-Y9vtiVJiLxahgKL-2Q7IqdBRwgtzk9Ey_Syz7d6ZDnbpi_lFUEUKiG-4dcm6DrBxEh1WkthR_iAdvDEBsL5aq2WPLZPDd-Py3uGvaXcD7a78A/s462/amazon-dog+supplies.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="Dog Supplies?" border="0" data-original-height="462" data-original-width="400" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglVCoUNxLg-GyKHsrW_T_W72JPzSKSb-Y9vtiVJiLxahgKL-2Q7IqdBRwgtzk9Ey_Syz7d6ZDnbpi_lFUEUKiG-4dcm6DrBxEh1WkthR_iAdvDEBsL5aq2WPLZPDd-Py3uGvaXcD7a78A/w277-h320/amazon-dog+supplies.PNG" title="Dog Supplies?" width="277" /></a></div><p></p><p>Wait. What?<br /></p>.http://www.blogger.com/profile/16490250863425476101noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-66745471657000860702021-02-26T14:06:00.006-05:002021-03-11T07:30:41.628-05:00Microsoft EOL'd Windows 7 during a pandemic & its hurting medical practices<div class="separator" style="text-align: center;"><p style="margin-left: 1em; margin-right: 1em;"></p><p style="text-align: left;">Microsoft fully <a href="https://www.techradar.com/how-to/how-to-prepare-for-windows-7-end-of-life">ended support for their Windows 7 product</a>
in January of last year. The change is primarily administrative:
Microsoft will no longer distribute security patches for free with
Window 7 or guarantee its functionality.</p></div><p>It is not a sudden move
by Microsoft: the company has a well-documented support cycle for all
versions of Windows, and Windows 7 customers were given plenty of
notification, including from pop-ups from within Windows that users have
to minimize to continue using the computers. This post is not meant to
imply that Microsoft did not make a good faith effort to notify their
users.</p><p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://upload.wikimedia.org/wikipedia/commons/thumb/0/0a/Unofficial_Windows_logo_variant_-_2002%E2%80%932012_(Multicolored).svg/544px-Unofficial_Windows_logo_variant_-_2002%E2%80%932012_(Multicolored).svg.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="480" data-original-width="544" height="295" src="https://upload.wikimedia.org/wikipedia/commons/thumb/0/0a/Unofficial_Windows_logo_variant_-_2002%E2%80%932012_(Multicolored).svg/544px-Unofficial_Windows_logo_variant_-_2002%E2%80%932012_(Multicolored).svg.png" width="334" /></a></div><p style="text-align: center;"><br /></p><p>Also: Windows 7 is not a good product at this point. Windows 7 is fundamentally insecure and unstable with modern applications (and has been for years), regardless of what support cycle it is in.</p><p>That said, there are many unique situations for which running an out-of-date version of Windows is the only practical option. I've found this to be particularly true in the medical field, among practices of all sizes. The reasons why this is the case aren't widely understood, even by the physicians and executives who ultimately make purchases for things like office software licenses, so I thought I might talk about why I believe this to be the case, and why I think that Microsoft should loosen EOL restrictions on medical users of Windows 7.<br /></p><p>It is fairly common knowledge that the FDA is responsible for manufacturing medical devices. When most people think about medical devices, they might think about an artificial implant, or a surgical tool, or maybe an X-Ray machine. Software is not widely understood as being a medical device, but if that software is included as part of a medical device, or is used by a company that makes medical devices, that software is regulated under rules just as stringent as those for the "device" itself. There are many differences between a software program and a physical medical device; the difference that is important for our discussion today is that a physical medical device is a static item that exists largely in the same state over a long period of time. It might be necessary to maintain the device, or the device might expire, but it is unusual to think of a medical device as <i>requiring substantial modification over time in order to continue providing the same behavior over time</i>. Yet, this is precisely the case with medical software.</p><p>The term "software for medical devices" is a very wide net, but as it relates to the issue of Windows licensing and support, I am thinking primarily about a specific type of software called a driver that is designed to allow a physical device like a camera or x-ray machine to "talk" to a computer. It is common for, say, a medical imaging device to be connected to a Windows PC in order to actually work. For regulatory purposes, the device doesn't include the PC. But practically, the Windows PC is required for the device to function. The "device" is a package deal between the medical device, the medical software, the PC and Windows.</p><p>When Windows is in its normal support cycle, Microsoft is regularly pushing a variety of updates and patches to Windows that change Windows in ways both large and small. These changes often have a direct influence on medical devices or their software. It is a common experience in a medical office for a Windows update to break something. Physicians in this situation are left with a choice: roll back the Windows update or do without the medical device.</p><p>In order to simply continue to provide the same functionality, medical device manufacturers must release updates for the software drivers that accompany their products. This is common practice for manufacturers of the vast majority of computer peripheral manufacturers, whether they make Mice or cameras or keyboards or printers. However, only medical device manufacturers must request FDA authorization for that software update. Regardless of whether that authorization process is overly burdensome or not, it exists, it costs medical device manufacturers significant additional expense, and it takes time. Many medical device manufacturers and software makers recoup these costs by charging licensing fees to allow device driver downloads. To an end-user physician, the results can be absurd situations where the level of service decreases with the level of investment: why pay thousands of dollars for a "medical" camera's software updates when a nearly identical camera that is not a medical device will tend to "just work" for years to come?</p><p>Several years ago, the FDA began to try to streamline the process of approving medical device software updates to address this issue. <a href="https://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM514737.pdf">Revised guidance released</a> in late 2017 for the 510k software approval form was designed to expedite approvals of medical device software updates, particularly for updates that do not change the functionality of the device. This course of action isn't risk-free from the regulator's perspective, and future administrations will certainly continue to modify this part of the approvals process. Even if a software update is not <i>intended</i> to fundamentally change a medical device's behavior doesn't mean that a bug included with that update might prove dangerous to a patient. There are no easy answers to this problem.</p><p style="text-align: center;"><img alt="Quite a few doctors have seen this before" class="n3VNCb" data-noaft="1" src="https://errorcodespro.com/wp-content/uploads/2017/08/DRIVER_CORRUPTED_EXPOOL-Cover-BSoD-Windows-Wally-450x316.jpg" style="height: 316px; margin: 0px; width: 450px;" title="Quite a few doctors have seen this before" /> <br /></p><p>So let's return to Windows 7's end of support. Because of the software device driver regulatory burden described above, physicians - particularly those providing services in underprivileged communities and with patients of less means - just don't have the capitol to afford annual repurchases of all of the various medical applications *and* Windows *and* regular PC and equipment updates. In the case of imaging devices, upgrading the OS of an attached PC can mean repurchasing new physical components for that X-Ray machine. These aren't small costs we are talking about.</p><p>Let's also not forget that Microsoft ended support for Windows 7 the same month that COVID-19 pandemic began rearing its head in the US: January 2020. Yes, doctors ignored repeated update notifications. <i>They may have had other things on their mind at the time</i>.<br /></p><p>Microsoft also made a slight modification to their prior end of support cycle rules. For Windows 7, Microsoft is enabling customers to purchase ongoing security patches for an annual fee. The fee starts at a couple of hundred dollars a year, increasing every year. Microsoft can't claim that they do not have the security patches available.</p><p>Given the unique situation facing medical professionals, between the extraordinary costs associated with upgrades to facing down the beginning of a once-in-a-century pandemic the same month that Microsoft ended Windows 7 support, would it be so much to ask for Microsoft to extend the Windows 7 security patches to medical offices? Call it a PR move/tax deduction/whatever. Doctors had a rough year. Give them a break.</p>Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-13591418587474159282021-02-25T13:34:00.010-05:002021-02-25T13:39:26.573-05:00EU says Pee-Yoo to transatlantic data sharingOver in Europe, <a href="https://www.reuters.com/article/us-facebook-privacy-dixon-interview/eu-u-s-data-flows-could-face-massive-disruption-irish-regulator-idUSKBN2AP009">Irish Data Protection Commissioner Helen Dixon has just succeeded in her push to prevent Facebook from transferring data on European users back to the United States</a>. <br /><p>At issue is a series of data transfer agreements between the EU and the US; it is the (reasonable) contention of Commissioner Dixon that the United State's regime of warrantless spying makes Facebook unable to comply with the data sharing arrangement required by the EU. Ireland may not seem like a major global hub for Big Data and telecom firms, but it very much is. The Emerald Isle's famous "Celtic Tiger" economic push successfully attracted some of the largest technology firms in the world with low taxes and easy regulations.</p><p>But the times, they are a-changin'.</p><div style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTMr0VFGd2LVphlfG2TLY3wOWm84Dt-otnzlq0vaLyN181Ub20JcHicakp_k60iPVKrCTcBOQEX4HyyW_V0X_MV2w8mHShvJvu2xN7KG0xV_5dDskmuse3nM8wHfV3NhAYTS67H0aVlYo/s290/critic.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="honey, when did you buy this data? it smells "off"" border="0" data-original-height="290" data-original-width="290" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTMr0VFGd2LVphlfG2TLY3wOWm84Dt-otnzlq0vaLyN181Ub20JcHicakp_k60iPVKrCTcBOQEX4HyyW_V0X_MV2w8mHShvJvu2xN7KG0xV_5dDskmuse3nM8wHfV3NhAYTS67H0aVlYo/s16000/critic.jpg" title="honey, when did you buy this data? it smells "off"" /></a></div><p></p><p>This is the latest wrinkle in an ongoing battle over privacy regulations responsible for protecting European user data that traverses the Atlantic, but not the first. In September of last year, <a href="https://epic.org/privacy/intl/privacy-shield/">Commissioner Dixon had released an initial injunction against Facebook on precisely this issue that was eventually blocked by the Irish High Court</a>.</p><p>Facebook is the target of this specific move on the part of the Irish Data Protection Commission, however other firms can expect bad news from Ireland in the coming days and months. The Commission has a total of 27 pending investigations on this issue targeted a swath of firms that range from Twitter to Google to Verizon. <br /></p><p>The implications for firms that depend on collecting massive troves of data about an international user basis could be dire - and the companies that serve them - are potentially dire. One of the most obvious consequences of the move is that it presents these firms with an offer they can't refuse: move all of your data to a European hosting facility, or get lost.</p><p>Setting aside the many basic problems with warrantless surveillance in the context of human rights, one of the repeated concerns voiced about the over-reliance on NSA global surveillance by members of the IT and telecom industry is the unintended consequence that surveillance has on global confidence in US IT products. For decades, the US has been the undisputed leader in all manner of computer technology. Many professionals - including myself - long avoided IT products from authoritarian countries precisely because their governments could not be trusted. But thanks to the post-9/11 "collect it all" methodology implemented during the Bush administration, continued by the Obama administration & forgotten during the Trump administration, it is becoming more and more difficult to tell exactly why an IT product manufactured or hosted in the US is fundamentally more trust-worthy than a similar product in, say, China.</p><p>Whether or not you support global warrantless surveillance, the EU's actions are making it clear that there will be a substantial economic cost to totalitarian surveillance as well as a moral one.</p>Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comIreland53.41291 -8.2438925.102676163821151 -43.40014 81.723143836178849 26.91236tag:blogger.com,1999:blog-4411720504608505363.post-16795394830387198852021-02-24T09:56:00.003-05:002021-02-25T14:11:05.977-05:00Web Cruisin'<p> </p><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="266" src="https://www.youtube.com/embed/842oaGAK_1g" width="320" youtube-src-id="842oaGAK_1g"></iframe></div><br /><p></p>
<p>The internet was a lot more fun in the 90s.</p>Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-74139847508132830142020-12-16T13:13:00.003-05:002021-02-25T13:41:46.077-05:00What is SolarWinds Orion and why should I care that it was hacked?<p><i>Full disclosure: I've been employed by several companies that were customers and/or vendors of SolarWinds. However, I have never been employed by SolarWinds and I was not compensated for this post.</i><br /></p><p>On December 13th, digital security firm FireEye published a post to their blog with the comprehensive title "<a href="https://www.fireeye.com/blog/threat-research/2020/12/evasive-attacker-leverages-solarwinds-supply-chain-compromises-with-sunburst-backdoor.html">Highly Evasive Attacker Leverages SolarWinds Supply Chain to Compromise Multiple Global Victims With SUNBURST Backdoor</a>". The post identified a digitally-signed component of the Orion software, SolarWinds.Orion.Core.BusinessLayer.dll, that contained a backdoor. Multiple signed updates contained additional malware. Traffic from infected hosts was disguised using traffic resembling normal SolarWinds activity and avoided using IPs that were part of non-U.S. netblocks or assignments registered to "bullet proof" hosts that are frequented by criminals.<br /></p><p>Orion's compromised distribution platform was then leveraged to infect a wide variety of organizations. According to FireEye, the "victims have included government, consulting, technology, telecom and
extractive entities in North America, Europe, Asia and the Middle East".</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://d15shllkswkct0.cloudfront.net/wp-content/blogs.dir/1/files/2020/12/solarwinds.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="solarwinds" border="0" data-original-height="375" data-original-width="750" height="213" src="https://d15shllkswkct0.cloudfront.net/wp-content/blogs.dir/1/files/2020/12/solarwinds.png" title="solarwinds" width="425" /></a></div>SolarWinds was founded 21 years ago by brothers David and Donald Nonce around a pair of network monitoring products named "Trace Route" and Ping Sweep. The programs were just what they sounded like - simple ICMP packets to identify hosts. The novelty came from their availability within Windows within a GUI and these features would be a mainstay of SolarWinds and an early differentiator from linux-based products that tended to offer better functionality but a higher bar for usability. Over time, SolarWinds began to integrate more advanced monitoring capabilities, for example by using SNMP. SolarWinds offered an easy-to-install, Windows-based way to monitor lots of servers. The company grew quickly and has remained aggressive in its acquisition of smaller firms to introduce additional functionality to their monitoring products. <p></p><p>Before virtualization hit the scene, one of the big value adds for system administrators were applications that allowed for centralized management of server OS and applications. Creating, managing and fixing the often highly-customized systems that allow for that centralized management to happen is a large part of the system administrator's job. Monitoring systems like those offered by SolarWinds are a logical place to integrate that kind of functionality; monitoring software was already designed to communicate with large numbers of servers using protocols such as SNMP. Even better, monitoring systems tend to be modular i.e. supporting custom plugins for monitoring custom applications. The same plugin structure is used for management purposes. Eventually, companies began offering software services to replace these custom data center control systems and it is in this context that Orion should be understood. <a href="https://www.solarwinds.com/solutions/orion">The SolarWinds product page explains</a> that Orion provides "centralized monitoring and management of your entire IT stack, from infrastructure to application". Furthermore, Orion is marketed as an enterprise-scale application that can monitor or control "400,000 elements on a single Orion Platform instance" (an <i>element</i> is a single network node, interface or volume ... a single server of virtual machine may contain many elements, but even with this being the case the software is designed to support environments of 1000+ servers). <br /></p><p>Arguably, a system like this violates some of the principal rules of data security best practices. Like many people, one of the first things that I do when I install linux on a new computer is that I set up a new user account for myself; I then disable the root account (or at least its access to SSH). The idea is that, even when I am the only person using the computer, I am always trying to use the least access necessary to accomplish the task at hand. This isn't an idea specific to user account management or even security best practices. For example, minimizing access also forces users to evaluate their decisions prior to committing to them. The "Do you REALLY want to do this? Y/N" prompt isn't necessarily a part of user interfaces for security purposes, but you are still minimizing access to resources, making the user jump through a hoop.</p><p>SolarWinds repeatedly makes it clear that Orion is designed to be a
"full stack" solution that offers monitoring and management of all of
the OSI layers. But products like Orion, by centralizing so many functions, make it more likely that user error, system failure or security breaches have devastating, systemic consequences. <a href="https://www.newsweek.com/solarwinds-update-server-could-accessed-2019-using-password-solarwinds123-report-1554986">It is being reported that a SolarWinds update server could be accessed using the password "solarwinds123"</a>; breaking into the update servers was a key part of the Orion hack. Assuming this is true, I would bet a dollar that password was an internal placeholder and some poor technician just forgot to reset it before placing it production. But it is would also mean that SolarWinds wasn't doing their diligence in terms of security auditing. Apparently, SolarWinds was warned of the password issue by a researcher, but it also would not have been difficult to set up SolarWinds' own monitoring software to check for insecure password hashes.<br /></p><p></p><p>Outside of data center administrators, few people even in the IT industry will have much experience with products with Orion. But that doesn't mean that you or your business is not impacted by this compromise. The list of organizations that use Orion includes a laundry list of US Federal government agencies, including the Department of Homeland Security and the Centers for Disease Control. There is reason to believe a state actor - most likely Russia - was behind the attack. A fair number of hosting providers and data centers use Orion; Orion may not be hooked into your website, but the bare metal running the VM that hosts your website could very well be hooked into Orion. The list of effected entities I've seen recently numbers around 18,000 - but many of those entities host infrastructure responsible for thousands of other organizations. As a result, it may not be possible to have a full appreciation for the scale of this attack at the moment. </p><p>FireEye's research indicated the compromise has most likely been in place since March, or nearly nine months. This is both an extremely long and an extremely short period of time. It is a long period of time in that it is extremely difficult to keep a compromise of this depth and breadth alive for this long, and an enormous amount of information can be obtained in that period of time. It is a short period of time in that the sophistication needed to keep the operation going also limited the amount of data that could be obtained. Massive increases in network throughput would (hopefully) have been identified sooner by traditional intrusion detection systems. This means it is more likely they were looking for private encryption keys, SSL certificates, things that are relatively small, easy to obfuscate, and that can be re-weaponized to obtain access to additional systems. It is less likely they were scraping patient health data from the CDC.</p><p>I wish I could wrap this up with some quick recommendations that could have avoided this situation, but I'm not sure I have any easy solutions for this one. Everything is driving the industry toward further consolidation; admins are responsible for more and more devices per staff member. COVID and the remote working explosion it has caused are driving this. There has been some attention on how working from home has introduced systemic strain to the "last mile" of internet service. All of those miles of beautiful fiber and metro-ethernet that has been run to commercial real estate over the years has been replaced by 1,000 shaky home broadband connections. The success that ISPs have had in adjusting their networks to this unprecedented change in user behavior is one of the few brightspots in the US response to the disease, and the government had nothing to do with it.</p><p>But even before the epidemic, there has been a general assumption that the ratio of managed devices to administrator staff will simply continue to increase, forever. Why even have a skilled systems administrator on staff at all, when it is possible to have a control panel system installed for a one-time fee that entry-level employees could be trained on the job to use? This sort of centralization is essentially automating the most difficult tasks.<br /></p><p>Ultimately, Orion exemplifies the risk in this approach. Unskilled, inexperienced staff can be easily trained for single products, but will not be able to think critically about how that product works in order to improve it, fix it, or secure it. Likewise, increasing the workload of skilled workers increases the risk of careless errors that have real costs. I'm afraid there may not be an "app for that".<br /></p>Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-25278624145084342762020-12-15T14:42:00.006-05:002021-08-31T11:53:36.690-05:00Google Workspace Outage<p>Yesterday, on December 14th, all services associated with Google Workspace (AKA GSuite - or for those who aren't familiar with it, what is essentially Google's paid "business" services) went offline for roughly an hour from 7AM to 8AM Eastern time.</p><p>Users typically first encountered the error when attempting to send email or after receiving an error indicating that their account could "not be found" when attempting to login to Google services. </p><br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFHQaBkiwdwqJkTyvFWbs5ccL_qUDM5WOQa0EA3gsU70Z9TTh1v2yHzyDxrVqgzn24roVZn7cJM-RRXsNdw5Em87cZBr7wUHxcWF31IZBAjfOPNEgLcdrmBikpqXUe2RM1a-AMLYQXYmE/s998/workspace+status.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="Google's App Status dashboard" border="0" data-original-height="942" data-original-width="998" height="302" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFHQaBkiwdwqJkTyvFWbs5ccL_qUDM5WOQa0EA3gsU70Z9TTh1v2yHzyDxrVqgzn24roVZn7cJM-RRXsNdw5Em87cZBr7wUHxcWF31IZBAjfOPNEgLcdrmBikpqXUe2RM1a-AMLYQXYmE/w320-h302/workspace+status.PNG" title="Google's App Status dashboard" width="320" /></a></div><br />
<p>Other impacted services include Youtube and the Google Nest home security service. </p><p>Google's official statement to the press described the cause in extremely general terms:</p>
<blockquote><i>Services requiring users to log in experienced high error rates during this period,” a Google spokesperson said. “The authentication system issue was resolved at (7:32 a.m. EST). All services are now restored. We apologize to everyone affected, and we will conduct a thorough follow-up review to ensure this problem cannot recur in the future.</i></blockquote>
<p>This outage follows similar major failures with Microsoft's Office services and Windows Updates as well as service interruptions with Amazon's EC2 and related cloud products.<br /></p>Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-35838200825628365142019-04-16T12:13:00.003-05:002021-02-25T14:11:56.339-05:00Botnet spamming The Pirate Bay with malwareOver the last few weeks, a botnet has been mass-uploading a specific package of what appears to be malware (I haven't had time to look at the payload itself yet).<br />
<br />
Cleverly, the person(s) behind this effort have appeared to scrape filenames from titles that have already been pirated by popular uploaders.<br />
<br />
Stupidly, each download uses an obviously fraudulent filesize of 8.04MB. Videogames have not been that small for decades. This mistake would have been less obvious if not for the fact that the same user account - <a href="https://thepiratebay.org/user/halfax/1/3" rel="nofollow">halfax </a>- has uploaded dozens and dozens of games with the exact same filesize.<br />
<br />
Adding to the obvious fraud behind this effort is the number of nodes sharing these bad files. A screenshot of the current front page of the Games listing for TPB shows the disparity in the number of Seeders and Leachers between files shared by actual pirates and those shared by "halfax":<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMX6I0nko3FOXut5t57z89M8ZQNXnSA5JVy4xKPa0R59z98UI8KnqsVUPMU5qN360wcgMBDU4R8PkdBtDSZqG7SlDhevOyoM8x8o4VG5Yqvtz5AjuKdFIYE7TjJS70DQQHyWvZbSOPW28/s1600/TPB-badware.PNG" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="841" data-original-width="1148" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMX6I0nko3FOXut5t57z89M8ZQNXnSA5JVy4xKPa0R59z98UI8KnqsVUPMU5qN360wcgMBDU4R8PkdBtDSZqG7SlDhevOyoM8x8o4VG5Yqvtz5AjuKdFIYE7TjJS70DQQHyWvZbSOPW28/s400/TPB-badware.PNG" width="400" /></a></div>
<br />
Notice how, although there is variation in the number of seeders and leechers, the variation is +/-50 seeders (of a total ~2750 total). The balance between seeders & leechers is also essentially non-variable. Compare the numbers above with those from TPB's Top 40 listing from the last 48 hours:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiG3xVQzzfLxpY9LcNXu5_nWSxo1kG97Tg88G0RRSk8lwzwAwPoj8vCGbYBklYjt3DZy1esXKSHs4juVgaJjk4cKiGgaePinc5ymUz4_q0h6p35S_5MtcfBexRwV_QZbHxkcmN6FhLed7A/s1600/TPB-top40.PNG" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="881" data-original-width="1142" height="307" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiG3xVQzzfLxpY9LcNXu5_nWSxo1kG97Tg88G0RRSk8lwzwAwPoj8vCGbYBklYjt3DZy1esXKSHs4juVgaJjk4cKiGgaePinc5ymUz4_q0h6p35S_5MtcfBexRwV_QZbHxkcmN6FhLed7A/s400/TPB-top40.PNG" width="400" /></a></div>
<br />
Notice the high-degree of variability between both the ratio of seeders & leechers and the overal numbers of seeders & leechers.<br />
<br />
As time permits I will update the blog with the results of a review of the payload. For the time being, I would recommend any torrent users to avoid uploads that share similar odd patterns like those made apparent here.<br />
<br />
Just a quick note: I don't support software piracy (I don't support the current copyright laws either, but that's a different blog post). That doesn't mean that software pirates should serve as breeding grounds for malware. Preventing epidemics - whether physical or technical - requires going to where the viruses are and dealing with the behavior of patients or targets or malware in an objective, non-judgmental manner. Just like needle exchange programs do not "encourage" drug use, attempts to investigate and publicize predatory malware in torrent sharing communities does not "encourage" piracy.Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-5696616051334158372018-11-27T22:05:00.001-05:002018-11-27T22:05:24.937-05:00A Shame With No EndThree years ago I wrote <a href="https://www.joshwieder.net/2015/11/international-business-times-thepiratebay-malvertising-exoclick.html">a blog post demonstrating how the International Business Times appeared to be associated with some extremely shady online advertising networks</a>, resulting in International Business Times article links being advertised on places like malware-filled mirrors of The Pirate Bay. The presence of IB Times on a Pirate Bay clone site was particularly ironic as the online news outlet had recently published <a href="http://www.ibtimes.com/pirate-bay-malware-torrent-site-has-free-movies-virus-might-be-included-1697095">several</a> <a href="http://www.ibtimes.com.au/pirate-bay-contains-malicious-advertisements-using-tpb-can-infect-your-computer-malware-1375547">articles</a> detailing exactly how terrible the ads on Pirate Bay were.<br />
<br />
To be clear: I never found any indication of any malfeasance on IB Times' part. In fact, I think it is much more likely to be the fault of some affiliate marketing firm that did a poor job of tracking its purchases. As such, I contacted IB Times via Twitter to inform them of my findings. I was contacted by a representative of IBT Media, during which I offered (for free) to walk their marketing staff on how to identify the affiliate responsible for the ad placement. IBT declined - instead suddenly switching gears and demanding that I publish a statement from IBT on my site (which I did).<br />
<br />
That is where I left things.<br />
<br />
Today, I was sent a copy of <a href="https://www.nytimes.com/interactive/2018/11/27/style/what-is-inside-this-internet-rabbit-hole.html#ch-10">an excellent piece of investigative journalism from New York Times that has uncovered a sprawling series of extremely shady online storefronts engaging in dropshipping that very much appear to be affiliated with the current ownership of the International Business Times and Newsweek</a> - David Jang, his religion (called "<a href="http://motherjones.com/media/2014/03/newsweek-ibt-olivet-david-jang/">The Community</a>" - nothing creepy about that), his private college Ollivet University and some 140 LLCs registered for the store fronts.<br />
<br />
The article is long but very much worth an entire read.Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-64836598298852553032018-06-28T15:32:00.001-05:002018-06-28T15:34:49.550-05:00Palm Beach Post covers the Heroin EpidemicFor over a year I assisted the <a href="http://palmbeachpost.com/">Palm Beach Post</a> with an in-depth investigation on the State of Florida's involvement with sparking the current heroin epidemic. <a href="https://heroin.palmbeachpost.com/">That investigation has just been published</a>.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://heroin.palmbeachpost.com/wp-content/uploads/2018/06/FloridaPillsAndNeedlesWide3-1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="500" data-original-width="800" height="200" src="https://heroin.palmbeachpost.com/wp-content/uploads/2018/06/FloridaPillsAndNeedlesWide3-1.jpg" width="320" /></a></div>
Among other things, I assisted the Post by creating a custom player to support a carousel-style multimedia presentation with full-screen video encoded using Azure Media Services and distributed through Verizon CDN. It just occurred to me while writing this that my very first work with streaming was Windows Media Services 4.1 on IIS 5.0 (I missed <a href="https://www.youtube.com/watch?v=1fidxAhqZ3k">the NetShow party</a>). It sucked. AMS isn't too bad.<br />
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
For over a century, the United States' narrative surrounding its continuing war on its own drug-addicted citizens has been a jingoistic heap of catch-phrases and rationalizations that have resulted in <a href="http://news.bbc.co.uk/2/shared/spl/hi/uk/06/prisons/html/nn2page1.stm">the US incarcerating a larger share of our citizens in both absolute and per capita terms than any other nation in the world</a>. News organizations (particularly - but not solely - television news) have been instrumental in getting us to where we are now, gormlessly repeating the most outrageous claims of prohibitionists & law enforcement without context or criticism (consider the <a href="https://mises.org/wire/origins-crack-baby-myth">myth of the Crack Baby</a> or the <a href="https://www.forbes.com/sites/jacobsullum/2016/05/05/the-legend-of-the-miami-cannabil-provides-lessons-in-shoddy-drug-journalism">hysterical coverage surrounding so-called "bath salts"</a>).<br />
<br />
The Post avoids the pitfalls of yellow drug journalism in this investigation. Through careful review of public records, the Palm Beach Post team uncovered cozy financial relationships between oxycodone drug manufacturers and the drug warrior politicians who make a career out of imprisoning drug addicts. My favorite finding involved the identification of systemic errors in cause of death reporting by Florida medical examiners that in turn is used to create CDC's WONDER database. While dozens of researchers rely on WONDER to study trends in morbidity and mortality, there do not appear to be sufficient data quality controls in place to identify, for example, situations in which medical examiners report the same deaths multiple times. I have not found any evidence that independent researchers have attempted an audit. The world relies on this data to create policy of all kinds, yet the quality of the data is *extremely* poor when compared to the sort of bulk datasets one might find collected by an internet advertising firm.<br />
<br />
The entire series is worth a read, but expect to be a little disgusted with your politicians, regulators, police and drug companies by the time you're done.Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.comtag:blogger.com,1999:blog-4411720504608505363.post-69791120424246182922017-06-07T12:29:00.001-05:002017-06-07T12:29:59.773-05:00NSA Leak Bust Points to State Surveillance Deal with Printing FirmsEarlier this week a young government contractor named Reality Winner was accused by police of leaking an internal NSA document to news outlet The Intercept. The documents outline the intelligence community's take on Russian efforts to hack a variety of companies responsible for facilitating US election voting. You can <a href="https://theintercept.com/2017/06/05/top-secret-nsa-report-details-russian-hacking-effort-days-before-2016-election/">read the documents here</a>.<br />
<br />
Despite what anyone might have to say about the issue on Twitter, an arrest involving an accusation of any crime by any law enforcement agency in any country is not evidence of guilt. Even the most circumspect appraisal of the US justice system will reveal that tens of thousands of individuals are arrested every year only to have those charges *immediately* dismissed by a court, while nearly everyone who actually is *convicted* of a crime in this country has their charges reduced. Even in cases in which individuals have been convicted of the the most serious capitol crimes, courts have been forced to release dozens of individuals <a href="https://www.innocenceproject.org/">after DNA testing offered conclusive proof of innocence</a>.<br />
<br />
The point is this: being arrested is not being convicted. And being convicted is not proof-positive of guilt.<br />
<br />
For the purposes of this post I will set aside the substance of the leak itself; again, I recommend reading the Intercept's initial reporting. This post is focused on reports of how law enforcement is claiming that it identified young Ms Winner and the consequences of these reports for computer users with an interest in privacy. The <a href="https://www.eff.org/issues/printers">Electronic Frontier Foundation</a> (EFF) describes the purported technique involved as follows:<br />
<br />
<blockquote class="tr_bq">
<i>Imagine that every time you printed a document it automatically
included a secret code that could be used to identify the printer - and
potentially the person who used it. Sounds like something from an
episode of "Alias" right? Unfortunately the scenario isn't fictional. In a purported effort to
identify counterfeiters the US government has succeeded in persuading
some color laser printer manufacturers to encode each page with
identifying information. That means that without your knowledge or
consent an act you assume is private could become public. A
communication tool you're using in everyday life could become a tool for
government surveillance. And what's worse there are no laws to prevent
abuse.</i></blockquote>
<br />
The term for this technique is "forensic watermarking", "printer stenography" or "counterfeit deterrence system". The EFF definitively uncovered that a wide array of some of the most popular modern printers now print some form of watermark that can be used to definitively identify the device that printed a given document after <a href="https://www.eff.org/foia/foia-printer-dots">a series of FOIA requests to some 10 US government agencies in 2008</a>. The documents recovered through that FOIA request (some of whom date back to the 1990's) reveal that the watermarking techniques have been available since at least the 1980's, that printer manufacturers "voluntarily" adopted forensic watermarking under the ostensible justification of fighting counterfeiters, and that efforts to proliferate the use of watermarking involved the EU as well as the US.<br />
<br />
The watermark involved in the documents published by the Intercept consists of a pattern of yellow dots that, when translated, identifies the serial number of the printer used and the date & time the document was printed. Here are those dots, made more visible by introducing additional contrast (images c/o <a href="http://blog.erratasec.com/2017/06/how-intercept-outed-reality-winner.html#comment-form_7890864371470519628">Errata Sec's excellent post on this topic</a>): <br />
<div style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgpV-yDfD8xr8FDqzi-h-fn1QO6bExeIxZrlb5IMoAQHgf6HP1FqJMypHgs5kSGhyCoanpF7_Lw1UvC2PxmSHtx7aQWLKP0ZIICCQWXi3SRGIpU7gWevyLD_l8HNVxAzaEaVa5w1SAkuA/s1600/nsadots.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="223" data-original-width="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgpV-yDfD8xr8FDqzi-h-fn1QO6bExeIxZrlb5IMoAQHgf6HP1FqJMypHgs5kSGhyCoanpF7_Lw1UvC2PxmSHtx7aQWLKP0ZIICCQWXi3SRGIpU7gWevyLD_l8HNVxAzaEaVa5w1SAkuA/s1600/nsadots.png" /></a> </div>
And here is the data gleaned from translating that watermark:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixPxNcWj1tSj1mluoG80m2yDIHLk-jOwHBHkqk4oNh-ER_WpUrVtT1a0xASTC_nlaaI6rxBXRZkVFdqcBDZDoGW77Pn-DPrG6hmLscC9XswRJf00d4DpaS2YReM01Pp6uNl1OQhoNB9-0/s1600/docucolor.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="732" data-original-width="684" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixPxNcWj1tSj1mluoG80m2yDIHLk-jOwHBHkqk4oNh-ER_WpUrVtT1a0xASTC_nlaaI6rxBXRZkVFdqcBDZDoGW77Pn-DPrG6hmLscC9XswRJf00d4DpaS2YReM01Pp6uNl1OQhoNB9-0/s320/docucolor.png" width="299" /></a></div>
When this information is combined with the a standard corporate asset tagging system and printer logs, this watermarking can easily identify the workstation that printed a given document. The same technique can be used to create evidence that +a printer seized from a defendant's property generated a given document, as well.<br />
<br />
So how does a privacy-conscience printer-user avoid this watermarking technique?<br />
<br />
For one thing, do not assume that because you are unable to see any visible watermarking on documents from your printer that you are safe. Here is a photograph of a watermarked document taken in tandem with a Digital Blue QX5 microscope:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEifRPCsStSdXabYXsAdviedkx-Z9qvoHheHA56zDcL8aez4kg7UdGsVFG8ak9zlepLsYYXy5kksfEKh3X5N2l2ubZLH-Dbowrg_a5IXvLhWcHL43XlQ4PmeLYwBSf_f9bVP_lr3c815MdA/s1600/digblue.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="384" data-original-width="512" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEifRPCsStSdXabYXsAdviedkx-Z9qvoHheHA56zDcL8aez4kg7UdGsVFG8ak9zlepLsYYXy5kksfEKh3X5N2l2ubZLH-Dbowrg_a5IXvLhWcHL43XlQ4PmeLYwBSf_f9bVP_lr3c815MdA/s400/digblue.jpg" width="400" /></a></div>
<br />
Even with the microscope the forensic dots are barely visible. Attempting to view the pattern without any form of artificial enhancement is a fool's errand.<br />
<br />
A user can avoid purchasing one of the printers that <a href="https://www.eff.org/pages/list-printers-which-do-or-do-not-display-tracking-dots">EFF has tested and confirmed generates watermarks</a>. Unfortunately, this list is not up-to-date; and as time goes on, the likelihood that *all* manufacturers will produce some form of watermarking increases.<br />
<br />
The specific technique that involved the leaked documents published by the Intercept requires the use of color: the dots are a pale shade of yellow that is not easily visible without some form of digital enhancement. Avoiding the use of a color printer can avoid this specific technique. I am very skeptical of claims online that printing documents in "black & white" mode on color printers provides any form of protection: watermarks can easily be imprinted in greyscale (see "binary image watermarks"), and I have yet to see confirmation that this technique is anything other than *not* effective.<br />
<br />
Even the use of a modern black and white printer leaves me uncomfortable. There are numerous means of imprinting imperceptible watermarks; the popular yellow dots are simply one technique of many. DCT based watermarking techniques are significantly more complex to identify than just adding some document contrast; until now the computational expense required for DCT was likely cost prohibitive for manufacturers. This is certain to change over time.<br />
<br />
Tools designed specifically to protect users from this manner of technology are few to non-existent. I can't point the finger; I have not worked on this problem. I do have some ideas. Given that all water-marking techniques in use are unknown, it would likely be more reliable & perhaps cross-device-compatible to spoof identifying device information prior to reaching the printed document than attempting to identify & remove or modify the watermark itself. I have only marginal experience with peripheral firmware or drivers, but if anyone is interested in this type of project I learn fast & would be happy to help.Josh Wiederhttp://www.blogger.com/profile/11800273950071585348noreply@blogger.com