Wiki of Jeremy Bryan Smith 🌚

From Jeremy Bryan Smith
Jump to navigation Jump to search

Welcome to the Wiki of Jeremy Bryan Smith. Here I shall disseminate knowledge that I deem important enough that I may need to recall at a later point in time and/or that I believe may be useful for others out there.
Feel free to poke around, make comments, suggestions, and ingest the bits of information I have to share.

Regards,
Jeremy Bryan Smith


Find Me On

Veganism

Green Living

Air

TODO: Add air filtration stuff here...

Water

You can never trust your municipal water supply. Nor can you trust that any source of water (including bottled) has not been contaminated. Do not drink unfiltered tap water. Just don't.

I have used a reverse osmosis water filter system since 2014 and have never looked back. The one I happened to have purchased was the:
Home Master TMAFC Artesian Full Contact Undersink Reverse Osmosis Water Filter System

And the following replacement filters:

Along with this UV sterilizer:
Bluonics Ultraviolet Light Water Purifier UV Sterilizer

Super easy to install and maintain. I removed the tank once the bladder was no longer holding water and instead attached it to an ice maker:
Countertop Hot & Cold Water Dispenser and Ice Maker

Composting

I use this: Envirocycle USA-made, Food Safe, BPA and Rust Free Composting Tumbler and Compost Tea Maker

Special:IframePage

Technology

Random

At some point I will organize this stuff better.

Applications and Plug-ins

Here I will provide my opinions, recommendations on usage and optimal configuration, links to related third-party tools, and my own related tools

  • Web Browsers - The web browsers and related extensions / plug-ins, and tweaks for usability and security, etc. that I recommend
  • vim - My text editor of choice

Devices

  • Cameras
    • Canon PowerShot G9
    • Raspberry PI 5MP Camera Board Module
    • Raspberry Pi HQ Camera Module
    • Arducam Day and Night Vision IMX477 HQ Camera
    • Logitech HD Pro Webcam C920 (960-000764)
    • LOGITECH QUICKCAM PRO 9000 R Webcam
    • Logitech Quickcam Express

Software

Programming

Android

Microsoft Windows

E-mail

Storage

RAID

From Hardware to Software

If you've ever been forced to use hardware RAID and then need to migrate from hardware RAID to software RAID ...
I had 6 disks behind a Dell PERC 6/i hardware RAID controller and wanted to set up a ZFS pool. But for reasons beyond my control at the time, changing out the controller for a proper HBA was not an option. So, I did the unthinkable and configured 6 RAID-0 devices on the PERC controller, one for each physical disk, and then created a zpool from the virtual disks the controller exposed to the OS. It worked well enough for my purposes for years, but now I have the ability to swap in a real HBA. But the question is, do I have to copy all of the data off, swap the controller, then copy all data back? I was surprised to discover that the data on the disks, with the exception of a small section of metadata at the end of each disk, was in-tact just as if the disks were being used directly. In fact, the metadata was in Linux MDADM RAID format! So that must be what those PERC controllers are doing behind the scene. No surprise there, taking advantage of open source. I wonder if they are violating the GPL. I'll have to look at the firmware. Anyway, This may only work if you configured each disk as a RAID-0. I haven't tested other configurations yet.
Resources:


This is what I saw: 

root@ubuntu-server:~# lsblk --nodeps -I 8 -o name,maj:min,rm,size,ro,type,pttype,ptuuid,fstype,uuid,label,model,vendor
NAME MAJ:MIN RM   SIZE RO TYPE PTTYPE PTUUID                               FSTYPE          UUID                                 LABEL MODEL            VENDOR
sda    8:0    0 232.9G  0 disk gpt    5ace5e66-37f2-4d44-b518-f2a22834c3ab                                                            Samsung_SSD_860  ATA     
sdb    8:16   1  29.3G  0 disk dos    e16cd6ae                                                                                        Cruzer_Glide     SanDisk 
sdc    8:32   0 465.8G  0 disk                                             ddf_raid_member Dell    \x10                               ST9500530NS      ATA     
sdd    8:48   0 465.8G  0 disk                                             ddf_raid_member Dell    \x10                               ST9500530NS      ATA     
sde    8:64   0 465.8G  0 disk                                             ddf_raid_member Dell    \x10                               ST9500530NS      ATA     
sdf    8:80   0 465.8G  0 disk                                             ddf_raid_member Dell    \x10                               ST9500530NS      ATA     
sdg    8:96   0 465.8G  0 disk                                             ddf_raid_member Dell    \x10                               ST9500530NS      ATA     
sdh    8:112  0 465.8G  0 disk                                             ddf_raid_member Dell    \x10                               ST9500530NS      ATA     
sdi    8:128  1 235.8G  0 disk dos    be730627                                                                                        Voyager_SliderX2 Corsair 
sdj    8:144  0   3.7T  0 disk gpt    b5dcd625-9112-4f29-9b67-64e935bf15b3                                                            My_Passport_25E2 WD      
sdk    8:160  0   3.7T  0 disk gpt    31d5cc46-33c7-4806-80a6-57aaba69b38e                                                            My_Passport_25E2 WD      
sdl    8:176  1  58.9G  0 disk dos    d02d781a                             zfs_member      17764476958699951102                 upool Voyager_SliderX2 Corsair 
sdm    8:192  1  58.9G  0 disk dos    642fafbd                             zfs_member      12354630986627173815                 upool Voyager_SliderX2 Corsair 
sdn    8:208  1   118G  0 disk dos    604144b7                                                                                        Voyager_SliderX2 Corsair 
sdo    8:224  1   118G  0 disk dos    8f6dbdcd                                                                                        Voyager_SliderX2 Corsair 
sdp    8:240  1 116.5G  0 disk dos                                                                                                    Cruzer_Glide     SanDisk

You can see the 6 500GB disks as well as a few USB drives attached. We'll ignore those and list any Linux RAID devices... 

root@ubuntu-server:~# mdadm --query /dev/sd{c,d,e,f,g,h}
/dev/sdc: is not an md array
/dev/sdc: device -1393767213 in 6 device undetected container /dev/md/ddf0.  Use mdadm --examine for more detail.
/dev/sdd: is not an md array
/dev/sdd: device -1811426553 in 6 device undetected container /dev/md/ddf0.  Use mdadm --examine for more detail.
/dev/sde: is not an md array
/dev/sde: device 764605963 in 6 device undetected container /dev/md/ddf0.  Use mdadm --examine for more detail.
/dev/sdf: is not an md array
/dev/sdf: device 1584381922 in 6 device undetected container /dev/md/ddf0.  Use mdadm --examine for more detail.
/dev/sdg: is not an md array
/dev/sdg: device 896654598 in 6 device undetected container /dev/md/ddf0.  Use mdadm --examine for more detail.
/dev/sdh: is not an md array
/dev/sdh: device -2108342859 in 6 device undetected container /dev/md/ddf0.  Use mdadm --examine for more detail.

Hmm, not quite right, let's try dmraid ... 

root@ubuntu-server:~#root@ubuntu-server:~# dmraid -r
/dev/sdh: ddf1, ".ddf1_disks", GROUP, ok, 975699968 sectors, data@ 0
/dev/sdc: ddf1, ".ddf1_disks", GROUP, ok, 975699968 sectors, data@ 0
/dev/sde: ddf1, ".ddf1_disks", GROUP, ok, 975699968 sectors, data@ 0
/dev/sdg: ddf1, ".ddf1_disks", GROUP, ok, 975699968 sectors, data@ 0
/dev/sdd: ddf1, ".ddf1_disks", GROUP, ok, 975699968 sectors, data@ 0
/dev/sdf: ddf1, ".ddf1_disks", GROUP, ok, 975699968 sectors, data@ 0

We see that the Ok, now let's inspect one of those disks' partition table... 

root@ubuntu-server:~# parted /dev/sdg unit b p
Warning: Not all of the space available to /dev/sdg appears to be used, you can fix the GPT to use all of the space (an extra 1073200 blocks) or continue with the current setting? 
Fix/Ignore? Ignore                                                        
Model: ATA ST9500530NS (scsi)
Disk /dev/sdg: 500107862016B
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start      End            Size           File system  Name      Flags
 1      1048576B   9437183B       8388608B                    Reserved
 2      10485760B  18874367B      8388608B                    GRUB      bios_grub
 3      20971520B  499558366719B  499537395200B  zfs          ZFS

Let's step back and take a look at how the disks, partitions, and RAID devices are related ... 

 lsblk -I 8,9 -o name,maj:min,size,type /dev/sdg
 NAME        MAJ:MIN   SIZE TYPE
sdg           8:96  465.8G disk
|-sdg1        8:97      8M part
|-sdg2        8:98      8M part
|-sdg3        8:99  465.2G part
`-md125       9:125 465.3G raid0
  |-md125p1 259:3       8M part
  |-md125p2 259:4       8M part
  `-md125p3 259:5   465.2G part

Inspecting the corresponding MDADM RAID(s): 

root@ubuntu-server:~# grep sdg /proc/mdstat 
md125 : active raid0 sdg[0]
md127 : inactive sdf[5](S) sdh[4](S) sde[3](S) sdd[2](S) sdg[1](S) sdc[0](S)

root@ubuntu-server:~# mdadm --detail /dev/md125
/dev/md125:
         Container : /dev/md/ddf0, member 0
        Raid Level : raid0
        Array Size : 487849984 (465.25 GiB 499.56 GB)
      Raid Devices : 1
     Total Devices : 1

             State : clean 
    Active Devices : 1
   Working Devices : 1
    Failed Devices : 0
     Spare Devices : 0

        Chunk Size : 128K

Consistency Policy : none

    Container GUID : 44656C6C:20202020:10000079:10281F17:4B825EED:49CB41EA
                  (Dell     02/22/20 10:39:41)
               Seq : 00000004
     Virtual Disks : 6

    Number   Major   Minor   RaidDevice State
       0       8       96        0      active sync   /dev/sdg

root@ubuntu-server:~# mdadm --detail /dev/md127
/dev/md127:
           Version : ddf
        Raid Level : container
     Total Devices : 6

   Working Devices : 6

    Container GUID : 44656C6C:20202020:10000079:10281F17:4B825EED:49CB41EA
                  (Dell     02/22/20 10:39:41)
               Seq : 00000004
     Virtual Disks : 6

     Member Arrays : /dev/md/disk1_0 /dev/md/disk5_0 /dev/md/disk3_0 /dev/md/disk4_0 /dev/md/disk2_0 /dev/md122

    Number   Major   Minor   RaidDevice

       -       8       64        -        /dev/sde
       -       8       32        -        /dev/sdc
       -       8      112        -        /dev/sdh
       -       8       80        -        /dev/sdf
       -       8       48        -        /dev/sdd
       -       8       96        -        /dev/sdg

And the partitions on the RAID device itself: 

root@ubuntu-server:~# parted /dev/md125 unit b p
Model: Linux Software RAID Array (md)
Disk /dev/md125: 499558383616B
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start      End            Size           File system  Name      Flags
 1      1048576B   9437183B       8388608B                    Reserved
 2      10485760B  18874367B      8388608B                    GRUB      bios_grub
 3      20971520B  499558366719B  499537395200B  zfs          ZFS

So, capacity of the disk itself is 500,107,862,016 bytes and the capacity of the RAID-0 device presented is 499,558,383,616 bytes.
So, there is 500,107,862,016 - 499,558,383,616 = 549,478,400 bytes / 1024 = 536,600 KiB / 1024 = 524.0234375 MiB used up by the MDADM RAID signature at the end of the disk.
Cool! So, can we just zero out that mdadm signature and use the disks normally? Let's see...

Controllers

See Serve the Home's Top Picks for FreeNAS HBAs for a good overview.
Basically:

  • Stay away from hardware RAID
  • There are a few RAID cards that can be flashed to IT mode to support proper HBA (passthrough/JBOD) mode.
  • There are many re-branded / OEM versions that should also work fine (Dell, IBM)
  • There are many being sold used

Check these out:

  • Internal
    • LSI SAS HBA 9201-8i: 8 port, SATA III, 6 Gbps, ~ $40 - $50 USD on ebay as of 2020-03-02
      Good replacement for PERC 6/i
    • IBM M1015 (LSI 9220-8i): 8 port, SATA III, 6 Gbps,
    • Dell H200: 8 port, SATA III, 6 Gbps, $30 - $40 USD on ebay as of 2020-03-02
      Good replacement for PERC 6/i
  • External

Filesystems

Data Integrity

Storage Analysis

Misc

Security/Privacy

Admirable People

Life Lessons

Lists

  • Travel - Lists for things to do before and things to bring when travelling.
  • Playlists - Playlists of music, movies, etc.
Retrieved from "https://wiki.jeremybryansmith.com/index.php?title=Main_Page&oldid=441"