====== Setup LVMCache for monitor.zappa.cx with RAID-1 on SSD ======
===== Orig LV: monitor-disk F3_2TB -wi-ao---- 6.00g =====
SSD PV's has been added to F3_2TB VG and SSD tags has been added.
bigbang:~# pvs -o +tags
PV VG Fmt Attr PSize PFree PV Tags
/dev/cciss/c0d0p2 F3_2TB lvm2 a-- 1.79t 734.65g HDD
/dev/sdb1 F3_2TB lvm2 a-- 400.00g 400.00g SSD
/dev/sdc1 F3_2TB lvm2 a-- 400.00g 400.00g SSD
bigbang:~#
Create a 2-way RAID1 cache data LV (1GB)
bigbang:~# lvcreate --type raid1 -m 1 -n monitorCDLV -L 1G F3_2TB @SSD
Logical volume "monitorCDLV" created.
bigbang:~#
Create a 2-way RAID1 cache metadata LV (8MB minimum or 1000 times smaller than cache LV)
bigbang:~# lvcreate --type raid1 -m 1 -n monitorCMLV -L 8M F3_2TB @SSD
Logical volume "monitorCMLV" created.
bigbang:~#
Create a cache pool LV combining cache data LV and cache metadata LV
bigbang:~# lvconvert --type cache-pool --poolmetadata F3_2TB/monitorCMLV F3_2TB/monitorCDLV
WARNING: Converting logical volume F3_2TB/monitorCDLV and F3_2TB/monitorCMLV to cache pool's data and metadata volumes with metadata wiping.
THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.)
Do you really want to convert F3_2TB/monitorCDLV and F3_2TB/monitorCMLV? [y/n]: y
Converted F3_2TB/monitorCDLV to cache pool.
bigbang:~#
Create a cached LV by combining the cache pool LV and origin LV
bigbang:~# lvconvert --type cache --cachepool F3_2TB/monitorCDLV F3_2TB/monitor-disk
Do you want wipe existing metadata of cache pool volume F3_2TB/monitorCDLV? [y/n]: y
Logical volume F3_2TB/monitor-disk is now cached.
bigbang:~#
===== Done =====
====== To change cache mode ======
bigbang:~# lvs -o +cachemode F3_2TB/monitor-disk
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert CacheMode
monitor-disk F3_2TB Cwi-aoC--- 6.00g [monitorCDLV] [monitor-disk_corig] 13.33 1.90 0.00 writethrough
bigbang:~#
bigbang:~# lvchange --cachemode writeback F3_2TB/monitor-disk
Logical volume F3_2TB/monitor-disk changed.
bigbang:~#
bigbang:~# lvs -o +cachemode F3_2TB/monitor-disk
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert CacheMode
monitor-disk F3_2TB Cwi-aoC--- 6.00g [monitorCDLV] [monitor-disk_corig] 13.37 2.83 5.02 writeback
bigbang:~#
bigbang:~# lvchange --cachemode writethrough F3_2TB/monitor-disk
Flushing 107 blocks for cache F3_2TB/monitor-disk.
Flushing 79 blocks for cache F3_2TB/monitor-disk.
Logical volume F3_2TB/monitor-disk changed.
bigbang:~#
bigbang:~# lvs -o +cachemode F3_2TB/monitor-disk
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert CacheMode
monitor-disk F3_2TB Cwi-aoC--- 6.00g [monitorCDLV] [monitor-disk_corig] 13.37 2.83 0.00 writethrough
bigbang:~#
====== To remove cache ======
Make sure it is in writethrough - just to be safe.
Also check that no blocks are dirty.
bigbang:~# lvconvert --uncache F3_2TB/monitor-disk
Logical volume "monitorCDLV" successfully removed
Logical volume F3_2TB/monitor-disk is not cached.
bigbang:~#
====== Setup LVMCache and ThinPool for caching multiple LV's with RAID-1 on SSD ======
===== Setting up ThinPool (size 500GB on the HW RAID-1 HDD and metadata on LVM mirrored SSD's) =====
Create 500GB LV for ThinPool0 on the HW raided HDD's
bigbang:~# lvcreate -n ThinPool0 -L 500G F3_2TB @HDD
Logical volume "ThinPool0" created.
bigbang:~#
Create 1GB LV for ThinPool0_meta using LVM RAID-1 on the SSD's
bigbang:~# lvcreate --type raid1 -m 1 -n ThinPool0_meta -L 1G F3_2TB @SSD
Logical volume "ThinPool0_meta" created.
bigbang:~#
Create the ThinPool0 by combining data data and metadata LV's
bigbang:~# lvconvert --type thin-pool --poolmetadata F3_2TB/ThinPool0_meta F3_2TB/ThinPool0
Thin pool volume with chunk size 64.00 KiB can address at most 15.81 TiB of data.
WARNING: Converting F3_2TB/ThinPool0 and F3_2TB/ThinPool0_meta to thin pool's data and metadata volumes with metadata wiping.
THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.)
Do you really want to convert F3_2TB/ThinPool0 and F3_2TB/ThinPool0_meta? [y/n]: y
Converted F3_2TB/ThinPool0 and F3_2TB/ThinPool0_meta to thin pool.
bigbang:~#
ThinPool0 has now been created, LV's using it can be created like so
# lvcreate -n ThinLV -V VirtualSize --thinpool ThinPool0 F3_2TB
Snapshots are created without specifying the size as it is automatically
using the data from the ThinPool0 LV for storing the difference.
# lvcreate -n ThinLV_s1 -s F3_2TB/ThinLV
===== Adding caching for the ThinPool0 =====
Create a 2-way RAID1 cache data LV (50GB)
bigbang:~# lvcreate --type raid1 -m 1 -n ThinPool0_CDLV -L 50G F3_2TB @SSD
Logical volume "ThinPool0_CDLV" created.
bigbang:~#
Create a 2-way RAID1 cache metadata LV (8MB minimum or 1000 times smaller than cache LV)
bigbang:~# lvcreate --type raid1 -m 1 -n ThinPool0_CMLV -L 50M F3_2TB @SSD
Rounding up size to full physical extent 52.00 MiB
Logical volume "ThinPool0_CMLV" created.
bigbang:~#
Create a writeback cached LV by combining the cache pool LV cache metadata and origin LV
bigbang:~# lvconvert --type cache --cachemode writeback --cachepool ThinPool0_CDLV --poolmetadata ThinPool0_CMLV F3_2TB/ThinPool0
WARNING: Converting F3_2TB/ThinPool0_CDLV and F3_2TB/ThinPool0_CMLV to cache pool's data and metadata volumes with metadata wiping.
THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.)
Do you really want to convert F3_2TB/ThinPool0_CDLV and F3_2TB/ThinPool0_CMLV? [y/n]: y
Converted F3_2TB/ThinPool0_CDLV and F3_2TB/ThinPool0_CMLV to cache pool.
WARNING: Cached thin pool's data cannot be currently resized and require manual uncache before resize!
Logical volume F3_2TB/ThinPool0_tdata is now cached.
bigbang:~#
===== Removing cache from ThinPool0 =====
Change to writethrough before separating cache pool from cache LV
bigbang:~# lvchange --cachemode writethrough F3_2TB/ThinPool0_tdata
Flushing 0 blocks for cache F3_2TB/ThinPool0_tdata.
Logical volume F3_2TB/ThinPool0_tdata changed.
Separate and keep the cache pool from a cache LV.
bigbang:~# lvconvert --splitcache F3_2TB/ThinPool0
Logical volume F3_2TB/ThinPool0_tdata is not cached and cache pool F3_2TB/ThinPool0_CDLV is unused.
Attach cache LV to previously separated cache pool
bigbang:~# lvconvert --type cache --cachemode writeback --cachepool F3_2TB/ThinPool0_CDLV F3_2TB/ThinPool0
Do you want wipe existing metadata of cache pool F3_2TB/ThinPool0_CDLV? [y/n]: y
WARNING: Cached thin pool's data cannot be currently resized and require manual uncache before resize!
Logical volume F3_2TB/ThinPool0_tdata is now cached.
===== Extending size of ThinPool0 =====
==== Remove caching from the thinpool ====
As it does not seem to be possible to extend a cached thinpool, we first need to remove the cache.
Follow first two steps above from section "Removing cache from ThinPool0" (do not re-attach the cache of course)
Some - before information
bigbang:~# lvdisplay -a F3_2TB/ThinPool0_tdata
--- Logical volume ---
Internal LV Name ThinPool0_tdata
VG Name F3_2TB
LV UUID Fyc9Md-c1XD-xpib-dJDw-Obr4-SYL5-G88yqS
LV Write Access read/write
LV Creation host, time bigbang, 2019-10-12 14:17:30 +0200
LV Cache pool name ThinPool0_CDLV
LV Cache origin name ThinPool0_tdata_corig
LV Status available
# open 1
LV Size 500.00 GiB
Cache used blocks 99.99%
Cache metadata blocks 18.54%
Cache dirty blocks 0.00%
Cache read hits/misses 644218163 / 453243621
Cache wrt hits/misses 1046297083 / 31441523
Cache demotions 1455727
Cache promotions 1455727
Current LE 128000
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:36
bigbang:~# lvchange --cachemode writethrough F3_2TB/ThinPool0_tdata
Flushing 55 blocks for cache F3_2TB/ThinPool0_tdata.
Logical volume F3_2TB/ThinPool0_tdata changed.
bigbang:~# lvconvert --splitcache F3_2TB/ThinPool0
Logical volume F3_2TB/ThinPool0_tdata is not cached and cache pool F3_2TB/ThinPool0_CDLV is unused.
==== Now extend the ThinPool0 LV ====
bigbang:~# lvextend -L 1t F3_2TB/ThinPool0
Size of logical volume F3_2TB/ThinPool0_tdata changed from 500.00 GiB (128000 extents) to 1.00 TiB (262144 extents).
Logical volume F3_2TB/ThinPool0_tdata successfully resized.
==== And now reattach the cache ====
Do the last step from section "Removing cache from ThinPool0" to reattach the cache to the thinpool
bigbang:~# lvconvert --type cache --cachemode writeback --cachepool F3_2TB/ThinPool0_CDLV F3_2TB/ThinPool0
Do you want wipe existing metadata of cache pool F3_2TB/ThinPool0_CDLV? [y/n]: y
WARNING: Cached thin pool's data cannot be currently resized and require manual uncache before resize!
Logical volume F3_2TB/ThinPool0_tdata is now cached.
And some "after" information
bigbang:~# lvdisplay -a F3_2TB/ThinPool0_tdata
--- Logical volume ---
Internal LV Name ThinPool0_tdata
VG Name F3_2TB
LV UUID Fyc9Md-c1XD-xpib-dJDw-Obr4-SYL5-G88yqS
LV Write Access read/write
LV Creation host, time bigbang, 2019-10-12 14:17:30 +0200
LV Cache pool name ThinPool0_CDLV
LV Cache origin name ThinPool0_tdata_corig
LV Status available
# open 1
LV Size 1.00 TiB
Cache used blocks 0.13%
Cache metadata blocks 12.46%
Cache dirty blocks 10.12%
Cache read hits/misses 0 / 36
Cache wrt hits/misses 3663 / 1530
Cache demotions 0
Cache promotions 1067
Current LE 262144
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:36