general:linux:lvmcache
Table of Contents
Setup LVMCache for monitor.zappa.cx with RAID-1 on SSD
Orig LV: monitor-disk F3_2TB -wi-ao---- 6.00g
SSD PV's has been added to F3_2TB VG and SSD tags has been added.
bigbang:~# pvs -o +tags PV VG Fmt Attr PSize PFree PV Tags /dev/cciss/c0d0p2 F3_2TB lvm2 a-- 1.79t 734.65g HDD /dev/sdb1 F3_2TB lvm2 a-- 400.00g 400.00g SSD /dev/sdc1 F3_2TB lvm2 a-- 400.00g 400.00g SSD bigbang:~#
Create a 2-way RAID1 cache data LV (1GB)
bigbang:~# lvcreate --type raid1 -m 1 -n monitorCDLV -L 1G F3_2TB @SSD Logical volume "monitorCDLV" created. bigbang:~#
Create a 2-way RAID1 cache metadata LV (8MB minimum or 1000 times smaller than cache LV)
bigbang:~# lvcreate --type raid1 -m 1 -n monitorCMLV -L 8M F3_2TB @SSD Logical volume "monitorCMLV" created. bigbang:~#
Create a cache pool LV combining cache data LV and cache metadata LV
bigbang:~# lvconvert --type cache-pool --poolmetadata F3_2TB/monitorCMLV F3_2TB/monitorCDLV WARNING: Converting logical volume F3_2TB/monitorCDLV and F3_2TB/monitorCMLV to cache pool's data and metadata volumes with metadata wiping. THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.) Do you really want to convert F3_2TB/monitorCDLV and F3_2TB/monitorCMLV? [y/n]: y Converted F3_2TB/monitorCDLV to cache pool. bigbang:~#
Create a cached LV by combining the cache pool LV and origin LV
bigbang:~# lvconvert --type cache --cachepool F3_2TB/monitorCDLV F3_2TB/monitor-disk Do you want wipe existing metadata of cache pool volume F3_2TB/monitorCDLV? [y/n]: y Logical volume F3_2TB/monitor-disk is now cached. bigbang:~#
Done
To change cache mode
bigbang:~# lvs -o +cachemode F3_2TB/monitor-disk LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert CacheMode monitor-disk F3_2TB Cwi-aoC--- 6.00g [monitorCDLV] [monitor-disk_corig] 13.33 1.90 0.00 writethrough bigbang:~#
bigbang:~# lvchange --cachemode writeback F3_2TB/monitor-disk Logical volume F3_2TB/monitor-disk changed. bigbang:~#
bigbang:~# lvs -o +cachemode F3_2TB/monitor-disk LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert CacheMode monitor-disk F3_2TB Cwi-aoC--- 6.00g [monitorCDLV] [monitor-disk_corig] 13.37 2.83 5.02 writeback bigbang:~#
bigbang:~# lvchange --cachemode writethrough F3_2TB/monitor-disk Flushing 107 blocks for cache F3_2TB/monitor-disk. Flushing 79 blocks for cache F3_2TB/monitor-disk. Logical volume F3_2TB/monitor-disk changed. bigbang:~#
bigbang:~# lvs -o +cachemode F3_2TB/monitor-disk LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert CacheMode monitor-disk F3_2TB Cwi-aoC--- 6.00g [monitorCDLV] [monitor-disk_corig] 13.37 2.83 0.00 writethrough bigbang:~#
To remove cache
Make sure it is in writethrough - just to be safe. Also check that no blocks are dirty.
bigbang:~# lvconvert --uncache F3_2TB/monitor-disk Logical volume "monitorCDLV" successfully removed Logical volume F3_2TB/monitor-disk is not cached. bigbang:~#
Setup LVMCache and ThinPool for caching multiple LV's with RAID-1 on SSD
Setting up ThinPool (size 500GB on the HW RAID-1 HDD and metadata on LVM mirrored SSD's)
Create 500GB LV for ThinPool0 on the HW raided HDD's
bigbang:~# lvcreate -n ThinPool0 -L 500G F3_2TB @HDD Logical volume "ThinPool0" created. bigbang:~#
Create 1GB LV for ThinPool0_meta using LVM RAID-1 on the SSD's
bigbang:~# lvcreate --type raid1 -m 1 -n ThinPool0_meta -L 1G F3_2TB @SSD Logical volume "ThinPool0_meta" created. bigbang:~#
Create the ThinPool0 by combining data data and metadata LV's
bigbang:~# lvconvert --type thin-pool --poolmetadata F3_2TB/ThinPool0_meta F3_2TB/ThinPool0 Thin pool volume with chunk size 64.00 KiB can address at most 15.81 TiB of data. WARNING: Converting F3_2TB/ThinPool0 and F3_2TB/ThinPool0_meta to thin pool's data and metadata volumes with metadata wiping. THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.) Do you really want to convert F3_2TB/ThinPool0 and F3_2TB/ThinPool0_meta? [y/n]: y Converted F3_2TB/ThinPool0 and F3_2TB/ThinPool0_meta to thin pool. bigbang:~#
ThinPool0 has now been created, LV's using it can be created like so
# lvcreate -n ThinLV -V VirtualSize --thinpool ThinPool0 F3_2TB
Snapshots are created without specifying the size as it is automatically using the data from the ThinPool0 LV for storing the difference.
# lvcreate -n ThinLV_s1 -s F3_2TB/ThinLV
Adding caching for the ThinPool0
Create a 2-way RAID1 cache data LV (50GB)
bigbang:~# lvcreate --type raid1 -m 1 -n ThinPool0_CDLV -L 50G F3_2TB @SSD Logical volume "ThinPool0_CDLV" created. bigbang:~#
Create a 2-way RAID1 cache metadata LV (8MB minimum or 1000 times smaller than cache LV)
bigbang:~# lvcreate --type raid1 -m 1 -n ThinPool0_CMLV -L 50M F3_2TB @SSD Rounding up size to full physical extent 52.00 MiB Logical volume "ThinPool0_CMLV" created. bigbang:~#
Create a writeback cached LV by combining the cache pool LV cache metadata and origin LV
bigbang:~# lvconvert --type cache --cachemode writeback --cachepool ThinPool0_CDLV --poolmetadata ThinPool0_CMLV F3_2TB/ThinPool0 WARNING: Converting F3_2TB/ThinPool0_CDLV and F3_2TB/ThinPool0_CMLV to cache pool's data and metadata volumes with metadata wiping. THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.) Do you really want to convert F3_2TB/ThinPool0_CDLV and F3_2TB/ThinPool0_CMLV? [y/n]: y Converted F3_2TB/ThinPool0_CDLV and F3_2TB/ThinPool0_CMLV to cache pool. WARNING: Cached thin pool's data cannot be currently resized and require manual uncache before resize! Logical volume F3_2TB/ThinPool0_tdata is now cached. bigbang:~#
Removing cache from ThinPool0
Change to writethrough before separating cache pool from cache LV
bigbang:~# lvchange --cachemode writethrough F3_2TB/ThinPool0_tdata Flushing 0 blocks for cache F3_2TB/ThinPool0_tdata. Logical volume F3_2TB/ThinPool0_tdata changed.
Separate and keep the cache pool from a cache LV.
bigbang:~# lvconvert --splitcache F3_2TB/ThinPool0 Logical volume F3_2TB/ThinPool0_tdata is not cached and cache pool F3_2TB/ThinPool0_CDLV is unused.
Attach cache LV to previously separated cache pool
bigbang:~# lvconvert --type cache --cachemode writeback --cachepool F3_2TB/ThinPool0_CDLV F3_2TB/ThinPool0 Do you want wipe existing metadata of cache pool F3_2TB/ThinPool0_CDLV? [y/n]: y WARNING: Cached thin pool's data cannot be currently resized and require manual uncache before resize! Logical volume F3_2TB/ThinPool0_tdata is now cached.
Extending size of ThinPool0
Remove caching from the thinpool
As it does not seem to be possible to extend a cached thinpool, we first need to remove the cache. Follow first two steps above from section “Removing cache from ThinPool0” (do not re-attach the cache of course)
Some - before information
bigbang:~# lvdisplay -a F3_2TB/ThinPool0_tdata --- Logical volume --- Internal LV Name ThinPool0_tdata VG Name F3_2TB LV UUID Fyc9Md-c1XD-xpib-dJDw-Obr4-SYL5-G88yqS LV Write Access read/write LV Creation host, time bigbang, 2019-10-12 14:17:30 +0200 LV Cache pool name ThinPool0_CDLV LV Cache origin name ThinPool0_tdata_corig LV Status available # open 1 LV Size 500.00 GiB Cache used blocks 99.99% Cache metadata blocks 18.54% Cache dirty blocks 0.00% Cache read hits/misses 644218163 / 453243621 Cache wrt hits/misses 1046297083 / 31441523 Cache demotions 1455727 Cache promotions 1455727 Current LE 128000 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:36
bigbang:~# lvchange --cachemode writethrough F3_2TB/ThinPool0_tdata Flushing 55 blocks for cache F3_2TB/ThinPool0_tdata. Logical volume F3_2TB/ThinPool0_tdata changed.
bigbang:~# lvconvert --splitcache F3_2TB/ThinPool0 Logical volume F3_2TB/ThinPool0_tdata is not cached and cache pool F3_2TB/ThinPool0_CDLV is unused.
Now extend the ThinPool0 LV
bigbang:~# lvextend -L 1t F3_2TB/ThinPool0 Size of logical volume F3_2TB/ThinPool0_tdata changed from 500.00 GiB (128000 extents) to 1.00 TiB (262144 extents). Logical volume F3_2TB/ThinPool0_tdata successfully resized.
And now reattach the cache
Do the last step from section “Removing cache from ThinPool0” to reattach the cache to the thinpool
bigbang:~# lvconvert --type cache --cachemode writeback --cachepool F3_2TB/ThinPool0_CDLV F3_2TB/ThinPool0 Do you want wipe existing metadata of cache pool F3_2TB/ThinPool0_CDLV? [y/n]: y WARNING: Cached thin pool's data cannot be currently resized and require manual uncache before resize! Logical volume F3_2TB/ThinPool0_tdata is now cached.
And some “after” information
bigbang:~# lvdisplay -a F3_2TB/ThinPool0_tdata --- Logical volume --- Internal LV Name ThinPool0_tdata VG Name F3_2TB LV UUID Fyc9Md-c1XD-xpib-dJDw-Obr4-SYL5-G88yqS LV Write Access read/write LV Creation host, time bigbang, 2019-10-12 14:17:30 +0200 LV Cache pool name ThinPool0_CDLV LV Cache origin name ThinPool0_tdata_corig LV Status available # open 1 LV Size 1.00 TiB Cache used blocks 0.13% Cache metadata blocks 12.46% Cache dirty blocks 10.12% Cache read hits/misses 0 / 36 Cache wrt hits/misses 3663 / 1530 Cache demotions 0 Cache promotions 1067 Current LE 262144 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:36
general/linux/lvmcache.txt · Last modified: 2021/02/11 08:25 by sunkan