[rp-]

Description

This PR makes the LINSTOR plugin record the size of a volume's DRBD device as the volume size in CloudStack.

LINSTOR rounds the storage of a volume up (e.g. to the LVM extent size, plus DRBD metadata) and DRBD makes all of
it usable, so the block device a VM sees is usually a few MiB larger than the size CloudStack requested and
records. Examples from a 3-node LVM-thin setup:

requested / recorded by CloudStack	device seen by the guest
536870912 (512 MiB, template clone)	540909568
1073741824 (1 GiB data disk)	1077665792

Filesystems the guest creates on the whole disk, or grows to its end (e.g. cloud-init growpart), use that space,
but the copies CloudStack makes of the volume are cut at the recorded size: snapshot backups to secondary storage
are shrunk to it, so the last few MiB of the disk are missing in the backup, and a volume or template created from
such a copy then holds a filesystem that is larger than its disk.

The driver now:

• records the device size after creating a volume (new, cloned from a template, or from a snapshot) and after a
  resize, and passes the device size to the hypervisor in the resize notification,
• corrects existing volumes before taking a snapshot and before copying a volume to secondary storage, so
  snapshot backups and volume copies cover the whole device,
• only ever grows the recorded size; if the device size can't be read from LINSTOR the recorded size is kept,
• counts the difference against the account's primary storage resource count, except on resize, where the
  orchestrator already counts the (now recorded) size.

The device size is the smallest usable size of the top layer (DRBD, LUKS or storage) over the diskful resources;
diskless resources only report the size of the volume definition. With DrbdOptions/ExactSize set on the resource
definition the device has the size of the volume definition, which is used as is.

Volumes created on the agent side (e.g. migrated into LINSTOR) are corrected at their next snapshot or copy.

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)
• Build/CI
• Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Screenshots (if appropriate):

How Has This Been Tested?

Unit tests (LinstorUtilTest, LinstorPrimaryDataStoreDriverImplTest) for the device size (smallest diskful
DRBD device, no DRBD layer, DrbdOptions/ExactSize, fallback to the volume definition, missing resource
definition) and for recording it (larger device, no resource count adjustment on resize, never shrinking, LINSTOR
API failure).

3-node KVM cluster with CloudStack 4.23 and this patch applied to the LINSTOR plugin, LINSTOR primary storage
(LVM-thin, DRBD 9.3, LINSTOR 1.35):

• snapshot of an existing 1 GiB data volume created before the patch: the recorded size changed from 1073741824 to
  1077665792, the account's primary storage count rose by exactly the difference, and the backed up qcow2 on
  secondary storage has a virtual size of 1077665792 bytes
• new data volume, attached and resized: the recorded size equals the size the guest sees after the resize, the
  resource count matches
• new VM: the ROOT volume cloned from the template records 540909568 bytes, the size the guest sees

How did you try to break this feature and the system with this change?

Covered by the unit tests:

• Resources with diskless replicas (they only report the volume definition size and are ignored) and resource
  definitions with DrbdOptions/ExactSize.
• LINSTOR API failures while reading the size: the recorded size is kept and the operation continues.
• A volume whose device size is already recorded: no further size or resource count changes.

[codecov]

Codecov Report

❌ Patch coverage is 68.96552% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 18.32%. Comparing base (d02adcd) to head (f0ec553).
⚠️ Report is 1 commits behind head on 4.22.

Files with missing lines	Patch %	Lines
...tore/driver/LinstorPrimaryDataStoreDriverImpl.java	64.00%	8 Missing and 1 partial ⚠️
...cloudstack/storage/datastore/util/LinstorUtil.java	72.72%	2 Missing and 7 partials ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##               4.22   #14658   +/-   ##
=========================================
  Coverage     18.32%   18.32%           
- Complexity    16644    16657   +13     
=========================================
  Files          5939     5939           
  Lines        537768   537843   +75     
  Branches      65929    65943   +14     
=========================================
+ Hits          98551    98569   +18     
- Misses       428008   428063   +55     
- Partials      11209    11211    +2     

Flag	Coverage Δ
uitests	4.08% <ø> (ø)
unittests	19.41% <68.96%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.