๐ Introduction
Welcome to Day 5 of our Linux Administration series!
Disk and storage management is one of the most important responsibilities of a Linux System Administrator. In enterprise environments, administrators regularly perform tasks such as:
- Checking disk and filesystem utilization
- Creating and managing partitions
- Creating and extending LVM volumes
- Increasing filesystem capacity
- Migrating data between disks
- Replacing disks without moving application data manually
- Resizing cloud disks in AWS
- Troubleshooting filesystem and mount-related issues
In this article, we will start with traditional partitioning and then move to LVM (Logical Volume Manager), which provides much greater flexibility for enterprise storage management.
Let's begin! ๐
๐ 1️⃣ Types of File Systems in Linux
Linux supports many filesystem types. Some commonly encountered filesystems in enterprise environments are:
| Filesystem | Common Usage |
|---|---|
| ext4 | General-purpose Linux filesystem |
| XFS | Common/default filesystem in RHEL-based enterprise systems |
| vfat | EFI partitions and removable/USB storage |
| NFS | Network-based Linux/Unix storage |
| CIFS/SMB | Windows file shares |
Check Filesystem Type
Use lsblk -f:
lsblk -fExample:
NAME FSTYPE LABEL UUID MOUNTPOINT
sda
├─sda1 xfs 1234-ABCD /boot
└─sda2 LVM2_member xxxx-xxxx
├─rl-root xfs xxxx-xxxx /
└─rl-home xfs xxxx-xxxx /homeOther useful commands:
blkidand:
df -Thdf -Th is particularly useful because it displays both filesystem type and disk usage.
๐ 2️⃣ Essential Disk Management Commands
๐น Check Disk Space
df -h
df -hThe -h option displays sizes in a human-readable format.
Example:
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/rl-root 20G 12G 8.0G 60% /๐น Check Filesystem Type and Usage
df -Th
df -Th๐น Check Directory Size
du -sh /var
du -sh /varFor example:
du -sh /var/log๐น Identify Large Directories
To check directories under /var:
du -h --max-depth=1 /var | sort -hrThis is extremely useful when troubleshooting a filesystem that is running out of space.
Example:
8.5G /var
5.2G /var/log
2.1G /var/cache
800M /var/lib๐น Check Inode Usage
Sometimes a filesystem has free disk space but cannot create new files because all inodes have been consumed.
Check inode usage:
df -iExample:
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda2 5242880 524000 4720880 10% /๐น List Block Devices
lsblk
lsblkFor more information:
lsblk -fYou can also display disk topology:
lsblk -o NAME,SIZE,TYPE,FSTYPE,MOUNTPOINTS๐งฑ 3️⃣ Traditional Partitioning – fdisk
Before working with LVM, it is important to understand traditional disk partitioning.
fdisk can be used to create and manage partitions on disks.
First identify the disk:
lsblkSuppose the new disk is /dev/sdb.
Start fdisk:
fdisk /dev/sdbCommon fdisk options include:
| Option | Function |
|---|---|
n | Create a new partition |
p | Display partition table |
d | Delete a partition |
t | Change partition type |
w | Write changes and exit |
q | Quit without saving |
For example:
Command (m for help): nCreate the partition and then write the changes:
Command (m for help): wVerify:
lsblkYou may now see:
sdb
└─sdb1Create a Filesystem
For ext4:
mkfs.ext4 /dev/sdb1For XFS:
mkfs.xfs /dev/sdb1⚠️ Warning:
mkfsformats the specified device. Formatting the wrong device can result in data loss. Always verify the device withlsblkbefore running the command.
Mount the Partition
Create a mount point:
mkdir /dataMount:
mount /dev/sdb1 /dataVerify:
df -h /data๐ 4️⃣ What is LVM?
LVM stands for Logical Volume Manager.
LVM provides a flexible way to manage storage by separating the physical disk from the logical filesystem layout.
Instead of directly creating a filesystem on a disk or partition, LVM introduces an additional storage-management layer.
LVM Architecture
Physical Disk
↓
Physical Volume (PV)
↓
Volume Group (VG)
↓
Logical Volume (LV)
↓
Filesystem
↓
Mount Point
Physical Disk
↓
Physical Volume (PV)
↓
Volume Group (VG)
↓
Logical Volume (LV)
↓
Filesystem
↓
Mount PointFor example:
/dev/sdb
↓
PV: /dev/sdb
↓
VG: vg01
↓
LV: lv_data
↓
XFS
↓
/dataWhy Use LVM?
LVM provides:
✔ Flexible storage management
✔ Storage pooling
✔ Online filesystem expansion
✔ Logical volume resizing
✔ Disk migration using pvmove
✔ Snapshot capabilities
✔ Easier storage administration
๐ 5️⃣ Create LVM Step-by-Step
Let's create an LVM configuration using /dev/sdb.
⚠️ Make sure
/dev/sdbis the correct empty disk before starting.
๐น Step 1: Create Physical Volume
Create the PV:
pvcreate /dev/sdbVerify:
pvsDetailed information:
pvdisplay๐น Step 2: Create Volume Group
Create a volume group named vg01:
vgcreate vg01 /dev/sdbVerify:
vgsDetailed information:
vgdisplay vg01๐น Step 3: Create Logical Volumes
Create a 1 GB ext4 LV:
lvcreate -L 1G -n lv_ext4 vg01Create another 1 GB LV:
lvcreate -L 1G -n lv_xfs vg01Verify:
lvsor:
lvdisplay๐ 6️⃣ Create Filesystems
Create an ext4 filesystem:
mkfs.ext4 /dev/vg01/lv_ext4Create an XFS filesystem:
mkfs.xfs /dev/vg01/lv_xfsVerify:
lsblk -f๐ 7️⃣ Mount Logical Volumes
Create mount points:
mkdir /lvm_ext4
mkdir /lvm_xfsMount the filesystems:
mount /dev/vg01/lv_ext4 /lvm_ext4
mount /dev/vg01/lv_xfs /lvm_xfsVerify:
df -hYou can also verify with:
lsblk -f๐ 8️⃣ Make LVM Mounts Persistent Using /etc/fstab
A mount performed manually will not necessarily survive a reboot.
Edit:
vi /etc/fstabInstead of relying on device names, using UUIDs is generally preferred.
Find the UUID:
blkid /dev/vg01/lv_ext4
blkid /dev/vg01/lv_xfsExample:
/dev/vg01/lv_ext4: UUID="1111-2222" TYPE="ext4"
/dev/vg01/lv_xfs: UUID="3333-4444" TYPE="xfs"Add entries such as:
UUID=1111-2222 /lvm_ext4 ext4 defaults 0 0
UUID=3333-4444 /lvm_xfs xfs defaults 0 0Test the configuration:
mount -aThen verify:
df -h๐ก Best Practice: Always run
mount -aafter modifying/etc/fstab. A syntax error infstabcan cause boot-related problems.
๐ 9️⃣ Extend an LVM Volume
One of the biggest advantages of LVM is the ability to increase storage capacity.
Suppose /dev/sdc is a new disk.
First verify:
lsblk๐น Step 1: Create a New Physical Volume
pvcreate /dev/sdc
pvcreate /dev/sdcVerify:
pvs๐น Step 2: Add the PV to the Existing VG
vgextend vg01 /dev/sdc
vgextend vg01 /dev/sdcVerify:
vgsThe additional capacity is now available inside vg01.
๐น Step 3: Extend the Logical Volume
For example, increase lv_ext4 by 1 GB:
lvextend -L +1G /dev/vg01/lv_ext4Verify:
lvs๐ 1️⃣0️⃣ Resize the Filesystem
Increasing the LV does not automatically mean the filesystem has been expanded unless you use an option such as lvextend -r.
For ext4:
resize2fs /dev/vg01/lv_ext4For XFS:
xfs_growfs /lvm_xfsCheck the final size:
df -h๐ก Easier Method
For supported filesystems, you can extend the LV and filesystem together:
lvextend -r -L +1G /dev/vg01/lv_ext4This is convenient because the filesystem resize is performed as part of the operation.
๐ 1️⃣1️⃣ Migrating Data Using pvmove
pvmove is extremely useful when replacing a disk in an LVM environment.
For example:
Old Disk New Disk
/dev/sdc /dev/sdd
↓ ↓
PV PV
\ /
\ /
Volume GroupSuppose /dev/sdc needs to be replaced.
Step 1: Add the New Disk
pvcreate /dev/sdd
pvcreate /dev/sddStep 2: Add It to the Volume Group
vgextend vg01 /dev/sdd
vgextend vg01 /dev/sddStep 3: Move LVM Extents
pvmove /dev/sdc /dev/sdd
pvmove /dev/sdc /dev/sddThis moves allocated extents from the old PV to the new PV.
Monitor the PVs:
pvsYou can also inspect LV-to-PV allocation:
lvs -a -o +devicesStep 4: Remove the Old Disk from the VG
Once all data has been moved:
vgreduce vg01 /dev/sdcVerify:
pvsStep 5: Remove the PV Signature
pvremove /dev/sdc
pvremove /dev/sdcThe disk can now be decommissioned or reused.
๐ก
pvmoveis particularly useful in enterprise environments when disks need to be replaced while keeping the logical volume configuration intact.
๐ 1️⃣2️⃣ Decommission LVM
When an LVM configuration is no longer required, it can be removed.
First unmount the filesystem:
umount /lvm_ext4Remove the logical volume:
lvremove /dev/vg01/lv_ext4If the volume group is no longer required:
vgremove vg01Finally remove the PV:
pvremove /dev/sdbVerify:
pvs
vgs
lvs⚠️ Warning: Removing LVs, VGs, or PVs can destroy access to data. Always confirm the storage layout and obtain the required approval before performing decommissioning activities on production systems.
☁️ 1️⃣3️⃣ Resizing a Disk in AWS
Cloud environments make disk expansion easier because an EBS volume can be increased without physically replacing the disk.
A typical Linux workflow is:
AWS EBS Volume
↓
Operating System detects increased disk
↓
Partition / LVM extended
↓
Filesystem extended๐น Step 1: Increase the EBS Volume
Increase the EBS volume size through the AWS management interface or approved AWS automation.
๐น Step 2: Verify the Disk
On the Linux server:
lsblkYou may see the disk size has increased while the partition is still smaller.
Example:
NAME SIZE TYPE MOUNTPOINT
xvda 50G disk
└─xvda1 30G part /๐น Step 3: Extend the Partition
If the root filesystem is on /dev/xvda1:
growpart /dev/xvda 1Verify:
lsblk⚠️ The exact device name may be
/dev/xvda,/dev/nvme0n1, or another name depending on the EC2 instance and storage configuration.
๐น Step 4: Extend the Filesystem
For ext4:
resize2fs /dev/xvda1For XFS:
xfs_growfs -d /Verify:
df -h๐งฉ AWS + LVM Example
If the AWS disk contains LVM, the process is slightly different.
For example:
AWS EBS
↓
/dev/nvme1n1
↓
Partition
↓
PV
↓
VG
↓
LV
↓
XFS
↓
/dataAfter increasing the EBS volume, you may need to extend each relevant layer:
lsblkThen, depending on the layout:
growpart /dev/nvme1n1 1Extend the PV:
pvresize /dev/nvme1n1p1Extend the LV:
lvextend -r -L +10G /dev/vg01/lv_dataFinally verify:
df -h /data๐ก The exact commands depend on whether the EBS volume is used as a whole-disk PV or contains a partition.
๐ฅ Troubleshooting Section
❌ Problem 1: Mount Fails
Check filesystem information:
blkidCheck /etc/fstab:
cat /etc/fstabCheck whether the mount point already exists:
mount | grep /dataTest:
mount -aCheck system logs:
journalctl -xe❌ Problem 2: LV Removal Fails – Device Busy
If you receive:
device is busyCheck which processes are using the mount point:
lsof +D /lvm_ext4or:
fuser -vm /lvm_ext4Check whether the filesystem is mounted:
findmnt /lvm_ext4After stopping the required processes, unmount:
umount /lvm_ext4Then retry:
lvremove /dev/vg01/lv_ext4❌ Problem 3: Filesystem Not Expanding
Check the disk layout:
lsblkCheck the LV:
lvdisplayCheck the filesystem:
df -ThFor LVM:
pvs
vgs
lvsFor XFS:
xfs_info /mountpointFor ext4:
tune2fs -l /dev/vg01/lv_ext4 | grep -i 'block count'๐ง Important Linux Storage Commands – Quick Reference
| Task | Command |
|---|---|
| List disks | lsblk |
| List filesystems | lsblk -f |
| Check disk usage | df -h |
| Check filesystem type | df -Th |
| Check inode usage | df -i |
| Check directory size | du -sh /path |
| Check partitions | fdisk -l |
| Create PV | pvcreate /dev/sdb |
| Display PVs | pvs |
| Create VG | vgcreate vg01 /dev/sdb |
| Display VGs | vgs |
| Create LV | lvcreate -L 1G -n lv01 vg01 |
| Display LVs | lvs |
| Extend VG | vgextend vg01 /dev/sdc |
| Extend PV | pvresize /dev/sdb |
| Extend LV | lvextend -L +1G /dev/vg01/lv01 |
| Extend LV + filesystem | lvextend -r -L +1G /dev/vg01/lv01 |
| Move data between PVs | pvmove /dev/sdc /dev/sdd |
| Remove PV from VG | vgreduce vg01 /dev/sdc |
| Remove PV | pvremove /dev/sdc |
| Grow ext4 | resize2fs |
| Grow XFS | xfs_growfs |
๐ฏ Key Takeaways
In this Day 5 Linux Administration lesson, we learned:
✅ Linux filesystem types
✅ Disk and filesystem monitoring commands
✅ Traditional partitioning with fdisk
✅ LVM architecture
✅ Creating PV, VG and LV
✅ Creating ext4 and XFS filesystems
✅ Mounting and configuring /etc/fstab
✅ Extending LVM storage
✅ Resizing ext4 and XFS filesystems
✅ Migrating data using pvmove
✅ Decommissioning LVM
✅ Resizing AWS EBS volumes
✅ Troubleshooting common storage problems
๐ Hands-On Practice
To reinforce today's concepts, create a Linux VM and practice the following:
Lab 1 – Traditional Partition
Create /dev/sdb1
↓
Create XFS filesystem
↓
Mount on /data
↓
Configure /etc/fstab
Create /dev/sdb1
↓
Create XFS filesystem
↓
Mount on /data
↓
Configure /etc/fstabLab 2 – LVM
/dev/sdc
↓
PV
↓
VG: vg01
↓
LV: lv_data
↓
XFS
↓
/data
/dev/sdc
↓
PV
↓
VG: vg01
↓
LV: lv_data
↓
XFS
↓
/dataLab 3 – Extend LVM
Add /dev/sdd and:
Create PV
↓
Extend VG
↓
Extend LV
↓
Grow filesystem
↓
Verify with df -hLab 4 – Disk Migration
/dev/sdd → /dev/sde
/dev/sdd → /dev/sdeUse:
pvmoveThen remove the old disk from the volume group.
๐งช Lab – Fill Inodes on a Linux Filesystem
A filesystem can have plenty of free disk space but still become unusable when all available inodes are consumed.
This happens when a very large number of small files are created.
Step 1 – Check Current Inode Usage
df -i /data
df -i /dataExample:
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/mapper/vg01-lv_data
655360 12000 643360 2% /dataStep 2 – Create a Test Directory
⚠️ Perform this only on a lab/test filesystem. Do not run it on /, /var, /home, or a production filesystem.
mkdir -p /data/inode-test
⚠️ Perform this only on a lab/test filesystem. Do not run it on /, /var, /home, or a production filesystem.
mkdir -p /data/inode-testStep 3 – Create a Large Number of Small Files
The following loop creates 100,000 empty files:
for i in $(seq 1 100000); do
touch /data/inode-test/file_$i
doneEach file consumes an inode even though the files contain no data.
Check inode usage:
df -i /dataYou should see the IUsed value increasing.
๐ Faster Method
For a larger test, use:
seq 1 500000 | xargs -n1 -P4 -I{} touch /data/inode-test/file_{}Check:
df -i /dataThe number of files required to exhaust the filesystem depends on how many inodes were created when the filesystem was formatted.
๐ Step 4 – Find the Directory Consuming Inodes
Use:
find /data -xdev -type f | cut -d/ -f1-3 | sort | uniq -c | sort -nr | headA simpler approach is:
find /data/inode-test -type f | wc -lExample:
100000๐งน Step 5 – Remove the Test Files
After completing the lab:
rm -rf /data/inode-testThen verify:
df -i /dataThe inode usage should drop back down.
๐ก Important Difference
Disk Space Exhaustion
Check:
df -hExample:
Use% = 95%There may still be free inodes.
Inode Exhaustion
Check:
df -iExample:
IUse% = 100%Even if:
df -hshows:
Use% = 40%you may still be unable to create new files.
This is because every file and directory requires an inode.
๐ Troubleshooting Inode Exhaustion
If a filesystem reaches 100% inode usage:
1. Identify the affected filesystem
df -i
df -i2. Find directories containing many files
find /data -xdev -type f | cut -d/ -f1-3 | sort | uniq -c | sort -nr | head
find /data -xdev -type f | cut -d/ -f1-3 | sort | uniq -c | sort -nr | head3. Count files in a suspected directory
find /data/inode-test -type f | wc -l
find /data/inode-test -type f | wc -l4. Identify files that can be safely removed
For example:
find /data/inode-test -type f -mtime +305. Remove only after validation
find /data/inode-test -type f -mtime +30 -delete
find /data/inode-test -type f -mtime +30 -deleteThen check:
df -i /data๐ฏ Interview Question
Q: Can a Linux filesystem be full even when df -h shows free space?
Answer: Yes.
A filesystem can run out of inodes even when it still has free disk blocks.
Check inode availability using:
df -iA common cause is a huge number of small files, such as:
- Application temporary files
- Cache files
- Session files
- Mail queues
- Log files
- Application-generated metadata
Remember:
Disk Space → df -h
Inodes → df -i
No comments:
Post a Comment