为什么df仅显示RAID10阵列大小的一半?


9

我使用4个75G驱动器创建了RAID10阵列,以创建150G的存储。
一切完成之后(包括初始同步),一切看起来都不错,除了输出df -h显示仅在指定安装点上有73G的存储。

细节:

  • 该计算机是Amazon EC2上的m1.large Ubuntu 11.10实例。
  • 4个驱动器是EBS驱动器,每个驱动器的大小为75G。
  • RAID10阵列是使用以下脚本创建的:

--

#!/bin/sh

disk1="/dev/sdh1"
disk2="/dev/sdh2"
disk3="/dev/sdh3"
disk4="/dev/sdh4"

echo "*** Verifying existence of 4 volumes $disk1, $disk2, $disk3 and $disk4"
if [ -b "$disk1" -a -b "$disk2" -a -b "$disk3" -a -b "$disk4" ]; then
    echo "# Found expected block devices."
else
    echo "!!! Did not find expected block devices.  Error."
    exit -1
fi
until read -p "??? - How big (in GB) are the disks (They should be the same size)?  " disk_size && [ $disk_size ]; do
    echo "Please enter a disk size."
done 

lv_size=$(echo "scale=2; $disk_size * 2.0" | bc)
echo "*** Assuming a per disk size of $disk_size gigs, will create a logical volume of $lv_size gigs, with $lv_size reserved for snapshots"

echo "*** Partitioning disks..."

echo "~ Partitioning $disk1"
echo ',,L' | sfdisk $disk1
echo "~ Partitioning $disk2"
echo ',,L' | sfdisk $disk2
echo "~ Partitioning $disk3"
echo ',,L' | sfdisk $disk3
echo "~ Partitioning $disk4"
echo ',,L' | sfdisk $disk4

sleep 6
echo "*** Creating /dev/md0 as a RAID 10"

/sbin/mdadm /dev/md0 --create --level=10 --raid-devices=4 $disk1 $disk2 $disk3 $disk4 

echo " ~ Allocating /dev/md0 as a physical volume."

/sbin/pvcreate /dev/md0

echo " ~ Allocating a Volume Group 'mongodb_vg'"

/sbin/vgcreate -s 64M mongodb_vg /dev/md0

echo " ~ Creating a Logical Volume 'mongodb_lv'"

num_extents=$(echo "$disk_size * 1000 / 64" | bc)

/sbin/lvcreate -l $num_extents -nmongodb_lv mongodb_vg

echo " ~ Formatting the new volume (/dev/mongodb_vg/mongodb_lv) with EXT4"

/sbin/mkfs.ext4 /dev/mongodb_vg/mongodb_lv

echo " ~ Done! Go ahead and mount the new filesystem.  Suggested FStab: "
echo " /dev/mongodb_vg/mongodb_lv /data ext4 defaults,noatime 0 0"

这是我得到的输出:

*** Verifying existence of 4 volumes /dev/xvdh1, /dev/xvdh2, /dev/xvdh3 and /dev/xvdh4
# Found expected block devices.
??? - How big (in GB) are the disks (They should be the same size)?  75
*** Assuming a per disk size of 75 gigs, will create a logical volume of 150.0 gigs, with 150.0 reserved for snapshots
*** Partitioning disks...
~ Partitioning /dev/xvdh1
Checking that no-one is using this disk right now ...
BLKRRPART: Invalid argument
OK

Disk /dev/xvdh1: 9790 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
 /dev/xvdh1: unrecognized partition table type
Old situation:   
No partitions found
New situation:   
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
/dev/xvdh1p1          0+   9789    9790-  78638174+  83  Linux
/dev/xvdh1p2          0       -       0          0    0  Empty
/dev/xvdh1p3          0       -       0          0    0  Empty
/dev/xvdh1p4          0       -       0          0    0  Empty
Warning: no primary partition is marked bootable (active)
This does not matter for LILO, but the DOS MBR will not boot this disk.
Successfully wrote the new partition table

Re-reading the partition table ...
BLKRRPART: Invalid argument

If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)  
~ Partitioning /dev/xvdh2
Checking that no-one is using this disk right now ...
BLKRRPART: Invalid argument
OK

Disk /dev/xvdh2: 9790 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
 /dev/xvdh2: unrecognized partition table type
   Old situation:   
No partitions found
New situation:   
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
/dev/xvdh2p1          0+   9789    9790-  78638174+  83  Linux
/dev/xvdh2p2          0       -       0          0    0  Empty
/dev/xvdh2p3          0       -       0          0    0  Empty
/dev/xvdh2p4          0       -       0          0    0  Empty
Warning: no primary partition is marked bootable (active)
This does not matter for LILO, but the DOS MBR will not boot this disk.
Successfully wrote the new partition table

Re-reading the partition table ...
BLKRRPART: Invalid argument

If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)  
~ Partitioning /dev/xvdh3
Checking that no-one is using this disk right now ...
BLKRRPART: Invalid argument
OK

Disk /dev/xvdh3: 9790 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
 /dev/xvdh3: unrecognized partition table type
Old situation:   
No partitions found
New situation:   
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
/dev/xvdh3p1          0+   9789    9790-  78638174+  83  Linux
/dev/xvdh3p2          0       -       0          0    0  Empty
/dev/xvdh3p3          0       -       0          0    0  Empty
/dev/xvdh3p4          0       -       0          0    0  Empty
Warning: no primary partition is marked bootable (active)
This does not matter for LILO, but the DOS MBR will not boot this disk.
Successfully wrote the new partition table

Re-reading the partition table ...
BLKRRPART: Invalid argument
If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)  
~ Partitioning /dev/xvdh4
Checking that no-one is using this disk right now ...
BLKRRPART: Invalid argument
OK

Disk /dev/xvdh4: 9790 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
 /dev/xvdh4: unrecognized partition table type
Old situation:   
No partitions found
New situation:   
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
/dev/xvdh4p1          0+   9789    9790-  78638174+  83  Linux
/dev/xvdh4p2          0       -       0          0    0  Empty
/dev/xvdh4p3          0       -       0          0    0  Empty
/dev/xvdh4p4          0       -       0          0    0  Empty
Warning: no primary partition is marked bootable (active)
This does not matter for LILO, but the DOS MBR will not boot this disk.
Successfully wrote the new partition table

Re-reading the partition table ...
BLKRRPART: Invalid argument

If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)  
*** Creating /dev/md0 as a RAID 10
mdadm: partition table exists on /dev/xvdh1 but will be lost or
       meaningless after creating array
mdadm: partition table exists on /dev/xvdh2 but will be lost or
       meaningless after creating array
mdadm: partition table exists on /dev/xvdh3 but will be lost or
       meaningless after creating array
mdadm: partition table exists on /dev/xvdh4 but will be lost or
       meaningless after creating array
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
 ~ Allocating /dev/md0 as a physical volume.

  Physical volume "/dev/md0" successfully created
 ~ Allocating a Volume Group 'mongodb_vg'
  Volume group "mongodb_vg" successfully created
 ~ Creating a Logical Volume 'mongodb_lv'
  Logical volume "mongodb_lv" created
 ~ Formatting the new volume (/dev/mongodb_vg/mongodb_lv) with EXT4
mke2fs 1.41.14 (22-Dec-2010)
Filesystem label=
OS type: Linux   
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=128 blocks, Stripe width=256 blocks
4800512 inodes, 19185664 blocks
959283 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
586 block groups 
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 36 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
 ~ Done! Go ahead and mount the new filesystem.  Suggested FStab:
 /dev/mongodb_vg/mongodb_lv /data ext4 defaults,noatime 0 0

这是有关的输出df -h

Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/mongodb_vg-mongodb_lv
                       73G  180M   69G   1% /ebsRaid

这是输出 mdadm --detail /dev/md0

/dev/md0:
        Version : 1.2
  Creation Time : Wed Feb 29 10:14:39 2012
     Raid Level : raid10
     Array Size : 157283328 (150.00 GiB 161.06 GB)
  Used Dev Size : 78641664 (75.00 GiB 80.53 GB)
   Raid Devices : 4
  Total Devices : 4
    Persistence : Superblock is persistent

    Update Time : Wed Feb 29 13:21:49 2012
          State : clean
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0

         Layout : near=2
     Chunk Size : 512K

           Name : my.site.com:0  (local to host my.site.com)
           UUID : CENSORED
         Events : 19

    Number   Major   Minor   RaidDevice State
       0     202      113        0      active sync   /dev/xvdh1
       1     202      114        1      active sync   /dev/xvdh2
       2     202      115        2      active sync   /dev/xvdh3
       3     202      116        3      active sync   /dev/xvdh4

这是输出cat /proc/mdstat

Personalities : [raid10] 
md0 : active raid10 xvdh4[3] xvdh3[2] xvdh2[1] xvdh1[0]
      157283328 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]

unused devices: <none>

编辑1:

这是lvdisplay -m的输出:

  --- Logical volume ---
  LV Name                /dev/mongodb_vg/mongodb_lv
  VG Name                mongodb_vg
  LV UUID                SEpGth-cXd3-ZFhy-XLHo-T5pV-gEd1-Tgancs
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                73.19 GiB
  Current LE             1171
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     4096
  Block device           252:0

  --- Segments ---
  Logical extent 0 to 1170:
    Type                linear
    Physical volume     /dev/md0
    Physical extents    0 to 1170

编辑2:

这是输出vgdisplay

  --- Volume group ---
  VG Name               mongodb_vg
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  2
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               1
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               149.94 GiB
  PE Size               64.00 MiB
  Total PE              2399
  Alloc PE / Size       1171 / 73.19 GiB
  Free  PE / Size       1228 / 76.75 GiB
  VG UUID               CENSORED

除非您的工作是基于高度详细的RAID 1 + 0性能考虑,否则,一种更加符合KISS的方法将是创建两个不同的镜像并将它们加入LVM层。
unixtippse 2012年

你能做一个lvdisply -m并发布吗?
webtoe 2012年

顺便说一句,我认为脚本无法正确确定扩展区数。恕我直言,应该乘以1024 $ DISK_SIZE,而不是1000
路易斯·费尔南多·阿伦

@webtoe-我将其添加到了问题中。
多伦

3
我无法回答您的问题,但是令人欣喜的是看到类似这样的内容以及相关的详细信息和输出。先生,请坚持我们。
MDMarra 2012年

Answers:


6

您的卷组未使用为其创建的全部范围:

VG Size               149.94 GiB
PE Size               64.00 MiB
Total PE              2399
Alloc PE / Size       1171 / 73.19 GiB
Free  PE / Size       1228 / 76.75 GiB

您可以使用以下命令添加更多范围:

lvextend -l +100%FREE /dev/mongodb_vg/mongodb_lv /dev/md0

在输入此内容之前,请先阅读手册页

此命令将扩展卷组以使用剩余的所有FREE扩展区(如果要保持某些扩展区可用,也可以选择较少的空间)。它将利用md0上的扩展区来执行此操作。

然后,您可以使用以下方法在线调整分区大小:

resize2fs  /dev/mongodb_vg/mongodb_lv

它应该说它正在在线调整大小。我相信这可以解决您的问题,但是在尝试之前,请阅读手册页并了解他们在做什么。我不负责您破坏磁盘。

顺便说一句,在EBS上进行RAID,然后在顶部进行lvm似乎对磁盘进行虚拟化是不必要的。通过添加额外的RAID,您不会提高性能或数据安全性。如果我没记错的话,LVM已经可以进行镜像/分割。尽管从技术上讲,您可以在RAID广告无限时在LVM上的RAID上运行LVM,但我不确定这样做是否会带来很多好处(尽管我很高兴指出这一点是错误的)。


谢谢。-我上面显示的脚本,从10gen的员工(该公司背后的MongoDB)的职位采取groups.google.com/group/mongodb-user/msg/4d74b8283e4da79d -关于EBS上的mongodb。我并不是说mongodb或10gen是linux / system / ebs专家,但这听起来像是一个扎实且经过充分解释的建议。由于我尚未在该计算机上执行任何其他操作,因此我愿意接受(并将高度接受并感谢)其他/更好的解决方案。这是我第一次真正使用任何RAID设置进行任何操作,因此我不太了解LVM设置。
多伦
By using our site, you acknowledge that you have read and understand our Cookie Policy and Privacy Policy.
Licensed under cc by-sa 3.0 with attribution required.