[20201104]關於稀疏檔案(sparse files).txt

[20201104]關於稀疏檔案(sparse files).txt

--//前幾天看連結：
--//裡面提到/var/log/lastlog檔案是稀疏檔案。對方大小達到273G，複製時導致檔案系統快取緊張。
--//家裡沒有這個環境，上班看看。

--//問題1：如何知道檔案是稀疏檔案.

# cat /etc/issue
Oracle Linux Server release 5.9
Kernel \r on an \m

# ls -lh /var/log/lastlog
-rw-r--r-- 1 root root 144K 2020-11-04 15:38:11 /var/log/lastlog

# du -sk /var/log/lastlog
44      /var/log/lastlog

--//很明顯兩者看到的大小不一致。我僅僅知道oracle 的臨時表空間檔案是稀疏檔案。但是如何確定linux檔案系統裡面那個檔案是稀疏檔案
--//呢？ 找到如下連結：

#  find /var/log/lastlog  -type f -printf "%S\t%p\n"
find: warning: unrecognized format directive `%S'
S       /var/log/lastlog

--//很明顯我使用版本不支援這個引數%S。看了man find文件發現：

%b     The amount of disk space used for this file in 512-byte blocks. Since disk space is allocated in multiples of the
       filesystem block size this is usually greater than %s/1024, but it can also be smaller  if  the  file  is  a
       sparse file.

%k     The  amount  of  disk  space used for this file in 1K blocks. Since disk space is allocated in multiples of the
       filesystem block size this is usually greater than %s/1024, but it can also be smaller if the file is a sparse
       file.

#  find /var/log/lastlog  -type f -printf "%k %s\t%p\n"
44 146876       /var/log/lastlog

--//換另外的機器：

# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.8 (Maipo)

# find /var/log/lastlog  -type f -printf "%k %s %S\t%p\n"
52 15862316 0.00335689  /var/log/lastlog
--//52*1024/15862316 = .00335688685057087502,也就是高版本的find才支援%S引數。
--//舊版本改寫如下：

# find / -type f -printf "%k %s\t%p\n" | awk '{ if ( $2 >0 && $1*1024/$2 < 1.0 && $2>0 ) {print } }'

--//問題2:如何建立稀疏檔案：


# dd if=/dev/zero of=sparse_file bs=1 count=0 seek=512M
0+0 records in
0+0 records out
0 bytes (0 B) copied, 1.6717e-05 seconds, 0.0 kB/s

# find /tmp/sparse_file -type f -printf "%k %s\t%p\n" | awk '{ if ( $2 >0 && $1*1024/$2 < 1.0 && $2>0 ) {print } }'
0 536870912     /tmp/sparse_file

# ls -lh sparse_file
-rw-r--r-- 1 root root 512M 2020-11-04 16:50:45 sparse_file

# du -sk sparse_file
0       sparse_file

# du -h --apparent-size sparse_file
512M    sparse_file

--//連結還提到複製問題，我記憶裡以前也遇到類似問題，使用lftp客戶端傳輸臨時檔案到另外的機器卡死。
http://blog.itpub.net/267265/viewspace-752384/

How to copy a sparse file

Copying a sparse file with a program that does not explicitly support them may copy the entire, uncompressed size of the
file, including the sparse. So, when you copy a sparse file using cp command, the destination file gets changed to a
fully allocated file. To copy a sparse file by keeping the destination copy as a sparse file, use any of the below
commands.

# cp --sparse=always source_file new_file
# rsync --sparse source_file new_file
# cpio --sparse
# tar --sparse

--//我看了cp的man文件，發現預設是--sparse=auto。
# man cp
...
By  default,  sparse  SOURCE files are detected by a crude heuristic and the corresponding DEST file is made sparse as
well.  That is the behavior selected by --sparse=auto.  Specify --sparse=always to create a sparse DEST file whenever
the SOURCE file contains a long enough sequence of zero bytes.  Use --sparse=never to inhibit creation of sparse files.

# cp --sparse=never sparse_file aa
/bin/cp: overwrite `aa'? y

# du  sparse_file aa
0       sparse_file
524804  aa

--//很明顯這樣複製過來的檔案就不再是稀疏檔案。