Android 多執行緒下載，斷點續傳，執行緒池

weixin_34259232發表於2017-07-09
Android執行緒斷點

你可以在這裡看到這個demo的原始碼: 
https://github.com/onlynight/MultiThreadDownloader

效果圖

這張效果圖是同時開啟三個下載任務，限制下載執行緒數量的效果圖。

多執行緒下載原理

多執行緒下載的原理就是將下載任務分割成一個個小片段再將每個小片段分配給各個執行緒進行下載。 
例如一個檔案大小為100M，我們決定使用4個執行緒下載，那麼每個執行緒下載的大小即為25M，每個執行緒的起始以及結束位置依次如下：

0: 0-25M 
1: 25-50M 
2: 50-75M 
3: 75-END

下載請求使用的是Http GET方法，GET請求允許我們設定引數請求的資料的範圍：

HttpURLConnection conn = (HttpURLConnection)
                        new URL("download url").openConnection();
// 設定http為get方法
conn.setRequestMethod("GET");
// 設定http請求的範圍
conn.setRequestProperty("Range", "bytes=" + startPos + "-" + endPos);

多執行緒訪問檔案需要建立RandomAccessFile：

RandomAccessFile threadFile = new RandomAccessFile(
                        fileDownloader.getFileSavePath() + File.separator +
                                fileDownloader.getFilename(), "rwd");
threadFile.seek(startPos);

斷點續傳原理

端點續傳要求我們將之前下載過的檔案片段儲存下來，並且記錄上次最後的下載位置。下次繼續下載的時候從上次記錄的位置開始下載即可，無需再從頭下載。

執行緒池作用

執行緒池的原理分析和使用請檢視者幾篇文章： 
http://blog.csdn.net/u010687392/article/details/49850803 
http://blog.csdn.net/qq_17250009/article/details/50906508

試想如果我們有100個下載任務，我們讓每個任務分成3個執行緒下載，那麼每個任務都需要4個執行緒，如果100個任務同時開啟下載那麼就意味著需要同時啟動400個執行緒執行下載任務，這樣勢必會影響app效能。

像上面這樣大量的建立執行緒的操作勢必會影響作業系統的效能，等這些任務執行完成後銷燬執行緒同樣也會消耗很多的系統資源，所以Java中提出了執行緒池的概念。

下面我們來分析執行緒池是如何節約系統資源的。

從時間角度：

  • 創執行緒的時間我們假定它為: t1
  • 執行緒執行的時間我們假定為: t2
  • 銷燬執行緒的時間我們假定為: t3

就用上例我們計算其耗時為：

T = 100(t1+t2+t3)

我們使用固定上限執行緒數量的執行緒池耗時為：

T1 = 5(t1+t3)+100t2

顯然固定執行緒上限執行緒池所需的時間短了很多，固定數量執行緒池節約了執行緒建立和銷燬的時間，使用執行緒複用方法避免了執行緒的頻繁建立和銷燬，不僅節約了時間同時節約了系統資源。

關鍵程式碼

下載執行緒：

import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.RandomAccessFile;
import java.net.HttpURLConnection;
import java.net.URL;

/**
 * Created by lion on 2017/2/10.
 */

public class DownloadRunnable implements Runnable {
    private int threadId = -1;
    private FileDownloader fileDownloader;
    private int downloadedSize = 0;

    private int startPos = -1;
    private int endPos = -1;
    private int downloadLength = 0;

    private boolean isFinish;
    private boolean isStart;

    public DownloadRunnable(FileDownloader fileDownloader, int threadId, int blockSize,
                            int downloadedSize) {
        this.fileDownloader = fileDownloader;
        this.threadId = threadId;
        int fileSize = fileDownloader.getFileSize();
        this.startPos = blockSize * threadId + downloadedSize;
        this.endPos = blockSize * (threadId + 1) < fileSize ?
                blockSize * (threadId + 1) : fileSize;
        this.downloadedSize = downloadedSize;
    }

    @Override
    public void run() {
        if (startPos >= endPos) {
            isFinish = true;
        } else {
            try {
                isStart = true;
                isFinish = false;
                HttpURLConnection conn = (HttpURLConnection)
                        new URL(fileDownloader.getDownloadUrl()).openConnection();
                conn.setConnectTimeout(FileDownloader.getConnectionTimeOut());
                conn.setRequestMethod("GET");

                //set accept file meta-data type
                conn.setRequestProperty("Accept", "image/gif, image/jpeg, image/pjpeg," +
                        " image/pjpeg, application/x-shockwave-flash, application/xaml+xml, " +
                        "application/vnd.ms-xpsdocument, application/x-ms-xbap, " +
                        "application/x-ms-application, application/vnd.ms-excel, " +
                        "application/vnd.ms-powerpoint, application/msword, */*");

                conn.setRequestProperty("Accept-Language", "zh-CN");
                conn.setRequestProperty("Referer", fileDownloader.getDownloadUrl());
                conn.setRequestProperty("Charset", "UTF-8");
                conn.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 8.0; " +
                        "Windows NT 5.2; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; " +
                        ".NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)");
                conn.setRequestProperty("Connection", "Keep-Alive");

                conn.setRequestProperty("Range", "bytes=" + startPos + "-" + endPos);
                conn.connect();

                RandomAccessFile threadFile = new RandomAccessFile(
                        fileDownloader.getFileSavePath() + File.separator +
                                fileDownloader.getFilename(), "rwd");
                threadFile.seek(startPos);
                InputStream inputStream = conn.getInputStream();
                byte[] buffer = new byte[10240];
                int offset;
                downloadLength = downloadedSize;
                while ((offset = inputStream.read(buffer, 0, 10240)) != -1) {
                    threadFile.write(buffer, 0, offset);
                    downloadLength += offset;
                    fileDownloader.appendDownloadSize(offset);
                }
                threadFile.close();
                inputStream.close();

                isFinish = true;
                isStart = false;
            } catch (IOException e) {
                e.printStackTrace();
                downloadLength = -1;
            }
        }
    }

    public int getDownloadLength() {
        return downloadLength;
    }

    public int getThreadId() {
        return threadId;
    }

    public boolean isFinish() {
        return isFinish;
    }

    public boolean isStart() {
        return isStart;
    }
}

下載器：

import android.content.Context;
import android.util.SparseIntArray;

import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.Date;
import java.util.UUID;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

/**
 * Created by lion on 2017/2/7.
 */

public class FileDownloader {

    public static final String TAG = "FileDownloader";

    /**
     * http connection timeout
     */
    private static int CONNECTION_TIME_OUT = 10 * 1000;

    private DownloadProgressManager downloadProgressManager;

    private DownloadRunnable[] downloadThreads;

    private String tagName = "";

    private String downloadUrl;
    private String fileSavePath;
    private String filename;
    private int threadNum = 1;
    private int fileSize = 0;
    private int currentDownloadSize = 0;

    private SparseIntArray currentDownloads;

    public DownloadRunnable[] getDownloadThreads() {
        return downloadThreads;
    }

    public DownloadProgressManager getDownloadProgressManager() {
        return downloadProgressManager;
    }

    public int getFileSize() {
        return fileSize;
    }

    public String getFileSavePath() {
        return fileSavePath;
    }

    public String getFilename() {
        return filename;
    }

    public String getDownloadUrl() {
        return downloadUrl;
    }

    public int getCurrentDownloadSize() {
        return currentDownloadSize;
    }

    public int getThreadNum() {
        return threadNum;
    }

    synchronized int appendDownloadSize(int size) {
        currentDownloadSize += size;
        return currentDownloadSize;
    }

    public FileDownloader(Context context) {
        this.currentDownloads = new SparseIntArray();
        this.downloadProgressManager = new DownloadProgressManager(context);
    }

    private void requestFileInfo(String downloadUrl) throws RuntimeException {
        try {
            HttpURLConnection connection = (HttpURLConnection)
                    new URL(downloadUrl).openConnection();
            connection.setConnectTimeout(CONNECTION_TIME_OUT);
            connection.setRequestMethod("GET");

            //set accept file meta-data type
            connection.setRequestProperty("Accept", "image/gif, image/jpeg, image/pjpeg," +
                    " image/pjpeg, application/x-shockwave-flash, application/xaml+xml, " +
                    "application/vnd.ms-xpsdocument, application/x-ms-xbap, " +
                    "application/x-ms-application, application/vnd.ms-excel, " +
                    "application/vnd.ms-powerpoint, application/msword, */*");

            connection.setRequestProperty("Accept-Language", "zh-CN");
            connection.setRequestProperty("Referer", downloadUrl);
            connection.setRequestProperty("Charset", "UTF-8");
            connection.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 8.0; " +
                    "Windows NT 5.2; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; " +
                    ".NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)");
//            connection.setRequestProperty("Connection", "Keep-Alive");

            connection.connect();

            if (connection.getResponseCode() == 200) {
                fileSize = connection.getContentLength();
                if (fileSize <= 0) {
                    throw new RuntimeException(TAG + " Unknown file size");
                }

                filename = getFilename(connection);
            } else {
                throw new RuntimeException(TAG + " Server Response Code is "
                        + connection.getResponseCode());
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    private String getFilename(HttpURLConnection connection) {
        String filename = downloadUrl != null ?
                downloadUrl.substring(downloadUrl.lastIndexOf("/") + 1) : null;
        if (filename == null || "".equals(filename.trim())) {//如果獲取不到檔名稱
            for (int i = 0; ; i++) {
                String mine = connection.getHeaderField(i);
                if (mine == null) break;
                if ("content-disposition".equals(connection.getHeaderFieldKey(i).toLowerCase())) {
                    Matcher m = Pattern.compile(".*filename=(.*)").
                            matcher(mine.toLowerCase());
                    if (m.find()) return m.group(1);
                }
            }
            filename = UUID.randomUUID() + ".tmp";//預設取一個檔名
        }
        return filename;
    }

    public void prepare(String downloadUrl, String fileSavePath, int threadNum) {
        this.downloadUrl = downloadUrl;
        this.fileSavePath = fileSavePath;
        requestFileInfo(downloadUrl);
        SparseIntArray progresses = downloadProgressManager.getProgress(downloadUrl);

        if (threadNum <= 0) {
            threadNum = this.threadNum;
        } else {
            this.threadNum = threadNum;
        }

        if (progresses != null && progresses.size() > 0) {
            threadNum = progresses.size();
            for (int i = 0; i < progresses.size(); i++) {
                currentDownloadSize += progresses.get(i);
            }
        }

        int block = fileSize % threadNum == 0 ?
                fileSize / threadNum : fileSize / threadNum + 1;

        downloadThreads = new DownloadRunnable[threadNum];

        for (int i = 0; i < threadNum; i++) {
            downloadThreads[i] = new DownloadRunnable(this, i, block,
                    progresses != null && progresses.size() == threadNum ?
                            progresses.valueAt(progresses.keyAt(i)) == -1 ? 0 :
                                    progresses.valueAt(progresses.keyAt(i)) : 0);
        }
    }

    public void start(OnDownloadListener listener) {
        boolean isFinish = false;
        int lastDownloadSize = 0;
        int speed = 0;
        Date current = new Date();
        while (!isFinish) {
            if (listener != null) {
                int percent = (int) (currentDownloadSize / (float) fileSize * 100);
                long time = new Date().getTime() - current.getTime();
                speed = (int) ((currentDownloadSize - lastDownloadSize) / 1024f / time * 1000f);
                listener.onUpdate(fileSize, currentDownloadSize, speed, percent);
                if (percent == 100) {
                    downloadProgressManager.finishDownload(downloadUrl);
                    break;
                }
            }
            current = new Date();
            lastDownloadSize = currentDownloadSize;
            updateProgress();
            isFinish = checkFinish();
            try {
                Thread.sleep(100);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }

//        System.out.println(tagName + " DOWNLOAD FINISH");
        if (listener != null) {
            listener.onUpdate(fileSize, fileSize, 0, 100);
        }
    }

    private boolean checkFinish() {
        if (downloadThreads != null && downloadThreads.length > 0) {
            for (DownloadRunnable downloadThread : downloadThreads) {
                if (!downloadThread.isFinish()) {
                    System.out.println("checkFinish false");
                    return false;
                }
            }

            return true;
        }
        System.out.println("checkFinish true");
        return false;
    }

    public boolean isFinish() {
        return checkFinish();
    }

    void updateProgress() {
        for (DownloadRunnable downloadThread : downloadThreads) {
            updateProgress(downloadThread.getThreadId(), downloadThread.getDownloadLength());
        }
    }

    synchronized void updateProgress(int threadId, int downloaded) {
        currentDownloads.put(threadId, downloaded);
        downloadProgressManager.saveProgress(downloadUrl, currentDownloads);
//        SparseIntArray progress = downloadProgressManager.getProgress(downloadUrl);
//        for (int i = 0; i < progress.size(); i++) {
//            System.out.println("prepare progress = " + progress.valueAt(progress.keyAt(i)));
//        }
    }

    public boolean isStart() {
        for (DownloadRunnable runnable : downloadThreads) {
            if (runnable.isStart()) {
                return true;
            }
        }

        return false;
    }

    static int getConnectionTimeOut() {
        return CONNECTION_TIME_OUT;
    }

    static void setConnectionTimeOut(int timeOut) {
        CONNECTION_TIME_OUT = timeOut;
    }

    public interface OnDownloadListener {
        void onUpdate(int totalSize, int currentSize, int speed, int percent);
    }

    public void setTagName(String tagName) {
        this.tagName = tagName;
    }
}

下載管理器：

import android.content.Context;

import java.util.ArrayList;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;

/**
 * Created by lion on 2017/2/8.
 */

public class DownloadManager {

    private static int PARALLEL_DOWNLOAD_SIZE = 6;
    private static DownloadManager instance;

    private Context context;
    private Executor downloadExecutor;
    private ArrayList<FileDownloader> fileDownloaders;

    public static DownloadManager getInstance(Context context) {
        if (instance == null) {
            instance = new DownloadManager(context);
        }
        return instance;
    }

    public DownloadManager(Context context) {
        this.context = context;
        downloadExecutor = Executors.newFixedThreadPool(PARALLEL_DOWNLOAD_SIZE);
//        downloadExecutor = Executors.newCachedThreadPool();
        fileDownloaders = new ArrayList<>();
    }

    public void download(String name, final String downloadUrl, final String fileSavePath, final int threadNum,
                         final FileDownloader.OnDownloadListener listener) {
        for (FileDownloader downloader : fileDownloaders) {
            if (downloader.isFinish()) {
                downloader.setTagName(name);
                startDownload(downloader, downloadUrl, fileSavePath, threadNum, listener);
                return;
            }
        }

        FileDownloader currentDownloader = new FileDownloader(context);
        currentDownloader.setTagName(name);
        fileDownloaders.add(currentDownloader);
        startDownload(currentDownloader, downloadUrl, fileSavePath, threadNum, listener);
    }

    public void download(final String downloadUrl, final String fileSavePath, final int threadNum,
                         final FileDownloader.OnDownloadListener listener) {
        for (FileDownloader downloader : fileDownloaders) {
            if (downloader.isFinish()) {
                startDownload(downloader, downloadUrl, fileSavePath, threadNum, listener);
                return;
            }
        }

        FileDownloader currentDownloader = new FileDownloader(context);
        fileDownloaders.add(currentDownloader);
        startDownload(currentDownloader, downloadUrl, fileSavePath, threadNum, listener);
    }

    private synchronized void startDownload(final FileDownloader currentDownloader,
                                            final String downloadUrl, final String fileSavePath,
                                            final int threadNum,
                                            final FileDownloader.OnDownloadListener listener) {
        downloadExecutor.execute(new Runnable() {
            @Override
            public void run() {
                currentDownloader.prepare(downloadUrl, fileSavePath,
                        threadNum);
                if (currentDownloader.getDownloadThreads() != null) {
                    for (DownloadRunnable runnable :
                            currentDownloader.getDownloadThreads()) {
                        downloadExecutor.execute(runnable);
                    }
                }
                currentDownloader.start(listener);
            }
        });
    }

    public static void setConnectionTimeOut(int timeOut) {
        FileDownloader.setConnectionTimeOut(timeOut);
    }

    public static void setParallelDownloadSize(int size) {
        PARALLEL_DOWNLOAD_SIZE = size;
    }
}

 

相關文章