如何縮小物件體積

李銘昕發表於2019-08-08
物件

有時候，我們會有一種需求，那就是把一個 物件 序列化放到佇列裡，然後消費者取出物件，並處理一定邏輯。但如果物件體積十分龐大，比如一個 Model，裡面有個欄位 text，而儲存的資料是個極大的富文字。這就導致我們的佇列體積過大，造成一定不穩定因素。

所以，接下來，我們來實現一個邏輯，來處理這個問題。

相關 PR #356 #359

定義 Interface

首先，我們定義 CodeDegenerateInterface 和 CodeGenerateInterface，他們可以呼叫對應方法，完成互相轉化。

<?php

declare(strict_types=1);
/**
 * This file is part of Hyperf.
 *
 * @link     https://www.hyperf.io
 * @document https://doc.hyperf.io
 * @contact  group@hyperf.io
 * @license  https://github.com/hyperf-cloud/hyperf/blob/master/LICENSE
 */

namespace Hyperf\Contract;

interface CodeDegenerateInterface
{
    public function degenerate(): CodeGenerateInterface;
}

<?php

declare(strict_types=1);
/**
 * This file is part of Hyperf.
 *
 * @link     https://www.hyperf.io
 * @document https://doc.hyperf.io
 * @contact  group@hyperf.io
 * @license  https://github.com/hyperf-cloud/hyperf/blob/master/LICENSE
 */

namespace Hyperf\Contract;

interface CodeGenerateInterface
{
    public function generate(): CodeDegenerateInterface;
}

測試 generate 和 degenerate

首先我們寫一個可以互相轉化的 Model 和 Meta

<?php

declare(strict_types=1);
/**
 * This file is part of Hyperf.
 *
 * @link     https://www.hyperf.io
 * @document https://doc.hyperf.io
 * @contact  group@hyperf.io
 * @license  https://github.com/hyperf-cloud/hyperf/blob/master/LICENSE
 */

namespace HyperfTest\AsyncQueue\Stub;

use Hyperf\Contract\CodeDegenerateInterface;
use Hyperf\Contract\CodeGenerateInterface;
use Hyperf\Utils\Context;

class DemoModel implements CodeGenerateInterface
{
    public $id;

    public $name;

    public $gendar;

    public $signature;

    public function __construct($id, $name, $gendar, $signature)
    {
        $this->id = $id;
        $this->name = $name;
        $this->gendar = $gendar;
        $this->signature = $signature;
    }

    public function generate(): CodeDegenerateInterface
    {
        Context::set('test.async-queue.demo.model.' . $this->id, [
            $this->name, $this->gendar, $this->signature,
        ]);

        return new DemoModelMeta($this->id);
    }
}

<?php

declare(strict_types=1);
/**
 * This file is part of Hyperf.
 *
 * @link     https://www.hyperf.io
 * @document https://doc.hyperf.io
 * @contact  group@hyperf.io
 * @license  https://github.com/hyperf-cloud/hyperf/blob/master/LICENSE
 */

namespace HyperfTest\AsyncQueue\Stub;

use Hyperf\Contract\CodeDegenerateInterface;
use Hyperf\Contract\CodeGenerateInterface;
use Hyperf\Utils\Context;

class DemoModelMeta implements CodeDegenerateInterface
{
    public $id;

    public function __construct($id)
    {
        $this->id = $id;
    }

    public function degenerate(): CodeGenerateInterface
    {
        $data = Context::get('test.async-queue.demo.model.' . $this->id);

        return new DemoModel($this->id, ...$data);
    }
}

然後編寫對應單元測試

use HyperfTest\AsyncQueue\Stub\DemoModel;
use HyperfTest\AsyncQueue\Stub\DemoModelMeta;

public function testDemoModelGenerate()
{
    $content = Str::random(1000);

    $model = new DemoModel(1, 'Hyperf', 1, $content);
    $s1 = serialize($model);
    $this->assertSame(1128, strlen($s1));

    $meta = $model->generate();
    $s2 = serialize($meta);
    $this->assertSame(65, strlen($s2));
    $this->assertInstanceOf(DemoModelMeta::class, $meta);

    $model2 = $meta->degenerate();
    $this->assertEquals($model, $model2);
}

改造 AsyncQueue

接下來，我們需要改造 AsyncQueue，這裡的處理就很簡單了，我們在壓入佇列前，和彈出佇列後，分別進行處理。

首先我們改造一下 Job 基類。

<?php

declare(strict_types=1);
/**
 * This file is part of Hyperf.
 *
 * @link     https://www.hyperf.io
 * @document https://doc.hyperf.io
 * @contact  group@hyperf.io
 * @license  https://github.com/hyperf-cloud/hyperf/blob/master/LICENSE
 */

namespace Hyperf\AsyncQueue;

use Hyperf\Contract\CodeDegenerateInterface;
use Hyperf\Contract\CodeGenerateInterface;

abstract class Job implements JobInterface, CodeGenerateInterface, CodeDegenerateInterface
{
    /**
     * @var int
     */
    protected $maxAttempts = 0;

    public function getMaxAttempts(): int
    {
        return $this->maxAttempts;
    }

    /**
     * @return JobInterface
     */
    public function degenerate(): CodeGenerateInterface
    {
        foreach ($this as $key => $value) {
            if ($value instanceof CodeDegenerateInterface) {
                $this->{$key} = $value->degenerate();
            }
        }

        return $this;
    }

    /**
     * @return JobInterface
     */
    public function generate(): CodeDegenerateInterface
    {
        foreach ($this as $key => $value) {
            if ($value instanceof CodeGenerateInterface) {
                $this->{$key} = $value->generate();
            }
        }

        return $this;
    }
}

接下來修改我們的 訊息 類，每當我們序列化時，壓縮 Job，反序列化時解壓縮 Job 即可。

<?php

declare(strict_types=1);
/**
 * This file is part of Hyperf.
 *
 * @link     https://www.hyperf.io
 * @document https://doc.hyperf.io
 * @contact  group@hyperf.io
 * @license  https://github.com/hyperf-cloud/hyperf/blob/master/LICENSE
 */

namespace Hyperf\AsyncQueue;

use Hyperf\Contract\CodeDegenerateInterface;
use Hyperf\Contract\CodeGenerateInterface;
use Serializable;

class Message implements MessageInterface, Serializable
{
    /**
     * @var JobInterface
     */
    protected $job;

    /**
     * @var int
     */
    protected $attempts = 0;

    public function __construct(JobInterface $job)
    {
        $this->job = $job;
    }

    public function job(): JobInterface
    {
        return $this->job;
    }

    public function attempts(): bool
    {
        if ($this->job->getMaxAttempts() > $this->attempts++) {
            return true;
        }
        return false;
    }

    public function serialize()
    {
        if ($this->job instanceof CodeGenerateInterface) {
            $this->job = $this->job->generate();
        }

        return serialize($this->job);
    }

    public function unserialize($serialized)
    {
        $this->job = unserialize($serialized);
        if ($this->job instanceof CodeDegenerateInterface) {
            $this->job = $this->job->degenerate();
        }
    }
}

接下來編寫單元測試

<?php

declare(strict_types=1);
/**
 * This file is part of Hyperf.
 *
 * @link     https://www.hyperf.io
 * @document https://doc.hyperf.io
 * @contact  group@hyperf.io
 * @license  https://github.com/hyperf-cloud/hyperf/blob/master/LICENSE
 */

namespace HyperfTest\AsyncQueue\Stub;

use Hyperf\AsyncQueue\Job;

class DemoJob extends Job
{
    public $id;

    public $model;

    public function __construct($id, $model = null)
    {
        $this->id = $id;
        $this->model = $model;
    }

    public function handle()
    {
    }
}

use Hyperf\AsyncQueue\Driver\RedisDriver;
use Hyperf\AsyncQueue\Message;
use Hyperf\Utils\Context;
use Hyperf\Utils\Packer\PhpSerializerPacker;
use Hyperf\Utils\Str;
use HyperfTest\AsyncQueue\Stub\DemoJob;
use HyperfTest\AsyncQueue\Stub\DemoModel;
use HyperfTest\AsyncQueue\Stub\DemoModelMeta;
use HyperfTest\AsyncQueue\Stub\Redis;
use Mockery;
use PHPUnit\Framework\TestCase;
use Psr\Container\ContainerInterface;
use Psr\EventDispatcher\EventDispatcherInterface;

public function testAsyncQueueJobGenerate()
{
    $container = $this->getContainer();
    $packer = $container->get(PhpSerializerPacker::class);
    $driver = new RedisDriver($container, [
        'channel' => 'test',
    ]);

    $id = uniqid();
    $content = Str::random(1000);
    $model = new DemoModel(1, 'Hyperf', 1, $content);
    $driver->push(new DemoJob($id, $model));

    $serialized = (string) Context::get('test.async-queue.lpush.value');
    $this->assertSame(218, strlen($serialized));

    /** @var Message $class */
    $class = $packer->unpack($serialized);

    $this->assertSame($id, $class->job()->id);
    $this->assertEquals($model, $class->job()->model);

    $key = Context::get('test.async-queue.lpush.key');
    $this->assertSame('test:waiting', $key);
}

protected function getContainer()
{
    $packer = new PhpSerializerPacker();
    $container = Mockery::mock(ContainerInterface::class);
    $container->shouldReceive('get')->with(PhpSerializerPacker::class)->andReturn($packer);
    $container->shouldReceive('get')->once()->with(EventDispatcherInterface::class)->andReturn(null);
    $container->shouldReceive('get')->once()->with(\Redis::class)->andReturn(new Redis());

    return $container;
}

這樣，我們的需求就算完成了。

本作品採用《CC 協議》，轉載必須註明作者和本文連結
Any fool can write code that a computer can understand. Good programmers write code that humans can understand.

相關文章