ZStack原始碼剖析之二次開發——在Utility上堆程式碼

泊浮目發表於2019-02-16

本文首發於泊浮目的專欄:https://segmentfault.com/blog…

背景

在上篇文章中(ZStack原始碼剖析之二次開發——可擴充套件框架
),我們簡單的瞭解了一下ZStack核心引擎的二次開發技巧。在這篇文章中,我們將一起來了解ZStack-Utility(即ZStack的Agent端)的二開姿勢。

例子

我們以ZStack管理節點呼叫startVm這個api為例子,一起來看一下在agent上的執行邏輯。

    def start(self):
        http_server = kvmagent.get_http_server()

        http_server.register_async_uri(self.KVM_START_VM_PATH, self.start_vm)

首先,得註冊一個http path用來接受reqeust。

    @kvmagent.replyerror
    def start_vm(self, req):
        cmd = jsonobject.loads(req[http.REQUEST_BODY])
        rsp = StartVmResponse()
        try:
            self._record_operation(cmd.vmInstanceUuid, self.VM_OP_START)

            self._start_vm(cmd)
            logger.debug(`successfully started vm[uuid:%s, name:%s]` % (cmd.vmInstanceUuid, cmd.vmName))
        except kvmagent.KvmError as e:
            e_str = linux.get_exception_stacktrace()
            logger.warn(e_str)
            if "burst" in e_str and "Illegal" in e_str and "rate" in e_str:
                rsp.error = "QoS exceed max limit, please check and reset it in zstack"
            elif "cannot set up guest memory" in e_str:
                logger.warn(`unable to start vm[uuid:%s], %s` % (cmd.vmInstanceUuid, e_str))
                rsp.error = "No enough physical memory for guest"
            else:
                rsp.error = e_str
            err = self.handle_vfio_irq_conflict(cmd.vmInstanceUuid)
            if err != "":
                rsp.error = "%s, details: %s" % (err, rsp.error)
            rsp.success = False
        return jsonobject.dumps(rsp)

直接進入主幹邏輯,self._start_vm(cmd)

    @lock.lock(`libvirt-startvm`)
    def _start_vm(self, cmd):
        try:
            vm = get_vm_by_uuid_no_retry(cmd.vmInstanceUuid, False)

            if vm:
                if vm.state == Vm.VM_STATE_RUNNING:
                    raise kvmagent.KvmError(
                        `vm[uuid:%s, name:%s] is already running` % (cmd.vmInstanceUuid, vm.get_name()))
                else:
                    vm.destroy()

            vm = Vm.from_StartVmCmd(cmd)
            vm.start(cmd.timeout)
        except libvirt.libvirtError as e:
            logger.warn(linux.get_exception_stacktrace())
            if "Device or resource busy" in str(e.message):
                raise kvmagent.KvmError(
                    `unable to start vm[uuid:%s, name:%s], libvirt error: %s` % (
                    cmd.vmInstanceUuid, cmd.vmName, str(e)))

            try:
                vm = get_vm_by_uuid(cmd.vmInstanceUuid)
                if vm and vm.state != Vm.VM_STATE_RUNNING:
                    raise kvmagent.KvmError(
                       `vm[uuid:%s, name:%s, state:%s] is not in running state, libvirt error: %s` % (
                        cmd.vmInstanceUuid, cmd.vmName, vm.state, str(e)))

            except kvmagent.KvmError:
                raise kvmagent.KvmError(
                    `unable to start vm[uuid:%s, name:%s], libvirt error: %s` % (cmd.vmInstanceUuid, cmd.vmName, str(e)))

關鍵邏輯:

            vm = Vm.from_StartVmCmd(cmd)
            vm.start(cmd.timeout)

先看from_StartVmCmd

    @staticmethod
    def from_StartVmCmd(cmd):
        use_virtio = cmd.useVirtio
        use_numa = cmd.useNuma

        elements = {}

        def make_root():
            root = etree.Element(`domain`)
            root.set(`type`, `kvm`)
            # self._root.set(`type`, `qemu`)
            root.set(`xmlns:qemu`, `http://libvirt.org/schemas/domain/qemu/1.0`)
            elements[`root`] = root

        def make_cpu():
            if use_numa:
                root = elements[`root`]
                e(root, `vcpu`, `128`, {`placement`: `static`, `current`: str(cmd.cpuNum)})
                # e(root,`vcpu`,str(cmd.cpuNum),{`placement`:`static`})
                tune = e(root, `cputune`)
                e(tune, `shares`, str(cmd.cpuSpeed * cmd.cpuNum))
                # enable nested virtualization
                if cmd.nestedVirtualization == `host-model`:
                    cpu = e(root, `cpu`, attrib={`mode`: `host-model`})
                    e(cpu, `model`, attrib={`fallback`: `allow`})
                elif cmd.nestedVirtualization == `host-passthrough`:
                    cpu = e(root, `cpu`, attrib={`mode`: `host-passthrough`})
                    e(cpu, `model`, attrib={`fallback`: `allow`})
                elif IS_AARCH64:
                    cpu = e(root, `cpu`, attrib={`mode`: `host-passthrough`})
                    e(cpu, `model`, attrib={`fallback`: `allow`})
                else:
                    cpu = e(root, `cpu`)
                    # e(cpu, `topology`, attrib={`sockets`: str(cmd.socketNum), `cores`: str(cmd.cpuOnSocket), `threads`: `1`})
                mem = cmd.memory / 1024
                e(cpu, `topology`, attrib={`sockets`: str(32), `cores`: str(4), `threads`: `1`})
                numa = e(cpu, `numa`)
                e(numa, `cell`, attrib={`id`: `0`, `cpus`: `0-127`, `memory`: str(mem), `unit`: `KiB`})
            else:
                root = elements[`root`]
                # e(root, `vcpu`, `128`, {`placement`: `static`, `current`: str(cmd.cpuNum)})
                e(root, `vcpu`, str(cmd.cpuNum), {`placement`: `static`})
                tune = e(root, `cputune`)
                e(tune, `shares`, str(cmd.cpuSpeed * cmd.cpuNum))
                # enable nested virtualization
                if cmd.nestedVirtualization == `host-model`:
                    cpu = e(root, `cpu`, attrib={`mode`: `host-model`})
                    e(cpu, `model`, attrib={`fallback`: `allow`})
                elif cmd.nestedVirtualization == `host-passthrough`:
                    cpu = e(root, `cpu`, attrib={`mode`: `host-passthrough`})
                    e(cpu, `model`, attrib={`fallback`: `allow`})
                elif IS_AARCH64:
                    cpu = e(root, `cpu`, attrib={`mode`: `host-passthrough`})
                    e(cpu, `model`, attrib={`fallback`: `allow`})
                else:
                    cpu = e(root, `cpu`)
                e(cpu, `topology`, attrib={`sockets`: str(cmd.socketNum), `cores`: str(cmd.cpuOnSocket), `threads`: `1`})

        def make_memory():
            root = elements[`root`]
            mem = cmd.memory / 1024
            if use_numa:
                e(root, `maxMemory`, str(68719476736), {`slots`: str(16), `unit`: `KiB`})
                # e(root,`memory`,str(mem),{`unit`:`k`})
                e(root, `currentMemory`, str(mem), {`unit`: `k`})
            else:
                e(root, `memory`, str(mem), {`unit`: `k`})
                e(root, `currentMemory`, str(mem), {`unit`: `k`})

        def make_os():
            root = elements[`root`]
            os = e(root, `os`)
            if IS_AARCH64:
                e(os, `type`, `hvm`, attrib={`arch`: `aarch64`})
                e(os, `loader`, `/usr/share/edk2.git/aarch64/QEMU_EFI-pflash.raw`, attrib={`readonly`: `yes`, `type`: `pflash`})
            else:
                e(os, `type`, `hvm`, attrib={`machine`: `pc`})
            # if not booting from cdrom, don`t add any boot element in os section
            if cmd.bootDev[0] == "cdrom":
                for boot_dev in cmd.bootDev:
                    e(os, `boot`, None, {`dev`: boot_dev})

            if cmd.useBootMenu:
                e(os, `bootmenu`, attrib={`enable`: `yes`})

        def make_features():
            root = elements[`root`]
            features = e(root, `features`)
            for f in [`acpi`, `apic`, `pae`]:
                e(features, f)
            if cmd.kvmHiddenState == True:
                kvm = e(features, "kvm")
                e(kvm, `hidden`, None, {`state`: `on`})

        def make_devices():
            root = elements[`root`]
            devices = e(root, `devices`)
            if cmd.addons and cmd.addons[`qemuPath`]:
                e(devices, `emulator`, cmd.addons[`qemuPath`])
            else:
                e(devices, `emulator`, kvmagent.get_qemu_path())
            tablet = e(devices, `input`, None, {`type`: `tablet`, `bus`: `usb`})
            e(tablet, `address`, None, {`type`:`usb`, `bus`:`0`, `port`:`1`})
            if IS_AARCH64:
                keyboard = e(devices, `input`, None, {`type`: `keyboard`, `bus`: `usb`})
            elements[`devices`] = devices

        def make_cdrom():
            devices = elements[`devices`]

            MAX_CDROM_NUM = len(Vm.ISO_DEVICE_LETTERS)
            EMPTY_CDROM_CONFIGS = None

            if IS_AARCH64:
                # AArch64 Does not support the attachment of multiple iso
                EMPTY_CDROM_CONFIGS = [
                    EmptyCdromConfig(None, None, None)
                ]
            else:
                # bus 0 unit 0 already use by root volume
                EMPTY_CDROM_CONFIGS = [
                    EmptyCdromConfig(`hd%s` % Vm.ISO_DEVICE_LETTERS[0], `0`, `1`),
                    EmptyCdromConfig(`hd%s` % Vm.ISO_DEVICE_LETTERS[1], `1`, `0`),
                    EmptyCdromConfig(`hd%s` % Vm.ISO_DEVICE_LETTERS[2], `1`, `1`)
                ]

            if len(EMPTY_CDROM_CONFIGS) != MAX_CDROM_NUM:
                logger.error(`ISO_DEVICE_LETTERS or EMPTY_CDROM_CONFIGS config error`)

            def makeEmptyCdrom(targetDev, bus, unit):
                cdrom = e(devices, `disk`, None, {`type`: `file`, `device`: `cdrom`})
                e(cdrom, `driver`, None, {`name`: `qemu`, `type`: `raw`})
                if IS_AARCH64:
                    e(cdrom, `target`, None, {`dev`: `sdc`, `bus`: `scsi`})
                else:
                    e(cdrom, `target`, None, {`dev`: targetDev, `bus`: `ide`})
                    e(cdrom, `address`, None,{`type` : `drive`, `bus` : bus, `unit` : unit})
                e(cdrom, `readonly`, None)
                return cdrom

            if not cmd.bootIso:
                for config in EMPTY_CDROM_CONFIGS:
                    makeEmptyCdrom(config.targetDev, config.bus, config.unit)
                return

            notEmptyCdrom = set([])
            for iso in cmd.bootIso:
                notEmptyCdrom.add(iso.deviceId)
                cdromConfig = EMPTY_CDROM_CONFIGS[iso.deviceId]
                if iso.path.startswith(`ceph`):
                    ic = IsoCeph()
                    ic.iso = iso
                    devices.append(ic.to_xmlobject(cdromConfig.targetDev, cdromConfig.bus , cdromConfig.unit))
                elif iso.path.startswith(`fusionstor`):
                    ic = IsoFusionstor()
                    ic.iso = iso
                    devices.append(ic.to_xmlobject(cdromConfig.targetDev, cdromConfig.bus , cdromConfig.unit))
                else:
                    cdrom = makeEmptyCdrom(cdromConfig.targetDev, cdromConfig.bus , cdromConfig.unit)
                    e(cdrom, `source`, None, {`file`: iso.path})

            emptyCdrom = set(range(MAX_CDROM_NUM)).difference(notEmptyCdrom)
            for i in emptyCdrom:
                cdromConfig = EMPTY_CDROM_CONFIGS[i]
                makeEmptyCdrom(cdromConfig.targetDev, cdromConfig.bus, cdromConfig.unit)

        def make_volumes():
            devices = elements[`devices`]
            volumes = [cmd.rootVolume]
            volumes.extend(cmd.dataVolumes)

            def filebased_volume(_dev_letter, _v):
                disk = etree.Element(`disk`, {`type`: `file`, `device`: `disk`, `snapshot`: `external`})
                e(disk, `driver`, None, {`name`: `qemu`, `type`: linux.get_img_fmt(_v.installPath), `cache`: _v.cacheMode})
                e(disk, `source`, None, {`file`: _v.installPath})

                if _v.shareable:
                    e(disk, `shareable`)

                if _v.useVirtioSCSI:
                    e(disk, `target`, None, {`dev`: `sd%s` % _dev_letter, `bus`: `scsi`})
                    e(disk, `wwn`, _v.wwn)
                    e(disk, `address`, None, {`type`: `drive`, `controller`: `0`, `unit`: str(_v.deviceId)})
                    return disk

                if _v.useVirtio:
                    e(disk, `target`, None, {`dev`: `vd%s` % _dev_letter, `bus`: `virtio`})
                elif IS_AARCH64:
                    e(disk, `target`, None, {`dev`: `sd%s` % _dev_letter, `bus`: `scsi`})
                else:
                    e(disk, `target`, None, {`dev`: `sd%s` % _dev_letter, `bus`: `ide`})
                return disk

            def iscsibased_volume(_dev_letter, _v):
                def blk_iscsi():
                    bi = BlkIscsi()
                    portal, bi.target, bi.lun = _v.installPath.lstrip(`iscsi://`).split(`/`)
                    bi.server_hostname, bi.server_port = portal.split(`:`)
                    bi.device_letter = _dev_letter
                    bi.volume_uuid = _v.volumeUuid
                    bi.chap_username = _v.chapUsername
                    bi.chap_password = _v.chapPassword

                    return bi.to_xmlobject()

                def virtio_iscsi():
                    vi = VirtioIscsi()
                    portal, vi.target, vi.lun = _v.installPath.lstrip(`iscsi://`).split(`/`)
                    vi.server_hostname, vi.server_port = portal.split(`:`)
                    vi.device_letter = _dev_letter
                    vi.volume_uuid = _v.volumeUuid
                    vi.chap_username = _v.chapUsername
                    vi.chap_password = _v.chapPassword

                    return vi.to_xmlobject()

                if _v.useVirtio:
                    return virtio_iscsi()
                else:
                    return blk_iscsi()

            def ceph_volume(_dev_letter, _v):
                def ceph_virtio():
                    vc = VirtioCeph()
                    vc.volume = _v
                    vc.dev_letter = _dev_letter
                    return vc.to_xmlobject()

                def ceph_blk():
                    if not IS_AARCH64:
                        ic = IdeCeph()
                    else:
                        ic = ScsiCeph()
                    ic.volume = _v
                    ic.dev_letter = _dev_letter
                    return ic.to_xmlobject()

                def ceph_virtio_scsi():
                    vsc = VirtioSCSICeph()
                    vsc.volume = _v
                    vsc.dev_letter = _dev_letter
                    return vsc.to_xmlobject()

                if _v.useVirtioSCSI:
                    disk = ceph_virtio_scsi()
                    if _v.shareable:
                        e(disk, `shareable`)
                    return disk

                if _v.useVirtio:
                    return ceph_virtio()
                else:
                    return ceph_blk()

            def fusionstor_volume(_dev_letter, _v):
                def fusionstor_virtio():
                    vc = VirtioFusionstor()
                    vc.volume = _v
                    vc.dev_letter = _dev_letter
                    return vc.to_xmlobject()

                def fusionstor_blk():
                    ic = IdeFusionstor()
                    ic.volume = _v
                    ic.dev_letter = _dev_letter
                    return ic.to_xmlobject()

                def fusionstor_virtio_scsi():
                    vsc = VirtioSCSIFusionstor()
                    vsc.volume = _v
                    vsc.dev_letter = _dev_letter
                    return vsc.to_xmlobject()

                if _v.useVirtioSCSI:
                    disk = fusionstor_virtio_scsi()
                    if _v.shareable:
                        e(disk, `shareable`)
                    return disk

                if _v.useVirtio:
                    return fusionstor_virtio()
                else:
                    return fusionstor_blk()

            def volume_qos(volume_xml_obj):
                if not cmd.addons:
                    return

                vol_qos = cmd.addons[`VolumeQos`]
                if not vol_qos:
                    return

                qos = vol_qos[v.volumeUuid]
                if not qos:
                    return

                if not qos.totalBandwidth and not qos.totalIops:
                    return

                iotune = e(volume_xml_obj, `iotune`)
                if qos.totalBandwidth:
                    e(iotune, `total_bytes_sec`, str(qos.totalBandwidth))
                if qos.totalIops:
                    # e(iotune, `total_iops_sec`, str(qos.totalIops))
                    e(iotune, `read_iops_sec`, str(qos.totalIops))
                    e(iotune, `write_iops_sec`, str(qos.totalIops))
                    # e(iotune, `read_iops_sec_max`, str(qos.totalIops))
                    # e(iotune, `write_iops_sec_max`, str(qos.totalIops))
                    # e(iotune, `total_iops_sec_max`, str(qos.totalIops))

            volumes.sort(key=lambda d: d.deviceId)
            scsi_device_ids = [v.deviceId for v in volumes if v.useVirtioSCSI]
            for v in volumes:
                if v.deviceId >= len(Vm.DEVICE_LETTERS):
                    err = "exceeds max disk limit, it`s %s but only 26 allowed" % v.deviceId
                    logger.warn(err)
                    raise kvmagent.KvmError(err)

                dev_letter = Vm.DEVICE_LETTERS[v.deviceId]
                if v.useVirtioSCSI:
                    dev_letter = Vm.DEVICE_LETTERS[scsi_device_ids.pop()]

                if v.deviceType == `file`:
                    vol = filebased_volume(dev_letter, v)
                elif v.deviceType == `iscsi`:
                    vol = iscsibased_volume(dev_letter, v)
                elif v.deviceType == `ceph`:
                    vol = ceph_volume(dev_letter, v)
                elif v.deviceType == `fusionstor`:
                    vol = fusionstor_volume(dev_letter, v)
                else:
                    raise Exception(`unknown volume deviceType: %s` % v.deviceType)

                assert vol is not None, `vol cannot be None`
                # set boot order for root volume when boot from hd
                if v.deviceId == 0 and cmd.bootDev[0] == `hd` and cmd.useBootMenu:
                    e(vol, `boot`, None, {`order`: `1`})
                volume_qos(vol)
                devices.append(vol)

        def make_nics():
            if not cmd.nics:
                return

            def nic_qos(nic_xml_object):
                if not cmd.addons:
                    return

                nqos = cmd.addons[`NicQos`]
                if not nqos:
                    return

                qos = nqos[nic.uuid]
                if not qos:
                    return

                if not qos.outboundBandwidth and not qos.inboundBandwidth:
                    return

                bandwidth = e(nic_xml_object, `bandwidth`)
                if qos.outboundBandwidth:
                    e(bandwidth, `outbound`, None, {`average`: str(qos.outboundBandwidth / 1024 / 8)})
                if qos.inboundBandwidth:
                    e(bandwidth, `inbound`, None, {`average`: str(qos.inboundBandwidth / 1024 / 8)})

            devices = elements[`devices`]
            for nic in cmd.nics:
                interface = e(devices, `interface`, None, {`type`: `bridge`})
                e(interface, `mac`, None, {`address`: nic.mac})
                if nic.ip is not None and nic.ip != "":
                    filterref = e(interface, `filterref`, None, {`filter`:`clean-traffic`})
                    e(filterref, `parameter`, None, {`name`:`IP`, `value`: nic.ip})
                e(interface, `alias`, None, {`name`: `net%s` % nic.nicInternalName.split(`.`)[1]})
                e(interface, `source`, None, {`bridge`: nic.bridgeName})
                if use_virtio:
                    e(interface, `model`, None, {`type`: `virtio`})
                else:
                    e(interface, `model`, None, {`type`: `e1000`})
                e(interface, `target`, None, {`dev`: nic.nicInternalName})

                nic_qos(interface)

        def make_meta():
            root = elements[`root`]

            e(root, `name`, cmd.vmInstanceUuid)
            e(root, `uuid`, uuidhelper.to_full_uuid(cmd.vmInstanceUuid))
            e(root, `description`, cmd.vmName)
            e(root, `on_poweroff`, `destroy`)
            e(root, `on_crash`, `restart`)
            e(root, `on_reboot`, `restart`)
            meta = e(root, `metadata`)
            zs = e(meta, `zstack`, usenamesapce=True)
            e(zs, `internalId`, str(cmd.vmInternalId))
            e(zs, `hostManagementIp`, str(cmd.hostManagementIp))
            clock = e(root, `clock`, None, {`offset`: cmd.clock})
            if cmd.clock == `localtime`:
                e(clock, `timer`, None, {`name`: `rtc`, `tickpolicy`: `catchup`})
                e(clock, `timer`, None, {`name`: `pit`, `tickpolicy`: `delay`})
                e(clock, `timer`, None, {`name`: `hpet`, `present`: `no`})
                e(clock, `timer`, None, {`name`: `hypervclock`, `present`: `yes`})

        def make_vnc():
            devices = elements[`devices`]
            if cmd.consolePassword == None:
                vnc = e(devices, `graphics`, None, {`type`: `vnc`, `port`: `5900`, `autoport`: `yes`})
            else:
                vnc = e(devices, `graphics`, None,
                        {`type`: `vnc`, `port`: `5900`, `autoport`: `yes`, `passwd`: str(cmd.consolePassword)})
            e(vnc, "listen", None, {`type`: `address`, `address`: `0.0.0.0`})

        def make_spice():
            devices = elements[`devices`]
            spice = e(devices, `graphics`, None, {`type`: `spice`, `port`: `5900`, `autoport`: `yes`})
            e(spice, "listen", None, {`type`: `address`, `address`: `0.0.0.0`})
            e(spice, "image", None, {`compression`: `auto_glz`})
            e(spice, "jpeg", None, {`compression`: `always`})
            e(spice, "zlib", None, {`compression`: `never`})
            e(spice, "playback", None, {`compression`: `off`})
            e(spice, "streaming", None, {`mode`: cmd.spiceStreamingMode})
            e(spice, "mouse", None, {`mode`: `client`})
            e(spice, "filetransfer", None, {`enable`: `no`})
            e(spice, "clipboard", None, {`copypaste`: `no`})

        def make_usb_redirect():
            if cmd.usbRedirect == "true":
                devices = elements[`devices`]
                e(devices, `controller`, None, {`type`: `usb`, `model`: `ich9-ehci1`})
                e(devices, `controller`, None, {`type`: `usb`, `model`: `ich9-uhci1`, `multifunction`: `on`})
                e(devices, `controller`, None, {`type`: `usb`, `model`: `ich9-uhci2`})
                e(devices, `controller`, None, {`type`: `usb`, `model`: `ich9-uhci3`})

                chan = e(devices, `channel`, None, {`type`: `spicevmc`})
                e(chan, `target`, None, {`type`: `virtio`, `name`: `com.redhat.spice.0`})
                e(chan, `address`, None, {`type`: `virtio-serial`})

                redirdev2 = e(devices, `redirdev`, None, {`type`: `spicevmc`, `bus`: `usb`})
                e(redirdev2, `address`, None, {`type`: `usb`, `bus`: `0`, `port`: `2`})
                redirdev3 = e(devices, `redirdev`, None, {`type`: `spicevmc`, `bus`: `usb`})
                e(redirdev3, `address`, None, {`type`: `usb`, `bus`: `0`, `port`: `3`})
                redirdev4 = e(devices, `redirdev`, None, {`type`: `spicevmc`, `bus`: `usb`})
                e(redirdev4, `address`, None, {`type`: `usb`, `bus`: `0`, `port`: `4`})
                redirdev5 = e(devices, `redirdev`, None, {`type`: `spicevmc`, `bus`: `usb`})
                e(redirdev5, `address`, None, {`type`: `usb`, `bus`: `0`, `port`: `6`})
            else:
                # make sure there are three default usb controllers, for usb 1.1/2.0/3.0
                devices = elements[`devices`]
                e(devices, `controller`, None, {`type`: `usb`, `index`: `0`})
                if not IS_AARCH64:
                    e(devices, `controller`, None, {`type`: `usb`, `index`: `1`, `model`: `ehci`})
                    e(devices, `controller`, None, {`type`: `usb`, `index`: `2`, `model`: `nec-xhci`})

        def make_video():
            devices = elements[`devices`]
            if IS_AARCH64:
                video = e(devices, `video`)
                e(video, `model`, None, {`type`: `virtio`})
            elif cmd.videoType != "qxl":
                video = e(devices, `video`)
                e(video, `model`, None, {`type`: str(cmd.videoType)})
            else:
                for monitor in range(cmd.VDIMonitorNumber):
                    video = e(devices, `video`)
                    e(video, `model`, None, {`type`: str(cmd.videoType)})


        def make_audio_microphone():
            if cmd.consoleMode == `spice`:
                devices = elements[`devices`]
                e(devices, `sound`,None,{`model`:`ich6`})
            else:
                return

        def make_graphic_console():
            if cmd.consoleMode == `spice`:
                make_spice()
            else:
                make_vnc()

        def make_addons():
            if not cmd.addons:
                return

            devices = elements[`devices`]
            channel = cmd.addons[`channel`]
            if channel:
                basedir = os.path.dirname(channel.socketPath)
                linux.mkdir(basedir, 0777)
                chan = e(devices, `channel`, None, {`type`: `unix`})
                e(chan, `source`, None, {`mode`: `bind`, `path`: channel.socketPath})
                e(chan, `target`, None, {`type`: `virtio`, `name`: channel.targetName})

            cephSecretKey = cmd.addons[`ceph_secret_key`]
            cephSecretUuid = cmd.addons[`ceph_secret_uuid`]
            if cephSecretKey and cephSecretUuid:
                VmPlugin._create_ceph_secret_key(cephSecretKey, cephSecretUuid)

            pciDevices = cmd.addons[`pciDevice`]
            if pciDevices:
                make_pci_device(pciDevices)

            usbDevices = cmd.addons[`usbDevice`]
            if usbDevices:
                make_usb_device(usbDevices)

        def make_pci_device(addresses):
            devices = elements[`devices`]
            for addr in addresses:
                if match_pci_device(addr):
                    hostdev = e(devices, "hostdev", None, {`mode`: `subsystem`, `type`: `pci`, `managed`: `yes`})
                    e(hostdev, "driver", None, {`name`: `vfio`})
                    source = e(hostdev, "source")
                    e(source, "address", None, {
                        "domain": hex(0) if len(addr.split(":")) == 2 else hex(int(addr.split(":")[0], 16)),
                        "bus": hex(int(addr.split(":")[-2], 16)),
                        "slot": hex(int(addr.split(":")[-1].split(".")[0], 16)),
                        "function": hex(int(addr.split(":")[-1].split(".")[1], 16))
                    })
                else:
                    raise kvmagent.KvmError(
                       `can not find pci device for address %s` % addr)

        def make_usb_device(usbDevices):
            next_uhci_port = 2
            next_ehci_port = 1
            next_xhci_port = 1
            devices = elements[`devices`]
            for usb in usbDevices:
                if match_usb_device(usb):
                    hostdev = e(devices, "hostdev", None, {`mode`: `subsystem`, `type`: `usb`, `managed`: `yes`})
                    source = e(hostdev, "source")
                    e(source, "address", None, {
                        "bus": str(int(usb.split(":")[0])),
                        "device": str(int(usb.split(":")[1]))
                    })
                    e(source, "vendor", None, {
                        "id": hex(int(usb.split(":")[2], 16))
                    })
                    e(source, "product", None, {
                        "id": hex(int(usb.split(":")[3], 16))
                    })

                    # get controller index from usbVersion
                    # eg. 1.1 -> 0
                    # eg. 2.0.0 -> 1
                    # eg. 3 -> 2
                    bus = int(usb.split(":")[4][0]) - 1
                    if bus == 0:
                        address = e(hostdev, "address", None, {`type`: `usb`, `bus`: str(bus), `port`: str(next_uhci_port)})
                        next_uhci_port += 1
                    elif bus == 1:
                        address = e(hostdev, "address", None, {`type`: `usb`, `bus`: str(bus), `port`: str(next_ehci_port)})
                        next_ehci_port += 1
                    elif bus == 2:
                        address = e(hostdev, "address", None, {`type`: `usb`, `bus`: str(bus), `port`: str(next_xhci_port)})
                        next_xhci_port += 1
                    else:
                        raise kvmagent.KvmError(`unknown usb controller %s`, bus)
                else:
                    raise kvmagent.KvmError(`cannot find usb device %s`, usb)

        # TODO(WeiW) Validate here
        def match_pci_device(addr):
            return True

        def match_usb_device(addr):
            if len(addr.split(`:`)) == 5:
                return True
            else:
                return False

        def make_balloon_memory():
            devices = elements[`devices`]
            b = e(devices, `memballoon`, None, {`model`: `virtio`})
            e(b, `stats`, None, {`period`: `10`})

        def make_console():
            devices = elements[`devices`]
            serial = e(devices, `serial`, None, {`type`: `pty`})
            e(serial, `target`, None, {`port`: `0`})
            console = e(devices, `console`, None, {`type`: `pty`})
            e(console, `target`, None, {`type`: `serial`, `port`: `0`})

        def make_sec_label():
            root = elements[`root`]
            e(root, `seclabel`, None, {`type`: `none`})

        def make_controllers():
            devices = elements[`devices`]
            e(devices, `controller`, None, {`type`: `scsi`, `model`: `virtio-scsi`})

        make_root()
        make_meta()
        make_cpu()
        make_memory()
        make_os()
        make_features()
        make_devices()
        make_video()
        make_audio_microphone()
        make_nics()
        make_volumes()
        make_cdrom()
        make_graphic_console()
        make_usb_redirect()
        make_addons()
        make_balloon_memory()
        make_console()
        make_sec_label()
        make_controllers()

        root = elements[`root`]
        xml = etree.tostring(root)

        vm = Vm()
        vm.uuid = cmd.vmInstanceUuid
        vm.domain_xml = xml
        vm.domain_xmlobject = xmlobject.loads(xml)
        return vm

顯然,上述邏輯是在組裝一份xml,便於之後的libvirt使用。

然後是

 vm.start(cmd.timeout)

可以看到,這裡是直接呼叫了libvirt的sdk。

這僅僅是一個呼叫流程。而在很多地方,來自MN的請求會直接呼叫linux的shell命令,詳情見linux.py。(獲取雲盤大小、主儲存容量等)。

問題

在基於擴充套件ZStack的Agent時,如果是一個全新的功能模組,可能並不會造成和原有程式碼的深度耦合。但如果在原有功能上的增強, 對原有程式碼進行修改可能會導致我們的業務邏輯和Utility的上游程式碼耦合。而在沒有足夠人力來維護、開發ZStack時,我們會將目標定為能夠及時跟上釋出版本。 因此,我們要儘量減少衝突。

舉個例子:我們要對啟動vm的邏輯進行增強,新增一個自己的配置寫入xml。這段程式碼如果寫進了vm_plugin.py,那麼就是一個耦合。耦合多了以後,跟上釋出版本就會很困難。

解決方案

這是一個參考方案:

如果是引入一個全新的功能模組,建議重寫一個專案。無論是程式碼規範還是自動化測試,都可以有一個很好的實踐。

如果是基於Utility的擴充套件,比如對於擴充套件的api——APIStartVmInstanceExMsg。由上游傳送http request時,將指定v2版本的agent。比如原有start vm會傳送至path:AGENT_IP:7070/vm/start;而如果我們增強了這部分邏輯,將這段程式碼copy至vm_plugin_ex.py,並註冊一個path,ex/vm/start。當然port也要重新註冊一個,就像這樣::AGENT_IP:7071/ex/vm/start

同樣的,對linux.py擴充套件時,複製一個linux2.py來存放屬於我們自己的擴充套件邏輯。

相關文章