7.6.8. 模型编译

horizon_plugin_pytorch.quantization.check_model(module: Union[torch.jit._script.ScriptModule, torch.nn.modules.module.Module], example_inputs: tuple, march: Optional[str] = None, input_source: Union[Sequence[str], str] = 'ddr', advice: Optional[int] = None, check_quanti_param: bool = True)
Check if nn.Module or jit.ScriptModule can be compiled by HBDK.

Dump advices for improving performance on BPU.

参数

  • module (nn.Module or jit.ScriptModule.) –

  • example_inputs (A tuple of example inputs, in torch.tensor format.) – For jit.trace and shape inference.

  • march (Specify the target march of bpu.) – Valid options are bayes and bernoulli2. If not provided, use horizon plugin global march.

  • input_source (Specify input features' sources(ddr/resizer/pyramid)) –

  • advice (Print HBDK compiler advices for improving the utilization of the) – model on bpu if layers of the model become slow by more than the specified time (in microseconds)

  • check_quanti_param (Check quanti param) –

返回

flag – 0 if pass, otherwise not.

返回类型

int

horizon_plugin_pytorch.quantization.compile_model(module: Union[torch.jit._script.ScriptModule, torch.nn.modules.module.Module], example_inputs: tuple, hbm: str, march: Optional[str] = None, name: Optional[str] = None, input_source: Union[Sequence[str], str] = 'ddr', input_layout: Optional[str] = None, output_layout: str = 'NCHW', opt: Union[str, int] = 'O2', balance_factor: int = 2, progressbar: bool = True, jobs: int = 16, debug: bool = True, extra_args: Optional[list] = None)

Compile the nn.Module or jit.ScriptModule.

参数

  • module (nn.Module or jit.ScriptModule.) –

  • example_inputs (A tuple of example inputs, in torch.tensor format.) – For jit.trace and shape inference.

  • hbm (Specify the output path of hbdk-cc.) –

  • march (Specify the target march of bpu.) – Valid options are bayes and bernoulli2. If not provided, use horizon plugin global march.

  • name (Name of the model, recorded in hbm.) – Can be obtained by hbdk-disas or hbrtGetModelNamesInHBM in runtime.

  • input_source (Specify input features' sources(ddr/resizer/pyramid)) –

  • input_layout (Specify input layout of all model inputs.) – Available layouts are NHWC, NCHW, BPU_RAW.

  • output_layout (Specify input layout of all model inputs.) – Available layouts are NHWC, NCHW, BPU_RAW.

  • opt (Specify optimization options.) – Available options are O0, O1, O2, O3, ddr, fast, balance.

  • balance_factor (Specify the balance ratio when optimization options is) – ‘balance’.

  • progressbar (Show compilation progress to alleviate anxiety.) –

  • jobs (Specify number of threads launched during compiler optimization.) – Default is ‘16’. 0 means use all available hardware concurrency.

  • debug (Enable debugging info in hbm.) –

  • extra_args (specify extra args listed in "hbdk-cc -h".) – format in list of string: e.g. [’–ability-entry’, str(entry_value), …]

返回

flag – 0 if pass, otherwise not.

返回类型

int

horizon_plugin_pytorch.quantization.export_hbir(module: Union[torch.jit._script.ScriptModule, torch.nn.modules.module.Module], example_inputs: tuple, hbir: str, march: Optional[str] = None)

Export the nn.Module or jit.ScriptModule to hbdk3.HBIR.

参数

  • module (nn.Module or jit.ScriptModule.) –

  • example_inputs (A tuple of example inputs, in torch.tensor format.) – For jit.trace and shape inference.

  • hbir (Specify the output path of hbir.) –

  • march (Specify march to export hbir.) – Valid options are bayes and bernoulli2. If not provided, use horizon plugin global march.

返回

返回类型

input names and output names

horizon_plugin_pytorch.quantization.perf_model(module: Union[torch.jit._script.ScriptModule, torch.nn.modules.module.Module], example_inputs: tuple, march: Optional[str] = None, out_dir: str = '.', name: Optional[str] = None, hbm: Optional[str] = None, input_source: Union[Sequence[str], str] = 'ddr', input_layout: Optional[str] = None, output_layout: str = 'NCHW', opt: Union[str, int] = 'O3', balance_factor: int = 2, progressbar: bool = True, jobs: int = 16, layer_details: bool = False, extra_args: Optional[list] = None)

Estimate the performance of nn.Module or jit.ScriptModule.

参数

  • module (nn.Module or jit.ScriptModule.) –

  • example_inputs (A tuple of example inputs, in torch.tensor format.) – For jit.trace and shape inference.

  • march (Specify the target march of bpu.) – Valid options are bayes and bernoulli2. If not provided, use horizon plugin global march.

  • out_dir (Specify the output directry to hold the performance results.) –

  • name (Name of the model, recorded in hbm.) – Can be obtained by hbdk-disas or hbrtGetModelNamesInHBM in runtime.

  • hbm (Specify the output path of hbdk-cc.) –

  • input_source (Specify input features' sources(ddr/resizer/pyramid)) –

  • input_layout (Specify input layout of all model inputs.) – Available layouts are NHWC, NCHW, BPU_RAW.

  • output_layout (Specify input layout of all model inputs.) – Available layouts are NHWC, NCHW, BPU_RAW.

  • opt (Specify optimization options.) – Available options are O0, O1, O2, O3, ddr, fast, balance.

  • balance_factor (Specify the balance ratio when optimization options is) – ‘balance’.

  • progressbar (Show compilation progress to alleviate anxiety.) –

  • jobs (Specify number of threads launched during compiler optimization.) – Default is ‘16’. 0 means use all available hardware concurrency.

  • layer_details (show layer performance details. (dev use only)) –

  • extra_args (specify extra args listed in "hbdk-cc -h".) – format in list of string: e.g. [’–ability-entry’, str(entry_value), …]

返回

返回类型

Performance details in json dict. Or error code when fail.

horizon_plugin_pytorch.quantization.visualize_model(module: Union[torch.jit._script.ScriptModule, torch.nn.modules.module.Module], example_inputs: tuple, march: Optional[str] = None, save_path: Optional[str] = None, show: bool = True)

Visualize nn.Module or jit.ScriptModule at the view of HBDK.

参数

  • module (nn.Module or jit.ScriptModule.) –

  • example_inputs (A tuple of example inputs, in torch.tensor format.) – For jit.trace and shape inference.

  • march (Specify the target march of bpu.) – Valid options are bayes and bernoulli2. If not provided, use horizon plugin global march.

  • save_path (Specify path to save the plot image.) –

  • show (Display the plotted image via display.) – Make sure X-server is correctly configured.

返回

返回类型

None