6.1.5.4. hat.models¶

Models widely used in upper module in HAT.

6.1.5.4.1. Models¶

embeddings

6.1.5.4.1.1. backbones¶

`EfficientNet`	A module of EfficientNet.
`efficientnet`	A module of efficientnet.
`efficientnet_lite`	A module of efficientnet_lite.
`MobileNetV1`	A module of mobilenetv1.
`ResNet18`	A module of resnet18.
`ResNet50`	A module of resnet50.
`VargConvNet`	A module of vargconvnet.
`ResNet50V2`	A module of resnet50V2.
`VargNetV2`	A module of vargnetv2.
`TinyVargNetV2`	A module of TinyVargNetv2.
`get_vargnetv2_stride2channels`	Get vargnet v2 stride to channel dict with giving channels and strides.
`CocktailVargNetV2`	CocktailVargNetV2.
`MixVarGENet`	Module of MixVarGENet.
`get_mixvargenet_stride2channels`	Get mixvargenet stride to channel dict with giving channels and strides.

6.1.5.4.1.2. losses¶

`CEWithHardMining`	CE loss with online hard negative mining and auto average factor.
`CEWithLabelSmooth`	The losses of cross-entropy with label smooth.
`CrossEntropyLoss`	Calculate cross entropy loss of multi stride output.
`SoftTargetCrossEntropy`	The losses of cross-entropy with soft target.
`CEWithWeightMap`	Crossentropy loss with image-specfic class weighted map within batch.
`ElementwiseL1HingeLoss`	Elementwise L1 Hinge Loss.
`ElementwiseL2HingeLoss`	Elementwise L2 Hinge Loss.
`FocalLoss`	Sigmoid focal loss.
`FocalLossV2`	Focal Loss.
`SoftmaxFocalLoss`	Focal Loss.
`LaneFastFocalLoss`	Modified focal loss. Exactly the same as CornerNet,
`GaussianFocalLoss`	Guassian focal loss.
`GIoULoss`	Generalized Intersection over Union Loss.
`MSELoss`	MSE (mean squared error) loss with clip value.
`SegLoss`	Segmentation loss wrapper.
`SmoothL1Loss`	Smooth L1 Loss.
`WeightedSquaredHingeLoss`	Weighted Squared ElementWiseHingeLoss.
`MixSegLossMultipreds`	Calculate multi-losses with multi-preds and correspondence targets.
`MixSegLoss`	Calculate multi-losses with same prediction and target.
`LnNormLoss`	LnNorm loss.
`L1Loss`	Smooth L1 Loss.
`MultiStrideLosses`	Multiple Stride Losses.
`SegEdgeLoss`

6.1.5.4.1.3. necks¶

`BiFPN`	Weighted Bi-directional Feature Pyramid Network(BiFPN).
`DwUnet`	Unet segmentation neck structure.
`FPN`
`RetinaNetFPN`	FPN for RetinaNet.
`Unet`	Unet neck module.
`PAFPN`	Path Aggregation Network for Instance Segmentation.
`FastSCNNNeck`	Upper neck module for segmentation.
`VargPAFPN`	Path Aggregation Network with BasicVargNetBlock or BasicMixVargNetBlock.
`SECONDNeck`	Second FPN modules.

6.1.5.4.1.4. structures¶

`Classifier`	The basic structure of classifier.
`EncoderDecoder`	The basic structure of encoder decoder.
`BMSegmentor`	The segmentor structure that inputs image metas into postprocess.
`Segmentor`	The basic structure of segmentor.
`SegmentorV2`	The basic structure of segmentor.

6.1.5.4.1.4.1. detectors¶

`RetinaNet`	The basic structure of retinanet.
`FCOS`	The basic structure of fcos.
`FCOS3D`	The basic structure of fcos3d.
`PointPillarsDetector`	The basic structure of PointPillars.

6.1.5.4.1.4.2. fcos¶

`FCOSConverter`
`FCOSDecoder`	param num_classes Number of categories excluding the background category.
`FCOSDecoder4RCNN`	Decoder for FCOS+RCNN Architecture.
`FCOSLoss`	FCOS loss wrapper.
`FCOSMultiStrideFilter`	Filter used for post-processing of FCOS.
`FCOSMultiStrideCatFilter`	A modified Filter used for post-processing of FCOS.
`FCOSHead`	Anchor-free head used in FCOS <https://arxiv.org/abs/1904.01355>.
`FCOSTarget`	Generate cls and reg targets for FCOS in training stage.
`DynamicFcosTarget`	Generate cls and box training targets for FCOS based on simOTA label assignment strategy used in YOLO-X.
`multiclass_nms`	NMS for multi-class bboxes.
`get_points`	Generate points according to feat_sizes.
`distance2bbox`	Decode distance prediction to bounding box.

6.1.5.4.1.4.3. lidar¶

`PillarFeatureNet`
`PointPillarScatter`
`Anchor3DGeneratorStride`	Lidar 3D Anchor Generator by stride.
`LidarTargetAssigner`	TargetAssigner for Lidar.
`GroundBox3dCoder`	Box3d Coder for Lidar.

6.1.5.4.1.4.4. retinanet¶

RetinaNetHead

An anchor-based head used in RetinaNet.

RetinaNetPostProcess

The postprocess of RetinaNet.

RetinanetMultiStrideFilter

6.1.5.4.1.4.5. seg¶

`SegDecoder`	Semantic Segmentation Decoder.
`VargNetSegDecoder`	Semantic Segmentation Decoder.
`SegHead`	Head Module for segmentation task.
`SegTarget`	Generate training targets for Seg task.
`FRCNNSegHead`	FRCNNSegHead module for segmentation task.

6.1.5.4.1.4.6. deeplab¶

Deeplabv3plusHead

Head Module for FCN.

6.1.5.4.1.4.7. fcn¶

`FCNHead`	Head Module for FCN.
`DepthwiseSeparableFCNHead`
`FCNTarget`	Generate Target for FCN.
`FCNDecoder`	FCN Decoder.

6.1.5.4.1.4.8. pointpillars¶

`BatchVoxelization`	Batch voxelization.
`PointPillarsHead`	Basic module of PointPillarsHead.
`PointPillarsPostProcess`	PointPillars PostProcess Module.
`PointPillarsPreProcess`	Point Pillars preprocess, include voxelization and extend features.
`PointPillarsLoss`	PointPillars Loss Module.

6.1.5.4.2. API Reference¶

class hat.models.backbones.CocktailVargNetV2(bn_kwargs: dict, model_type: str = 'VargNetV2', alpha: float = 1.0, group_base: int = 8, factor: int = 2, bias: bool = True, disable_quanti_input: bool = False, flat_output: bool = True, input_channels: int = 3, head_factor: int = 1, input_resize_scale: int = None, top_layer: Optional[torch.nn.modules.module.Module] = None)¶

CocktailVargNetV2.

对 VargNetV2 进行了简单魔改. 主要是去掉对 num_classes 作为 args 的要求和支持 top_layer 自定义.

TODO(ziyang01.wang) 重构计划, 将相应的修改吸收到 VargNetV2 中.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.backbones.EfficientNet(model_type: str, coefficient_params: tuple, num_classes: int, bn_kwargs: dict = None, bias: bool = False, drop_connect_rate: float = None, depth_division: int = 8, activation: str = 'relu', use_se_block: bool = False, blocks_args: Sequence[Dict] = (BlockArgs(kernel_size=3, num_repeat=1, in_filters=32, out_filters=16, expand_ratio=1, id_skip=True, strides=1, se_ratio=0.25), BlockArgs(kernel_size=3, num_repeat=2, in_filters=16, out_filters=24, expand_ratio=6, id_skip=True, strides=2, se_ratio=0.25), BlockArgs(kernel_size=5, num_repeat=2, in_filters=24, out_filters=40, expand_ratio=6, id_skip=True, strides=2, se_ratio=0.25), BlockArgs(kernel_size=3, num_repeat=3, in_filters=40, out_filters=80, expand_ratio=6, id_skip=True, strides=2, se_ratio=0.25), BlockArgs(kernel_size=5, num_repeat=3, in_filters=80, out_filters=112, expand_ratio=6, id_skip=True, strides=1, se_ratio=0.25), BlockArgs(kernel_size=5, num_repeat=4, in_filters=112, out_filters=192, expand_ratio=6, id_skip=True, strides=2, se_ratio=0.25), BlockArgs(kernel_size=3, num_repeat=1, in_filters=192, out_filters=320, expand_ratio=6, id_skip=True, strides=1, se_ratio=0.25)), include_top: bool = True, flat_output: bool = True, input_channels: int = 3, resolution: int = 0, split_expand_conv: bool = False, quant_input: bool = True)¶

A module of EfficientNet.

参数

model_type (str) – Select to use which EfficientNet(B0-B7 or lite0-4), for EfficientNet model, model_type must be one of: [‘b0’, ‘b1’, ‘b2’, ‘b3’, ‘b4’, ‘b5’, ‘b6’, ‘b7’], for EfficientNet-lite model, model_type must be one of: [‘lite0’, ‘lite1’, ‘lite2’, ‘lite3’, ‘lite4’].
coefficient_params (tuple) – Parameter coefficients of EfficientNet, include: width_coefficient(float): scaling coefficient for net width. depth_coefficient(float): scaling coefficient for net depth. default_resolution(int): default input image size. dropout_rate(float): dropout rate for final classifier layer. num_classes (int): Num classes of output layer.
bn_kwargs (dict) – Dict for Bn layer.
bias (bool) – Whether to use bias in module.
drop_connect_rate (float) – Dropout rate at skip connections.
depth_division (int) – Depth division, Defaults to 8.
activation (str) – Activation layer, defaults to ‘relu’.
use_se_block (bool) – Whether to use SEBlock in module.
blocks_args (list) – A list of BlockArgs to MBConvBlock modules.
include_top (bool) – Whether to include output layer.
flat_output (bool) – Whether to view the output tensor.
input_channels (int) – Input channels of first conv.
split_expand_conv (bool) – Whether split expand conv into two conv. Set to true when expand conv is too large to deploy on xj3.
quant_input (bool) – Whether quant input.

forward(inputs)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.backbones.MixVarGENet(net_config: List[hat.models.backbones.mixvargenet.MixVarGENetConfig], num_classes: int, bn_kwargs: dict, output_list: Union[List[int], Tuple[int]] = (), disable_quanti_input: bool = False, fc_filter: int = 1024, include_top: bool = True, flat_output: bool = True, bias: bool = False, input_channels: int = 3, input_sequence_length: int = 1, input_resize_scale: int = None, warping_module: torch.nn.modules.module.Module = None)¶

Module of MixVarGENet.

参数

net_config (List[MixVarGENetConfig]) – network setting.
num_classes (int) – Num classes.
bn_kwargs (dict) – Kwargs of bn layer.
output_list (List[int]) – Output id of net_config blocks. The output of these block will be the output of this net. Set output_list as [] would export all block’s output.
disable_quanti_input (bool) – whether quanti input.
fc_filter (int) – the out_channels of the last_conv.
include_top (bool) – Whether to include output layer.
flat_output (bool) – Whether to view the output tensor.
bias (bool) – Whehter to use bias.
input_channels (int) – Input image channels, first conv input channels is input_channels times input_sequence_length.
input_resize_scale – This will resize the input image with the scale value.

forward(x, uv_map=None)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

process_sequence_input(x: List) → Union[torch.Tensor, horizon_plugin_pytorch.qtensor.QTensor]¶: Process sequence input with cat.

class hat.models.backbones.MobileNetV1(num_classes: int, bn_kwargs: dict, alpha: float = 1.0, bias: bool = True, dw_with_relu: bool = True, include_top: bool = True, flat_output: bool = True)¶

A module of mobilenetv1.

参数

num_classes (int) – Num classes of output layer.
bn_kwargs (dict) – Dict for BN layer.
alpha (float) – Alpha for mobilenetv1.
bias (bool) – Whether to use bias in module.
dw_with_relu (bool) – Whether to use relu in dw conv.
include_top (bool) – Whether to include output layer.
flat_output (bool) – Whether to view the output tensor.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.backbones.ResNet18(num_classes: int, bn_kwargs: dict, bias: bool = True, include_top: bool = True, flat_output: bool = True)¶

A module of resnet18.

参数

num_classes (int) – Num classes of output layer.
bn_kwargs (dict) – Dict for BN layer.
bias (bool) – Whether to use bias in module.
include_top (bool) – Whether to include output layer.
flat_output (bool) – Whether to view the output tensor.

class hat.models.backbones.ResNet50(num_classes: int, bn_kwargs: dict, bias: bool = True, include_top: bool = True, flat_output: bool = True, stride_change: bool = False)¶

A module of resnet50.

参数

num_classes (int) – Num classes of output layer.
bn_kwargs (dict) – Dict for BN layer.
bias (bool) – Whether to use bias in module.
include_top (bool) – Whether to include output layer.
flat_output (bool) – Whether to view the output tensor.

class hat.models.backbones.ResNet50V2(num_classes: int, group_base: int, bn_kwargs: dict, bias: bool = True, extend_features: bool = False, include_top: bool = True, flat_output: bool = True)¶

A module of resnet50V2.

参数

num_classes – Num classes of output layer.
group_base – Group base for ExtendVarGNetFeatures.
bn_kwargs – Dict for BN layer.
bias – Whether to use bias in module.
extend_features – Whether to extend features.
include_top – Whether to include output layer.
flat_output – Whether to view the output tensor.

class hat.models.backbones.TinyVargNetV2(num_classes, bn_kwargs: dict, alpha: float = 1.0, group_base: int = 8, factor: int = 2, bias: bool = True, extend_features: bool = False, disable_quanti_input: bool = False, include_top: bool = True, flat_output: bool = True, input_channels: int = 3, input_sequence_length: int = 1, head_factor: int = 1, input_resize_scale: int = None, channel_list: Tuple[int] = (32, 32, 64, 128, 256))¶

A module of TinyVargNetv2.

参数

num_classes (int) – Num classes of output layer.
bn_kwargs (dict) – Dict for BN layer.
alpha (float) – Alpha for tinyvargnetv2.
group_base (int) – Group base for tinyvargnetv2.
factor (int) – Factor for channel expansion in basic block.
bias (bool) – Whether to use bias in module.
extend_features (bool) – Whether to extend features.
include_top (bool) – Whether to include output layer.
flat_output (bool) – Whether to view the output tensor.
input_channels (int) – Input channels of first conv.
input_sequence_length (int) – Length of input sequence.
head_factor (int) – Factor for channels expansion of stage1(mod2).
input_resize_scale (int) – Narrow_model need resize input 0.65 scale, While int_infer or visualize or eval
channel_list (tuple) – Number of channels in each stage.

class hat.models.backbones.VargConvNet(num_classes: int, bn_kwargs: dict, channels_list: list, repeats: list, group_list: int, factor_list: int, out_channels: int = 1024, bias: bool = True, include_top: bool = True, flat_output: bool = True, input_channels: int = 3, deep_stem: bool = True)¶

A module of vargconvnet.

参数

num_classes – Num classes of output layer.
bn_kwargs – Dict for BN layer.
channels_list – List for output channels
repeats – Depth of each stage.
group_list – Group of each stage.
factor_list – Factor for each stage.
out_channels – Output channels.
bias – Whether to use bias in module.
include_top – Whether to include output layer.
flat_output – Whether to view the output tensor.
input_channels – Input channels of first conv.
deep_stem – Whether use deep stem.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.backbones.VargNetV2(num_classes, bn_kwargs: dict, model_type: str = 'VargNetV2', alpha: float = 1.0, group_base: int = 8, factor: int = 2, bias: bool = True, extend_features: bool = False, disable_quanti_input: bool = False, include_top: bool = True, flat_output: bool = True, input_channels: int = 3, input_sequence_length: int = 1, head_factor: int = 1, input_resize_scale: int = None, channel_list: Tuple[int] = (32, 32, 64, 128, 256))¶

A module of vargnetv2.

参数

num_classes (int) – Num classes of output layer.
bn_kwargs (dict) – Dict for BN layer.
model_type (str) – Choose to use VargNetV2 or TinyVargNetV2.
alpha (float) – Alpha for vargnetv2.
group_base (int) – Group base for vargnetv2.
factor (int) – Factor for channel expansion in basic block.
bias (bool) – Whether to use bias in module.
extend_features (bool) – Whether to extend features.
include_top (bool) – Whether to include output layer.
flat_output (bool) – Whether to view the output tensor.
input_channels (int) – Input channels of first conv.
input_sequence_length (int) – Length of input sequence.
head_factor (int) – Factor for channels expansion of stage1(mod2).
input_resize_scale (int) – Narrow_model need resize input 0.65 scale, While int_infer or visualize or eval
channel_list (tuple) – Number of channels in each stage.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

process_sequence_input(x: List) → Union[torch.Tensor, horizon_plugin_pytorch.qtensor.QTensor]¶: Process sequence input with cat.

hat.models.backbones.efficientnet(model_type, **kwargs)¶: A module of efficientnet.

hat.models.backbones.efficientnet_lite(model_type, **kwargs)¶: A module of efficientnet_lite.

hat.models.backbones.get_mixvargenet_stride2channels(net_config: Union[List[List[hat.models.backbones.mixvargenet.MixVarGENetConfig]], List[List[hat.models.backbones.mixvargenet.IdentityConfig]]], strides: Optional[List[int]] = None) → Dict¶

Get mixvargenet stride to channel dict with giving channels and strides.

参数

net_config – network setting
strides – stride list corresponding to channels.

Returns: strides2channels: a stride to channel dict.

hat.models.backbones.get_vargnetv2_stride2channels(alpha: float, channels: Optional[List[int]] = None, strides: Optional[List[int]] = None) → Dict¶

Get vargnet v2 stride to channel dict with giving channels and strides.

参数

alpha – channel multipler.
channels – base channel of each stride.
strides – stride list corresponding to channels.

Returns: strides2channels: a stride to channel dict.

class hat.models.losses.CEWithHardMining(use_sigmoid: bool = False, ignore_index: int = - 1, norm_type: str = 'none', reduction: str = 'mean', loss_weight: float = 1.0, class_weight: Optional[torch.Tensor] = None, hard_neg_mining_cfg: Optional[Dict] = None)¶

CE loss with online hard negative mining and auto average factor.

参数

use_sigmoid – Whether logits tensor is converted to probability through sigmoid, Defaults to False. If True, use F.binary_cross_entropy_with_logits. If False, use F.cross_entropy.
ignore_index – Specifies a target value that is ignored and does not contribute to the loss.
norm_type – Normalization method, can be “fg_elt”, in which normalization factor is the number of foreground elements, “fbg_elt” the number of foreground and background element. “none” no normalize on loss. Defaults to “none”.
reduction – The method used to reduce the loss. Options are [“none”, “mean”, “sum”]. Default to “mean”.
loss_weight – Global weight of loss. Defaults is 1.0.
class_weight – Weight of each class. If given must be a vector with length equal to the number of classes. Default to None.
hard_neg_mining_cfg – hard negative mining config. Please refer to LossHardNegativeMining.

forward(pred, target, weight=None, avg_factor=None)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.CEWithLabelSmooth(smooth_alpha=0.1)¶

The losses of cross-entropy with label smooth.

参数: smooth_alpha (float) – Alpha of label smooth.

forward(input, target)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.CEWithWeightMap(weight_min: float = 0.5, remap_params: Optional[Dict] = None, **kwargs)¶

Crossentropy loss with image-specfic class weighted map within batch.

参数

weight_min – Min weight for each label.
remap_params – Params for remap label.

forward(pred, target, weight=None, avg_factor=None)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.CrossEntropyLoss(use_sigmoid: bool = False, reduction: str = 'mean', class_weight: Optional[List[float]] = None, loss_weight: float = 1.0, ignore_index: int = - 1, loss_name: Optional[str] = None, auto_class_weight: Optional[bool] = False, weight_min: Optional[float] = None, weight_noobj: Optional[float] = None, num_class: int = 0)¶

Calculate cross entropy loss of multi stride output.

参数

use_sigmoid (bool) – Whether logits tensor is converted to probability through sigmoid, Defaults to False. If True, use F.binary_cross_entropy_with_logits. If False, use F.cross_entropy.
reduction (str) – The method used to reduce the loss. Options are [none, mean, sum].
class_weight (list[float]) – Weight of each class. Defaults is None.
loss_weight (float) – Global weight of loss. Defaults is 1.
ignore_index (int) – Only works when using cross_entropy.
loss_name (str) – The key of loss in return dict. If None, return loss directly.

返回

cross entropy loss

forward(pred, target, weight=None, avg_factor=None)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.ElementwiseL1HingeLoss(loss_bound_l1: float = 0.0, pos_label: int = 1, neg_label: int = 0, norm_type: str = 'positive_label_elt', loss_weight: float = 1.0, reduction: Optional[str] = None, hard_neg_mining_cfg: Optional[Dict] = None)¶

Elementwise L1 Hinge Loss.

参数

loss_bound_l1 – Upper bound of l1 loss value in each entry.
pos_label – Value in label that represents positive entries.
neg_label – Value in label that represents negative entries.
norm_type – Normalization method, can be “positive_label_elt”, in which normalization factor is the number of positive elements, or “positive_loss_elt”, the number of positive losses.
loss_weight – Global weight of loss. Defaults is 1.0.
reduction – The method used to reduce the loss. Options are [none, mean, sum]. By default and recommended to be ‘mean’.
hard_neg_mining_cfg – hard negative mining config. Please refer to LossHardNegativeMining.

返回

loss value

返回类型

torch.Tensor

class hat.models.losses.ElementwiseL2HingeLoss(loss_bound_l1: float = 0.0, pos_label: int = 1, neg_label: int = 0, norm_type: str = 'positive_label_elt', loss_weight: float = 1.0, reduction: Optional[str] = None, hard_neg_mining_cfg: Optional[Dict] = None)¶

Elementwise L2 Hinge Loss.

参数

loss_bound_l1 – Upper bound of l1 loss value in each entry.
pos_label – Value in label that represents positive entries.
neg_label – Value in label that represents negative entries.
norm_type – Normalization method, can be “positive_label_elt”, in which normalization factor is the number of positive elements, or “positive_loss_elt”, the number of positive losses.
loss_weight – Global weight of loss. Defaults is 1.0.
reduction – The method used to reduce the loss. Options are [none, mean, sum]. By default and recommended to be ‘mean’.
hard_neg_mining_cfg – hard negative mining config. Please refer to LossHardNegativeMining.

返回

loss value

返回类型

torch.Tensor

class hat.models.losses.FocalLoss(loss_name, num_classes, alpha=0.25, gamma=2.0, loss_weight=1.0, eps=1e-12, reduction='mean')¶

Sigmoid focal loss.

参数

loss_name (str) – The key of loss in return dict.
num_classes (int) – Num_classes including background, C+1, C is number of foreground categories.
alpha (float) – A weighting factor for pos-sample, (1-alpha) is for neg-sample.
gamma (float) – Gamma used in focal loss to compress the contribution of easy examples.
loss_weight (float) – Global weight of loss. Defaults is 1.0.
eps (float) – A small value to avoid zero denominator.
reduction (str) – The method used to reduce the loss. Options are [none, mean, sum].

返回

A dict containing the calculated loss, the key of loss is loss_name.

返回类型

dict

forward(pred, target, weight=None, avg_factor=None, points_per_strides=None, valid_classes_list=None)¶

Forward method.

参数

pred (Tensor) – Cls pred, with shape(N, C), C is num_classes of foreground.
target (Tensor) – Cls target, with shape(N,), values in [0, C-1] represent the foreground, C or negative value represent the background.
weight (Tensor) – The weight of loss for each prediction. Default is None.
avg_factor (float) – Normalized factor.

class hat.models.losses.FocalLossV2(alpha: float = 0.25, gamma: float = 2.0, loss_weight: float = 1.0, eps: float = 1e-12, from_logits: bool = True, reduction: str = 'mean')¶

Focal Loss.

参数

alpha – A weighting factor for pos-sample, (1-alpha) is for neg-sample.
gamma – Gamma used in focal loss to compress the contribution of easy examples.
loss_weight – Global weight of loss. Defaults to 1.0.
eps – A small value to avoid zero denominator.
from_logits – Whether the input prediction is logits (before sigmoid).
reduction – The method to reduce the loss. Options are “none”, “mean” and “sum”. Defaults to “mean”.

forward(pred: torch.Tensor, target: torch.Tensor, weight: Optional[torch.Tensor] = None, avg_factor: Optional[Union[float, torch.Tensor]] = None)¶

Forward method.

参数

pred – cls pred, with shape (B, N, C), C is num_classes of foreground.
target – cls target, with shape (B, N, C), C is num_classes of foreground.
weight – The weight of loss for each prediction. It is mainly used to filter the ignored box. Default is None.
avg_factor – Normalized factor.

class hat.models.losses.GIoULoss(loss_name, loss_weight=1.0, eps=1e-06, reduction='mean')¶

Generalized Intersection over Union Loss.

参数

loss_name (str) – The key of loss in return dict.
loss_weight (float) – Global weight of loss. Defaults is 1.0.
eps (float) – A small value to avoid zero denominator.
reduction (str) – The method used to reduce the loss. Options are [none, mean, sum].

返回

A dict containing the calculated loss, the key of loss is loss_name.

返回类型

dict

forward(pred, target, weight=None, avg_factor=None)¶

Forward method.

参数

pred (torch.Tensor) – Predicted bboxes of format (x1, y1, x2, y2), represent upper-left and lower-right point, with shape(N, 4).
target (torch.Tensor) – Corresponding gt_boxes, the same shape as pred.
weight (torch.Tensor) – Element-wise weight loss weight, with shape(N,).
avg_factor (float) – Average factor that is used to average the loss.

class hat.models.losses.GaussianFocalLoss(alpha: float = 2.0, gamma: float = 4.0, loss_weight: float = 1.0)¶

Guassian focal loss.

参数

alpha – A weighting factor for positive sample.
gamma – Used in focal loss to balance contribution of easy examples and hard examples.
loss_weight – Weight factor for guassian focal loss.

forward(logits, labels, grad_tensor=None)¶

Forward function.

参数

pred (torch.Tensor) – The prediction.
target (torch.Tensor) – The learning target of the prediction in gaussian distribution.

class hat.models.losses.L1Loss(beta: float = 1.0, reduction: str = 'mean', loss_weight: Optional[float] = None)¶

Smooth L1 Loss.

参数

beta – The threshold in the piecewise function. Defaults to 1.0.
reduction – The method to reduce the loss. Options are “none”, “mean” and “sum”. Defaults to “mean”.
loss_weight – Loss weight.

forward(pred: torch.Tensor, target: torch.Tensor, weight: Optional[torch.Tensor] = None, avg_factor: Optional[Union[float, torch.Tensor]] = None)¶

Forward function.

参数

pred – The prediction.
target – The learning target of the prediction.
weight – The weight of loss for each prediction. Defaults to None.
avg_factor – Normalized factor.

class hat.models.losses.LaneFastFocalLoss(alpha: float = 2.0, gamma: float = 4.0, loss_weight: float = 1.0)¶

Modified focal loss. Exactly the same as CornerNet,: Runs faster and costs a little bit more memory, For Lane task, return effective loss when num_pos > 2.

参数

alpha – A weighting factor for positive sample.
gamma – Used in focal loss to balance contribution of easy examples and hard examples.
loss_weight – Weight factor for guassian focal loss.

class hat.models.losses.LnNormLoss(norm_order: int = 2, epsilon: float = 0.0, power: float = 1.0, reduction: Optional[str] = None, loss_weight: Optional[float] = None)¶

LnNorm loss.

Different from torch.nn.L1Loss, the loss function uses Ln norm to calculate the distance of two feature maps.

参数

norm_order – The order of norm.
epsilon – A small constant for finetune.
power – A power num of norm + epsilon of loss.
reduction – Reduction mode.
loss_weight – If present, it will be used to weight the output.

forward(pred: torch.Tensor, target: torch.Tensor, weight: Optional[torch.Tensor] = None, avg_factor: Optional[Union[float, torch.Tensor]] = None) → torch.Tensor¶

Forward method.

参数

pred (Tensor) – Optical flow pred, with shape(N, 2, H, W).
target (Tensor) – Optical flow target, with shape(N, 2, H, W),
sampling. (which obtained by ground truth) –
weight (Tensor) – The weight of loss for each prediction. Default is None.
avg_factor (float) – Normalized factor.

class hat.models.losses.MSELoss(clip_val: Optional[float] = None, reduction: Optional[str] = None, loss_weight: Optional[float] = None)¶

MSE (mean squared error) loss with clip value.

参数

clip_val – Clip value. If present, it is used to constrain the unweighted loss value between (-clip_val, clip_val). For the clipped entries, the gradient is calculated as if label value equals to predication +- clip_val.
reduction – Reduction mode.
loss_weight – If present, it will be used to weight the output.

forward(pred: torch.Tensor, target: torch.Tensor, weight: Optional[torch.Tensor] = None, valid_mask: Optional[torch.Tensor] = None, avg_factor: Optional[Union[float, torch.Tensor]] = None) → torch.Tensor¶

Mse loss between pred and target items.

参数

pred – Predict output.
target – Target ground truth.
weight – Weight of loss, shape like pred.
valid_mask – Valid mask of loss.
avg_factor – Avg factor of loss.

class hat.models.losses.MixSegLoss(losses: List[torch.nn.modules.module.Module], losses_weight: List[float] = None)¶

Calculate multi-losses with same prediction and target.

参数

losses – List of losses with the same input pred and target.
losses_weight – List of weights used for loss calculation. Default: None

forward(pred, target)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.MixSegLossMultipreds(losses: List[torch.nn.modules.module.Module], losses_weight: List[float] = None, loss_name: str = 'multipredsloss')¶

Calculate multi-losses with multi-preds and correspondence targets.

参数

losses – List of losses with different prediction and target.
losses_weight – List of weights used for loss calculation. Default: None
loss_name – Name of output loss

forward(pred, target)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.MultiStrideLosses(num_classes: int, out_strides: List[int], loss: torch.nn.modules.module.Module, loss_weights: Optional[List[float]] = None)¶

Multiple Stride Losses.

Apply the same loss function with different loss weights to multiple outputs.

参数

num_classes – Number of classes.
out_strides – strides of output feature maps
loss – Loss module.
loss_weights – Loss weight.

forward(preds: List[torch.Tensor], targets: List[torch.Tensor]) → Dict[str, torch.Tensor]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.SegEdgeLoss(edge_graph: List[List[int]], kernel_half_size: int = 2, ignore_index: int = 255, loss_name: Optional[str] = None, loss_weight: float = 1e-05)¶

forward(pred, target, weight=None, avg_factor=None)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.SegLoss(loss: List[torch.nn.modules.module.Module])¶

Segmentation loss wrapper.

参数: loss (dict) – loss config.

注解

This class is not universe. Make sure you know this class limit before using it.

forward(pred: Any, target: List[Dict]) → Dict¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.SmoothL1Loss(beta: float = 1.0, reduction: str = 'mean', loss_weight: Optional[float] = None, hard_neg_mining_cfg: Optional[Dict] = None)¶

Smooth L1 Loss.

参数

beta – The threshold in the piecewise function. Defaults to 1.0.
reduction – The method to reduce the loss. Options are “none”, “mean” and “sum”. Defaults to “mean”.
loss_weight – Loss weight.
hard_neg_mining_cfg – Hard negative mining cfg.

forward(pred: torch.Tensor, target: torch.Tensor, weight: Optional[torch.Tensor] = None, avg_factor: Optional[Union[float, torch.Tensor]] = None)¶

Forward function.

参数

pred – The prediction.
target – The learning target of the prediction.
weight – The weight of loss for each prediction. Defaults to None.
avg_factor – Normalized factor.

class hat.models.losses.SoftTargetCrossEntropy(loss_name=None)¶

The losses of cross-entropy with soft target.

参数: loss_name (str) – The name of returned losses.

forward(input, target)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.SoftmaxFocalLoss(loss_name: str, num_classes: int, alpha: float = 0.25, gamma: float = 2.0, reduction: str = 'mean', weight: Union[float, Sequence] = 1.0)¶

Focal Loss.

参数

loss_name (str) – The key of loss in return dict.
num_classes (int) – Class number.
alpha (float, optional) – Alpha. Defaults to 0.25.
gamma (float, optional) – Gamma. Defaults to 2.0.
reduction (str, optional) – Specifies the reduction to apply to the output: 'mean' | 'sum'. Defaults to 'mean'.
weight (Union[float, Sequence], optional) – Weight to be applied to the loss of each input. Defaults to 1.0.

forward(logits, labels)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.WeightedSquaredHingeLoss(reduction: str, loss_weight: float = 1.0, weight_low_thr: float = 0.1, weight_high_thr: float = 1.0, hard_neg_mining_cfg: Optional[Dict] = None)¶

Weighted Squared ElementWiseHingeLoss.

参数

reduction (str) – Possible values are {‘mean’, ‘sum’, ‘sum_mean’, ‘none’}
loss_weight (float) – by default 1.0
weight_low_thr (float) – Lower threshold for elementwise weight, by default 0.1
weight_high_thr (float) – Upper threshold for pixel-wise weight, by default 1.0
hard_neg_mining_cfg (dict) – Hard negative mining cfg

forward(pred, target, weight=None, avg_factor=None)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.necks.BiFPN(in_strides: List[int], out_strides: int, stride2channels: Dict, out_channels: Union[int, Dict], num_outs: int, stack: int = 3, start_level: int = 0, end_level: int = - 1, fpn_name: str = 'bifpn_sum', upsample_type: str = 'module', use_fx: bool = False)¶

Weighted Bi-directional Feature Pyramid Network(BiFPN).

This is an implementation of - EfficientDet: Scalable and Efficient Object Detection (https://arxiv.org/abs/1911.09070)

参数

in_strides – Stride of input feature map
out_strides – Stride of output feature map
stride2channels – The key:value is stride:channel , the channles have been multipified by alpha
out_channels – Channel number of output layer, the key:value is stride:channel.
num_outs – Number of BifpnLayer’s input, the value is must 5, because the bifpn layer is fixed
stack – Number of BifpnLayer
start_level – Index of the start input backbone level used to build the feature pyramid. Default: 0.
end_level – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, means the last level.
fpn_name – the value is mutst between with ‘bifpn_sum’, ‘bifpn_fa’.
upsample_type – use module or function unsample, the candidate is [‘module’, ‘function’].
use_fx – Whether use fx mode qat. Default: False.

forward(inputs)¶

Forward features.

参数: inputs (list[tensor]) – Input tensors

Returns (list[tensor]): Output tensors

class hat.models.necks.DwUnet(base_channels: int, bn_kwargs: Dict = None, act_type: torch.nn.modules.module.Module = <class 'torch.nn.modules.activation.ReLU'>, use_deconv: bool = False, dw_with_act: bool = False, output_scales: Sequence = (4, 8, 16, 32, 64))¶

Unet segmentation neck structure.

Built with separable convolution layers.

参数

base_channels (int) – Output channel number of the output layer of scale 1.
bn_kwargs (Dict, optional) – Keyword arguments for BN layer. Defaults to {}.
use_deconv (bool, optional) – Whether user deconv for upsampling layer. Defaults to False.
dw_with_act (bool, optional) – Whether user relu after the depthwise conv in SeparableConv. Defaults to False.
output_scales (Sequence, optional) – The scale of each output layer. Defaults to (4, 8, 16, 32, 64).

forward(inputs)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.necks.FPN(in_strides: List[int], in_channels: List[int], out_strides: List[int], out_channels: List[int], fix_out_channel: Optional[int] = None, bn_kwargs: Optional[Dict] = None)¶

forward(features: List[torch.Tensor]) → List[torch.Tensor]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.necks.FastSCNNNeck(in_channels: List[int], feat_channels: List[int], indexes: List[int], bn_kwargs: Optional[Dict] = None, scale_factor: int = 4, split_pooling: bool = False)¶

Upper neck module for segmentation.

参数

in_channels – channels of each input feature map
feat_channels – channels for featture maps.
indexes – indexes of inputs.
bn_kwargs – Dict for Bn layer.
scale_factor – scale factor for fusion.
split_pooling – Whehter split pooling. For bernoulli2.

forward(inputs)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.necks.PAFPN(in_channels, out_channels, out_strides, num_outs, start_level=0, end_level=- 1, add_extra_convs=False, relu_before_extra_convs=False)¶

Path Aggregation Network for Instance Segmentation.

This is an implementation of the PAFPN in Path Aggregation Network <https://arxiv.org/abs/1803.01534>.

参数

in_channels (List[int]) – Number of input channels per scale.
out_channels (int | Dict) – Output channels of each scale
out_strides (List[int]) – Stride of output feature map
num_outs (int) – Number of output scales.
start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.
end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, which means the last level.
add_extra_convs (bool | str) –
If bool, it decides whether to add conv layers on top of the original feature maps. Default to False. If True, it is equivalent to add_extra_convs=’on_input’. If str, it specifies the source feature map of the extra convs. Only the following options are allowed:
- ’on_input’: Last feat map of neck inputs (i.e. backbone feature).
- ’on_lateral’: Last feature map after lateral convs.
- ’on_output’: The last output feature map after fpn convs.
relu_before_extra_convs (bool) – Whether to apply relu before the extra conv. Default: False.

forward(inputs)¶: Forward function.

class hat.models.necks.RetinaNetFPN(in_strides: List[int], in_channels: List[int], out_strides: List[int], out_channels: List[int], fix_out_channel: Optional[int] = None)¶

FPN for RetinaNet.

The difference with FPN is that RetinaNetFPN has two extra convs correspond to stride 64 and stride 128 except the lateral convs.

参数

in_strides (list) – strides of each input feature map
in_channels (list) – channels of each input feature map, the length of in_channels should be equal to in_strides
out_strides (list) – strides of each output feature map, should be a subset of in_strides, and continuous (any subsequence of 2, 4, 8, 16, 32, 64 …). The largest stride in in_strides and out_strides should be equal
out_channels (list) – channels of each output feature maps the length of out_channels should be equal to out_strides
fix_out_channel (int, optional) – if set, there will be a 1x1 conv following each output feature map so that each final output has fix_out_channel channels

forward(features: List[torch.Tensor]) → List[torch.Tensor]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_weights()¶: Initialize the weights of FPN module.

class hat.models.necks.SECONDNeck(in_feature_channel: int, down_layer_nums: List[int], down_layer_strides: List[int], down_layer_channels: List[int], up_layer_strides: List[int], up_layer_channels: List[int], bn_kwargs: Optional[Dict] = None, use_relu6: bool = False, quantize: bool = False)¶

Second FPN modules.

Implements the network structure of PointPillars: <https://arxiv.org/abs/1812.05784>

Although the structure is called backbone in the original paper, we follow the publicly available code structure and use it as a neck module.

Adapted from GitHub second.pytorch: <https://github.com/traveller59/second.pytorch>

参数

in_feature_channel – number of input feature channels.
down_layer_nums – number of layers for each down-sample stage.
down_layer_strides – stride for each down-sampling stage.
down_layer_channels – number of filters for each down-sample stage.
up_layer_strides – stride for each up-sample stage.
up_layer_channels – number of filters for each up-sampling stage.
bn_kwargs – batch norm kwargs.
use_relu6 – whether to use relu6.
quantize – whether to quantize the module.

forward(x: torch.Tensor, quant: bool = False)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.necks.Unet(in_strides: List[int], out_strides: List[int], stride2channels: Dict[int, int], out_stride2channels: Dict[int, int] = None, factor: int = 2, use_bias: bool = False, bn_kwargs: Optional[Dict] = None, group_base: int = 8, fusion_block_name: str = 'default')¶

Unet neck module.

参数

in_strides – contains the strides of feature maps from backbone.
out_strides – contains the strides of feature maps the neck output.
out_stride2channels – output stride to channel dict.
stride2channels – input stride to channel dict.
fusion_block_name – support FusionBlock and OnePathFusionBlock.

forward(features)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.necks.VargPAFPN(in_channels: List[int], out_channels: int, out_strides: List[int], num_outs: int, bn_kwargs: Dict, start_level: int = 0, end_level: int = - 1, with_pafpn_conv: bool = False, varg_block_type: str = 'BasicMixVarGEBlock', group_base: int = 16)¶

Path Aggregation Network with BasicVargNetBlock or BasicMixVargNetBlock.

参数

in_channels – Number of input channels per scale.
out_channels – Output channels of each scale
out_strides – Stride of output feature map
num_outs – Number of output scales.
bn_kwargs – Dict for Bn layer.
start_level – Index of the start input backbone level used to build the feature pyramid. Default is 0.
level (end_level Index of the end input backbone) – build the feature pyramid. Default is -1, which means the last level.
with_pafpn_conv – Choice whether to use a extra 3x3 conv_block to the out features. Default is False.
varg_block_type – Choice varg block type from [“BasicVarGBlock”, “BasicMixVarGEBlock”], Default is “BasicMixVarGEBlock”.
group_base – groupbase for varg block. Default is 16.

forward(inputs)¶: Forward function.

class hat.models.structures.BMSegmentor(backbone: torch.nn.modules.module.Module, neck: torch.nn.modules.module.Module, head: torch.nn.modules.module.Module, target: Optional[torch.nn.modules.module.Module] = None, loss: Optional[torch.nn.modules.module.Module] = None, desc: Optional[torch.nn.modules.module.Module] = None, postprocess: Optional[torch.nn.modules.module.Module] = None)¶

The segmentor structure that inputs image metas into postprocess.

forward(data: dict)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.structures.Classifier(backbone, losses=None)¶

The basic structure of classifier.

参数

backbone (torch.nn.Module) – Backbone module.
losses (torch.nn.Module) – Losses module.

forward(data)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.structures.EncoderDecoder(backbone: torch.nn.modules.module.Module, decode_head: torch.nn.modules.module.Module, target: Optional[object] = None, loss: Optional[torch.nn.modules.module.Module] = None, neck: Optional[torch.nn.modules.module.Module] = None, auxiliary_heads: Optional[List[Dict]] = None, decode: Optional[object] = None, with_target: Optional[torch.nn.modules.module.Module] = False)¶

The basic structure of encoder decoder.

参数

backbone – Backbone module.
decode_head – Decode head module.
target – Target module for decode head. Default: None.
loss – Loss module for decode head. Default: None.
neck – Neck module. Default: None.
auxiliary_heads – List of auxiliary head modules which contains of “head”, “target”, “loss”. Default: None.
decode – decode. Defualt: None.
with_target – Whether return target during inference.

forward(data: dict)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.structures.Segmentor(backbone, neck, head, losses=None)¶

The basic structure of segmentor.

参数

backbone (torch.nn.Module) – Backbone module.
neck (torch.nn.Module) – Neck module.
head (torch.nn.Module) – Head module.
losses (torch.nn.Module) – Losses module.

forward(data: dict)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.structures.SegmentorV2(backbone: torch.nn.modules.module.Module, neck: torch.nn.modules.module.Module, head: torch.nn.modules.module.Module, target: Optional[torch.nn.modules.module.Module] = None, loss: Optional[torch.nn.modules.module.Module] = None, desc: Optional[torch.nn.modules.module.Module] = None, postprocess: Optional[torch.nn.modules.module.Module] = None)¶

The basic structure of segmentor.

参数

backbone – Backbone module.
neck – Neck module.
head – Head module.
loss – Loss module.
desc – Desc module
postprocess – Postprocess module.

forward(data: dict)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.structures.detectors.FCOS(backbone: torch.nn.modules.module.Module, neck: torch.nn.modules.module.Module = None, head: torch.nn.modules.module.Module = None, targets: torch.nn.modules.module.Module = None, desc: torch.nn.modules.module.Module = None, post_process: torch.nn.modules.module.Module = None, loss_cls: torch.nn.modules.module.Module = None, loss_reg: torch.nn.modules.module.Module = None, loss_centerness: torch.nn.modules.module.Module = None)¶

The basic structure of fcos.

参数

backbone – Backbone module.
neck – Neck module.
head – Head module.
targets – Target module.
loss_cls – Classification loss module.
loss_reg – Regiression loss module.
loss_centerness – Centerness loss module.
desc – Description module.
postprocess – Postprocess module.

extract_feat(img, uv_map=None)¶: Directly extract features from the backbone + neck.

forward(data: Dict)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.structures.detectors.FCOS3D(backbone: torch.nn.modules.module.Module, neck: torch.nn.modules.module.Module = None, head: torch.nn.modules.module.Module = None, targets: torch.nn.modules.module.Module = None, post_process: torch.nn.modules.module.Module = None, loss: torch.nn.modules.module.Module = None)¶

The basic structure of fcos3d.

参数

backbone – Backbone module.
neck – Neck module.
head – Head module.
targets – Target module.
post_process – post_process module.
loss – loss module.

extract_feat(img)¶: Directly extract features from the backbone + neck.

forward(data)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.structures.detectors.PointPillarsDetector(feature_map_shape: List[int], pre_process: Optional[torch.nn.modules.module.Module] = None, reader: Optional[torch.nn.modules.module.Module] = None, backbone: Optional[torch.nn.modules.module.Module] = None, neck: Optional[torch.nn.modules.module.Module] = None, head: Optional[torch.nn.modules.module.Module] = None, anchor_generator: Optional[torch.nn.modules.module.Module] = None, targets: Optional[torch.nn.modules.module.Module] = None, loss: Optional[torch.nn.modules.module.Module] = None, postprocess: Optional[torch.nn.modules.module.Module] = None, quant_begin_neck: bool = False, is_deploy: bool = False)¶

The basic structure of PointPillars.

参数

feature_map_shape – Feature map shape, in (W, H, 1) format.
out_size_factor – Downsample factor.
reader – Reader module.
backbone – Backbone module.
neck – Neck module.
head – Head module.
anchor_generator – Anchor generator module.
targets – Target generator module.
loss – Loss module.
postprocess – Postprocess module.

forward(example)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.structures.detectors.RetinaNet(backbone: torch.nn.modules.module.Module, neck: Optional[torch.nn.modules.module.Module] = None, head: Optional[torch.nn.modules.module.Module] = None, filter_module: Optional[torch.nn.modules.module.Module] = None, anchors: Optional[torch.nn.modules.module.Module] = None, targets: Optional[torch.nn.modules.module.Module] = None, post_process: Optional[torch.nn.modules.module.Module] = None, loss_cls: Optional[torch.nn.modules.module.Module] = None, loss_reg: Optional[torch.nn.modules.module.Module] = None)¶

The basic structure of retinanet.

参数

backbone – backbone module or dict for building backbone module.
neck – neck module or dict for building neck module.
head – head module or dict for building head module.
anchors – anchors module or dict for building anchors module.
targets – targets module or dict for building target module.
post_process – post_process module or dict for building post_process module.
loss_cls – loss_cls module or dict for building loss_cls module.
loss_reg – loss_reg module or dict for building loss_reg module.

forward(data: Dict)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.fcos.DynamicFcosTarget(strides: Sequence[int], topK: int, loss_cls: torch.nn.modules.module.Module, loss_reg: torch.nn.modules.module.Module, cls_out_channels: int, background_label: int, center_sampling: bool = False, center_sampling_radius: float = 2.5, bbox_relu: bool = False)¶

Generate cls and box training targets for FCOS based on simOTA label assignment strategy used in YOLO-X.

参数

strides – Strides of points in multiple feature levels.
topK – Number of positive sample for each ground truth to keep.
cls_out_channels – Out_channels of cls_score.
background_label – Label ID of background, set as num_classes.
loss_cls – Loss for cls to choose positive target.
loss_reg – Loss for reg to choose positive target.
center_sampling – Whether to perform center sampling.
center_sampling_radius – The radius of the center sampling area.
bbox_relu – Whether apply relu to bbox preds.

forward(label, pred, *args)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.fcos.FCOSConverter(task_name: str, cls_name_mapping: dict)¶

forward(preds)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.fcos.FCOSDecoder(num_classes: int, strides: Sequence[int], transforms: Optional[Sequence[dict]] = None, inverse_transform_key: Optional[Sequence[str]] = None, nms_use_centerness: bool = True, nms_sqrt: bool = True, test_cfg: Optional[dict] = None, input_resize_scale: Optional[Union[float, torch.Tensor]] = None, truncate_bbox: bool = True, filter_score_mul_centerness: bool = False, meta_data_bool: bool = True, label_offset: int = 0, upscale_bbox_pred: bool = False, bbox_relu: bool = False)¶

参数

num_classes – Number of categories excluding the background category.
strides – A list contains the strides of fcos_head output.
transforms – A list contains the transform config.
inverse_transform_key – A list contains the inverse transform info key.
nms_use_centerness – If True, use centerness as a factor in nms post-processing.
nms_sqrt – If True, sqrt(score_thr * score_factors).
test_cfg – Cfg dict, including some configurations of nms.
input_resize_scale – The scale to resize bbox.
truncate_bbox – If True, truncate the predictive bbox out of image boundary. Default True.
filter_score_mul_centerness – If True, filter out bbox by score multiply centerness, else filter out bbox by score. Default False.
meta_data_bool – Whether get shape info from meta data.
label_offset – label offset.
upscale_bbox_pred – Whether upscale bbox preds.
bbox_relu – Whether apply relu to bbox preds.

forward(pred: Sequence[torch.Tensor], meta_data: Dict[str, Any])¶

Do post process for model predictions.

参数

pred – Prediction tensors.
meta_data – Meta data used in post processor, e.g. image width, height.

class hat.models.task_modules.fcos.FCOSDecoder4RCNN(num_classes: int, strides: Sequence[int], input_shape: Tuple[int], nms_use_centerness: bool = True, nms_sqrt: bool = True, test_cfg: Optional[Dict] = None, input_resize_scale: Optional[Union[float, torch.Tensor]] = None)¶

Decoder for FCOS+RCNN Architecture.

参数

num_classes – Number of categories excluding the background category.
strides – A list contains the strides of fcos_head output.
input_shape – The shape of input_image.
nms_use_centerness – If True, use centerness as a factor in nms post-processing.
nms_sqrt – If True, sqrt(score_thr * score_factors).
rescale – Whether to map the prediction result to the orig img.
test_cfg – Cfg dict, including some configurations of nms.
input_resize_scale – The scale to resize bbox.

forward(pred: collections.OrderedDict)¶

Do post process for model predictions.

参数

pred – Prediction tensors.
meta_data – Meta data used in post processor, e.g. image width, height.

class hat.models.task_modules.fcos.FCOSHead(num_classes: int, in_strides: Sequence[int], out_strides: Sequence[int], stride2channels: dict, upscale_bbox_pred: bool, feat_channels: int = 256, stacked_convs: int = 4, use_sigmoid: bool = True, share_bn: bool = False, dequant_output: bool = True, int8_output: bool = True, int16_output=False, nhwc_output=False, share_conv: bool = True, bbox_relu=True, use_plain_conv: bool = False, use_gn: bool = False, use_scale: bool = False, add_stride: bool = False, output_dict: bool = False, set_all_int16_qconfig=False)¶

Anchor-free head used in FCOS <https://arxiv.org/abs/1904.01355>.

参数

num_classes – Number of categories excluding the background category.
in_strides – A list contains the strides of feature maps from backbone or neck.
out_strides – A list contains the strides of this head will output.
stride2channels – A stride to channel dict.
upscale_bbox_pred – If true, upscale bbox pred by FPN strides.
feat_channels – Number of hidden channels.
stacked_convs – Number of stacking convs of the head.
use_sigmoid – Whether the classification output is obtained using sigmoid.
share_bn – Whether to share bn between multiple levels, default is share_bn.
dequant_output – Whether to dequant output. Default: True
int8_output – If True, output int8, otherwise output int32. Default: True.
int16_output – If True, output int16, otherwise output int32. Default: False.
nhwc_output – transpose output layout to nhwc.
share_conv – Only the number of all stride channels is the same, share_conv can be True, branches share conv, otherwise not. Default: True.
bbox_relu – Whether use relu for bbox. Default: True.
use_plain_conv – If True, use plain conv rather than depth-wise conv in some conv layers. This argument works when share_conv=True. Default: False.
use_gn – If True, use group normalization instead of batch normalization in some conv layers. This argument works when share_conv=True. Default: False.
use_scale – If True, add a scale layer to scale the predictions like what original FCOS does. This argument works when share_conv=True. Default: False.
add_stride – If True, add extra out_strides. Sometimes the out_strides is not a subset of in_strides, for example, the in_strides is [4, 8, 16, 32, 64] but the out_strides is [8, 16, 32, 64, 128], then we need to add an extra stride 128 in this head. This argument works when share_conv=True. Default: False.
output_dict – If True, forward(self) will output a dict.

forward(feats)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

forward_single(x, i, stride)¶

Forward features of a single scale level.

参数

x (Tensor) – FPN feature maps of the specified stride.
i (int) – Index of feature level.
stride (int) – The corresponding stride for feature maps, only used to upscale bbox pred when self.upscale_bbox_pred is True.

class hat.models.task_modules.fcos.FCOSLoss(cls_loss: torch.nn.modules.module.Module, reg_loss: torch.nn.modules.module.Module, centerness_loss: Optional[torch.nn.modules.module.Module] = None)¶

FCOS loss wrapper.

参数: losses (list) – loss configs.

注解

This class is not universe. Make sure you know this class limit before using it.

forward(pred: Tuple, target: Tuple[Dict]) → Dict¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.fcos.FCOSMultiStrideCatFilter(strides: Sequence[int], threshold: float, task_strides: Sequence[Sequence[int]], idx_range: Optional[Tuple[int, int]] = None)¶

A modified Filter used for post-processing of FCOS.

In each stride, concatenate the scores of each task as the first input of FilterModule, which can reduce latency in BPU.

参数

strides (Sequence[int]) – A list contains the strides of feature maps.
idx_range (Optional[Tuple[int, int]], optional) – The index range of values counted in compare of the first input. Defaults to None which means use all the values.
threshold (float) – The lower bound of output.
task_strides (Sequence[Sequence[int]]) – A list of out_stirdes of each task.

forward(preds: Sequence[torch.Tensor], **kwargs) → Sequence[torch.Tensor]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.fcos.FCOSMultiStrideFilter(strides: Sequence[int], threshold: float, idx_range: Optional[Tuple[int, int]] = None, for_compile: bool = False)¶

Filter used for post-processing of FCOS.

参数

strides – A list contains the strides of feature maps.
idx_range – The index range of values counted in compare of the first input. Defaults to None which means use all the values.
threshold – The lower bound of output.
for_compile – Whether used for compile. if true, should not include postprocess.

forward(preds: Sequence[torch.Tensor], meta_and_label: Optional[Dict] = None, **kwargs) → Sequence[torch.Tensor]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.fcos.FCOSTarget(strides: Tuple[int, ...], regress_ranges: Tuple[Tuple[int, int], ...], cls_out_channels: int, background_label: int, norm_on_bbox: bool = True, center_sampling: bool = True, center_sample_radius: float = 1.5, use_iou_replace_ctrness: bool = False, task_batch_list: Optional[List[int]] = None)¶

Generate cls and reg targets for FCOS in training stage.

参数

strides – Strides of points in multiple feature levels.
regress_ranges – Regress range of multiple level points.
cls_out_channels – Out_channels of cls_score.
background_label – Label ID of background, set as num_classes.
center_sampling – If true, use center sampling.
center_sample_radius – Radius of center sampling. Default: 1.5.
norm_on_bbox – If true, normalize the regression targets with FPN strides.
use_iou_replace_ctrness – If true, use iou as box quality assessment method, else use ctrness. Default: false.
task_batch_list – Mask for different label source dataset.

forward(label, pred, *args)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

hat.models.task_modules.fcos.distance2bbox(points: torch.Tensor, distance: torch.Tensor, max_shape: Optional[Tuple] = None) → torch.Tensor¶

Decode distance prediction to bounding box.

参数

points – Shape (n, 2), [x, y].
distance – Distance from the given point to 4 boundaries (left, top, right, bottom).
max_shape – Shape of the image, used to clamp decoded bbox in max_shape range.

返回

Decoded bbox, with shape (n, 4).

hat.models.task_modules.fcos.get_points(feat_sizes: List[Tuple], strides: List[int], dtype: torch.dtype, device: torch.device, flatten: bool = False) → List[torch.Tensor]¶

Generate points according to feat_sizes.

参数

feat_sizes – Multi-level feature map sizes, the value is the HW of a certain layer.
strides – Multi-level feature map strides.
dtype – Type of points should be.
device – Device of points should be.
flatten – Whether to flatten 2D coordinates into 1D dimension.

返回

Points of multiple levels belong to each image,: the value in mlvl_points is [Tensor(H1W1, 2), Tensor(H2W2, 2), …]

hat.models.task_modules.fcos.multiclass_nms(multi_bboxes, multi_scores, score_thr, nms, iou_threshold, max_per_img=- 1, score_factors=None, nms_sqrt=False, filter_score_mul_centerness=False, label_offset=0)¶

NMS for multi-class bboxes.

参数

multi_bboxes (Tensor) – shape (n, #class*4) or (n, 4)
multi_scores (Tensor) – shape (n, #class), where the last column contains scores of the background class, but this will be ignored.
score_thr (float) – bbox threshold, bboxes with scores lower than it will not be considered.
nms (str) – nms type, candidate values are [‘nms’, ‘soft_nms’].
iou_threshold (float) – NMS IoU threshold
max_per_img (int) – if there are more than max_num bboxes after NMS, only top max_num will be kept.
score_factors (Tensor) – The factors multiplied to scores before applying NMS
nms_sqrt (bool) – If True, sqrt(score_thr * score_factors)

class hat.models.task_modules.lidar.Anchor3DGeneratorStride(class_names: List[str], anchor_sizes: List[List[float]], anchor_strides: List[List[float]], anchor_offsets: List[List[float]], rotations: List[List[float]], match_thresholds: List[float], unmatch_thresholds: List[float], dtype: Any = torch.float32)¶

Lidar 3D Anchor Generator by stride.

参数

anchor_sizes – 3D sizes of anchors.
anchor_strides – Strides of anchors.
anchor_offsets – Offsets of anchors.
rotations – Rotations of anchors in a feature grid.
class_names – Class names of data.
match_thresholds – Match thresholds of IoU.
unmatch_thresholds – Unmatch thresholds of IoU.

forward(feature_map_size, device)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.lidar.GroundBox3dCoder(linear_dim: bool = False, vec_encode: bool = False, n_dim: int = 7, norm_velo: bool = False)¶

Box3d Coder for Lidar.

参数

linear_dim – Whether to smooth dimension. Defaults to False.
vec_encode – Whether encode angle to vector. Defaults to False.
n_dim – dims of bbox3d. Defaults to 7.
norm_velo – Whether to normalize. Defaults to False.

decode(box_encodings, anchors)¶

Box decode for lidar bbox.

参数

boxes – normal boxes, shape [N, 7]: (x, y, z, w, l, h, r)
anchors – anchors, shape [N, 7]: (x, y, z, w, l, h, r)

encode(boxes: torch.Tensor, anchors: torch.Tensor)¶

Box encode for Lidar boxes.

参数

boxes – normal boxes, shape [N, 7]: x, y, z, l, w, h, r
anchors – anchors, shape [N, 7]: x, y, z, l, w, h, r

class hat.models.task_modules.lidar.LidarTargetAssigner(box_coder: hat.models.task_modules.lidar.box_coders.GroundBox3dCoder, class_names: List[str], positive_fraction: int = None, sample_size: int = 512)¶

TargetAssigner for Lidar.

参数

box_coder – BoxCoder.
class_names – Class names.
positive_fraction – Positive fraction.
sample_size – Sample size.

create_targets_single(all_anchors, gt_boxes, similarity_fn, box_encoding_fn, gt_classes=None, matched_threshold=0.6, unmatched_threshold=0.45, positive_fraction=None, sample_size=300, norm_by_num_examples=False, box_code_size=7)¶

Create targets.

参数

all_anchors – [num_of_anchors, box_ndim] float tensor.
gt_boxes – [num_gt_boxes, box_ndim] float tensor.
similarity_fn – a function, accept anchors and gt_boxes, return similarity matrix(such as IoU).
box_encoding_fn – a function, accept gt_boxes and anchors, return box encodings(offsets).
prune_anchor_fn – a function, accept anchors, return indices that indicate valid anchors.
gt_classes – [num_gt_boxes] int tensor. indicate gt classes, must start with 1.
matched_threshold – float, iou greater than matched_threshold will be treated as positives.
unmatched_threshold – float, iou smaller than unmatched_threshold will be treated as negatives.
positive_fraction – [0-1] float or None. if not None, we will try to keep ratio of pos/neg equal to positive_fraction when sample. if there is not enough positives, it fills the rest with negatives.
rpn_batch_size – int. sample size.
norm_by_num_examples – bool. norm box_weight by number of examples.

返回

targets params(labels, bbox_targets, bbox_outside_weights).

返回类型

dict

forward(anchors_list, matched_thresholds, unmatched_thresholds, annos, device=None)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

nearest_iou_similarity(boxes1, boxes2)¶

Compute matrix of (negated) sq distances.

参数

boxlist1 – BoxList holding N boxes.
boxlist2 – BoxList holding M boxes.

返回

A tensor with shape [N, M] representing negated pairwise squared distance.

class hat.models.task_modules.lidar.PillarFeatureNet(num_input_features: int, num_filters: Tuple[int, ...] = (64,), with_distance: bool = False, voxel_size: Tuple[float, float, int] = (0.2, 0.2, 4), pc_range: Tuple[float, ...] = (0.0, - 40.0, - 3.0, 70.4, 40.0, 1.0), bn_kwargs: dict = None, quantize: bool = False, use_4dim: bool = False, use_conv: bool = False, pool_size: Tuple[int, int] = (1, 1), normalize_xyz: bool = False, hw_reverse: bool = False)¶

forward(features: torch.Tensor, num_voxels: torch.Tensor, coors: torch.Tensor, horizon_preprocess: bool = False)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.lidar.PointPillarScatter(num_input_features: int, use_horizon_pillar_scatter: bool = False, quantize=False, **kwargs)¶

forward(voxel_features: torch.Tensor, coords: torch.Tensor, batch_size: int, input_shape: torch.Tensor)¶

Forward pass of the scatter module.

Note: batch_size has to be passed in additionally, because voxel features are concatenated on the M-channel since the number of voxels in each frame differs and there is no easy way we concat them same as image (CHW -> NCHW). M-channel concatenation would require another tensor to record number of voxels per frame, which indicates batch_size consequently.

参数

voxel_features (torch.Tensor) – MxC tensor of pillar features, where M is number of pillars, C is each pillar’s feature dim.
coords (torch.Tensor) – each pillar’s original BEV coordinate.
batch_size (int) – batch size of the feature.
input_shape (torch.Tensor) – shape of the expected BEC map. Derived from point-cloud range and voxel size.

返回

a BEV view feature tensor with point features: scattered on it.

返回类型

[torch.Tensor]

class hat.models.task_modules.retinanet.RetinaNetHead(num_classes: int, num_anchors: int, in_channels: int, feat_channels: int, stacked_convs: int = 4, int16_output: bool = True, dequant_output: bool = True)¶

An anchor-based head used in RetinaNet.

The head contains two subnetworks. The first classifies anchor boxes and the second regresses deltas for the anchors.

参数

num_classes (int) – Number of categories excluding the background category.
num_anchors (int) – Number of anchors for each pixel.
in_channels (int) – Number of channels in the input feature map.
feat_channels (int) – Number of hidden channels.
stacked_convs (int) – Number of convs before cls and reg.
int16_output (bool) – If True, output int16, otherwise output int32. Default: True

forward(features)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

forward_single(x)¶

Forward feature of a single scale level.

参数

x (Tensor) – Feature of a single scale level.

返回

cls_score (Tensor): Cls scores for a single scale level: the channels number is num_anchors * num_classes.
bbox_pred (Tensor): Box energies / deltas for a single scale: level, the channels number is num_anchors * 4.

返回类型

tuple

init_weights()¶: Initialize weights of the head.

class hat.models.task_modules.retinanet.RetinaNetPostProcess(score_thresh: float, nms_thresh: float, detections_per_img: int, topk_candidates: int = 1000)¶

The postprocess of RetinaNet.

参数

score_thresh (float) – Filter boxes whose score is lower than this.
nms_thresh (float) – thresh for nms.
detections_per_img (int) – Get top n boxes by score after nms.
topk_candidates (int) – Get top n boxes by score after decode.

forward(boxes, preds, image_shapes)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.retinanet.RetinanetMultiStrideFilter(strides: Sequence[int], threshold: float)¶

forward(cls_scores, bbox_preds)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.seg.FRCNNSegHead(group_base: int, in_strides: List, in_channels: List, out_strides: List, out_channels: List, bn_kwargs: Dict, proj_channel_multiplier: float = 1.0, with_extra_conv: bool = False, use_bias: bool = True, linear_out: bool = True, argmax_output: bool = False, with_score: bool = False, rle_label: bool = False, dequant_output: bool = True, int8_output: bool = False, no_upscale_infer: bool = False)¶

FRCNNSegHead module for segmentation task.

参数

group_base – Group base of group conv
in_strides – The strides corresponding to the inputs of seg_head, the inputs usually come from backbone or neck.
in_channels – Number of channels of each input stride.
out_strides – List of output strides.
out_channels – Number of channels of each output stride.
bn_kwargs – Extra keyword arguments for bn layers.
proj_channel_multiplier – Multiplier of channels of pw conv in block.
with_extra_conv – Whether to use extra conv module.
use_bias – Whether to use bias in conv module.
linear_out – Whether NOT to use to act of pw.
argmax_output – Whether conduct argmax on output.
with_score – Whether to keep score in argmax operation.
rle_label – Whether to calculate rle representation of label output.
dequant_output – Whether to dequant output.
int8_output – If True, output int8, otherwise output int32.
no_upscale_infer – Load params from x2 scale if True.

forward(x: List[torch.Tensor]) → List[torch.Tensor]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.seg.SegDecoder(out_strides: List[int], decode_strides: List[int], upscale_times: Optional[List[int]] = None, transforms: Optional[List[dict]] = None, inverse_transform_key: Optional[List[str]] = None, output_names: Optional[str] = 'pred_seg')¶

Semantic Segmentation Decoder.

参数

out_strides – List of output strides, represents the strides of the output from seg_head.
output_names – Keys of returned results dict.
decode_strides – Strides that need to be decoded, should be a subset of out_strides.
upscale_times – Bilinear upscale times for each decode stride, default to None, which means same as decode stride.
transforms – A list contains the transform config.
inverse_transform_key – A list contains the inverse transform info key.

class hat.models.task_modules.seg.SegHead(num_classes, in_strides, out_strides, stride2channels, feat_channels=256, stride_loss_weights=None, stacked_convs=1, argmax_output=False, dequant_output=True, int8_output=True, upscale=False, upscale_stride=4, output_with_bn=False, bn_kwargs=None, upsample_output_scale=None, output_conf=False)¶

Head Module for segmentation task.

参数

num_classes (int) – Number of classes.
in_strides (list[int]) – The strides corresponding to the inputs of seg_head, the inputs usually come from backbone or neck.
out_strides (list[int]) – List of output strides.
stride2channels (dict) – A stride to channel dict.
feat_channels (int or list[int]) – Number of hidden channels (of each output stride).
stride_loss_weights (list[int]) – loss weight of each stride.
stacked_convs (int) – Number of stacking convs of head.
argmax_output (bool) – Whether conduct argmax on output. Default: False
dequant_output (bool) – Whether to dequant output. Default: True
int8_output (bool) – If True, output int8, otherwise output int32. Default: True
upscale (bool) – If True, stride{x}’s feature map is upsampled by 2x, then the upsampled feature is adding supervisory signal. Default is False.
upscale_stride (int) – Specify which stride’s feature need to be upsampled when upscale is True.
output_with_bn (bool) – Whether add bn layer to the output conv.
bn_kwargs (dict) – Extra keyword arguments for bn layers.
upsample_output_scale (int) – Output upsample scale, only used in qat model, default is None.

forward(feats)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

forward_single(x, stride_index=0)¶

Forward features of a single scale level.

参数

x (Tensor) – feature maps of the specified stride.
stride_index (int) – stride index of input feature map.

返回

seg predictions of input feature maps.

返回类型

tuple

class hat.models.task_modules.seg.SegTarget(ignore_index=255, label_name='gt_seg')¶

Generate training targets for Seg task.

参数

ignore_index (int, optional) – Index of ignore class.
label_name (str, optional) – The key corresponding to the gt seg in label.

class hat.models.task_modules.seg.VargNetSegDecoder(out_strides: List[int], input_padding: Sequence[int] = (0, 0, 0, 0))¶

Semantic Segmentation Decoder.

参数

out_strides (list[int]) – List of output strides, represents the strides of the output from seg_head.
output_names (str or list[str]) – Keys of returned results dict.
decode_strides (int or list[int]) – Strides that need to be decoded, should be a subset of out_strides.
transforms (Sequence[dict]) – A list contains the transform config.
inverse_transform_key (Sequence[str]) – A list contains the inverse transform info key.

forward(pred: Sequence[torch.Tensor])¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.deeplab.Deeplabv3plusHead(in_channels: int, c1_index: int, c1_in_channels: int, feat_channels: int, num_classes: int, dilations: List[int], num_repeats: List[int], argmax_output: Optional[bool] = False, dequant_output: Optional[bool] = True, int8_output: Optional[bool] = True, bn_kwargs: Optional[Dict] = None, dropout_ratio: Optional[float] = 0.1, upsample_output_scale: Optional[int] = None, upsample_decode_scale: Optional[int] = 4, bias=True)¶

Head Module for FCN.

参数

in_channels – Input channels.
c1_index – Index for c1 input.
c1_in_channels – In channels of c1.
feat_channels – Channels for the module.
num_classes – Number of classes.
dilations – List of dilations for aspp.
num_repeats – List of repeat for each branch of ASPP.
argmax_output – Whether conduct argmax on output. Default: False.
dequant_output – Whether to dequant output. Default: True
int8_output – If True, output int8, otherwise output int32. Default: False.
bn_kwargs – Extra keyword arguments for bn layers. Default: None.
dropout_ratio – Ratio for dropout during training. Default: 0.1.
upsample_decode_scale – upsample scale to c1. Default is 4.
upsample_output_scale – Output upsample scale, only used in qat model, default is None.
bias – Whether has bias. Default: True.

forward(inputs)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.fcn.DepthwiseSeparableFCNHead(in_channels, feat_channels, num_convs=1, **kwargs)¶

class hat.models.task_modules.fcn.FCNDecoder(upsample_output_scale: int = 8, use_bce: bool = False, bg_cls: int = 0, bg_threshold: float = 0.25)¶

FCN Decoder.

参数

upsample_output_scale – Output upsample scale. Default: 8.
use_bce – Whether using binary crosse entrypy. Default: False.
bg_cls – Background classes id. Default: 0.
bg_threshold – Background threshold. Default: 0.25.

forward(pred)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.fcn.FCNHead(input_index: int, in_channels: int, feat_channels: int, num_classes: int, dropout_ratio: Optional[float] = 0.1, int8_output: Optional[bool] = False, argmax_output: Optional[bool] = False, dequant_output: Optional[bool] = True, upsample_output_scale: Optional[int] = None, num_convs: Optional[int] = 2, bn_kwargs: Optional[Dict] = None)¶

Head Module for FCN.

参数

input_index – Index of inputs.
in_channels – Input channels.
feat_channels – Channels for the module.
num_classes – Number of classes.
dropout_ratio – Ratio for dropout during training. Default: 0.1.
int8_output – If True, output int8, otherwise output int32. Default: False.
argmax_output – Whether conduct argmax on output. Default: False.
dequant_output – Whether to dequant output. Default: True.
upsample_output_scale – Output upsample scale. Default: None.
num_convs – number of convs in head. Default: 2.
bn_kwargs – Extra keyword arguments for bn layers. Default: None.

forward(inputs: List[torch.Tensor])¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.fcn.FCNTarget(num_classes: Optional[int] = 19)¶

Generate Target for FCN.

参数: num_classes – Number of classes. Defualt: 19.

forward(label: torch.Tensor, pred: torch.Tensor) → dict¶

参数

label – data Tenser.(n, h, w)
pred – Output Tenser. (n, c, h, w).

返回

Loss inputs.

返回类型

dict

class hat.models.task_modules.pointpillars.BatchVoxelization(pc_range: List[float], voxel_size: List[float], max_voxels_num: Union[tuple, int] = 20000, max_points_in_voxel: int = 30)¶

Batch voxelization.

参数

pc_range – Point cloud range.
voxel_size – voxel size, (x, y, z) scale.
max_voxels_num – Max voxel number to use. Defaults to 20000.
max_points_in_voxel – Number of points in per voxel. Defaults to 30.

forward(points_lst: List[torch.Tensor], is_deploy=False)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.pointpillars.PointPillarsHead(in_channels: int = 128, num_classes: int = 1, anchors_num_per_class: int = 2, use_direction_classifier: bool = True, num_direction_bins: int = 2, box_code_size: int = 7)¶

Basic module of PointPillarsHead.

参数

in_channels – Channel number of input feature.
num_classes – Number of class.
anchors_num_per_class – Anchor number for per class.
use_direction_classifier – Whether to use direction.
num_direction_bin – Number of direction bins.
box_code_size – BoxCoder size.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.pointpillars.PointPillarsLoss(num_classes: int, loss_cls: Optional[torch.nn.modules.module.Module] = None, loss_bbox: Optional[torch.nn.modules.module.Module] = None, loss_dir: Optional[torch.nn.modules.module.Module] = None, pos_cls_weight: float = 1.0, neg_cls_weight: float = 1.0, num_direction_bins: int = 2, direction_offset: float = 0.0)¶

PointPillars Loss Module.

参数

num_classes – Number of classes
loss_cls – Classification loss module.
loss_bbox – Bbox regression loss module.
loss_dir – Direction loss module.
pos_cls_weight – Positive weight. Defaults to 1.0.
neg_cls_weight – Negative weight. Defaults to 1.0.
num_direction_bins – Number of direction. Defaults to 2.
direction_offset – Direction offset. Defaults to 0.0.

forward(anchors, box_cls_labels, reg_targets, box_preds, cls_preds, dir_preds)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.pointpillars.PointPillarsPostProcess(num_classes: int, box_coder: int, use_direction_classifier: bool = True, num_direction_bins: int = 2, direction_offset: float = 0.0, use_rotate_nms: bool = False, nms_pre_max_size: int = 1000, nms_post_max_size: int = 300, nms_iou_threshold: float = 0.5, score_threshold: float = 0.05, post_center_limit_range: List[float] = [0, - 39.68, - 5, 69.12, 39.68, 5], max_per_img: int = 100)¶

PointPillars PostProcess Module.

参数

num_classes – Number of classes.
box_coder – BoxCeder module.
use_direction_classifier – Whether to use direction.
num_direction_bins – Number of direction for per anchor. Defaults to 2.
direction_offset – Direction offset. Defaults to 0.0.
use_rotate_nms – Whether to use rotated nms.
nms_pre_max_size – Max size of nms preprocess.
nms_post_max_size – Max size of nms postprocess.
nms_iou_threshold – IoU threshold of nms.
score_threshold – Score threshold.
post_center_limit_range – PointCloud range.
max_per_img – Max number of object per image.

forward(box_preds: torch.Tensor, cls_preds: torch.Tensor, dir_preds: torch.Tensor, anchors: torch.Tensor)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

nms(boxes: torch.Tensor, scores: torch.Tensor, iou_threshold: float, pre_max_size: Optional[int] = None, post_max_size: Optional[int] = None)¶

NMS.

参数

boxes – Shape(N, 4), boxes in (x1, y1, x2, y2) format.
scores – Shape(N), scores.
iou_threshold – IoU threshold.
pre_nms_top_n – Get top n boxes by score before nms.
output_num – Get top n boxes by score after nms.

返回

Indices.

class hat.models.task_modules.pointpillars.PointPillarsPreProcess(pc_range: List[float], voxel_size: List[float], max_voxels_num: int = 20000, max_points_in_voxel: int = 30, norm_range: Optional[List] = None, norm_dims: Optional[List] = None)¶

Point Pillars preprocess, include voxelization and extend features.

参数

pc_range – Point cloud range.
voxel_size – voxel size, (x, y, z) scale.
max_voxels_num – Max voxel number to use. Defaults to 20000.
max_points_in_voxel – Number of points in per voxel. Defaults to 30.
norm_range – Feature range, like [x_min, y_min, z_min, …, x_max, y_max, z_max, …].
norm_dims – Dims to do normalize.

forward(points_lst, is_deploy=False)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.