maix.nn
maix.nn module
You can use
maix.nnto access this module with MaixPy
This module is generated from MaixPy and MaixCDK
Module
| module | brief |
|---|---|
| F | maix.nn.F module |
Enum
Variable
Function
Class
Retinaface
Retinaface class
C++ defination code:
class Retinaface
__init__
def __init__(self, model: str = '', dual_buff: bool = True) -> None
Constructor of Retinaface class
| item | description |
|---|---|
| type | func |
| param | model: model path, default empty, you can load model later by load function. dual_buff: direction [in], prepare dual input output buffer to accelarate forward, that is, when NPU is forwarding we not wait and prepare the next input buff. If you want to ensure every time forward output the input's result, set this arg to false please. Default true to ensure speed. |
| throw | If model arg is not empty and load failed, will throw err::Exception. |
| static | False |
C++ defination code:
Retinaface(const string &model = "", bool dual_buff = true)
load
def load(self, model: str) -> maix.err.Err
Load model from file
| item | description |
|---|---|
| type | func |
| param | model: Model path want to load |
| return | err::Err |
| static | False |
C++ defination code:
err::Err load(const string &model)
detect
def detect(self, img: maix.image.Image, conf_th: float = 0.4, iou_th: float = 0.45, fit: maix.image.Fit = ...) -> list[...]
Detect objects from image
| item | description |
|---|---|
| type | func |
| param | img: Image want to detect, if image's size not match model input's, will auto resize with fit method. conf_th: Confidence threshold, default 0.4. iou_th: IoU threshold, default 0.45. fit: Resize method, default image.Fit.FIT_CONTAIN. |
| throw | If image format not match model input format, will throw err::Exception. |
| return | Object list. In C++, you should delete it after use. |
| static | False |
C++ defination code:
std::vector<nn::Object> *detect(image::Image &img, float conf_th = 0.4, float iou_th = 0.45, maix::image::Fit fit = maix::image::FIT_CONTAIN)
input_size
def input_size(self) -> maix.image.Size
Get model input size
| item | description |
|---|---|
| type | func |
| return | model input size |
| static | False |
C++ defination code:
image::Size input_size()
input_width
def input_width(self) -> int
Get model input width
| item | description |
|---|---|
| type | func |
| return | model input size of width |
| static | False |
C++ defination code:
int input_width()
input_height
def input_height(self) -> int
Get model input height
| item | description |
|---|---|
| type | func |
| return | model input size of height |
| static | False |
C++ defination code:
int input_height()
input_format
def input_format(self) -> maix.image.Format
Get input image format
| item | description |
|---|---|
| type | func |
| return | input image format, image::Format type. |
| static | False |
C++ defination code:
image::Format input_format()
mean
Get mean value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> mean
scale
Get scale value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> scale
MUD
MUD(model universal describe file) class
C++ defination code:
class MUD
__init__
def __init__(self, model_path: str = None) -> None
MUD constructor
| item | description |
|---|---|
| type | func |
| param | model_path: direction [in], model file path, model format can be MUD(model universal describe file) file. If model_path set, will load model from file, load failed will raise err.Exception. If model_path not set, you can load model later by load function. |
| static | False |
C++ defination code:
MUD(const char *model_path = nullptr)
load
def load(self, model_path: str) -> maix.err.Err
Load model from file
| item | description |
|---|---|
| type | func |
| param | model_path: direction [in], model file path, model format can be MUD(model universal describe file) file. |
| return | error code, if load success, return err::ERR_NONE |
| static | False |
C++ defination code:
err::Err load(const std::string &model_path)
type
Model type, string type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::string type
items
Model config items, different model type has different config items
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::map<std::string, std::map<std::string, std::string>> items
LayerInfo
NN model layer info
C++ defination code:
class LayerInfo
__init__
def __init__(self, name: str = '', dtype: maix.tensor.DType = ..., shape: list[int] = []) -> None
LayerInfo constructor
| item | description |
|---|---|
| type | func |
| param | name: direction [in], layer name dtype: direction [in], layer data type shape: direction [in], layer shape |
| static | False |
C++ defination code:
LayerInfo(const std::string &name = "", tensor::DType dtype = tensor::DType::FLOAT32, std::vector<int> shape = std::vector<int>())
name
Layer name
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::string name
dtype
Layer data type
| item | description |
|---|---|
| type | var |
| attention | If model is quantized, this is the real quantized data type like int8 float16, in most scene, inputs and outputs we actually use float32 in API like forward. |
| static | False |
| readonly | False |
C++ defination code:
tensor::DType dtype
shape
Layer shape
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<int> shape
shape_int
def shape_int(self) -> int
Shape as one int type, multiply all dims of shape
| item | description |
|---|---|
| type | func |
| static | False |
C++ defination code:
int shape_int()
to_str
def to_str(self) -> str
To string
| item | description |
|---|---|
| type | func |
| static | False |
C++ defination code:
std::string to_str()
__str__
def __str__(self) -> str
To string
| item | description |
|---|---|
| type | func |
| static | False |
C++ defination code:
std::string __str__()
NN
Neural network class
C++ defination code:
class NN
__init__
def __init__(self, model: str = '', dual_buff: bool = True) -> None
Neural network constructor
| item | description |
|---|---|
| type | func |
| param | model: direction [in], model file path, model format can be MUD(model universal describe file) file. If model_path set, will load model from file, load failed will raise err.Exception. If model_path not set, you can load model later by load function. dual_buff: direction [in], prepare dual input output buffer to accelarate forward, that is, when NPU is forwarding we not wait and prepare the next input buff. If you want to ensure every time forward output the input's result, set this arg to false please. Default true to ensure speed. |
| static | False |
C++ defination code:
NN(const std::string &model = "", bool dual_buff = true)
load
def load(self, model: str) -> maix.err.Err
Load model from file
| item | description |
|---|---|
| type | func |
| param | model: direction [in], model file path, model format can be MUD(model universal describe file) file. |
| return | error code, if load success, return err::ERR_NONE |
| static | False |
C++ defination code:
err::Err load(const std::string &model)
loaded
def loaded(self) -> bool
Is model loaded
| item | description |
|---|---|
| type | func |
| return | true if model loaded, else false |
| static | False |
C++ defination code:
bool loaded()
set_dual_buff
def set_dual_buff(self, enable: bool) -> None
Enable dual buff or disable dual buff
| item | description |
|---|---|
| type | func |
| param | enable: true to enable, false to disable |
| static | False |
C++ defination code:
void set_dual_buff(bool enable)
inputs_info
def inputs_info(self) -> list[LayerInfo]
Get model input layer info
| item | description |
|---|---|
| type | func |
| return | input layer info |
| static | False |
C++ defination code:
std::vector<nn::LayerInfo> inputs_info()
outputs_info
def outputs_info(self) -> list[LayerInfo]
Get model output layer info
| item | description |
|---|---|
| type | func |
| return | output layer info |
| static | False |
C++ defination code:
std::vector<nn::LayerInfo> outputs_info()
extra_info
def extra_info(self) -> dict[str, str]
Get model extra info define in MUD file
| item | description |
|---|---|
| type | func |
| return | extra info, dict type, key-value object, attention: key and value are all string type. |
| static | False |
C++ defination code:
std::map<std::string, std::string> extra_info()
forward
def forward(self, inputs: maix.tensor.Tensors, copy_result: bool = True, dual_buff_wait: bool = False) -> maix.tensor.Tensors
forward run model, get output of model,\nthis is specially for MaixPy, not efficient, but easy to use in MaixPy
| item | description |
|---|---|
| type | func |
| param | input: direction [in], input tensor copy_result: If set true, will copy result to a new variable; else will use a internal memory, you can only use it until to the next forward. Default true to avoid problems, you can set it to false manually to make speed faster. dual_buff_wait: bool type, only for dual_buff mode, if true, will inference this image and wait for result, default false. |
| return | output tensor. In C++, you should manually delete tensors in return value and return value. If dual_buff mode, it can be NULL(None in MaixPy) means not ready. |
| throw | if error ocurrs like no memory or arg error, will raise err.Exception. |
| static | False |
C++ defination code:
tensor::Tensors *forward(tensor::Tensors &inputs, bool copy_result = true, bool dual_buff_wait = false)
forward_image
def forward_image(self, img: maix.image.Image, mean: list[float] = [], scale: list[float] = [], fit: maix.image.Fit = ..., copy_result: bool = True, dual_buff_wait: bool = False) -> maix.tensor.Tensors
forward model, param is image
| item | description |
|---|---|
| type | func |
| param | img: input image mean: mean value, a list type, e.g. [0.485, 0.456, 0.406], default is empty list means not normalize. scale: scale value, a list type, e.g. [1/0.229, 1/0.224, 1/0.225], default is empty list means not normalize. fit: fit mode, if the image size of input not equal to model's input, it will auto resize use this fit method, default is image.Fit.FIT_FILL for easy coordinate calculation, but for more accurate result, use image.Fit.FIT_CONTAIN is better. copy_result: If set true, will copy result to a new variable; else will use a internal memory, you can only use it until to the next forward. Default true to avoid problems, you can set it to false manually to make speed faster. dual_buff_wait: bool type, only for dual_buff mode, if true, will inference this image and wait for result, default false. |
| return | output tensor. In C++, you should manually delete tensors in return value and return value. If dual_buff mode, it can be NULL(None in MaixPy) means not ready. |
| throw | If error occurs, like arg error or alloc memory failed, will raise err.Exception. |
| static | False |
C++ defination code:
tensor::Tensors *forward_image(image::Image &img, std::vector<float> mean = std::vector<float>(), std::vector<float> scale = std::vector<float>(), image::Fit fit = image::Fit::FIT_FILL, bool copy_result = true, bool dual_buff_wait = false)
Classifier
Classifier
C++ defination code:
class Classifier
__init__
def __init__(self, model: str = '', dual_buff: bool = True) -> None
Construct a new Classifier object
| item | description |
|---|---|
| type | func |
| param | model: MUD model path, if empty, will not load model, you can call load() later. if not empty, will load model and will raise err::Exception if load failed. dual_buff: direction [in], prepare dual input output buffer to accelarate forward, that is, when NPU is forwarding we not wait and prepare the next input buff. If you want to ensure every time forward output the input's result, set this arg to false please. Default true to ensure speed. |
| static | False |
C++ defination code:
Classifier(const string &model = "", bool dual_buff = true)
load
def load(self, model: str) -> maix.err.Err
Load model from file, model format is .mud,\nMUD file should contain [extra] section, have key-values:\n- model_type: classifier\n- input_type: rgb or bgr\n- mean: 123.675, 116.28, 103.53\n- scale: 0.017124753831663668, 0.01750700280112045, 0.017429193899782137\n- labels: imagenet_classes.txt
| item | description |
|---|---|
| type | func |
| param | model: MUD model path |
| return | error code, if load failed, return error code |
| static | False |
C++ defination code:
err::Err load(const string &model)
classify
def classify(self, img: maix.image.Image, softmax: bool = True, fit: maix.image.Fit = ...) -> list[tuple[int, float]]
Forward image to model, get result. Only for image input, use classify_raw for tensor input.
| item | description |
|---|---|
| type | func |
| param | img: image, format should match model input_type， or will raise err.Exception softmax: if true, will do softmax to result, or will return raw value fit: image resize fit mode, default Fit.FIT_COVER, see image.Fit. |
| throw | If error occurred, will raise err::Exception, you can find reason in log, mostly caused by args error or hardware error. |
| return | result, a list of (label, score). If in dual_buff mode, value can be one element list and score is zero when not ready. In C++, you need to delete it after use. |
| static | False |
C++ defination code:
std::vector<std::pair<int, float>> *classify(image::Image &img, bool softmax = true, image::Fit fit = image::FIT_COVER)
classify_raw
def classify_raw(self, data: maix.tensor.Tensor, softmax: bool = True) -> list[tuple[int, float]]
Forward tensor data to model, get result
| item | description |
|---|---|
| type | func |
| param | data: tensor data, format should match model input_type， or will raise err.Excetion softmax: if true, will do softmax to result, or will return raw value |
| throw | If error occurred, will raise err::Exception, you can find reason in log, mostly caused by args error or hardware error. |
| return | result, a list of (label, score). In C++, you need to delete it after use. |
| static | False |
C++ defination code:
std::vector<std::pair<int, float>> *classify_raw(tensor::Tensor &data, bool softmax = true)
input_size
def input_size(self) -> maix.image.Size
Get model input size, only for image input
| item | description |
|---|---|
| type | func |
| return | model input size |
| static | False |
C++ defination code:
image::Size input_size()
input_width
def input_width(self) -> int
Get model input width, only for image input
| item | description |
|---|---|
| type | func |
| return | model input size of width |
| static | False |
C++ defination code:
int input_width()
input_height
def input_height(self) -> int
Get model input height, only for image input
| item | description |
|---|---|
| type | func |
| return | model input size of height |
| static | False |
C++ defination code:
int input_height()
input_format
def input_format(self) -> maix.image.Format
Get input image format, only for image input
| item | description |
|---|---|
| type | func |
| return | input image format, image::Format type. |
| static | False |
C++ defination code:
image::Format input_format()
input_shape
def input_shape(self) -> list[int]
Get input shape, if have multiple input, only return first input shape
| item | description |
|---|---|
| type | func |
| return | input shape, list type |
| static | False |
C++ defination code:
std::vector<int> input_shape()
labels
Labels list
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<string> labels
label_path
Label file path
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::string label_path
mean
Get mean value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> mean
scale
Get scale value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> scale
NanoTrack
NanoTrack class
C++ defination code:
class NanoTrack
__init__
def __init__(self, model: str = '') -> None
Constructor of NanoTrack class
| item | description |
|---|---|
| type | func |
| param | model: model path, default empty, you can load model later by load function. |
| throw | If model arg is not empty and load failed, will throw err::Exception. |
| static | False |
C++ defination code:
NanoTrack(const string &model = "")
load
def load(self, model: str) -> maix.err.Err
Load model from file
| item | description |
|---|---|
| type | func |
| param | model: Model path want to load |
| return | err::Err |
| static | False |
C++ defination code:
err::Err load(const string &model)
init
def init(self, img: maix.image.Image, x: int, y: int, w: int, h: int) -> None
Init tracker, give tacker first target image and target position.
| item | description |
|---|---|
| type | func |
| param | img: Image want to detect, target should be in this image. x: the target position left top coordinate x. y: the target position left top coordinate y. w: the target width. h: the target height. |
| throw | If image format not match model input format, will throw err::Exception. |
| static | False |
C++ defination code:
void init(image::Image &img, int x, int y, int w, int h)
track
def track(self, img: maix.image.Image, threshold: float = 0.9) -> ...
Track object acoording to last object position and the init function learned target feature.
| item | description |
|---|---|
| type | func |
| param | img: image to detect object and track, can be any resolution, before detect it will crop a area according to last time target's position. threshold: If score < threshold, will see this new detection is invalid, but remain return this new detecion, default 0.9. |
| return | object, position and score, and detect area in points's first 4 element(x, y, w, h, center_x, center_y, input_size, target_size) |
| static | False |
C++ defination code:
nn::Object track(image::Image &img, float threshold = 0.9)
input_size
def input_size(self) -> maix.image.Size
Get model input size
| item | description |
|---|---|
| type | func |
| return | model input size |
| static | False |
C++ defination code:
image::Size input_size()
input_width
def input_width(self) -> int
Get model input width
| item | description |
|---|---|
| type | func |
| return | model input size of width |
| static | False |
C++ defination code:
int input_width()
input_height
def input_height(self) -> int
Get model input height
| item | description |
|---|---|
| type | func |
| return | model input size of height |
| static | False |
C++ defination code:
int input_height()
input_format
def input_format(self) -> maix.image.Format
Get input image format
| item | description |
|---|---|
| type | func |
| return | input image format, image::Format type. |
| static | False |
C++ defination code:
image::Format input_format()
mean
Get mean value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> mean
scale
Get scale value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> scale
Object
Object for detect result
C++ defination code:
class Object
__init__
def __init__(self, x: int = 0, y: int = 0, w: int = 0, h: int = 0, class_id: int = 0, score: float = 0, points: list[int] = []) -> None
Constructor of Object for detect result
| item | description |
|---|---|
| type | func |
| param | x: left top x y: left top y w: width h: height class_id: class id score: score |
| static | False |
C++ defination code:
Object(int x = 0, int y = 0, int w = 0, int h = 0, int class_id = 0, float score = 0, std::vector<int> points = std::vector<int>())
__str__
def __str__(self) -> str
Object info to string
| item | description |
|---|---|
| type | func |
| return | Object info string |
| static | False |
C++ defination code:
std::string to_str()
x
Object left top coordinate x
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
int x
y
Object left top coordinate y
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
int y
w
Object width
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
int w
h
Object height
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
int h
class_id
Object class id
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
int class_id
score
Object score
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
float score
points
keypoints
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<int> points
seg_mask
segmentation mask, uint8 list type, shape is h * w but flattened to one dimension, value fron 0 to 255.
| item | description |
|---|---|
| type | var |
| attention | For efficiency, it's a pointer in C++, use this carefully! |
| static | False |
| readonly | False |
C++ defination code:
image::Image *seg_mask
ObjectFloat
Object for detect result
C++ defination code:
class ObjectFloat
__init__
def __init__(self, x: float = 0, y: float = 0, w: float = 0, h: float = 0, class_id: float = 0, score: float = 0, points: list[float] = []) -> None
Constructor of Object for detect result
| item | description |
|---|---|
| type | func |
| param | x: left top x y: left top y w: width h: height class_id: class id score: score |
| static | False |
C++ defination code:
ObjectFloat(float x = 0, float y = 0, float w = 0, float h = 0, float class_id = 0, float score = 0, std::vector<float> points = std::vector<float>())
__str__
def __str__(self) -> str
Object info to string
| item | description |
|---|---|
| type | func |
| return | Object info string |
| static | False |
C++ defination code:
std::string to_str()
x
Object left top coordinate x
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
float x
y
Object left top coordinate y
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
float y
w
Object width
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
float w
h
Object height
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
float h
class_id
Object class id
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
float class_id
score
Object score
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
float score
points
keypoints
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> points
Objects
Objects Class for detect result
C++ defination code:
class Objects
__init__
def __init__(self) -> None
Constructor of Objects class
| item | description |
|---|---|
| type | func |
| static | False |
C++ defination code:
Objects()
add
def add(self, x: int = 0, y: int = 0, w: int = 0, h: int = 0, class_id: int = 0, score: float = 0, points: list[int] = []) -> Object
Add object to objects
| item | description |
|---|---|
| type | func |
| static | False |
C++ defination code:
nn::Object *add(int x = 0, int y = 0, int w = 0, int h = 0, int class_id = 0, float score = 0, std::vector<int> points = std::vector<int>())
remove
def remove(self, idx: int) -> maix.err.Err
Remove object form objects
| item | description |
|---|---|
| type | func |
| static | False |
C++ defination code:
err::Err remove(int idx)
at
def at(self, idx: int) -> Object
Get object item
| item | description |
|---|---|
| type | func |
| static | False |
C++ defination code:
nn::Object *at(int idx)
__item__
def __item__(self, idx: int) -> Object
Get object item
| item | description |
|---|---|
| type | func |
| static | False |
C++ defination code:
nn::Object *operator[](int idx)
__len__
def __len__(self) -> int
Get size
| item | description |
|---|---|
| type | func |
| static | False |
C++ defination code:
size_t size()
__iter__
def __iter__(self) -> typing.Iterator
Begin
| item | description |
|---|---|
| type | func |
| static | False |
C++ defination code:
std::vector<Object*>::iterator begin()
YOLOv5
YOLOv5 class
C++ defination code:
class YOLOv5
__init__
def __init__(self, model: str = '', dual_buff: bool = True) -> None
Constructor of YOLOv5 class
| item | description |
|---|---|
| type | func |
| param | model: model path, default empty, you can load model later by load function. dual_buff: direction [in], prepare dual input output buffer to accelarate forward, that is, when NPU is forwarding we not wait and prepare the next input buff. If you want to ensure every time forward output the input's result, set this arg to false please. Default true to ensure speed. |
| throw | If model arg is not empty and load failed, will throw err::Exception. |
| static | False |
C++ defination code:
YOLOv5(const string &model = "", bool dual_buff = true)
load
def load(self, model: str) -> maix.err.Err
Load model from file
| item | description |
|---|---|
| type | func |
| param | model: Model path want to load |
| return | err::Err |
| static | False |
C++ defination code:
err::Err load(const string &model)
detect
def detect(self, img: maix.image.Image, conf_th: float = 0.5, iou_th: float = 0.45, fit: maix.image.Fit = ...) -> list[Object]
Detect objects from image
| item | description |
|---|---|
| type | func |
| param | img: Image want to detect, if image's size not match model input's, will auto resize with fit method. conf_th: Confidence threshold, default 0.5. iou_th: IoU threshold, default 0.45. fit: Resize method, default image.Fit.FIT_CONTAIN. |
| throw | If image format not match model input format, will throw err::Exception. |
| return | Object list. In C++, you should delete it after use. |
| static | False |
C++ defination code:
std::vector<nn::Object> *detect(image::Image &img, float conf_th = 0.5, float iou_th = 0.45, maix::image::Fit fit = maix::image::FIT_CONTAIN)
input_size
def input_size(self) -> maix.image.Size
Get model input size
| item | description |
|---|---|
| type | func |
| return | model input size |
| static | False |
C++ defination code:
image::Size input_size()
input_width
def input_width(self) -> int
Get model input width
| item | description |
|---|---|
| type | func |
| return | model input size of width |
| static | False |
C++ defination code:
int input_width()
input_height
def input_height(self) -> int
Get model input height
| item | description |
|---|---|
| type | func |
| return | model input size of height |
| static | False |
C++ defination code:
int input_height()
input_format
def input_format(self) -> maix.image.Format
Get input image format
| item | description |
|---|---|
| type | func |
| return | input image format, image::Format type. |
| static | False |
C++ defination code:
image::Format input_format()
labels
Labels list
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<string> labels
label_path
Label file path
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::string label_path
mean
Get mean value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> mean
scale
Get scale value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> scale
anchors
Get anchors
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> anchors
FaceObject
Face object
C++ defination code:
class FaceObject
__init__
def __init__(self, x: int = 0, y: int = 0, w: int = 0, h: int = 0, class_id: int = 0, score: float = 0, points: list[int] = [], feature: list[float] = [], face: maix.image.Image = ...) -> None
Constructor
| item | description |
|---|---|
| type | func |
| static | False |
C++ defination code:
FaceObject(int x = 0, int y = 0, int w = 0, int h = 0, int class_id = 0, float score = 0, std::vector<int> points = std::vector<int>(), std::vector<float> feature = std::vector<float>(), image::Image face = image::Image())
__str__
def __str__(self) -> str
FaceObject info to string
| item | description |
|---|---|
| type | func |
| return | FaceObject info string |
| static | False |
C++ defination code:
std::string to_str()
x
FaceObject left top coordinate x
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
int x
y
FaceObject left top coordinate y
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
int y
w
FaceObject width
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
int w
h
FaceObject height
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
int h
class_id
FaceObject class id
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
int class_id
score
FaceObject score
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
float score
points
keypoints
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<int> points
feature
feature, float list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> feature
face
face image
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
image::Image face
FaceRecognizer
FaceRecognizer class
C++ defination code:
class FaceRecognizer
__init__
def __init__(self, detect_model: str = '', feature_model: str = '', dual_buff: bool = True) -> None
Constructor of FaceRecognizer class
| item | description |
|---|---|
| type | func |
| param | detect_model: face detect model path, default empty, you can load model later by load function. feature_model: feature extract model dual_buff: direction [in], prepare dual input output buffer to accelarate forward, that is, when NPU is forwarding we not wait and prepare the next input buff. If you want to ensure every time forward output the input's result, set this arg to false please. Default true to ensure speed. |
| throw | If model arg is not empty and load failed, will throw err::Exception. |
| static | False |
C++ defination code:
FaceRecognizer(const string &detect_model = "", const string &feature_model = "", bool dual_buff = true)
load
def load(self, detect_model: str, feature_model: str) -> maix.err.Err
Load model from file
| item | description |
|---|---|
| type | func |
| param | detect_model: face detect model path, default empty, you can load model later by load function. feature_model: feature extract model |
| return | err::Err |
| static | False |
C++ defination code:
err::Err load(const string &detect_model, const string &feature_model)
recognize
def recognize(self, img: maix.image.Image, conf_th: float = 0.5, iou_th: float = 0.45, compare_th: float = 0.8, get_feature: bool = False, get_face: bool = False, fit: maix.image.Fit = ...) -> list[FaceObject]
Detect objects from image
| item | description |
|---|---|
| type | func |
| param | img: Image want to detect, if image's size not match model input's, will auto resize with fit method. conf_th: Detect confidence threshold, default 0.5. iou_th: Detect IoU threshold, default 0.45. compare_th: Compare two face score threshold, default 0.8, if two faces' score < this value, will see this face fas unknown. get_feature: return feature or not, if true will copy features to result, if false will not copy feature to result to save time and memory. get_face: return face image or not, if true result object's face attribute will valid, or face sttribute is empty. Get face image will alloc memory and copy image, so will lead to slower speed. fit: Resize method, default image.Fit.FIT_CONTAIN. |
| throw | If image format not match model input format, will throw err::Exception. |
| return | FaceObject list. In C++, you should delete it after use. |
| static | False |
C++ defination code:
std::vector<nn::FaceObject> *recognize(image::Image &img, float conf_th = 0.5, float iou_th = 0.45, float compare_th = 0.8, bool get_feature = false, bool get_face = false, maix::image::Fit fit = maix::image::FIT_CONTAIN)
add_face
def add_face(self, face: FaceObject, label: str) -> maix.err.Err
Add face to lib
| item | description |
|---|---|
| type | func |
| param | face: face object, find by recognize label: face label(name) |
| static | False |
C++ defination code:
err::Err add_face(nn::FaceObject *face, const std::string &label)
remove_face
def remove_face(self, idx: int = -1, label: str = '') -> maix.err.Err
remove face from lib
| item | description |
|---|---|
| type | func |
| param | idx: index of face in lib, default -1 means use label, idx and label must have one, idx have high priotiry. label: which face to remove, default to empty string mean use idx, idx and label must have one, idx have high priotiry. |
| static | False |
C++ defination code:
err::Err remove_face(int idx = -1, const std::string &label = "")
save_faces
def save_faces(self, path: str) -> maix.err.Err
Save faces info to a file
| item | description |
|---|---|
| type | func |
| param | path: where to save, string type. |
| return | err.Err type |
| static | False |
C++ defination code:
err::Err save_faces(const std::string &path)
load_faces
def load_faces(self, path: str) -> maix.err.Err
Load faces info from a file
| item | description |
|---|---|
| type | func |
| param | path: from where to load, string type. |
| return | err::Err type |
| static | False |
C++ defination code:
err::Err load_faces(const std::string &path)
input_size
def input_size(self) -> maix.image.Size
Get model input size
| item | description |
|---|---|
| type | func |
| return | model input size |
| static | False |
C++ defination code:
image::Size input_size()
input_width
def input_width(self) -> int
Get model input width
| item | description |
|---|---|
| type | func |
| return | model input size of width |
| static | False |
C++ defination code:
int input_width()
input_height
def input_height(self) -> int
Get model input height
| item | description |
|---|---|
| type | func |
| return | model input size of height |
| static | False |
C++ defination code:
int input_height()
input_format
def input_format(self) -> maix.image.Format
Get input image format
| item | description |
|---|---|
| type | func |
| return | input image format, image::Format type. |
| static | False |
C++ defination code:
image::Format input_format()
mean_detector
Get mean value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> mean_detector
scale_detector
Get scale value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> scale_detector
mean_feature
Get mean value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> mean_feature
scale_feature
Get scale value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> scale_feature
labels
labels, list type, first is "unknown"
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<std::string> labels
features
features
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<std::vector<float>> features
SelfLearnClassifier
SelfLearnClassifier
C++ defination code:
class SelfLearnClassifier
__init__
def __init__(self, model: str = '', dual_buff: bool = True) -> None
Construct a new SelfLearnClassifier object
| item | description |
|---|---|
| type | func |
| param | model: MUD model path, if empty, will not load model, you can call load_model() later. if not empty, will load model and will raise err::Exception if load failed. dual_buff: direction [in], prepare dual input output buffer to accelarate forward, that is, when NPU is forwarding we not wait and prepare the next input buff. If you want to ensure every time forward output the input's result, set this arg to false please. Default true to ensure speed. |
| static | False |
C++ defination code:
SelfLearnClassifier(const std::string &model = "", bool dual_buff = true)
load_model
def load_model(self, model: str) -> maix.err.Err
Load model from file, model format is .mud,\nMUD file should contain [extra] section, have key-values:\n- model_type: classifier_no_top\n- input_type: rgb or bgr\n- mean: 123.675, 116.28, 103.53\n- scale: 0.017124753831663668, 0.01750700280112045, 0.017429193899782137
| item | description |
|---|---|
| type | func |
| param | model: MUD model path |
| return | error code, if load failed, return error code |
| static | False |
C++ defination code:
err::Err load_model(const string &model)
classify
def classify(self, img: maix.image.Image, fit: maix.image.Fit = ...) -> list[tuple[int, float]]
Classify image
| item | description |
|---|---|
| type | func |
| param | img: image, format should match model input_type， or will raise err.Exception fit: image resize fit mode, default Fit.FIT_COVER, see image.Fit. |
| throw | If error occurred, will raise err::Exception, you can find reason in log, mostly caused by args error or hardware error. |
| return | result, a list of (idx, distance), smaller distance means more similar. In C++, you need to delete it after use. |
| static | False |
C++ defination code:
std::vector<std::pair<int, float>> *classify(image::Image &img, image::Fit fit = image::FIT_COVER)
add_class
def add_class(self, img: maix.image.Image, fit: maix.image.Fit = ...) -> None
Add a class to recognize
| item | description |
|---|---|
| type | func |
| param | img: Add a image as a new class fit: image resize fit mode, default Fit.FIT_COVER, see image.Fit. |
| static | False |
C++ defination code:
void add_class(image::Image &img, image::Fit fit = image::FIT_COVER)
class_num
def class_num(self) -> int
Get class number
| item | description |
|---|---|
| type | func |
| static | False |
C++ defination code:
int class_num()
rm_class
def rm_class(self, idx: int) -> maix.err.Err
Remove a class
| item | description |
|---|---|
| type | func |
| param | idx: index, value from 0 to class_num(); |
| static | False |
C++ defination code:
err::Err rm_class(int idx)
add_sample
def add_sample(self, img: maix.image.Image, fit: maix.image.Fit = ...) -> None
Add sample, you should call learn method after add some samples to learn classes.\nSample image can be any of classes we already added.
| item | description |
|---|---|
| type | func |
| param | img: Add a image as a new sample. |
| static | False |
C++ defination code:
void add_sample(image::Image &img, image::Fit fit = image::FIT_COVER)
rm_sample
def rm_sample(self, idx: int) -> maix.err.Err
Remove a sample
| item | description |
|---|---|
| type | func |
| param | idx: index, value from 0 to sample_num(); |
| static | False |
C++ defination code:
err::Err rm_sample(int idx)
sample_num
def sample_num(self) -> int
Get sample number
| item | description |
|---|---|
| type | func |
| static | False |
C++ defination code:
int sample_num()
learn
def learn(self) -> int
Start auto learn class features from classes image and samples.\nYou should call this method after you add some samples.
| item | description |
|---|---|
| type | func |
| return | learn epoch(times), 0 means learn nothing. |
| static | False |
C++ defination code:
int learn()
clear
def clear(self) -> None
Clear all class and samples
| item | description |
|---|---|
| type | func |
| static | False |
C++ defination code:
void clear()
input_size
def input_size(self) -> maix.image.Size
Get model input size, only for image input
| item | description |
|---|---|
| type | func |
| return | model input size |
| static | False |
C++ defination code:
image::Size input_size()
input_width
def input_width(self) -> int
Get model input width, only for image input
| item | description |
|---|---|
| type | func |
| return | model input size of width |
| static | False |
C++ defination code:
int input_width()
input_height
def input_height(self) -> int
Get model input height, only for image input
| item | description |
|---|---|
| type | func |
| return | model input size of height |
| static | False |
C++ defination code:
int input_height()
input_format
def input_format(self) -> maix.image.Format
Get input image format, only for image input
| item | description |
|---|---|
| type | func |
| return | input image format, image::Format type. |
| static | False |
C++ defination code:
image::Format input_format()
input_shape
def input_shape(self) -> list[int]
Get input shape, if have multiple input, only return first input shape
| item | description |
|---|---|
| type | func |
| return | input shape, list type |
| static | False |
C++ defination code:
std::vector<int> input_shape()
save
def save(self, path: str, labels: list[str] = []) -> maix.err.Err
Save features and labels to a binary file
| item | description |
|---|---|
| type | func |
| param | path: file path to save, e.g. /root/my_classes.bin labels: class labels, can be None, or length must equal to class num, or will return err::Err |
| return | maix.err.Err if labels exists but length not equal to class num, or save file failed, or class num is 0. |
| static | False |
C++ defination code:
err::Err save(const std::string &path, const std::vector<std::string> &labels = std::vector<std::string>())
load
def load(self, path: str) -> list[str]
Load features info from binary file
| item | description |
|---|---|
| type | func |
| param | path: feature info binary file path, e.g. /root/my_classes.bin |
| static | False |
C++ defination code:
std::vector<std::string> load(const std::string &path)
labels
Labels list
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<string> labels
label_path
Label file path
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::string label_path
mean
Get mean value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> mean
scale
Get scale value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> scale
FaceDetector
FaceDetector class
C++ defination code:
class FaceDetector
__init__
def __init__(self, model: str = '', dual_buff: bool = True) -> None
Constructor of FaceDetector class
| item | description |
|---|---|
| type | func |
| param | model: model path, default empty, you can load model later by load function. dual_buff: direction [in], prepare dual input output buffer to accelarate forward, that is, when NPU is forwarding we not wait and prepare the next input buff. If you want to ensure every time forward output the input's result, set this arg to false please. Default true to ensure speed. |
| throw | If model arg is not empty and load failed, will throw err::Exception. |
| static | False |
C++ defination code:
FaceDetector(const string &model = "", bool dual_buff = true)
load
def load(self, model: str) -> maix.err.Err
Load model from file
| item | description |
|---|---|
| type | func |
| param | model: Model path want to load |
| return | err::Err |
| static | False |
C++ defination code:
err::Err load(const string &model)
detect
def detect(self, img: maix.image.Image, conf_th: float = 0.5, iou_th: float = 0.45, fit: maix.image.Fit = ...) -> list[Object]
Detect objects from image
| item | description |
|---|---|
| type | func |
| param | img: Image want to detect, if image's size not match model input's, will auto resize with fit method. conf_th: Confidence threshold, default 0.5. iou_th: IoU threshold, default 0.45. fit: Resize method, default image.Fit.FIT_CONTAIN. |
| throw | If image format not match model input format, will throw err::Exception. |
| return | Object list. In C++, you should delete it after use. |
| static | False |
C++ defination code:
std::vector<nn::Object> *detect(image::Image &img, float conf_th = 0.5, float iou_th = 0.45, maix::image::Fit fit = maix::image::FIT_CONTAIN)
input_size
def input_size(self) -> maix.image.Size
Get model input size
| item | description |
|---|---|
| type | func |
| return | model input size |
| static | False |
C++ defination code:
image::Size input_size()
input_width
def input_width(self) -> int
Get model input width
| item | description |
|---|---|
| type | func |
| return | model input size of width |
| static | False |
C++ defination code:
int input_width()
input_height
def input_height(self) -> int
Get model input height
| item | description |
|---|---|
| type | func |
| return | model input size of height |
| static | False |
C++ defination code:
int input_height()
input_format
def input_format(self) -> maix.image.Format
Get input image format
| item | description |
|---|---|
| type | func |
| return | input image format, image::Format type. |
| static | False |
C++ defination code:
image::Format input_format()
mean
Get mean value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> mean
scale
Get scale value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> scale
YOLOv8
YOLOv8 class
C++ defination code:
class YOLOv8
__init__
def __init__(self, model: str = '', dual_buff: bool = True) -> None
Constructor of YOLOv8 class
| item | description |
|---|---|
| type | func |
| param | model: model path, default empty, you can load model later by load function. dual_buff: direction [in], prepare dual input output buffer to accelarate forward, that is, when NPU is forwarding we not wait and prepare the next input buff. If you want to ensure every time forward output the input's result, set this arg to false please. Default true to ensure speed. |
| throw | If model arg is not empty and load failed, will throw err::Exception. |
| static | False |
C++ defination code:
YOLOv8(const string &model = "", bool dual_buff = true)
load
def load(self, model: str) -> maix.err.Err
Load model from file
| item | description |
|---|---|
| type | func |
| param | model: Model path want to load |
| return | err::Err |
| static | False |
C++ defination code:
err::Err load(const string &model)
detect
def detect(self, img: maix.image.Image, conf_th: float = 0.5, iou_th: float = 0.45, fit: maix.image.Fit = ..., keypoint_th: float = 0.5) -> Objects
Detect objects from image
| item | description |
|---|---|
| type | func |
| param | img: Image want to detect, if image's size not match model input's, will auto resize with fit method. conf_th: Confidence threshold, default 0.5. iou_th: IoU threshold, default 0.45. fit: Resize method, default image.Fit.FIT_CONTAIN. keypoint_th: keypoint threshold, default 0.5, only for yolov8-pose model. |
| throw | If image format not match model input format, will throw err::Exception. |
| return | Object list. In C++, you should delete it after use. If model is yolov8-pose, object's points have value, and if points' value < 0 means that point is invalid(conf < keypoint_th). |
| static | False |
C++ defination code:
nn::Objects *detect(image::Image &img, float conf_th = 0.5, float iou_th = 0.45, maix::image::Fit fit = maix::image::FIT_CONTAIN, float keypoint_th = 0.5)
input_size
def input_size(self) -> maix.image.Size
Get model input size
| item | description |
|---|---|
| type | func |
| return | model input size |
| static | False |
C++ defination code:
image::Size input_size()
input_width
def input_width(self) -> int
Get model input width
| item | description |
|---|---|
| type | func |
| return | model input size of width |
| static | False |
C++ defination code:
int input_width()
input_height
def input_height(self) -> int
Get model input height
| item | description |
|---|---|
| type | func |
| return | model input size of height |
| static | False |
C++ defination code:
int input_height()
input_format
def input_format(self) -> maix.image.Format
Get input image format
| item | description |
|---|---|
| type | func |
| return | input image format, image::Format type. |
| static | False |
C++ defination code:
image::Format input_format()
draw_pose
def draw_pose(self, img: maix.image.Image, points: list[int], radius: int = 4, color: maix.image.Color = ..., body: bool = True) -> None
Draw pose keypoints on image
| item | description |
|---|---|
| type | func |
| param | img: image object, maix.image.Image type. points: keypoits, int list type, [x, y, x, y ...] radius: radius of points. color: color of points. body: true, if points' length is 17*2 and body is ture, will draw lines as human body, if set to false won't draw lines, default true. |
| static | False |
C++ defination code:
void draw_pose(image::Image &img, std::vector<int> points, int radius = 4, image::Color color = image::COLOR_RED, bool body = true)
draw_seg_mask
def draw_seg_mask(self, img: maix.image.Image, x: int, y: int, seg_mask: maix.image.Image, threshold: int = 127) -> None
Draw segmentation on image
| item | description |
|---|---|
| type | func |
| param | img: image object, maix.image.Image type. seg_mask: segmentation mask image by detect method, a grayscale image threshold: only mask's value > threshold will be draw on image, value from 0 to 255. |
| static | False |
C++ defination code:
void draw_seg_mask(image::Image &img, int x, int y, image::Image &seg_mask, int threshold = 127)
labels
Labels list
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<string> labels
label_path
Label file path
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::string label_path
mean
Get mean value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> mean
scale
Get scale value, list type
| item | description |
|---|---|
| type | var |
| static | False |
| readonly | False |
C++ defination code:
std::vector<float> scale
