maix.nn

maix.nn module

You can use maix.nn to access this module with MaixPy
This module is generated from MaixPy and MaixCDK

Module

module	brief
F	maix.nn.F module

Enum

SpeechDevice

speech device

item	describe
values	DEVICE_NONE: DEVICE_PCM: DEVICE_MIC: DEVICE_WAV:

C++ defination code:

enum class SpeechDevice {
    DEVICE_NONE = -1,
    DEVICE_PCM,
    DEVICE_MIC,
    DEVICE_WAV,
}

SpeechDecoder

speech decoder type

item	describe
values	DECODER_RAW: DECODER_DIG: DECODER_LVCSR: DECODER_KWS: DECODER_ALL:

C++ defination code:

enum class SpeechDecoder {
    DECODER_RAW = 1,
    DECODER_DIG = 2,
    DECODER_LVCSR = 4,
    DECODER_KWS = 8,
    DECODER_ALL = 65535,
}

Variable

Function

Class

NanoTrack

NanoTrack class

C++ defination code:
class NanoTrack

init

def __init__(self, model: str = '') -> None

Constructor of NanoTrack class

item	description
type	func
param	model: model path, default empty, you can load model later by load function.
throw	If model arg is not empty and load failed, will throw err::Exception.
static	False

C++ defination code:
NanoTrack(const string &model = "")

load

def load(self, model: str) -> maix.err.Err

Load model from file

item	description
type	func
param	model: Model path want to load
return	err::Err
static	False

C++ defination code:
err::Err load(const string &model)

init

def init(self, img: maix.image.Image, x: int, y: int, w: int, h: int) -> None

Init tracker, give tacker first target image and target position.

item	description
type	func
param	img: Image want to detect, target should be in this image. x: the target position left top coordinate x. y: the target position left top coordinate y. w: the target width. h: the target height.
throw	If image format not match model input format, will throw err::Exception.
static	False

C++ defination code:

void init(image::Image &img, int x, int y, int w, int h)

track

def track(self, img: maix.image.Image, threshold: float = 0.9) -> ...

Track object acoording to last object position and the init function learned target feature.

item	description
type	func
param	img: image to detect object and track, can be any resolution, before detect it will crop a area according to last time target's position. threshold: If score < threshold, will see this new detection is invalid, but remain return this new detecion, default 0.9.
return	object, position and score, and detect area in points's first 4 element(x, y, w, h, center_x, center_y, input_size, target_size)
static	False

C++ defination code:

nn::Object track(image::Image &img, float threshold = 0.9)

input_size

def input_size(self) -> maix.image.Size

Get model input size

item	description
type	func
return	model input size
static	False

C++ defination code:
image::Size input_size()

input_width

def input_width(self) -> int

Get model input width

item	description
type	func
return	model input size of width
static	False

C++ defination code:
int input_width()

input_height

def input_height(self) -> int

Get model input height

item	description
type	func
return	model input size of height
static	False

C++ defination code:
int input_height()

input_format

def input_format(self) -> maix.image.Format

Get input image format

item	description
type	func
return	input image format, image::Format type.
static	False

C++ defination code:
image::Format input_format()

mean

Get mean value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> mean

scale

Get scale value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> scale

OCR_Box

Object for OCR detect box

C++ defination code:
class OCR_Box

init

def __init__(self, x1: int = 0, y1: int = 0, x2: int = 0, y2: int = 0, x3: int = 0, y3: int = 0, x4: int = 0, y4: int = 0) -> None

OCR_Box constructor

item	description
type	func
static	False

C++ defination code:

OCR_Box(int x1 = 0, int y1 = 0, int x2 = 0, int y2 = 0, int x3 = 0, int y3 = 0, int x4 = 0, int y4 = 0)

x1

left top point of box

item	description
type	var
static	False
readonly	False

C++ defination code:
int x1

y1

left top point of box

item	description
type	var
static	False
readonly	False

C++ defination code:
int y1

x2

right top point of box

item	description
type	var
static	False
readonly	False

C++ defination code:
int x2

y2

right top point of box

item	description
type	var
static	False
readonly	False

C++ defination code:
int y2

x3

right bottom point of box

item	description
type	var
static	False
readonly	False

C++ defination code:
int x3

y3

right bottom point of box

item	description
type	var
static	False
readonly	False

C++ defination code:
int y3

x4

left bottom point of box

item	description
type	var
static	False
readonly	False

C++ defination code:
int x4

y4

left bottom point of box

item	description
type	var
static	False
readonly	False

C++ defination code:
int y4

to_list

def to_list(self) -> list[int]

convert box point to a list type.

item	description
type	func
return	list type, element is int type, value [x1, y1, x2, y2, x3, y3, x4, y4].
static	False

C++ defination code:
std::vector<int> to_list()

OCR_Object

Object for OCR detect result

C++ defination code:
class OCR_Object

init

def __init__(self, box: OCR_Box, idx_list: list[int], char_list: list[str], score: float = 0, char_pos: list[int] = []) -> None

Constructor of Object for OCR detect result

item	description
type	func
param	score: score
static	False

C++ defination code:

OCR_Object(const nn::OCR_Box &box, const std::vector<int> &idx_list, const std::vector<std::string> &char_list, float score = 0, const std::vector<int> &char_pos = std::vector<int>())

box

OCR_Object box, 4 points box, first point at the left-top, clock-wise.

item	description
type	var
static	False
readonly	False

C++ defination code:
nn::OCR_Box box

score

Object score

item	description
type	var
static	False
readonly	False

C++ defination code:
float score

idx_list

chars' idx list, element is int type.

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<int> idx_list

char_pos

Chars' position relative to left

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<int> char_pos

char_str

def char_str(self) -> str

Get OCR_Object's charactors, return a string type.

item	description
type	func
return	All charactors in string type.
static	False

C++ defination code:
const std::string &char_str()

char_list

def char_list(self) -> list[str]

Get OCR_Object's charactors, return a list type.

item	description
type	func
return	All charactors in list type.
static	False

C++ defination code:
const std::vector<std::string> &char_list()

update_chars

def update_chars(self, char_list: list[str]) -> None

Set OCR_Object's charactors

item	description
type	func
param	char_list: All charactors in list type.
static	False

C++ defination code:

void update_chars(const std::vector<std::string> &char_list)

str

def __str__(self) -> str

OCR_Object info to string

item	description
type	func
return	OCR_Object info string
static	False

C++ defination code:
std::string to_str()

OCR_Objects

OCR_Objects Class for detect result

C++ defination code:
class OCR_Objects

init

def __init__(self) -> None

Constructor of OCR_Objects class

item	description
type	func
static	False

C++ defination code:
OCR_Objects()

add

def add(self, box: OCR_Box, idx_list: list[int], char_list: list[str], score: float = 0, char_pos: list[int] = []) -> OCR_Object

Add object to objects

item	description
type	func
throw	Throw exception if no memory
static	False

C++ defination code:

nn::OCR_Object &add(const nn::OCR_Box &box, const std::vector<int> &idx_list, const std::vector<std::string> &char_list, float score = 0, const std::vector<int> &char_pos = std::vector<int>())

remove

def remove(self, idx: int) -> maix.err.Err

Remove object form objects

item	description
type	func
static	False

C++ defination code:
err::Err remove(int idx)

at

def at(self, idx: int) -> OCR_Object

Get object item

item	description
type	func
static	False

C++ defination code:
nn::OCR_Object &at(int idx)

getitem

def __getitem__(self, idx: int) -> OCR_Object

Get object item

item	description
type	func
static	False

C++ defination code:
nn::OCR_Object &operator[](int idx)

len

def __len__(self) -> int

Get size

item	description
type	func
static	False

C++ defination code:
size_t size()

iter

def __iter__(self) -> typing.Iterator

Begin

item	description
type	func
static	False

C++ defination code:
std::vector<OCR_Object*>::iterator begin()

Speech

C++ defination code:
class Speech

init

def __init__(self, model: str = '') -> None

Construct a new Speech object

item	description
type	func
param	model: model path, default empty, you can load model later by load function.
throw	If model arg is not empty and load failed, will throw err::Exception.
static	False

C++ defination code:
Speech(const string &model = "")

init (overload 1)

Construct a new Speech object

item	description
type	func
param	model: model path, default empty, you can load model later by load function.
throw	If model arg is not empty and load failed, will throw err::Exception.
static	False

C++ defination code:
Speech(const string &model = "")

load

def load(self, model: str) -> maix.err.Err

Load model from file

item	description
type	func
param	model: Model path want to load
return	err::Err
static	False

C++ defination code:
err::Err load(const string &model)

load (overload 1)

Load model from file

item	description
type	func
param	model: Model path want to load
return	err::Err
static	False

C++ defination code:
err::Err load(const string &model)

init

def init(self, dev_type: SpeechDevice, device_name: str = '') -> maix.err.Err

Init the ASR library and select the type and name of the audio device.

item	description
type	func
param	dev_type: device type want to detect, can choose between WAV, PCM, or MIC. device_name: device name want to detect, can choose a WAV file, a PCM file, or a MIC device name.
throw	1. If am model is not loaded, will throw err::ERR_NOT_IMPL. 2. If device is not supported, will throw err::ERR_NOT_IMPL.
return	err::Err type, if init success, return err::ERR_NONE
static	False

C++ defination code:

err::Err init(nn::SpeechDevice dev_type, const string &device_name = "")

init (overload 1)

Init the ASR library and select the type and name of the audio device.

item	description
type	func
param	dev_type: device type want to detect, can choose between WAV, PCM, or MIC. device_name: device name want to detect, can choose a WAV file, a PCM file, or a MIC device name.
throw	1. If am model is not loaded, will throw err::ERR_NOT_IMPL. 2. If device is not supported, will throw err::ERR_NOT_IMPL.
return	err::Err type, if init success, return err::ERR_NONE
static	False

C++ defination code:

err::Err init(nn::SpeechDevice dev_type, const string &device_name = "")

devive

def devive(self, dev_type: SpeechDevice, device_name: str) -> maix.err.Err

Reset the device, usually used for PCM/WAV recognition,\nsuch as identifying the next WAV file.

item	description
type	func
param	dev_type: device type want to detect, can choose between WAV, PCM, or MIC. device_name: device name want to detect, can choose a WAV file, a PCM file, or a MIC device name.
throw	If device is not supported, will throw err::ERR_NOT_IMPL.
return	err::Err type, if init success, return err::ERR_NONE
static	False

C++ defination code:

err::Err devive(nn::SpeechDevice dev_type, const string &device_name)

devive (overload 1)

Reset the device, usually used for PCM/WAV recognition,\nsuch as identifying the next WAV file.

item	description
type	func
param	dev_type: device type want to detect, can choose between WAV, PCM, or MIC. device_name: device name want to detect, can choose a WAV file, a PCM file, or a MIC device name.
throw	If device is not supported, will throw err::ERR_NOT_IMPL.
return	err::Err type, if init success, return err::ERR_NONE
static	False

C++ defination code:

err::Err devive(nn::SpeechDevice dev_type, const string &device_name)

dec_deinit

def dec_deinit(self, decoder: SpeechDecoder) -> None

Deinit the decoder.

item	description
type	func
param	decoder: decoder type want to deinit can choose between DECODER_RAW, DECODER_DIG, DECODER_LVCSR, DECODER_KWS or DECODER_ALL.
throw	If device is not supported, will throw err::ERR_NOT_IMPL.
static	False

C++ defination code:
void dec_deinit(nn::SpeechDecoder decoder)

dec_deinit (overload 1)

Deinit the decoder.

item	description
type	func
param	decoder: decoder type want to deinit can choose between DECODER_RAW, DECODER_DIG, DECODER_LVCSR, DECODER_KWS or DECODER_ALL.
throw	If device is not supported, will throw err::ERR_NOT_IMPL.
static	False

C++ defination code:
void dec_deinit(nn::SpeechDecoder decoder)

raw

def raw(self, callback: typing.Callable[[list[tuple[int, float]], int], None]) -> maix.err.Err

Init raw decoder, it will output the prediction results of the original AM.

item	description
type	func
param	callback: raw decoder user callback.
return	err::Err type, if init success, return err::ERR_NONE
static	False

C++ defination code:

err::Err raw(std::function<void(std::vector<std::pair<int, float>>, int)> callback)

raw (overload 1)

Get raw decoder status

item	description
type	func
return	bool, raw decoder status
static	False

C++ defination code:
bool raw()

raw (overload 2)

Init raw decoder, it will output the prediction results of the original AM.

item	description
type	func
param	callback: raw decoder user callback.
return	err::Err type, if init success, return err::ERR_NONE
static	False

C++ defination code:

err::Err raw(std::function<void(std::vector<std::pair<int, float>>, int)> callback)

raw (overload 3)

Get raw decoder status

item	description
type	func
return	bool, raw decoder status
static	False

C++ defination code:
bool raw()

digit

def digit(self, blank: int, callback: typing.Callable[[str, int], None]) -> maix.err.Err

Init digit decoder, it will output the Chinese digit recognition results within the last 4 seconds.

item	description
type	func
param	blank: If it exceeds this value, insert a '_' in the output result to indicate idle mute. callback: digit decoder user callback.
return	err::Err type, if init success, return err::ERR_NONE
static	False

C++ defination code:

err::Err digit(int blank, std::function<void(char*, int)> callback)

digit (overload 1)

Get digit decoder status

item	description
type	func
return	bool, digit decoder status
static	False

C++ defination code:
bool digit()

digit (overload 2)

Init digit decoder, it will output the Chinese digit recognition results within the last 4 seconds.

item	description
type	func
param	blank: If it exceeds this value, insert a '_' in the output result to indicate idle mute. callback: digit decoder user callback.
return	err::Err type, if init success, return err::ERR_NONE
static	False

C++ defination code:

err::Err digit(int blank, std::function<void(char*, int)> callback)

digit (overload 3)

Get digit decoder status

item	description
type	func
return	bool, digit decoder status
static	False

C++ defination code:
bool digit()

kws

def kws(self, kw_tbl: list[str], kw_gate: list[float], callback: typing.Callable[[list[float], int], None], auto_similar: bool = True) -> maix.err.Err

Init kws decoder, it will output a probability list of all registered keywords in the latest frame,\nusers can set their own thresholds for wake-up.

item	description
type	func
param	kw_tbl: Keyword list, filled in with spaces separated by pinyin, for example: xiao3 ai4 tong2 xue2 kw_gate: kw_gate, keyword probability gate table, the number should be the same as kw_tbl auto_similar: Whether to perform automatic homophone processing, setting it to true will automatically calculate the probability by using pinyin with different tones as homophones callback: digit decoder user callback.
return	err::Err type, if init success, return err::ERR_NONE
static	False

C++ defination code:

err::Err kws(std::vector<string> kw_tbl, std::vector<float> kw_gate, std::function<void(std::vector<float>, int)> callback, bool auto_similar = true)

kws (overload 1)

Get kws decoder status

item	description
type	func
return	bool, kws decoder status
static	False

C++ defination code:
bool kws()

kws (overload 2)

Init kws decoder, it will output a probability list of all registered keywords in the latest frame,\nusers can set their own thresholds for wake-up.

item	description
type	func
param	kw_tbl: Keyword list, filled in with spaces separated by pinyin, for example: xiao3 ai4 tong2 xue2 kw_gate: kw_gate, keyword probability gate table, the number should be the same as kw_tbl auto_similar: Whether to perform automatic homophone processing, setting it to true will automatically calculate the probability by using pinyin with different tones as homophones callback: digit decoder user callback.
return	err::Err type, if init success, return err::ERR_NONE
static	False

C++ defination code:

err::Err kws(std::vector<string> kw_tbl, std::vector<float> kw_gate, std::function<void(std::vector<float>, int)> callback, bool auto_similar = true)

kws (overload 3)

Get kws decoder status

item	description
type	func
return	bool, kws decoder status
static	False

C++ defination code:
bool kws()

lvcsr

def lvcsr(self, sfst_name: str, sym_name: str, phones_txt: str, words_txt: str, callback: typing.Callable[[tuple[str, str], int], None], beam: float = 8, bg_prob: float = 10, scale: float = 0.5, mmap: bool = False) -> maix.err.Err

Init lvcsr decoder, it will output continuous speech recognition results (less than 1024 Chinese characters).

item	description
type	func
param	sfst_name: Sfst file path. sym_name: Sym file path (output symbol table). phones_txt: Path to phones.bin (pinyin table). words_txt: Path to words.bin (dictionary table). callback: lvcsr decoder user callback. beam: The beam size for WFST search is set to 8 by default, and it is recommended to be between 3 and 9. The larger the size, the larger the search space, and the more accurate but slower the search. bg_prob: The absolute value of the natural logarithm of the default probability value for background pinyin outside of BEAM-CNT is set to 10 by default. scale: acoustics_cost = log(pny_prob)scale. mmap*: use mmap to load the WFST decoding image, If set to true, the beam should be less than 5.
return	err::Err type, if init success, return err::ERR_NONE
static	False

C++ defination code:

err::Err lvcsr(const string &sfst_name, const string &sym_name,
                       const string &phones_txt, const string &words_txt, 
                       std::function<void(std::pair<char*, char*>, int)> callback,
                       float beam = 8, float bg_prob = 10, float scale = 0.5, bool mmap = false)

lvcsr (overload 1)

Get lvcsr decoder status

item	description
type	func
return	bool, lvcsr decoder status
static	False

C++ defination code:
bool lvcsr()

lvcsr (overload 2)

Init lvcsr decoder, it will output continuous speech recognition results (less than 1024 Chinese characters).

item	description
type	func
param	sfst_name: Sfst file path. sym_name: Sym file path (output symbol table). phones_txt: Path to phones.bin (pinyin table). words_txt: Path to words.bin (dictionary table). callback: lvcsr decoder user callback. beam: The beam size for WFST search is set to 8 by default, and it is recommended to be between 3 and 9. The larger the size, the larger the search space, and the more accurate but slower the search. bg_prob: The absolute value of the natural logarithm of the default probability value for background pinyin outside of BEAM-CNT is set to 10 by default. scale: acoustics_cost = log(pny_prob)scale. mmap*: use mmap to load the WFST decoding image, If set to true, the beam should be less than 5.
return	err::Err type, if init success, return err::ERR_NONE
static	False

C++ defination code:

err::Err lvcsr(const string &sfst_name, const string &sym_name,
                       const string &phones_txt, const string &words_txt, 
                       std::function<void(std::pair<char*, char*>, int)> callback,
                       float beam = 8, float bg_prob = 10, float scale = 0.5, bool mmap = false)

lvcsr (overload 3)

Get lvcsr decoder status

item	description
type	func
return	bool, lvcsr decoder status
static	False

C++ defination code:
bool lvcsr()

run

def run(self, frame: int) -> int

Run speech recognition, user can run 1 frame at a time and do other processing after running,\nor it can run continuously within a thread and be stopped by an external thread.

item	description
type	func
param	frame: The number of frames per run.
return	int type, return actual number of frames in the run.
static	False

C++ defination code:
int run(int frame)

run (overload 1)

Run speech recognition, user can run 1 frame at a time and do other processing after running,\nor it can run continuously within a thread and be stopped by an external thread.

item	description
type	func
param	frame: The number of frames per run.
return	int type, return actual number of frames in the run.
static	False

C++ defination code:
int run(int frame)

clear

def clear(self) -> None

Reset internal cache operation

item	description
type	func
static	False

C++ defination code:
void clear()

clear (overload 1)

Reset internal cache operation

item	description
type	func
static	False

C++ defination code:
void clear()

frame_time

def frame_time(self) -> int

Get the time of one frame.

item	description
type	func
return	int type, return the time of one frame.
static	False

C++ defination code:
int frame_time()

frame_time (overload 1)

Get the time of one frame.

item	description
type	func
return	int type, return the time of one frame.
static	False

C++ defination code:
int frame_time()

similar

def similar(self, pny: str, similar_pnys: list[str]) -> maix.err.Err

Manually register mute words, and each pinyin can register up to 10 homophones,\nplease note that using this interface to register homophones will overwrite,\nthe homophone table automatically generated in the "automatic homophone processing" feature.

item	description
type	func
param	dev_type: device type want to detect, can choose between WAV, PCM, or MIC. device_name: device name want to detect, can choose a WAV file, a PCM file, or a MIC device name.
return	err::Err type, if init success, return err::ERR_NONE
static	False

C++ defination code:

err::Err similar(const string &pny, std::vector<std::string> similar_pnys)

similar (overload 1)

item	description
type	func
param	dev_type: device type want to detect, can choose between WAV, PCM, or MIC. device_name: device name want to detect, can choose a WAV file, a PCM file, or a MIC device name.
return	err::Err type, if init success, return err::ERR_NONE
static	False

C++ defination code:

err::Err similar(const string &pny, std::vector<std::string> similar_pnys)

skip_frames

def skip_frames(self, num: int) -> None

Run some frames and drop, this can be used to avoid\nincorrect recognition results when switching decoders.

item	description
type	func
param	num: number of frames to run and drop
static	False

C++ defination code:
void skip_frames(int num)

skip_frames (overload 1)

Run some frames and drop, this can be used to avoid\nincorrect recognition results when switching decoders.

item	description
type	func
param	num: number of frames to run and drop
static	False

C++ defination code:
void skip_frames(int num)

mean

Get mean value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> mean

scale

Get scale value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> scale

dev_type

def dev_type(self) -> SpeechDevice

get device type

item	description
type	func
return	nn::SpeechDevice type, see SpeechDevice of this module
static	False

C++ defination code:
nn::SpeechDevice dev_type()

dev_type (overload 1)

get device type

item	description
type	func
return	nn::SpeechDevice type, see SpeechDevice of this module
static	False

C++ defination code:
nn::SpeechDevice dev_type()

YOLOv8

YOLOv8 class

C++ defination code:
class YOLOv8

init

def __init__(self, model: str = '', dual_buff: bool = True) -> None

Constructor of YOLOv8 class

item	description
type	func
param	model: model path, default empty, you can load model later by load function. dual_buff: direction [in], prepare dual input output buffer to accelarate forward, that is, when NPU is forwarding we not wait and prepare the next input buff. If you want to ensure every time forward output the input's result, set this arg to false please. Default true to ensure speed.
throw	If model arg is not empty and load failed, will throw err::Exception.
static	False

C++ defination code:

YOLOv8(const string &model = "", bool dual_buff = true)

load

def load(self, model: str) -> maix.err.Err

Load model from file

item	description
type	func
param	model: Model path want to load
return	err::Err
static	False

C++ defination code:
err::Err load(const string &model)

detect

def detect(self, img: maix.image.Image, conf_th: float = 0.5, iou_th: float = 0.45, fit: maix.image.Fit = ..., keypoint_th: float = 0.5) -> ...

Detect objects from image

item	description
type	func
param	img: Image want to detect, if image's size not match model input's, will auto resize with fit method. conf_th: Confidence threshold, default 0.5. iou_th: IoU threshold, default 0.45. fit: Resize method, default image.Fit.FIT_CONTAIN. keypoint_th: keypoint threshold, default 0.5, only for yolov8-pose model.
throw	If image format not match model input format, will throw err::Exception.
return	Object list. In C++, you should delete it after use. If model is yolov8-pose, object's points have value, and if points' value < 0 means that point is invalid(conf < keypoint_th).
static	False

C++ defination code:

nn::Objects *detect(image::Image &img, float conf_th = 0.5, float iou_th = 0.45, maix::image::Fit fit = maix::image::FIT_CONTAIN, float keypoint_th = 0.5)

input_size

def input_size(self) -> maix.image.Size

Get model input size

item	description
type	func
return	model input size
static	False

C++ defination code:
image::Size input_size()

input_width

def input_width(self) -> int

Get model input width

item	description
type	func
return	model input size of width
static	False

C++ defination code:
int input_width()

input_height

def input_height(self) -> int

Get model input height

item	description
type	func
return	model input size of height
static	False

C++ defination code:
int input_height()

input_format

def input_format(self) -> maix.image.Format

Get input image format

item	description
type	func
return	input image format, image::Format type.
static	False

C++ defination code:
image::Format input_format()

draw_pose

def draw_pose(self, img: maix.image.Image, points: list[int], radius: int = 4, color: maix.image.Color = ..., colors: list[maix.image.Color] = [], body: bool = True, close: bool = False) -> None

Draw pose keypoints on image

item	description
type	func
param	img: image object, maix.image.Image type. points: keypoits, int list type, [x, y, x, y ...] radius: radius of points. color: color of points. colors: assign colors for points, list type, element is image.Color object. body: true, if points' length is 172 and body is ture, will draw lines as human body, if set to false won't draw lines, default true. close*: connect all points to close a polygon, default false.
static	False

C++ defination code:

void draw_pose(image::Image &img, std::vector<int> points, int radius = 4, image::Color color = image::COLOR_RED, const std::vector<image::Color> &colors = std::vector<image::Color>(), bool body = true, bool close = false)

draw_seg_mask

def draw_seg_mask(self, img: maix.image.Image, x: int, y: int, seg_mask: maix.image.Image, threshold: int = 127) -> None

Draw segmentation on image

item	description
type	func
param	img: image object, maix.image.Image type. seg_mask: segmentation mask image by detect method, a grayscale image threshold: only mask's value > threshold will be draw on image, value from 0 to 255.
static	False

C++ defination code:

void draw_seg_mask(image::Image &img, int x, int y, image::Image &seg_mask, int threshold = 127)

labels

Labels list

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<string> labels

label_path

Label file path

item	description
type	var
static	False
readonly	False

C++ defination code:
std::string label_path

mean

Get mean value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> mean

scale

Get scale value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> scale

Object

Object for detect result

C++ defination code:
class Object

init

def __init__(self, x: int = 0, y: int = 0, w: int = 0, h: int = 0, class_id: int = 0, score: float = 0, points: list[int] = [], angle: float = -9999) -> None

Constructor of Object for detect result

item	description
type	func
param	x: left top x y: left top y w: width h: height class_id: class id score: score
static	False

C++ defination code:

Object(int x = 0, int y = 0, int w = 0, int h = 0, int class_id = 0, float score = 0, std::vector<int> points = std::vector<int>(), float angle = -9999)

str

def __str__(self) -> str

Object info to string

item	description
type	func
return	Object info string
static	False

C++ defination code:
std::string to_str()

get_obb_points

def get_obb_points(self) -> list[int]

Get OBB(oriented bounding box) points, auto calculated according to x,y,w,h,angle

item	description
type	func
static	False

C++ defination code:
std::vector<int> get_obb_points()

x

Object left top coordinate x

item	description
type	var
static	False
readonly	False

C++ defination code:
int x

y

Object left top coordinate y

item	description
type	var
static	False
readonly	False

C++ defination code:
int y

w

Object width

item	description
type	var
static	False
readonly	False

C++ defination code:
int w

h

Object height

item	description
type	var
static	False
readonly	False

C++ defination code:
int h

class_id

Object class id

item	description
type	var
static	False
readonly	False

C++ defination code:
int class_id

score

Object score

item	description
type	var
static	False
readonly	False

C++ defination code:
float score

points

keypoints

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<int> points

angle

Rotate angle, -9999 means not set, value is a percentage, need to multiply 180 to get the real angle or multiply PI to get the radian.

item	description
type	var
static	False
readonly	False

C++ defination code:
float angle

seg_mask

segmentation mask, uint8 list type, shape is h * w but flattened to one dimension, value fron 0 to 255.

item	description
type	var
attention	For efficiency, it's a pointer in C++, use this carefully!
static	False
readonly	False

C++ defination code:
image::Image *seg_mask

ObjectFloat

Object for detect result

C++ defination code:
class ObjectFloat

init

def __init__(self, x: float = 0, y: float = 0, w: float = 0, h: float = 0, class_id: float = 0, score: float = 0, points: list[float] = [], angle: float = -1) -> None

Constructor of Object for detect result

item	description
type	func
param	x: left top x y: left top y w: width h: height class_id: class id score: score
static	False

C++ defination code:

ObjectFloat(float x = 0, float y = 0, float w = 0, float h = 0, float class_id = 0, float score = 0, std::vector<float> points = std::vector<float>(), float angle = -1)

str

def __str__(self) -> str

Object info to string

item	description
type	func
return	Object info string
static	False

C++ defination code:
std::string to_str()

x

Object left top coordinate x

item	description
type	var
static	False
readonly	False

C++ defination code:
float x

y

Object left top coordinate y

item	description
type	var
static	False
readonly	False

C++ defination code:
float y

w

Object width

item	description
type	var
static	False
readonly	False

C++ defination code:
float w

h

Object height

item	description
type	var
static	False
readonly	False

C++ defination code:
float h

class_id

Object class id

item	description
type	var
static	False
readonly	False

C++ defination code:
float class_id

score

Object score

item	description
type	var
static	False
readonly	False

C++ defination code:
float score

points

keypoints

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> points

angle

Rotate angle

item	description
type	var
static	False
readonly	False

C++ defination code:
float angle

Objects

Objects Class for detect result

C++ defination code:
class Objects

init

def __init__(self) -> None

Constructor of Objects class

item	description
type	func
static	False

C++ defination code:
Objects()

add

def add(self, x: int = 0, y: int = 0, w: int = 0, h: int = 0, class_id: int = 0, score: float = 0, points: list[int] = [], angle: float = -1) -> Object

Add object to objects

item	description
type	func
throw	Throw exception if no memory
static	False

C++ defination code:

nn::Object &add(int x = 0, int y = 0, int w = 0, int h = 0, int class_id = 0, float score = 0, std::vector<int> points = std::vector<int>(), float angle = -1)

remove

def remove(self, idx: int) -> maix.err.Err

Remove object form objects

item	description
type	func
static	False

C++ defination code:
err::Err remove(int idx)

at

def at(self, idx: int) -> Object

Get object item

item	description
type	func
static	False

C++ defination code:
nn::Object &at(int idx)

getitem

def __getitem__(self, idx: int) -> Object

Get object item

item	description
type	func
static	False

C++ defination code:
nn::Object &operator[](int idx)

len

def __len__(self) -> int

Get size

item	description
type	func
static	False

C++ defination code:
size_t size()

iter

def __iter__(self) -> typing.Iterator

Begin

item	description
type	func
static	False

C++ defination code:
std::vector<Object*>::iterator begin()

MUD

MUD(model universal describe file) class

C++ defination code:
class MUD

init

def __init__(self, model_path: str = '') -> None

MUD constructor

item	description
type	func
param	model_path: direction [in], model file path, model format can be MUD(model universal describe file) file. If model_path set, will load model from file, load failed will raise err.Exception. If model_path not set, you can load model later by load function.
static	False

C++ defination code:
MUD(const std::string &model_path = "")

load

def load(self, model_path: str) -> maix.err.Err

Load model from file

item	description
type	func
param	model_path: direction [in], model file path, model format can be MUD(model universal describe file) file.
return	error code, if load success, return err::ERR_NONE
static	False

C++ defination code:

err::Err load(const std::string &model_path)

type

Model type, string type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::string type

items

Model config items, different model type has different config items

item	description
type	var
static	False
readonly	False

C++ defination code:

std::map<std::string, std::map<std::string, std::string>> items

model_path

Model path

item	description
type	var
static	False
readonly	False

C++ defination code:
std::string model_path

parse_labels

def parse_labels(self, key: str = 'labels') -> list[str]

Please load() first, parse labels in items["extra"]["labels"],\nif items["extra"]["labels"] is a file path: will parse file, every one line is a label;\nif items["extra"]["labels"] is a string, labels split by comma(",").\nExecute this method will replace items["extra"]["labels"];

item	description
type	func
param	key: parse from items[key], default "labels".
return	parsed labels list.
static	False

C++ defination code:

std::vector<std::string> parse_labels(const std::string key = "labels")

LayerInfo

NN model layer info

C++ defination code:
class LayerInfo

init

def __init__(self, name: str = '', dtype: maix.tensor.DType = ..., shape: list[int] = []) -> None

LayerInfo constructor

item	description
type	func
param	name: direction [in], layer name dtype: direction [in], layer data type shape: direction [in], layer shape
static	False

C++ defination code:

LayerInfo(const std::string &name =  "", tensor::DType dtype = tensor::DType::FLOAT32, std::vector<int> shape = std::vector<int>())

name

Layer name

item	description
type	var
static	False
readonly	False

C++ defination code:
std::string   name

dtype

Layer data type

item	description
type	var
attention	If model is quantized, this is the real quantized data type like int8 float16, in most scene, inputs and outputs we actually use float32 in API like forward.
static	False
readonly	False

C++ defination code:
tensor::DType dtype

shape

Layer shape

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<int> shape

shape_int

def shape_int(self) -> int

Shape as one int type, multiply all dims of shape

item	description
type	func
static	False

C++ defination code:
int shape_int()

to_str

def to_str(self) -> str

To string

item	description
type	func
static	False

C++ defination code:
std::string to_str()

str

def __str__(self) -> str

To string

item	description
type	func
static	False

C++ defination code:
std::string __str__()

NN

Neural network class

C++ defination code:
class NN

init

def __init__(self, model: str = '', dual_buff: bool = False) -> None

Neural network constructor

item	description
type	func
param	model: direction [in], model file path, model format can be MUD(model universal describe file) file. If model_path set, will load model from file, load failed will raise err.Exception. If model_path not set, you can load model later by load function. dual_buff: direction [in], prepare dual input output buffer to accelarate forward, that is, when NPU is forwarding we not wait and prepare the next input buff. If you want to ensure every time forward output the input's result, set this arg to false please. Default false to ensure easy use.
static	False

C++ defination code:

NN(const std::string &model = "", bool dual_buff = false)

load

def load(self, model: str) -> maix.err.Err

Load model from file

item	description
type	func
param	model: direction [in], model file path, model format can be MUD(model universal describe file) file.
return	error code, if load success, return err::ERR_NONE
static	False

C++ defination code:
err::Err load(const std::string &model)

loaded

def loaded(self) -> bool

Is model loaded

item	description
type	func
return	true if model loaded, else false
static	False

C++ defination code:
bool loaded()

set_dual_buff

def set_dual_buff(self, enable: bool) -> None

Enable dual buff or disable dual buff

item	description
type	func
param	enable: true to enable, false to disable
static	False

C++ defination code:
void set_dual_buff(bool enable)

inputs_info

def inputs_info(self) -> list[LayerInfo]

Get model input layer info

item	description
type	func
return	input layer info
static	False

C++ defination code:
std::vector<nn::LayerInfo> inputs_info()

outputs_info

def outputs_info(self) -> list[LayerInfo]

Get model output layer info

item	description
type	func
return	output layer info
static	False

C++ defination code:
std::vector<nn::LayerInfo> outputs_info()

extra_info

def extra_info(self) -> dict[str, str]

Get model extra info define in MUD file

item	description
type	func
return	extra info, dict type, key-value object, attention: key and value are all string type.
static	False

C++ defination code:

std::map<std::string, std::string> extra_info()

extra_info_labels

def extra_info_labels(self) -> list[str]

Get model parsed extra info labels define in MUD file

item	description
type	func
return	labels list in extra info, string list type.
static	False

C++ defination code:

std::vector<std::string> extra_info_labels()

forward

def forward(self, inputs: maix.tensor.Tensors, copy_result: bool = True, dual_buff_wait: bool = False) -> maix.tensor.Tensors

forward run model, get output of model,\nthis is specially for MaixPy, not efficient, but easy to use in MaixPy

item	description
type	func
param	input: direction [in], input tensor copy_result: If set true, will copy result to a new variable; else will use a internal memory, you can only use it until to the next forward. Default true to avoid problems, you can set it to false manually to make speed faster. dual_buff_wait: bool type, only for dual_buff mode, if true, will inference this image and wait for result, default false.
return	output tensor. In C++, you should manually delete tensors in return value and return value. If dual_buff mode, it can be NULL(None in MaixPy) means not ready.
throw	if error ocurrs like no memory or arg error, will raise err.Exception.
static	False

C++ defination code:

tensor::Tensors *forward(tensor::Tensors &inputs, bool copy_result = true, bool dual_buff_wait = false)

forward_image

def forward_image(self, img: maix.image.Image, mean: list[float] = [], scale: list[float] = [], fit: maix.image.Fit = ..., copy_result: bool = True, dual_buff_wait: bool = False) -> maix.tensor.Tensors

forward model, param is image

item	description
type	func
param	img: input image mean: mean value, a list type, e.g. [0.485, 0.456, 0.406], default is empty list means not normalize. scale: scale value, a list type, e.g. [1/0.229, 1/0.224, 1/0.225], default is empty list means not normalize. fit: fit mode, if the image size of input not equal to model's input, it will auto resize use this fit method, default is image.Fit.FIT_FILL for easy coordinate calculation, but for more accurate result, use image.Fit.FIT_CONTAIN is better. copy_result: If set true, will copy result to a new variable; else will use a internal memory, you can only use it until to the next forward. Default true to avoid problems, you can set it to false manually to make speed faster. dual_buff_wait: bool type, only for dual_buff mode, if true, will inference this image and wait for result, default false.
return	output tensor. In C++, you should manually delete tensors in return value and return value. If dual_buff mode, it can be NULL(None in MaixPy) means not ready.
throw	If error occurs, like arg error or alloc memory failed, will raise err.Exception.
static	False

C++ defination code:

tensor::Tensors *forward_image(image::Image &img, std::vector<float> mean = std::vector<float>(), std::vector<float> scale = std::vector<float>(), image::Fit fit = image::Fit::FIT_FILL, bool copy_result = true, bool dual_buff_wait = false)

FaceObject

Face object

C++ defination code:
class FaceObject

init

def __init__(self, x: int = 0, y: int = 0, w: int = 0, h: int = 0, class_id: int = 0, score: float = 0, points: list[int] = [], feature: list[float] = [], face: maix.image.Image = ...) -> None

Constructor

item	description
type	func
static	False

C++ defination code:

FaceObject(int x = 0, int y = 0, int w = 0, int h = 0, int class_id = 0, float score = 0, std::vector<int> points = std::vector<int>(), std::vector<float> feature = std::vector<float>(), image::Image face = image::Image())

str

def __str__(self) -> str

FaceObject info to string

item	description
type	func
return	FaceObject info string
static	False

C++ defination code:
std::string to_str()

x

FaceObject left top coordinate x

item	description
type	var
static	False
readonly	False

C++ defination code:
int x

y

FaceObject left top coordinate y

item	description
type	var
static	False
readonly	False

C++ defination code:
int y

w

FaceObject width

item	description
type	var
static	False
readonly	False

C++ defination code:
int w

h

FaceObject height

item	description
type	var
static	False
readonly	False

C++ defination code:
int h

class_id

FaceObject class id

item	description
type	var
static	False
readonly	False

C++ defination code:
int class_id

score

FaceObject score

item	description
type	var
static	False
readonly	False

C++ defination code:
float score

points

keypoints

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<int> points

feature

feature, float list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> feature

face

face image

item	description
type	var
static	False
readonly	False

C++ defination code:
image::Image face

FaceObjects

Objects Class for detect result

C++ defination code:
class FaceObjects

init

def __init__(self) -> None

Constructor of FaceObjects class

item	description
type	func
static	False

C++ defination code:
FaceObjects()

add

def add(self, x: int = 0, y: int = 0, w: int = 0, h: int = 0, class_id: int = 0, score: float = 0, points: list[int] = [], feature: list[float] = [], face: maix.image.Image = ...) -> FaceObject

Add object to FaceObjects

item	description
type	func
throw	Throw exception if no memory
static	False

C++ defination code:

nn::FaceObject &add(int x = 0, int y = 0, int w = 0, int h = 0, int class_id = 0, float score = 0, std::vector<int> points = std::vector<int>(), std::vector<float> feature = std::vector<float>(), image::Image face = image::Image())

remove

def remove(self, idx: int) -> maix.err.Err

Remove object form FaceObjects

item	description
type	func
static	False

C++ defination code:
err::Err remove(int idx)

at

def at(self, idx: int) -> FaceObject

Get object item

item	description
type	func
static	False

C++ defination code:
nn::FaceObject &at(int idx)

getitem

def __getitem__(self, idx: int) -> FaceObject

Get object item

item	description
type	func
static	False

C++ defination code:
nn::FaceObject &operator[](int idx)

len

def __len__(self) -> int

Get size

item	description
type	func
static	False

C++ defination code:
size_t size()

iter

def __iter__(self) -> typing.Iterator

Begin

item	description
type	func
static	False

C++ defination code:
std::vector<FaceObject*>::iterator begin()

FaceRecognizer

FaceRecognizer class

C++ defination code:
class FaceRecognizer

init

def __init__(self, detect_model: str = '', feature_model: str = '', dual_buff: bool = True) -> None

Constructor of FaceRecognizer class

item	description
type	func
param	detect_model: face detect model path, default empty, you can load model later by load function. feature_model: feature extract model dual_buff: direction [in], prepare dual input output buffer to accelarate forward, that is, when NPU is forwarding we not wait and prepare the next input buff. If you want to ensure every time forward output the input's result, set this arg to false please. Default true to ensure speed.
throw	If model arg is not empty and load failed, will throw err::Exception.
static	False

C++ defination code:

FaceRecognizer(const string &detect_model = "", const string &feature_model = "", bool dual_buff = true)

load

def load(self, detect_model: str, feature_model: str) -> maix.err.Err

Load model from file

item	description
type	func
param	detect_model: face detect model path, default empty, you can load model later by load function. feature_model: feature extract model
return	err::Err
static	False

C++ defination code:

err::Err load(const string &detect_model, const string &feature_model)

recognize

def recognize(self, img: maix.image.Image, conf_th: float = 0.5, iou_th: float = 0.45, compare_th: float = 0.8, get_feature: bool = False, get_face: bool = False, fit: maix.image.Fit = ...) -> FaceObjects

Detect objects from image

item	description
type	func
param	img: Image want to detect, if image's size not match model input's, will auto resize with fit method. conf_th: Detect confidence threshold, default 0.5. iou_th: Detect IoU threshold, default 0.45. compare_th: Compare two face score threshold, default 0.8, if two faces' score < this value, will see this face fas unknown. get_feature: return feature or not, if true will copy features to result, if false will not copy feature to result to save time and memory. get_face: return face image or not, if true result object's face attribute will valid, or face sttribute is empty. Get face image will alloc memory and copy image, so will lead to slower speed. fit: Resize method, default image.Fit.FIT_CONTAIN.
throw	If image format not match model input format, will throw err::Exception.
return	FaceObjects object. In C++, you should delete it after use.
static	False

C++ defination code:

nn::FaceObjects *recognize(image::Image &img, float conf_th = 0.5, float iou_th = 0.45, float compare_th = 0.8, bool get_feature = false, bool get_face = false, maix::image::Fit fit = maix::image::FIT_CONTAIN)

add_face

def add_face(self, face: FaceObject, label: str) -> maix.err.Err

Add face to lib

item	description
type	func
param	face: face object, find by recognize label: face label(name)
static	False

C++ defination code:

err::Err add_face(nn::FaceObject *face, const std::string &label)

remove_face

def remove_face(self, idx: int = -1, label: str = '') -> maix.err.Err

remove face from lib

item	description
type	func
param	idx: index of face in lib, default -1 means use label, value [0,face_num), idx and label must have one, idx have high priotiry. label: which face to remove, default to empty string mean use idx, idx and label must have one, idx have high priotiry.
static	False

C++ defination code:

err::Err remove_face(int idx = -1, const std::string &label = "")

save_faces

def save_faces(self, path: str) -> maix.err.Err

Save faces info to a file

item	description
type	func
param	path: where to save, string type.
return	err.Err type
static	False

C++ defination code:

err::Err save_faces(const std::string &path)

load_faces

def load_faces(self, path: str) -> maix.err.Err

Load faces info from a file

item	description
type	func
param	path: from where to load, string type.
return	err::Err type
static	False

C++ defination code:

err::Err load_faces(const std::string &path)

input_size

def input_size(self) -> maix.image.Size

Get model input size

item	description
type	func
return	model input size
static	False

C++ defination code:
image::Size input_size()

input_width

def input_width(self) -> int

Get model input width

item	description
type	func
return	model input size of width
static	False

C++ defination code:
int input_width()

input_height

def input_height(self) -> int

Get model input height

item	description
type	func
return	model input size of height
static	False

C++ defination code:
int input_height()

input_format

def input_format(self) -> maix.image.Format

Get input image format

item	description
type	func
return	input image format, image::Format type.
static	False

C++ defination code:
image::Format input_format()

mean_detector

Get mean value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> mean_detector

scale_detector

Get scale value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> scale_detector

mean_feature

Get mean value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> mean_feature

scale_feature

Get scale value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> scale_feature

labels

labels, list type, first is "unknown"

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<std::string> labels

features

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<std::vector<float>> features

SelfLearnClassifier

C++ defination code:
class SelfLearnClassifier

init

def __init__(self, model: str = '', dual_buff: bool = True) -> None

Construct a new SelfLearnClassifier object

item	description
type	func
param	model: MUD model path, if empty, will not load model, you can call load_model() later. if not empty, will load model and will raise err::Exception if load failed. dual_buff: direction [in], prepare dual input output buffer to accelarate forward, that is, when NPU is forwarding we not wait and prepare the next input buff. If you want to ensure every time forward output the input's result, set this arg to false please. Default true to ensure speed.
static	False

C++ defination code:

SelfLearnClassifier(const std::string &model = "", bool dual_buff = true)

load_model

def load_model(self, model: str) -> maix.err.Err

Load model from file, model format is .mud,\nMUD file should contain [extra] section, have key-values:\n- model_type: classifier_no_top\n- input_type: rgb or bgr\n- mean: 123.675, 116.28, 103.53\n- scale: 0.017124753831663668, 0.01750700280112045, 0.017429193899782137

item	description
type	func
param	model: MUD model path
return	error code, if load failed, return error code
static	False

C++ defination code:
err::Err load_model(const string &model)

classify

def classify(self, img: maix.image.Image, fit: maix.image.Fit = ...) -> list[tuple[int, float]]

Classify image

item	description
type	func
param	img: image, format should match model input_type， or will raise err.Exception fit: image resize fit mode, default Fit.FIT_COVER, see image.Fit.
throw	If error occurred, will raise err::Exception, you can find reason in log, mostly caused by args error or hardware error.
return	result, a list of (idx, distance), smaller distance means more similar. In C++, you need to delete it after use.
static	False

C++ defination code:

std::vector<std::pair<int, float>> *classify(image::Image &img, image::Fit fit = image::FIT_COVER)

add_class

def add_class(self, img: maix.image.Image, fit: maix.image.Fit = ...) -> None

Add a class to recognize

item	description
type	func
param	img: Add a image as a new class fit: image resize fit mode, default Fit.FIT_COVER, see image.Fit.
static	False

C++ defination code:

void add_class(image::Image &img, image::Fit fit = image::FIT_COVER)

class_num

def class_num(self) -> int

Get class number

item	description
type	func
static	False

C++ defination code:
int class_num()

rm_class

def rm_class(self, idx: int) -> maix.err.Err

Remove a class

item	description
type	func
param	idx: index, value from 0 to class_num();
static	False

C++ defination code:
err::Err rm_class(int idx)

add_sample

def add_sample(self, img: maix.image.Image, fit: maix.image.Fit = ...) -> None

Add sample, you should call learn method after add some samples to learn classes.\nSample image can be any of classes we already added.

item	description
type	func
param	img: Add a image as a new sample.
static	False

C++ defination code:

void add_sample(image::Image &img, image::Fit fit = image::FIT_COVER)

rm_sample

def rm_sample(self, idx: int) -> maix.err.Err

Remove a sample

item	description
type	func
param	idx: index, value from 0 to sample_num();
static	False

C++ defination code:
err::Err rm_sample(int idx)

sample_num

def sample_num(self) -> int

Get sample number

item	description
type	func
static	False

C++ defination code:
int sample_num()

learn

def learn(self) -> int

Start auto learn class features from classes image and samples.\nYou should call this method after you add some samples.

item	description
type	func
return	learn epoch(times), 0 means learn nothing.
static	False

C++ defination code:
int learn()

clear

def clear(self) -> None

Clear all class and samples

item	description
type	func
static	False

C++ defination code:
void clear()

input_size

def input_size(self) -> maix.image.Size

Get model input size, only for image input

item	description
type	func
return	model input size
static	False

C++ defination code:
image::Size input_size()

input_width

def input_width(self) -> int

Get model input width, only for image input

item	description
type	func
return	model input size of width
static	False

C++ defination code:
int input_width()

input_height

def input_height(self) -> int

Get model input height, only for image input

item	description
type	func
return	model input size of height
static	False

C++ defination code:
int input_height()

input_format

def input_format(self) -> maix.image.Format

Get input image format, only for image input

item	description
type	func
return	input image format, image::Format type.
static	False

C++ defination code:
image::Format input_format()

input_shape

def input_shape(self) -> list[int]

Get input shape, if have multiple input, only return first input shape

item	description
type	func
return	input shape, list type
static	False

C++ defination code:
std::vector<int> input_shape()

save

def save(self, path: str, labels: list[str] = []) -> maix.err.Err

Save features and labels to a binary file

item	description
type	func
param	path: file path to save, e.g. /root/my_classes.bin labels: class labels, can be None, or length must equal to class num, or will return err::Err
return	maix.err.Err if labels exists but length not equal to class num, or save file failed, or class num is 0.
static	False

C++ defination code:

err::Err save(const std::string &path, const std::vector<std::string> &labels = std::vector<std::string>())

load

def load(self, path: str) -> list[str]

Load features info from binary file

item	description
type	func
param	path: feature info binary file path, e.g. /root/my_classes.bin
static	False

C++ defination code:

std::vector<std::string> load(const std::string &path)

labels

Labels list

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<string> labels

label_path

Label file path

item	description
type	var
static	False
readonly	False

C++ defination code:
std::string label_path

mean

Get mean value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> mean

scale

Get scale value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> scale

YOLOv5

YOLOv5 class

C++ defination code:
class YOLOv5

init

def __init__(self, model: str = '', dual_buff: bool = True) -> None

Constructor of YOLOv5 class

item	description
type	func
param	model: model path, default empty, you can load model later by load function. dual_buff: direction [in], prepare dual input output buffer to accelarate forward, that is, when NPU is forwarding we not wait and prepare the next input buff. If you want to ensure every time forward output the input's result, set this arg to false please. Default true to ensure speed.
throw	If model arg is not empty and load failed, will throw err::Exception.
static	False

C++ defination code:

YOLOv5(const string &model = "", bool dual_buff = true)

load

def load(self, model: str) -> maix.err.Err

Load model from file

item	description
type	func
param	model: Model path want to load
return	err::Err
static	False

C++ defination code:
err::Err load(const string &model)

detect

def detect(self, img: maix.image.Image, conf_th: float = 0.5, iou_th: float = 0.45, fit: maix.image.Fit = ...) -> list[Object]

Detect objects from image

item	description
type	func
param	img: Image want to detect, if image's size not match model input's, will auto resize with fit method. conf_th: Confidence threshold, default 0.5. iou_th: IoU threshold, default 0.45. fit: Resize method, default image.Fit.FIT_CONTAIN.
throw	If image format not match model input format, will throw err::Exception.
return	Object list. In C++, you should delete it after use.
static	False

C++ defination code:

std::vector<nn::Object> *detect(image::Image &img, float conf_th = 0.5, float iou_th = 0.45, maix::image::Fit fit = maix::image::FIT_CONTAIN)

input_size

def input_size(self) -> maix.image.Size

Get model input size

item	description
type	func
return	model input size
static	False

C++ defination code:
image::Size input_size()

input_width

def input_width(self) -> int

Get model input width

item	description
type	func
return	model input size of width
static	False

C++ defination code:
int input_width()

input_height

def input_height(self) -> int

Get model input height

item	description
type	func
return	model input size of height
static	False

C++ defination code:
int input_height()

input_format

def input_format(self) -> maix.image.Format

Get input image format

item	description
type	func
return	input image format, image::Format type.
static	False

C++ defination code:
image::Format input_format()

labels

Labels list

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<string> labels

label_path

Label file path

item	description
type	var
static	False
readonly	False

C++ defination code:
std::string label_path

mean

Get mean value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> mean

scale

Get scale value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> scale

anchors

Get anchors

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> anchors

YOLO11

YOLO11 class

C++ defination code:
class YOLO11

init

def __init__(self, model: str = '', dual_buff: bool = True) -> None

Constructor of YOLO11 class

item	description
type	func
param	model: model path, default empty, you can load model later by load function. dual_buff: direction [in], prepare dual input output buffer to accelarate forward, that is, when NPU is forwarding we not wait and prepare the next input buff. If you want to ensure every time forward output the input's result, set this arg to false please. Default true to ensure speed.
throw	If model arg is not empty and load failed, will throw err::Exception.
static	False

C++ defination code:

YOLO11(const string &model = "", bool dual_buff = true)

load

def load(self, model: str) -> maix.err.Err

Load model from file

item	description
type	func
param	model: Model path want to load
return	err::Err
static	False

C++ defination code:
err::Err load(const string &model)

detect

def detect(self, img: maix.image.Image, conf_th: float = 0.5, iou_th: float = 0.45, fit: maix.image.Fit = ..., keypoint_th: float = 0.5) -> Objects

Detect objects from image

item	description
type	func
param	img: Image want to detect, if image's size not match model input's, will auto resize with fit method. conf_th: Confidence threshold, default 0.5. iou_th: IoU threshold, default 0.45. fit: Resize method, default image.Fit.FIT_CONTAIN. keypoint_th: keypoint threshold, default 0.5, only for yolo11-pose model.
throw	If image format not match model input format, will throw err::Exception.
return	Object list. In C++, you should delete it after use. If model is yolo11-pose, object's points have value, and if points' value < 0 means that point is invalid(conf < keypoint_th).
static	False

C++ defination code:

nn::Objects *detect(image::Image &img, float conf_th = 0.5, float iou_th = 0.45, maix::image::Fit fit = maix::image::FIT_CONTAIN, float keypoint_th = 0.5)

input_size

def input_size(self) -> maix.image.Size

Get model input size

item	description
type	func
return	model input size
static	False

C++ defination code:
image::Size input_size()

input_width

def input_width(self) -> int

Get model input width

item	description
type	func
return	model input size of width
static	False

C++ defination code:
int input_width()

input_height

def input_height(self) -> int

Get model input height

item	description
type	func
return	model input size of height
static	False

C++ defination code:
int input_height()

input_format

def input_format(self) -> maix.image.Format

Get input image format

item	description
type	func
return	input image format, image::Format type.
static	False

C++ defination code:
image::Format input_format()

draw_pose

def draw_pose(self, img: maix.image.Image, points: list[int], radius: int = 4, color: maix.image.Color = ..., colors: list[maix.image.Color] = [], body: bool = True, close: bool = False) -> None

Draw pose keypoints on image

item	description
type	func
param	img: image object, maix.image.Image type. points: keypoits, int list type, [x, y, x, y ...] radius: radius of points. color: color of points. colors: assign colors for points, list type, element is image.Color object. body: true, if points' length is 172 and body is ture, will draw lines as human body, if set to false won't draw lines, default true. close*: connect all points to close a polygon, default false.
static	False

C++ defination code:

void draw_pose(image::Image &img, std::vector<int> points, int radius = 4, image::Color color = image::COLOR_RED, const std::vector<image::Color> &colors = std::vector<image::Color>(), bool body = true, bool close = false)

draw_seg_mask

def draw_seg_mask(self, img: maix.image.Image, x: int, y: int, seg_mask: maix.image.Image, threshold: int = 127) -> None

Draw segmentation on image

item	description
type	func
param	img: image object, maix.image.Image type. seg_mask: segmentation mask image by detect method, a grayscale image threshold: only mask's value > threshold will be draw on image, value from 0 to 255.
static	False

C++ defination code:

void draw_seg_mask(image::Image &img, int x, int y, image::Image &seg_mask, int threshold = 127)

labels

Labels list

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<string> labels

label_path

Label file path

item	description
type	var
static	False
readonly	False

C++ defination code:
std::string label_path

mean

Get mean value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> mean

scale

Get scale value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> scale

Classifier

C++ defination code:
class Classifier

init

def __init__(self, model: str = '', dual_buff: bool = True) -> None

Construct a new Classifier object

item	description
type	func
param	model: MUD model path, if empty, will not load model, you can call load() later. if not empty, will load model and will raise err::Exception if load failed. dual_buff: direction [in], prepare dual input output buffer to accelarate forward, that is, when NPU is forwarding we not wait and prepare the next input buff. If you want to ensure every time forward output the input's result, set this arg to false please. Default true to ensure speed.
static	False

C++ defination code:

Classifier(const string &model = "", bool dual_buff = true)

load

def load(self, model: str) -> maix.err.Err

Load model from file, model format is .mud,\nMUD file should contain [extra] section, have key-values:\n- model_type: classifier\n- input_type: rgb or bgr\n- mean: 123.675, 116.28, 103.53\n- scale: 0.017124753831663668, 0.01750700280112045, 0.017429193899782137\n- labels: imagenet_classes.txt

item	description
type	func
param	model: MUD model path
return	error code, if load failed, return error code
static	False

C++ defination code:
err::Err load(const string &model)

classify

def classify(self, img: maix.image.Image, softmax: bool = True, fit: maix.image.Fit = ...) -> list[tuple[int, float]]

Forward image to model, get result. Only for image input, use classify_raw for tensor input.

item	description
type	func
param	img: image, format should match model input_type， or will raise err.Exception softmax: if true, will do softmax to result, or will return raw value fit: image resize fit mode, default Fit.FIT_COVER, see image.Fit.
throw	If error occurred, will raise err::Exception, you can find reason in log, mostly caused by args error or hardware error.
return	result, a list of (label, score). If in dual_buff mode, value can be one element list and score is zero when not ready. In C++, you need to delete it after use.
static	False

C++ defination code:

std::vector<std::pair<int, float>> *classify(image::Image &img, bool softmax = true, image::Fit fit = image::FIT_COVER)

classify_raw

def classify_raw(self, data: maix.tensor.Tensor, softmax: bool = True) -> list[tuple[int, float]]

Forward tensor data to model, get result

item	description
type	func
param	data: tensor data, format should match model input_type， or will raise err.Excetion softmax: if true, will do softmax to result, or will return raw value
throw	If error occurred, will raise err::Exception, you can find reason in log, mostly caused by args error or hardware error.
return	result, a list of (label, score). In C++, you need to delete it after use.
static	False

C++ defination code:

std::vector<std::pair<int, float>> *classify_raw(tensor::Tensor &data, bool softmax = true)

input_size

def input_size(self) -> maix.image.Size

Get model input size, only for image input

item	description
type	func
return	model input size
static	False

C++ defination code:
image::Size input_size()

input_width

def input_width(self) -> int

Get model input width, only for image input

item	description
type	func
return	model input size of width
static	False

C++ defination code:
int input_width()

input_height

def input_height(self) -> int

Get model input height, only for image input

item	description
type	func
return	model input size of height
static	False

C++ defination code:
int input_height()

input_format

def input_format(self) -> maix.image.Format

Get input image format, only for image input

item	description
type	func
return	input image format, image::Format type.
static	False

C++ defination code:
image::Format input_format()

input_shape

def input_shape(self) -> list[int]

Get input shape, if have multiple input, only return first input shape

item	description
type	func
return	input shape, list type
static	False

C++ defination code:
std::vector<int> input_shape()

labels

Labels list

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<string> labels

label_path

Label file path

item	description
type	var
static	False
readonly	False

C++ defination code:
std::string label_path

mean

Get mean value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> mean

scale

Get scale value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> scale

Retinaface

Retinaface class

C++ defination code:
class Retinaface

init

def __init__(self, model: str = '', dual_buff: bool = True) -> None

Constructor of Retinaface class

item	description
type	func
param	model: model path, default empty, you can load model later by load function. dual_buff: direction [in], prepare dual input output buffer to accelarate forward, that is, when NPU is forwarding we not wait and prepare the next input buff. If you want to ensure every time forward output the input's result, set this arg to false please. Default true to ensure speed.
throw	If model arg is not empty and load failed, will throw err::Exception.
static	False

C++ defination code:

Retinaface(const string &model = "", bool dual_buff = true)

load

def load(self, model: str) -> maix.err.Err

Load model from file

item	description
type	func
param	model: Model path want to load
return	err::Err
static	False

C++ defination code:
err::Err load(const string &model)

detect

def detect(self, img: maix.image.Image, conf_th: float = 0.4, iou_th: float = 0.45, fit: maix.image.Fit = ...) -> list[Object]

Detect objects from image

item	description
type	func
param	img: Image want to detect, if image's size not match model input's, will auto resize with fit method. conf_th: Confidence threshold, default 0.4. iou_th: IoU threshold, default 0.45. fit: Resize method, default image.Fit.FIT_CONTAIN.
throw	If image format not match model input format, will throw err::Exception.
return	Object list. In C++, you should delete it after use.
static	False

C++ defination code:

std::vector<nn::Object> *detect(image::Image &img, float conf_th = 0.4, float iou_th = 0.45, maix::image::Fit fit = maix::image::FIT_CONTAIN)

input_size

def input_size(self) -> maix.image.Size

Get model input size

item	description
type	func
return	model input size
static	False

C++ defination code:
image::Size input_size()

input_width

def input_width(self) -> int

Get model input width

item	description
type	func
return	model input size of width
static	False

C++ defination code:
int input_width()

input_height

def input_height(self) -> int

Get model input height

item	description
type	func
return	model input size of height
static	False

C++ defination code:
int input_height()

input_format

def input_format(self) -> maix.image.Format

Get input image format

item	description
type	func
return	input image format, image::Format type.
static	False

C++ defination code:
image::Format input_format()

mean

Get mean value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> mean

scale

Get scale value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> scale

FaceDetector

FaceDetector class

C++ defination code:
class FaceDetector

init

def __init__(self, model: str = '', dual_buff: bool = True) -> None

Constructor of FaceDetector class

item	description
type	func
param	model: model path, default empty, you can load model later by load function. dual_buff: direction [in], prepare dual input output buffer to accelarate forward, that is, when NPU is forwarding we not wait and prepare the next input buff. If you want to ensure every time forward output the input's result, set this arg to false please. Default true to ensure speed.
throw	If model arg is not empty and load failed, will throw err::Exception.
static	False

C++ defination code:

FaceDetector(const string &model = "", bool dual_buff = true)

load

def load(self, model: str) -> maix.err.Err

Load model from file

item	description
type	func
param	model: Model path want to load
return	err::Err
static	False

C++ defination code:
err::Err load(const string &model)

detect

def detect(self, img: maix.image.Image, conf_th: float = 0.5, iou_th: float = 0.45, fit: maix.image.Fit = ...) -> list[Object]

Detect objects from image

item	description
type	func
param	img: Image want to detect, if image's size not match model input's, will auto resize with fit method. conf_th: Confidence threshold, default 0.5. iou_th: IoU threshold, default 0.45. fit: Resize method, default image.Fit.FIT_CONTAIN.
throw	If image format not match model input format, will throw err::Exception.
return	Object list. In C++, you should delete it after use.
static	False

C++ defination code:

std::vector<nn::Object> *detect(image::Image &img, float conf_th = 0.5, float iou_th = 0.45, maix::image::Fit fit = maix::image::FIT_CONTAIN)

input_size

def input_size(self) -> maix.image.Size

Get model input size

item	description
type	func
return	model input size
static	False

C++ defination code:
image::Size input_size()

input_width

def input_width(self) -> int

Get model input width

item	description
type	func
return	model input size of width
static	False

C++ defination code:
int input_width()

input_height

def input_height(self) -> int

Get model input height

item	description
type	func
return	model input size of height
static	False

C++ defination code:
int input_height()

input_format

def input_format(self) -> maix.image.Format

Get input image format

item	description
type	func
return	input image format, image::Format type.
static	False

C++ defination code:
image::Format input_format()

mean

Get mean value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> mean

scale

Get scale value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> scale

PP_OCR

PP_OCR class

C++ defination code:
class PP_OCR

init

def __init__(self, model: str = '') -> None

Constructor of PP_OCR class

item	description
type	func
param	model: model path, default empty, you can load model later by load function.
throw	If model arg is not empty and load failed, will throw err::Exception.
static	False

C++ defination code:
PP_OCR(const string &model = "")

load

def load(self, model: str) -> maix.err.Err

Load model from file

item	description
type	func
param	model: Model path want to load
return	err::Err
static	False

C++ defination code:
err::Err load(const string &model)

detect

def detect(self, img: maix.image.Image, thresh: float = 0.3, box_thresh: float = 0.6, fit: maix.image.Fit = ..., char_box: bool = False) -> OCR_Objects

Detect objects from image

item	description
type	func
param	img: Image want to detect, if image's size not match model input's, will auto resize with fit method. thresh: Confidence threshold where pixels have charactor, default 0.3. box_thresh: Box threshold, the box prob higher than this value will be valid, default 0.6. fit: Resize method, default image.Fit.FIT_CONTAIN. char_box: Calculate every charactor's box, default false, if true then you can get charactor's box by nn.OCR_Object's char_boxes attribute.
throw	If image format not match model input format or no memory, will throw err::Exception.
return	nn.OCR_Objects type. In C++, you should delete it after use.
static	False

C++ defination code:

nn::OCR_Objects *detect(image::Image &img, float thresh = 0.3, float box_thresh = 0.6, maix::image::Fit fit = maix::image::FIT_CONTAIN, bool char_box = false)

recognize

def recognize(self, img: maix.image.Image, box_points: list[int] = []) -> OCR_Object

Only recognize, not detect

item	description
type	func
param	img: image to recognize chractors, can be a stanrd cropped charactors image, if crop image not standard, you can use box_points to assgin where the charactors' 4 corner is. box_points: list type, length must be 8 or 0, default empty means not transfer image to standard image. 4 points postiion, format: [x1, y1, x2, y2, x3, y3, x4, y4], point 1 at the left-top, point 2 right-top... char_box: Calculate every charactor's box, default false, if true then you can get charactor's box by nn.OCR_Object's char_boxes attribute.
static	False

C++ defination code:

nn::OCR_Object *recognize(image::Image &img, const std::vector<int> &box_points = std::vector<int>())

draw_seg_mask

def draw_seg_mask(self, img: maix.image.Image, x: int, y: int, seg_mask: maix.image.Image, threshold: int = 127) -> None

Draw segmentation on image

item	description
type	func
param	img: image object, maix.image.Image type. seg_mask: segmentation mask image by detect method, a grayscale image threshold: only mask's value > threshold will be draw on image, value from 0 to 255.
static	False

C++ defination code:

void draw_seg_mask(image::Image &img, int x, int y, image::Image &seg_mask, int threshold = 127)

input_size

def input_size(self) -> maix.image.Size

Get model input size

item	description
type	func
return	model input size
static	False

C++ defination code:
image::Size input_size()

input_width

def input_width(self) -> int

Get model input width

item	description
type	func
return	model input size of width
static	False

C++ defination code:
int input_width()

input_height

def input_height(self) -> int

Get model input height

item	description
type	func
return	model input size of height
static	False

C++ defination code:
int input_height()

input_format

def input_format(self) -> maix.image.Format

Get input image format

item	description
type	func
return	input image format, image::Format type.
static	False

C++ defination code:
image::Format input_format()

mean

Get mean value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> mean

scale

Get scale value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> scale

rec_mean

Get mean value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> rec_mean

rec_scale

Get scale value, list type

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<float> rec_scale

labels

labels (charactors)

item	description
type	var
static	False
readonly	False

C++ defination code:
std::vector<std::string> labels

det

model have detect model

item	description
type	var
static	False
readonly	False

C++ defination code:
bool det

rec

model have recognize model

item	description
type	var
static	False
readonly	False

C++ defination code:
bool rec

timer

MaixPy

maix.nn

Module

Enum

SpeechDevice

SpeechDecoder

Variable

Function

Class

NanoTrack

__init__

load

init

track

input_size

input_width

input_height

input_format

mean

scale

OCR_Box

__init__

x1

y1

x2

y2

x3

y3

x4

y4

to_list

OCR_Object

__init__

box

score

idx_list

char_pos

char_str

char_list

update_chars

__str__

OCR_Objects

__init__

add

remove

at

__getitem__

__len__

__iter__

Speech

__init__

__init__ (overload 1)

load

load (overload 1)

init

init (overload 1)

devive

devive (overload 1)

dec_deinit

dec_deinit (overload 1)

raw

raw (overload 1)

raw (overload 2)

raw (overload 3)

digit

digit (overload 1)

digit (overload 2)

digit (overload 3)

kws

kws (overload 1)

kws (overload 2)

kws (overload 3)

lvcsr

lvcsr (overload 1)

lvcsr (overload 2)

lvcsr (overload 3)

run

run (overload 1)

clear

clear (overload 1)

init

init

init

str

init

getitem

len

iter

init

init (overload 1)

init

init

str

init

str

init

getitem

len

iter

init

init