Python on Maix-III AXera-Pi
Date | Author | Change |
---|---|---|
2023.02.04 | wonder | Create file |
2023.03.01 | wonder | Add Yolo8 model |
Finish reading AXera-Pi Guide, we can start Python on MAIX-III AXera-Pi.
What is Python
Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation.It is dynamically typed and garbage-collected. It supports multiple programming paradigms, including structured (particularly procedural), object-oriented and functional programming. It is often described as a "batteries included" language due to its comprehensive standard library. wikipedia
Difference between Python and C++
Python is a kind of Scripting language, this means it is interpreted at runtime, users run the code files named with extension name
py
on Python interpreter, instaed of compiling them then run it. C++ is compiling language, it needs to be compiled before running, which takes a long time to compile. Python is easier for users to understand and start, while C++ is normally complex and difficult for users to start.Python tutorial
Python website provides many documents, which is enough for studying python. Besides, use Jupyter Notebook to start python about AI is a good idea, which you have to learn and try.
What is Jupyter Notebook
The Jupyter Notebook is the original web application for creating and sharing computational documents. It offers a simple, streamlined, document-centric experience. This document is created by Jupyter Notebook, saved all results of example codes and explanation for users to read.
We have set
Python3、Jupyter Notebook、Pinpong、Pillow
in AXera-Pi for users easy-use with python, here we takeJupyter Notebook
as the python program example.
Run Jupyter Notebook
To run Jupyter Notebook on AXera-Pi, we need to get the IP address of AXera-Pi, both RNDIS IP address or network IP address are OK, with this we can open the Jupyter Notebook website application running on AXera-Pi. Run ifconfig
to get the IP address of AXera-Pi, and run command jupyter notebook
on AXera-Pi via a terminal, then visit the IP:8888
in your computer browser (Attention:ifconfig
command result lo:127.0.0.1
is inaccessible), like 192.168.0.125:8888
(192.168.0.125
is a example IP address of AXera-Pi, see the following picture), then you can run your python code on Jupyter Notebook.
Note: Pay attention to the communication between computer and AXera-Pi, operator may fail due to bad communication quality.
Visit the IP:8888
, enter root
to login if required password.
Click New
and choose Python3 to new a file with python kernel.
- Python3:A file with python3 kernel
- Text File:A test file
- Folder:New a folder
- Terminal:New a terminal
Jupyter Notebook basic usage
In Jupyter Notebook, we edit by cell table, there are blue cell table
(Command mode) and green cell table
(Editing mode) two versions.
Blue cell table
(Command mode)
Green cell table
(Editing mode)
Common usage:
- Shift+ Enter: Run cell table, move to next cell table and enter into command mode.
- Ctrl + Enter: Run cell table, and enter into command mode.
Edit mode:
- Esc:Enter Command mode
Command mode:
- h: Open help
- Enter: Enter Editing mode
- x: Cut cell table
- c: Copy cell table
- v: Paste cell table
- dd: Delete this cell table
- ii: Stop running task
- a: Add cell table above
- b: Add cell table below
- m: Change cell table into markdown format
Run code
All examples in this document are run on
GC4653
camera sensor, visit Maix-III AXera-Pi (FAQ) to see how to switch camera sensor if you useOS04A10
night enhanced camera sensor.
Here are examples running code or command on Jupyter Notebook, the result is shown in Jupyter Notebook or onboard screen.
- Use command
! + cmd
to run command or script
!ls home/images # Run command
air.jpg carvana02.jpg face5.jpg o2_resize.jpg ssd_car.jpg
aoa-2.jpeg carvana03.jpg grace_hopper.jpg pineapple.jpg ssd_dog.jpg
aoa.jpeg carvana04.jpg mobileface01.jpg pose-1.jpeg ssd_horse.jpg
bike.jpg cat.jpg mobileface02.jpg pose-2.jpeg
bike2.jpg cityscape.png mtcnn_face4.jpg pose-3.jpeg
cable.jpg dog.jpg mtcnn_face6.jpg pose.jpg
carvana01.jpg efficientdet.png mv2seg.png selfie.jpg
!/home/ax-samples/build/install/bin/ax_yolov5s -m /home/models/yolov5s.joint -i /home/images/cat.jpg -r 10 # Run script
--------------------------------------
model file : /home/models/yolov5s.joint
image file : /home/images/cat.jpg
img_h, img_w : 640 640
[AX_SYS_LOG] AX_SYS_Log2ConsoleThread_Start
Run-Joint Runtime version: 0.5.10
--------------------------------------
[INFO]: Virtual npu mode is 1_1
Tools version: d696ee2f
run over: output len 3
--------------------------------------
Create handle took 487.99 ms (neu 22.29 ms, axe 0.00 ms, overhead 465.70 ms)
--------------------------------------
Repeat 10 times, avg time 22.57 ms, max_time 22.88 ms, min_time 22.46 ms
--------------------------------------
detection num: 1
15: 89%, [ 167, 28, 356, 353], cat
[AX_SYS_LOG] Waiting thread(2867848448) to exit
[AX_SYS_LOG] AX_Log2ConsoleRoutine terminated!!!
exit[AX_SYS_LOG] join thread(2867848448) ret:0
from IPython.display import Image # Run python code
Image("yolov5s_out.jpg")
- Use
%run
to run script file, here we usehello.py
as example.
%run hello.py
hello world!
- Download file from AXera-Pi to computer
Document edit on Jupyter Notebook can be downloaded to computer, the downloaded file format is .ipynb
, click File
->Download as
->your target format
if you want other file format.
ax-pipeline-api
ax-pipeline-api: is built on ax-pipeline, and it's suitable for pybind11
and ctypes
, which is used for Python API programming, with which users can load many built-in AI models by Python, or use python library like pinpong、opencv、numpy、pillow. These make AXera-Pi easier to use.
What is the difference between ctypes
and pybind11
ctypes
is what we first used on AXera-Pi, so it's more stable and has more interface than pybind11
, while pybind11
is easier to see the result from screen or website and it's easier for people to understand. Because of limited performance, they should not be used at the same time.
Before running ax-pipeline-api
, make sure you have installed it. And if you have trouble running this application, updating this application may be a good solution.
!pip3 install ax-pipeline-api -U
Requirement already satisfied: ax-pipeline-api in /usr/local/lib/python3.9/dist-packages (1.0.7)
Collecting ax-pipeline-api
Using cached ax-pipeline-api-1.0.7.tar.gz (15.5 MB)
Using cached ax-pipeline-api-1.0.6.tar.gz (19.5 MB)
import time
from ax import pipeline
pipeline.load([
'libsample_vin_ivps_joint_vo_sipy.so',
'-p', '/home/config/yolov8.json',
'-c', '2',
])
while pipeline.work():
time.sleep(0.001)
tmp = pipeline.result()
if tmp and tmp['nObjSize']:
for i in tmp['mObjects']:
x, y, w, h = i['bbox']['x'], i['bbox']['y'], i['bbox']['w'], i['bbox']['h']
objname, objprob = i['objname'], i['prob']
print(objname, objprob, x, y, w, h)
# if tmp['nObjSize'] > 10: # try exit
# pipeline.drop()
pipeline.drop()
b'toilet' 0.4541160762310028 0.602770209312439 0.9111631512641907 0.16810722649097443 0.08513855934143066
b'toilet' 0.6902503967285156 0.606963574886322 0.9117961525917053 0.16024480760097504 0.08727789670228958
b'toilet' 0.6852353811264038 0.6020327210426331 0.9118891358375549 0.16942621767520905 0.08718493580818176
b'toilet' 0.7014157176017761 0.6041151881217957 0.9120386242866516 0.16582755744457245 0.0863698348402977
b'cup' 0.46080872416496277 0.6049922108650208 0.9143685698509216 0.1643451750278473 0.08425315469503403
As the log of running yolov8 above, object detection result parameters are printed out. And the detection picture is shown onboard screen.
All examples in this document are run on
GC4653
camera sensor, change code'-c', '2'
into'-c', '0'
if you useOS04A10
night enhanced camera sensor.
pipeline.load([
'libsample_vin_ivps_joint_venc_rtsp_vo_sipy.so',
'-p', '/home/config/yolov5s.json',
'-c', '2', # GC4653 camera sensor
# '-c', '0', # OS04A10 camera sensor
])
We can also using different .so
library or other AI
models to do more AI usages. There are also useful information on ax-pipeline-api.
- Here are the
ibxxx*.so
library in AXera-Pi:
libsample_h264_ivps_joint_vo_sipy.so # input h264 video to ivps joint output screen vo
libsample_v4l2_user_ivps_joint_vo_sipy.so # input v4l2 /dev/videoX to ivps joint output screen vo
libsample_rtsp_ivps_joint_rtsp_vo_sipy.so # input video from rtsp to ivps joint output rtsp and screen vo
libsample_vin_ivps_joint_vo_sipy.so # input mipi sensor to ivps joint output screen vo
libsample_vin_ivps_joint_venc_rtsp_sipy.so # input mipi sensor to ivps joint output rtsp
libsample_vin_ivps_joint_venc_rtsp_vo_sipy.so # input mipi sensor to ivps joint output rtsp and screen vo
libsample_vin_ivps_joint_vo_h265_sipy.so # input mipi sensor to ivps joint output screen vo and save h265 video file
libsample_multi_rtsp_ivps_joint_multi_rtsp_sipy.so # input multi rtsp video to ivps joint output multi rtsp video\n",
libsample_rtsp_ivps_joint_rtsp_sipy.so # input video from rtsp to ivps joint output rtsp\n",
libsample_rtsp_ivps_joint_rtsp_vo_sipy.so # input video from rtsp to ivps joint output rtsp and screen vo\n",
libsample_rtsp_ivps_joint_vo_sipy.so # input video from rtsp to ivps joint output screen vo
This is the main code of changing ibxxx*.so
.
pipeline.load([
'libsample_vin_ivps_joint_venc_rtsp_vo_sipy.so',
'-p', '/home/config/yolov5s.json',
'-c', '2',
])
- Here are the AI models (which are in the folder
/home/config
) in AXera-Pi:
ax_bvc_det.json hrnet_pose_yolov8.json yolov5s_face_recognition.json
ax_person_det.json license_plate_recognition.json yolov5s_license_plate.json
ax_pose.json nanodet.json yolov6.json
ax_pose_yolov5s.json palm_hand_detection.json yolov7.json
ax_pose_yolov8.json pp_human_seg.json yolov7_face.json
crowdcount.json scrfd.json yolov7_palm_hand.json
hand_pose.json yolo_fastbody.json yolov8.json
hand_pose_yolov7_palm.json yolopv2.json yolov8_seg.json
hrnet_animal_pose.json yolov5_seg.json yolox.json
hrnet_pose.json yolov5s.json
hrnet_pose_ax_det.json yolov5s_face.json
This is the main code of changing AI models
.
pipeline.load([
'libsample_vin_ivps_joint_vo_sipy.so',
'-p', '/home/config/yolov5s_face.json',
'-c', '2',
])
import time
from ax import pipeline
pipeline.load([
'libsample_vin_ivps_joint_vo_sipy.so',
'-p', '/home/config/yolov8_seg.json',
'-c', '2',
])
while pipeline.work():
time.sleep(0.001)
tmp = pipeline.result()
if tmp and tmp['nObjSize']:
for i in tmp['mObjects']:
print(i)
# if tmp['nObjSize'] > 10: # try exit
# pipeline.drop()
pipeline.drop()
{'label': 39, 'prob': 0.41857901215553284, 'objname': b'bottle', 'bbox': {'x': 0.02848125249147415, 'y': 0.03796946257352829, 'w': 0.03146517649292946, 'h': 0.15615946054458618}, 'bHasMask': 1, 'mYolov5Mask': {'w': 6, 'h': 15, 'data': b'\x00\x00\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\x00'}}
{'label': 39, 'prob': 0.4042961299419403, 'objname': b'bottle', 'bbox': {'x': 0.027379659935832024, 'y': 0.037133704870939255, 'w': 0.033295709639787674, 'h': 0.15926814079284668}, 'bHasMask': 1, 'mYolov5Mask': {'w': 6, 'h': 15, 'data': b'\x00\x00\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\x00'}}
{'label': 39, 'prob': 0.4118087887763977, 'objname': b'bottle', 'bbox': {'x': 0.028065890073776245, 'y': 0.03647643327713013, 'w': 0.0326821468770504, 'h': 0.15858806669712067}, 'bHasMask': 1, 'mYolov5Mask': {'w': 6, 'h': 15, 'data': b'\x00\x00\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\x00\xff\xff\xff\xff\x00\x00'}}
import time
from ax import pipeline
pipeline.load([
'libsample_vin_ivps_joint_vo_sipy.so',
'-p', '/home/config/ax_bvc_det.json',
'-c', '2',
])
while pipeline.work():
time.sleep(0.001)
tmp = pipeline.result()
if tmp and tmp['nObjSize']:
for i in tmp['mObjects']:
x, y, w, h = i['bbox']['x'], i['bbox']['y'], i['bbox']['w'], i['bbox']['h']
objname, objprob = i['objname'], i['prob']
print(objname, objprob, x, y, w, h)
# if tmp['nObjSize'] > 10: # try exit
# pipeline.drop()
pipeline.drop()
b'vehicle' 0.9299032092094421 0.3565574288368225 0.44399410486221313 0.23071418702602386 0.2580929398536682
b'vehicle' 0.9225113391876221 0.357175350189209 0.44230249524116516 0.23054184019565582 0.2606807053089142
b'vehicle' 0.9186123609542847 0.3581112325191498 0.44336238503456116 0.22992925345897675 0.26163965463638306
b'vehicle' 0.5208129286766052 0.3618425130844116 0.4461480975151062 0.23065532743930817 0.2652992308139801
b'vehicle' 0.7194858193397522 0.3608142137527466 0.45302334427833557 0.23270295560359955 0.2703518867492676
b'vehicle' 0.8540934324264526 0.3617907166481018 0.4548843204975128 0.23152287304401398 0.27814221382141113
b'vehicle' 0.8912967443466187 0.3607788681983948 0.4583750367164612 0.2328854203224182 0.27056747674942017
b'vehicle' 0.8969211578369141 0.3522576689720154 0.45107507705688477 0.22706037759780884 0.26078158617019653
b'vehicle' 0.7722539901733398 0.32488399744033813 0.4447280168533325 0.23074783384799957 0.26997989416122437
b'vehicle' 0.8158320784568787 0.31976011395454407 0.46247294545173645 0.23671922087669373 0.28281715512275696
b'vehicle' 0.9343507289886475 0.332416832447052 0.5008321404457092 0.2320593148469925 0.26590874791145325
b'vehicle' 0.896917462348938 0.3266463875770569 0.48811814188957214 0.23444382846355438 0.266757994890213
b'vehicle' 0.7356315851211548 0.31006965041160583 0.45962005853652954 0.23436705768108368 0.26961490511894226
import time
from ax import pipeline
pipeline.load([
'libsample_vin_ivps_joint_vo_sipy.so',
'-p', '/home/config/crowdcount.json',
'-c', '2',
])
while pipeline.work():
time.sleep(0.001)
tmp = pipeline.result()
if tmp and tmp['nObjSize']:
for i in tmp['mObjects']:
x, y, w, h = i['bbox']['x'], i['bbox']['y'], i['bbox']['w'], i['bbox']['h']
objname, objprob = i['objname'], i['prob']
print(objname, objprob, x, y, w, h)
# if tmp['nObjSize'] > 10: # try exit
# pipeline.drop()
pipeline.drop()
Human posture detection on ax-pose-ppl
These are the detection points.
0: Nose 1: left eye 2: right eye 3: left ear 4: right ear 5: left shoulder 6: right shoulder 7: left elbow 8: right elbow 9: left wrist 10: right wrist 11: left hip 12: right hip 13: left knee 14: right knee 15: left ankle 16: right ankle
import time
from ax import pipeline
pipeline.load([
'libsample_vin_ivps_joint_vo_sipy.so',
'-p', '/home/config/ax_pose.json',
'-c', '2',
])
while pipeline.work():
time.sleep(0.001)
tmp = pipeline.result()
if tmp and tmp['nObjSize']:
for i in tmp['mObjects']:
print(i)
# if tmp['nObjSize'] > 10: # try exit
# pipeline.drop()
pipeline.drop()
{'label': 0, 'prob': 0.41659796237945557, 'objname': b'person', 'bbox': {'x': 0.01200273260474205, 'y': 0.0, 'w': 0.9315435290336609, 'h': 0.9421796798706055}, 'bHasBoxVertices': 0, 'bHasLandmark': 17, 'landmark': [{'x': 0.6708333492279053, 'y': 0.23333333432674408}, {'x': 0.6427083611488342, 'y': 0.16851851344108582}, {'x': 0.6520833373069763, 'y': 0.14629629254341125}, {'x': 0.7322916388511658, 'y': 0.5055555701255798}, {'x': 0.7614583373069763, 'y': 0.06481481343507767}, {'x': 0.7541666626930237, 'y': 0.09444444626569748}, {'x': 0.7541666626930237, 'y': 0.1518518477678299}, {'x': 0.7124999761581421, 'y': 0.15925925970077515}, {'x': 0.5041666626930237, 'y': 0.08703703433275223}, {'x': 0.6739583611488342, 'y': 0.07407407462596893}, {'x': 0.690625011920929, 'y': 0.6814814805984497}, {'x': 0.7833333611488342, 'y': 0.25}, {'x': 0.7614583373069763, 'y': 0.25}, {'x': 0.35104167461395264, 'y': 0.6074073910713196}, {'x': 0.3489583432674408, 'y': 0.5777778029441833}, {'x': 0.0572916679084301, 'y': 0.5185185074806213}, {'x': 0.0677083358168602, 'y': 0.5185185074806213}]}
import time
from ax import pipeline
from PIL import Image, ImageDraw
# ready sipeed logo canvas
lcd_width, lcd_height = 854, 480
img = Image.new('RGBA', (lcd_width, lcd_height), (255,0,0,200))
ui = ImageDraw.ImageDraw(img)
ui.rectangle((20,20,lcd_width-20,lcd_height-20), fill=(0,0,0,0), outline=(0,0,255,100), width=20)
logo = Image.open("/home/res/logo.png")
img.paste(logo, box=(lcd_width-logo.size[0], lcd_height-logo.size[1]), mask=None)
def rgba2argb(rgba):
r,g,b,a = rgba.split()
return Image.merge("RGBA", (a,b,g,r))
canvas_argb = rgba2argb(img)
pipeline.load([
'libsample_vin_ivps_joint_vo_sipy.so',
'-p', '/home/config/yolov5s.json',
# '-p', '/home/config/yolov8.json',
'-c', '2',
])
while pipeline.work():
time.sleep(0.001)
argb = canvas_argb.copy()
tmp = pipeline.result()
if tmp and tmp['nObjSize']:
ui = ImageDraw.ImageDraw(argb)
for i in tmp['mObjects']:
x = i['bbox']['x'] * lcd_width
y = i['bbox']['y'] * lcd_height
w = i['bbox']['w'] * lcd_width
h = i['bbox']['h'] * lcd_height
objlabel = i['label']
objprob = i['prob']
ui.rectangle((x,y,x+w,y+h), fill=(100,0,0,255), outline=(255,0,0,255))
ui.text((x,y), str(objlabel))
ui.text((x,y+20), str(objprob))
pipeline.config("ui_image", (lcd_width, lcd_height, "ARGB", argb.tobytes()))
pipeline.free()
import m3axpi
# m3axpi.camera(SysCase=0) # switch os04a10
# m3axpi.camera(SysCase=2) # default gc4653
# m3axpi.load("/home/config/yolov8.json")
from PIL import Image, ImageDraw, ImageFont
lcd_width, lcd_height, lcd_channel = 854, 480, 4
fnt = ImageFont.truetype("/home/res/sans.ttf", 20)
img = Image.new('RGBA', (lcd_width, lcd_height), (255,0,0,200))
ui = ImageDraw.ImageDraw(img)
ui.rectangle((20, 20, lcd_width-20, lcd_height-20), fill=(0,0,0,0), outline=(0,0,255,100), width=20)
logo = Image.open("/home/res/logo.png")
img.paste(logo, box=(lcd_width-logo.size[0], lcd_height-logo.size[1]), mask=None)
while True:
rgba = img.copy()
tmp = m3axpi.capture()
rgb = Image.frombuffer("RGB", (tmp[1], tmp[0]), tmp[3])
rgba.paste(rgb, box=(0, 0), mask=None) ## camera 320x180 paste 854x480
res = m3axpi.forward()
if 'nObjSize' in res:
ui = ImageDraw.ImageDraw(rgba)
ui.text((0, 0), "fps:%02d" % (res['niFps']), font=fnt)
for obj in res['mObjects']:
x, y, w, h = int(obj['bbox'][0]*lcd_width), int(obj['bbox'][1]*lcd_height), int(obj['bbox'][2]*lcd_width), int(obj['bbox'][3]*lcd_height)
ui.rectangle((x,y,x+w,y+h), fill=(255,0,0,100), outline=(255,0,0,255))
ui.text((x, y), "%s:%02d" % (obj['objname'], obj['prob']*100), font=fnt)
rgba.paste(logo, box=(x+w-logo.size[1], y+h-logo.size[1]), mask=None)
m3axpi.display([lcd_height, lcd_width, lcd_channel, rgba.tobytes()])
# display(rgb)显示到网页
!ls home/images
air.jpg carvana02.jpg face5.jpg o2_resize.jpg ssd_car.jpg
aoa-2.jpeg carvana03.jpg grace_hopper.jpg pineapple.jpg ssd_dog.jpg
aoa.jpeg carvana04.jpg mobileface01.jpg pose-1.jpeg ssd_horse.jpg
bike.jpg cat.jpg mobileface02.jpg pose-2.jpeg
bike2.jpg cityscape.png mtcnn_face4.jpg pose-3.jpeg
cable.jpg dog.jpg mtcnn_face6.jpg pose.jpg
carvana01.jpg efficientdet.png mv2seg.png selfie.jpg
from PIL import Image, ImageDraw
pil_im = Image.open('home/images/bike2.jpg', 'r')
draw = ImageDraw.Draw(pil_im)
draw.arc((0, 0,400,400) , start=0, end=300, fill='red',width=3)
draw.rectangle((20, 20, 200, 100), fill=(100, 20, 60), outline="#FF0000", width=3)
pil_im.show() # display(pil_im)