wfb-ng OpenHD
OpenHD:
Ruby HD
Обалденная статья как приручить сервы и смартфон и OpenCV
Классный обзор цифровой системы
EZ-WiFi Broadcast/Open.HD:
Camera:
OpenCV Android
Orange PI Zero Plus2 H2
Banana Open Cv:
OpenCv
OpenCV C++
RPi Cam Web Interface:
vcgencmd measure_temp
vcgencmd measure_clock arm
Сравнение PICAMERA and OpenCV
Изучаем Raspberry Pi. Часть 8. Модуль камеры Raspberry Pi
airsim.YawMode.Zero()
Mavlink
Drone Kit Python API:
Some installation details:
WFB-NG some installation detail:
EniSy, [12/26/2022 6:56 PM]
Друзья я аккуратно интересуюсь насколько сложная задача ретрансляции сигнала WFB как это может выглядеть пока кроме wifi репитеров и не пришло на ум
EniSy, [12/26/2022 6:57 PM]
... привык к беспроводному штативу в 100 метрах от меня
RD000000, [12/26/2022 8:27 PM]
Несложно. SBC, пара wifi на разных каналах, wfb_rx и wfb_tx, один стримит RTP напрямую на другой.
SBC - Single Board Computer
RTP: Real-time Transport Protocol
Либо просто wifi AP, если расстояние небольшое.
Qgroundcontrol tuning:
sudo systemctl enable wifibroadcast@drone
sudo systemctl enable fpv-camera
sudo raspi-config
ifconfig
WiFiTx Power here:
sudo nano /etc/modprobe.d/wfb.conf
sudo systemctl status wifibroadcast@drone
iwconfig
sudo systemctl status fpv-camera
sudo journalctl -xu wifibroadcast@drone
./wfb_keygen
Notes:
Twise check sequrity key!
For some reasons fpv-camera.service was not placed in systemd in AIR unit. It is source of video source.
Copy it from wfb-gn scripts.
UDP stream from python:
gst-launch-1.0 udpsrc port=5600 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' ! rtph264depay ! vaapih264dec ! fpsdisplaysink sync=false
Python + pyGObject: How to install
https://pygobject.readthedocs.io/en/latest/getting_started.html#ubuntu-getting-started
Next create PyCharm project with all inviroument.
It tested on
https://gist.github.com/patrickelectric/443645bb0fd6e71b34c504d20d475d5a
to use multi-udp, we need change 127.0.0.1 to 224.0.0.1 and it allow work two video-stream at same time:
wifibroadcast.cfg,
[gs_video]
peer = 'connect://224.0.0.1:5600' # outgoing connection for
# video sink (QGroundControl on GS)
- Use QGroundControl (for telemetry and control) + WifibroadcastOSD (for video and OSD) on the GS. You need to disable video display in QGroundcontrol to avoid conflict
https://forum.ubuntu.ru/index.php?PHPSESSID=9e42138cc3eda4c96a65268ed13fdeb5&topic=287533.msg2264684#msg2264684
Install OpenCv to RPI3
pip3 install opencv-python
sudo apt-get install libcblas-dev
sudo apt-get install libhdf5-dev
sudo apt-get install libhdf5-serial-dev
sudo apt-get install libatlas-base-dev
sudo apt-get install libjasper-dev
sudo apt-get install libqtgui4
sudo apt-get install libqt4-test
Setup:
Xubuntu 18.04 arm64
rtl - version
wfb-ng - stable.
Do Update.
Add some libs
UDP SEND FRAMES:
import cv2,socket,pickle,os
import numpy as np
s=socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.setsockopt(socket.SOL_SOCKET,socket.SO_SNDBUF,1000000)
server_ip = "192.168.11.241"
server_port = 6876
cap = cv2.VideoCapture(0, apiPreference=cv2.CAP_V4L2)
while True:
ret,photo = cap.read()
#cv2.imshow('streaming',photo)
ret,buffer = cv2.imencode(".jpg",photo,[int(cv2.IMWRITE_JPEG_QUALITY),30])
x_as_bytes = pickle.dumps(buffer)
s.sendto((x_as_bytes),(server_ip,server_port))
if cv2.waitKey(10)==13:
break
cv2.destroyAllWindows()
cap.release()
Tuning of WFB-NG:
Один из возможных решений в установке драйвера вайфай:
Алгоритм ключевых точек:
АйпиКамера сWFB-NG и с Flow control соединена с носителем разьемом по езернет. получив целеуказание. Отстреливается.
VERY USEFULL GSTREAMER HELP
https://thiblahute.github.io/GStreamer-doc/frequently-asked-questions/using.html?gi-language=c
check encoder or decoder , just start gst-inspect-1.0
Create Ubuntu BOOT usb flash:
dd if=ubuntu.iso of=/dev/sdx oflag=direct bs=1M status=progress
eraze:
sudo dd if=/dev/zero of=/dev/sdX bs=1M status=progress
Secure erase:
sudo shred -v -n1 -z /dev/sdX
hdparm -I /dev/X
gst-inspect-1.0 autovideosink
Building OpenCV with Gstreamer
https://ebenezertechs.com/optimizing-opencv-3-4-in-raspberry-pi-3-using-gstreamer/
###################################################################################
WFB-NG instalation
rtl8812au compiled on kernel v5.15.60 and has problem with 5.19.30
how to set kernel on Ubuntu:
Here is about how to set kernel version 5.13
but, for instalation wfb-ng we will need install kernel headers:
sudo apt install linux-generic
But it set headers for v5.15.60
so we should set kernel for 5.15.60. In this case we can set WiFi driver and compile wfb-ng .
wget https://raw.githubusercontent.com/pimlie/ubuntu-mainline-kernel.sh/master/ubuntu-mainline-kernel.sh
chmod +x ubuntu-mainline-kernel.sh
# search and find your wanted version
sudo ./ubuntu-mainline-kernel.sh -r | grep 5.15
(it will show us kernel version list. So do same for 5.15)
# install that version kernel
sudo ./ubuntu-mainline-kernel.sh -i v5.15.60
# get all menuentries
grep 'menuentry \|submenu ' /boot/grub/grub.cfg | cut -f2 -d "'"
# change the grub configuration
vi /etc/default/grub
from: GRUB_DEFAULT=0
change to:
GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 5.15.60-051560-generic"
# update grub
sudo update-grub
# reboot
sudo reboot now
# verify
uname -r
it must show 5.15.60-051560-generic
Now we can install WFB_NG.
##################################################################################
OSD compile:
svpcom/wfb-ng-osd
and copy to mavlink directory source from
opencv-python==4.5.3.56
docker ps -a
docker rm -f mycontainer
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5a8f89adeead ubuntu:14.04 "/bin/bash" About a minute ago Exited (0) About a minute ago agitated_newton
$ docker commit 5a8f89adeead newimagename
$ docker run -ti -v "$PWD/somedir":/somedir newimagename /bin/bashgit clone --recursive https://github.com/opencv/opencv-python.git
sudo ln -s /opt/opencv-4.1.0/lib/python3.7/dist-packages/cv2 /usr/local/lib/python3.7/site-packages/cv2
print(cv2.getBuildInformation())
how to install python3.7
-D WITH_QT=OFF \
-D WITH_GTK=ON \udpsrc port=5600 ! application/x-rtp, payload=96 ! rtph264depay ! h264parse ! avdec_h264 ! decodebin ! videoconvert ! video/x-raw,format=(string)BGR ! videoconvert ! appsink emit-signals=true sync=false max-buffers=2 drop=true
EXAMPLE WORK ON PC UBUNTU
########################################################################
import cv2
print(cv2.getBuildInformation())
gstreamer_str = "sudo gst-launch-1.0 udpsrc port=5600 ! application/x-rtp, payload=96 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! videoscale ! video/x-raw,width=640,height=480,format=BGR ! appsink drop=1"
cap = cv2.VideoCapture(gstreamer_str, cv2.CAP_GSTREAMER)
while(cap.isOpened()):
ret, frame = cap.read()
if ret:
cv2.imshow("Input via Gstreamer", frame)
if cv2.waitKey(25) & 0xFF == ord('q'):
break
else:
break
cap.release()
cv2.destroyAllWindows()
########################################################################
script_cv.sh
###
OPENCV_VER="master"
TMPDIR=$(mktemp -d)
# Build and install OpenCV from source.
cd "${TMPDIR}"
git clone --branch ${OPENCV_VER} --depth 1 --recurse-submodules --shallow-submodules https://github.com/opencv/opencv-python.git opencv-python-${OPENCV_VER}
cd opencv-python-${OPENCV_VER}
export ENABLE_CONTRIB=1
export ENABLE_HEADLESS=0
# We want GStreamer support enabled.
export CMAKE_ARGS="-DWITH_GSTREAMER=ON"
export CMAKE_ARGS="-DWITH_GTK=ON"
python3 -m pip wheel . --verbose
# Install OpenCV
python3 -m pip install opencv_python*.whl
###
sudo apt-get install libgtk2.0-dev libgtk-3-dev
sudo apt-get install pkg-config
pip3 list | grep opencv
pip uninstall opencv-contrib-python
nano script_cv.sh
chmod +x script_cv.sh
./script_cv.sh
pip install opencv_contrib_python*.whl
install opencv
Поиск изображения по фрагменту.
https://vc.ru/dev/249864-poisk-izobrazheniy-po-fragmentu-s-pomoshchyu-orb
алгоритм Фарнебека
задача класса Template matching
a =Process(x)
a.start()
a.join()
очень большая нагрузка при распаковке? Это очень странно.
sudo gst-launch-1.0 udpsrc port=5602 ! video/x-raw, format=BGR ! appsink drop=1
gst-launch-1.0 videotestsrc ! video/x-raw, format=BGR ! autovideoconvert ! videoconvert ! video/x-raw, width=144, height=80, framerate=1/2 ! udpsink host=127.0.0.1 port=5602 sync=false
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-plugins/html/gstreamer-plugins-tee.html
https://stackoverflow.com/questions/40672169/gstreamer-1-0-one-source-and-two-sink-display
gst-launch-1.0 v4l2src do-timestamp=true io-mode=mmap device=/dev/video0 extra-controls="controls, rotate=180,image_stabilization=1,h264_profile=4,h264_i_frame_period=60,h264_level=11,power_line_frequency=1,exposure_metering_mong=1,exposure_dynamic_framerate=0,scene_mode=0,video_bitrate_mode=1,video_bitrate=4000000,repeat_sequence_header=1" ! video/x-h264,profile=high,width=1280,height=720,framerate=30/1,stream-format=byte-stream ! h264parse disable-passthrough=true ! rtph264pay config-interval=1 pt=96 mtu=1400 aggregate-mode=zero-latency ! udpsink host=127.0.0.1 port=5602 sync=false
v4l2src can be used to capture video from v4l2 devices, like webcams and tv cards.
Example launch lines
gst-launch-1.0 v4l2src ! xvimagesink
This pipeline shows the video captured from /dev/video0 tv card and for webcams.
gst-launch-1.0 v4l2src ! jpegdec ! xvimagesink
####################################
import cv2
port = 5600
pipeline = ('udpsrc port={} ! '
'application/x-rtp, payload=96 ! '
'rtpjitterbuffer ! rtph264depay ! '
'decodebin ! videoconvert ! '
'appsink').format(port)
cap = cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER)
############################################
gst-launch-1.0
v4l2src do-timestamp=true io-mode=mmap
device=/dev/video0
extra-controls="controls,
rotate=180,
image_stabilization=1,
h264_profile=4,
h264_i_frame_period=60,
h264_level=11,
power_line_frequency=1,
exposure_metering_mong=1,
exposure_dynamic_framerate=0,
scene_mode=0,
video_bitrate_mode=1,
video_bitrate=4000000,
repeat_sequence_header=1" !
video/x-h264,
profile=high,
width=1280,
height=720,
framerate=30/1,
stream-format=byte-stream !
h264parse disable-passthrough=true !
rtph264pay
config-interval=1
pt=96
mtu=1400
aggregate-mode=zero-latency !
udpsink host=127.0.0.1
port=5602
sync=false
-e should be added to the end if you use ctrl+c to end capturing. gst-launch-1.0 v4l2src ! videoconvert ! x264enc ! flvmux ! filesink location=xyz.flvgst-launch-1.0 v4l2src ! ffmpegcolorspace ! jpegenc ! avimux ! filesink location=output.avigst-launch-1.0 videotestsrc num-buffers=100 ! tee name=t ! queue ! videoconvert ! xvimagesink sync=false t. ! queue ! videorate ! video/x-raw,framerate=25/1 ! videoconvert ! avenc_mpeg2video bitrate=3000000 ! avimux ! filesink location=test_file.avi async=0
gst-launch-1.0 imagesequencesrc location=image-%05d.jpg start-index=1 stop-index=50 framerate=24/1 ! decodebin ! videoconvert ! autovideosinkStop value of index. The special value -1 means no stop.
gst-launch-1.0 -ev 4l2src device=/dev/video0 ! video/x-raw, format=UYVY, width=2592, height=1944,framerate=28/1 ! tee name=cam ! queue ! nvvidconv ! 'video/x-raw(memory:NVMM),format=(string)I420' ! nvv4l2h265enc maxperf-enable=true insert-vui=true insert-sps-pps=1 ! tee name=h265_stream ! queue ! h265parse ! rtph265pay ! udpsink clients=192.168.18.18:5201 h265_stream. ! queue ! h265parse ! matroskamux ! filesink location=test_h265.mkv cam. ! queue ! fakesink
# Capture a single image and save it in JPEG format.
$ gst-launch-1.0 v4l2src num-buffers=1 ! jpegenc ! filesink location=/tmp/test.jpg
# Stream video from a webcam.
$ gst-launch-1.0 v4l2src ! xvimagesink
gst-launch-1.0 v4l2src num-buffers=1 ! image/jpeg,framerate=5/1,width=1280,height=960 ! jpegparse ! filesink location=/tmp/test2.jpg
https://gist.github.com/anselmobattisti/de244db254c5bd8116543d01c6714222
Список доступных девайсов для v4l2src. Этот элемент предназначен для открытия видеопотока через интерфейс Video4Linux2. :
ls -l /dev/v4l/by-id
Пример
gst-launch-1.0 v4l2src ! xvimagesink
Шаблон:
gst-launch-1.0 videotestsrc pattern=smpte ! xvimagesink
список всех доступных свойств, а так же допустимые для них значения:
gst-inspect-1.0 filesrc
Install camera loopback:
Problem after instalation:
Create virtual camera:
sudo modprobe v4l2loopback exclusive_caps=1 video_nr=5 #creates `/dev/video5`
gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=YUY2,width=640,height=480,framerate=30/1" ! tee name=t ! queue ! v4l2sink device=/dev/video1 t. ! queue ! v4l2sink device=/dev/video2
Study
implementation
PID controller:
Using DroneKit with keyboard:
visual navigation
Mission planner, Ubuntu. how to connect Gstreamer:
https://discuss.ardupilot.org/t/udp-gstreamer-video-feed-not-working-after-new-gcs-install/88280
Mavlink
Настройка Ardupilot
IT OPEN LOW DELAY VIDEO:
gst-launch-1.0 udpsrc port=5600 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' ! rtph264depay ! avdec_h264 ! clockoverlay valignment=bottom ! autovideosink sync=false
gst-launch-1.0 udpsrc port=5600 ! application/x-rtp, payload=96 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! videoscale ! video/x-raw,width=640,height=480,format=BGR ! appsink drop=1
gst-launch-1.0 udpsrc port=5600 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' ! rtph264depay ! avdec_h264 ! clockoverlay valignment=bottom ! appsink drop=1
WI-FI chanels description:
With default settings WFB use radio channel 165 (5825 MHz), width 20MHz, MCS #1 (QPSK 1/2) with long GI. This provides ~7 mbit/s of effective speed (i.e. usable speed after FEC and packet encoding) for both directions in sum, because WiFi is half-duplex. So it is suitable for video down stream 720p@49fps (4 mbit/s) + two full-speed telemetry streams (uplink and downlink). If you need a higher bandwidth you can use other MCS index (for example 2 or greater) and/or 40MHz channel.
Some mention about power:
/etc/modprobe.d/wfb.conf ethtool -i wlanXXsudo nano /etc/modprobe.d/wfb.conf
CSRT трекер
Photogrammometria.
Battery measurement
OrtoPhotoPlan
Open IPC how-to:
Bash OpenIPC:
https://www.youtube.com/watch?v=y-x9_n0fDMI
https://www.youtube.com/watch?v=y-x9_n0fDMI
OpticalFlow:
Object tracking using Homography:
KalmanFilter:
Налаштування швидкості для WFB_NG !!! важливо для дальності.
https://www.youtube.com/watch?v=JEqai5JKZws
https://www.youtube.com/watch?v=JEqai5JKZws
Комментарии
Отправить комментарий