Since my old video streaming tutorial get a lot of view I wanted to add some new info and an introduction. If you are building a robot and need to stream video this is for you!
Why android to desktop
-Android
The android device I use is an old old v21 phone. I can buy them for $5 a piece or just ask around for an old phone in someones junk drawer. I waned something cheap and available. Lots of sensors too. Compare that to a naked raspberry pi at $35 with no camera and much weaker software.
-Desktop
If you are using neural nets then you need some processing power. Sending video is a good option to avoid weighing down your bot.
Streaming from android
The older posts that you linked from used the basic camera api. DO NOT use the basic camera api.
The second thing I found is bigflake. Its useful but its complicated and I don’t think its that fast although it was difficult to benchmark.
Every app that uses an android is using the webrtc framework; google video, snapchat, signal etc. However webrtc has a very limiting api an is designed for video calls. However I found this library on github. It opens the api and allows you to extract video frame by frame. My remote computers start video command has a 740 ms turnaround before recieiving its first nal from the remote camera on my home network.
These two txt files are the classes that extract video from my android. I draw them on a texture but you dont have to if you want a blank screen. It takes 16ms to encode and 5 ms to packetize and send using the packetizer from the older tutorial.
FFMPEG
FFmpegframegrabber is used to grabframes and display them.
My first hang up was installing. You need to install the latest release as library.
https://github.com/bytedeco/javacv/releases
Instantiation
final FrameGrabber grabber; ... grabber = new FFmpegFrameGrabber(inputStream, 0);
Although I still cannot benchmark it properly I have got it working in “real time” . I had to download the snapshot, change a variable by hand, build it and then paste that jar to replace the javacv.jar that came in the release files you downloaded above/
line 926 change it from 0 to 1 to eliminate the latency
// Enable multithreading when available video_c.thread_count(1);
Heres my stack overflow question.
https://stackoverflow.com/questions/59185079/ffmpeg-javacv-latency-issue
The older section goes into detail and the packetization is still required. The other parts are worth a read too.