I am developing a camera application were I need to display the camera preview without conceivable latency, so when the user looks at the world and the preview together he shouldn’t conceive a time gap between them.
Every frame needs to be scaled, flipped, rotated and overlaid with a small image before being displayed but all these manipulations add less than 1ms to the computations so no problem there.
The minimal latency I measure on my app, on some example projects and on the built-in camera is 120ms (tested on Note4, Galaxy 4 etc) which is definitely noticeable.
I must reduce it to 60ms. The best frame rate I managed to get is a stable 31 fps. I tried both the old Camera API and Camera2 API both with SurfaceView and TextureView.
The app needs to run only on a single phone model which is based on the Snapdragon 615 SoC and the OmniVision 13850 camera running OS 5.0.2. I need to display the preview on a 1080x608 area of the display. So there is no shortage of CPU power, as can be seen from the attached Systrace the processor is idle most of the time:
https: // drive.google.com/open?id=0B780awz-gp4URDFPXzBNRVMwa0k
https: // drive.google.com/open?id=0B780awz-gp4UVEE2NGRmbEJPdlU
GPU profiling shows me that rendering is well below the 16ms mark. The 13850 can provide 30, 60 and even 120 fps.
If I understand the problem correctly, the limit comes from the camera VSync being capped at 32ms (to a maximum of 30 fps) by the Android system (maybe in the kernel drivers) or from the 13850 configuration.
According to Google Android is using triple buffering so at 32ms that’s 96ms which with some overheads can explains the 120ms.
I assume that if I could increase camera preview fps to 60 my latency will drop to 60ms.
There are many discussions how to achieve 60 fps for drawing graphics on the display but I can’t find much about getting 60 fps from the camera preview.
Is my assumption correct?
How do I decrease the camera preview latency to 60ms? (It’s a dedicated device so I am allowed to make changes to Linux and drivers).
Which files do I have to investigate?
Thanks
Every frame needs to be scaled, flipped, rotated and overlaid with a small image before being displayed but all these manipulations add less than 1ms to the computations so no problem there.
The minimal latency I measure on my app, on some example projects and on the built-in camera is 120ms (tested on Note4, Galaxy 4 etc) which is definitely noticeable.
I must reduce it to 60ms. The best frame rate I managed to get is a stable 31 fps. I tried both the old Camera API and Camera2 API both with SurfaceView and TextureView.
The app needs to run only on a single phone model which is based on the Snapdragon 615 SoC and the OmniVision 13850 camera running OS 5.0.2. I need to display the preview on a 1080x608 area of the display. So there is no shortage of CPU power, as can be seen from the attached Systrace the processor is idle most of the time:
https: // drive.google.com/open?id=0B780awz-gp4URDFPXzBNRVMwa0k
https: // drive.google.com/open?id=0B780awz-gp4UVEE2NGRmbEJPdlU
GPU profiling shows me that rendering is well below the 16ms mark. The 13850 can provide 30, 60 and even 120 fps.
If I understand the problem correctly, the limit comes from the camera VSync being capped at 32ms (to a maximum of 30 fps) by the Android system (maybe in the kernel drivers) or from the 13850 configuration.
According to Google Android is using triple buffering so at 32ms that’s 96ms which with some overheads can explains the 120ms.
I assume that if I could increase camera preview fps to 60 my latency will drop to 60ms.
There are many discussions how to achieve 60 fps for drawing graphics on the display but I can’t find much about getting 60 fps from the camera preview.
Is my assumption correct?
How do I decrease the camera preview latency to 60ms? (It’s a dedicated device so I am allowed to make changes to Linux and drivers).
Which files do I have to investigate?
Thanks
No comments:
Post a Comment