7:20 a.m.
Ben, thanks for the help.
For Sign Language, frame rate and resolution are likely the key factors. We'll likely
want to experiment with the a 320 by 400 window and the ability to turn off audio as
possible ways to improve performance. Thoughts???
Stephen Jacobs
Associate Professor
Interactive Games and Media
Rochester Institute of Technology
102 Lomb Memorial Drive
Bldg 70
Rochester, NY 14618
sj(a)mail.rit.edu
585-475-7803
9:31 a.m.
New subject: Codec speed optimizations
Stephen Jacobs wrote:
Ben, thanks for the help.
No problem.
For Sign Language, frame rate and resolution are likely the key
factors. We'll likely want to experiment with the a 320 by 400 window and the ability
to turn off audio as possible ways to improve performance. Thoughts???
If you don't need audio, there's no reason to enable it, and so far I
think your speed tests don't have it. Audio (specifically the Speex
codex) doesn't cost much CPU time or bandwidth, but it's not zero.
Real-time video encoding takes a lot of CPU, and the XO-1.5 is still very
limited in that department. Naturally, increasing the frame size
decreases the achievable FPS for a given processor and encode settings.
You'll have to find the best option within your performance envelope.
Remember that OLPC laptops are always about 10 years behind in terms of speed.
As for image size, I recommend that you stick to one of the sizes that the
camera can produce natively: 640x480, 352x288, 320x240, or 176x144 [1].
Choosing one of these sizes avoids the need for any scaling or other
processing on the CPU, because the scaling is instead done by dedicated
hardware inside the camera. The true sensor resolution of the camera is
640x480, so 352x288 may show interpolation aliasing artifacts ... but it
may not.
The Harvard Digital Literacy Club found that Nicaraguan students were able
to sign to each other even at the pretty awful video quality produced by
the XO-1's Record activity. It wasn't ideal, but the signing was
comprehensible.
Cropping the image would be interesting, as it requires much less CPU time
than scaling. You might want to experiment with using, say, just the
right half of the visual field, because the XO camera is off-center.
One final note: There are three common ways to code color in video [2].
The camera is shooting in YUYV 4:2:2, a.k.a. YUY2 [3]. It doesn't support
4:2:0, which is what my last pipeline tried to provide. You'd probably do
better to be encoding in 4:2:0, which for Theora means I420. To get
there, you can try using
GST_OUTPIPE_BASE = "v4l2src !
video/x-raw-yuv,width=320,height=240,framerate=15/1,format="(fourcc)YUY2"
! ffmpegcolorspace ! video/x-raw-yuv,format="(fourcc)I420" ! theoraenc
bitrate=50 speed-level=2 ! udpsink host=%s"
You should benchmark this to see if the cost of converting YUY2->I420 is
offset by the increased encode speed.
--Ben
[1]
http://dev.laptop.org/git/olpc-2.6/tree/drivers/media/video/ov7670.c?h=ol...
[2] http://en.wikipedia.org/wiki/Chroma_subsampling
[3]
http://dev.laptop.org/git/olpc-2.6/tree/drivers/media/video/ov7670.c?h=ol...
9:58 a.m.
New subject: Codec speed optimizations
352x288, 320x240
These two look like winners for testing with
The Harvard Digital Literacy Club found that Nicaraguan students were able
to sign to each other even at the pretty awful video quality produced by
the XO-1's Record activity. It wasn't ideal, but the signing was
comprehensible.
Yah, it's ok, but for long-term, sustained communication its pretty painful.
Could focus test as well, but I'm fluent and my wife is Deaf and my experience is
that
if there's a choice between the two, people would rather have a smaller window with
smoother signing.
Cropping the image would be interesting, as it requires much less CPU time
than scaling. You might want to experiment with using, say, just the
right half of the visual field, because the XO camera is off-center.
Also useful
You should benchmark this to see if the cost of converting YUY2->I420 is
offset by the increased encode speed.
Figure the team will give it a shot as well.
Thanks so much. This is extremely helpful!
5150
days inactive
5150
days old
2 comments
2 participants
participants (2)
-
Benjamin M. Schwartz -
Stephen Jacobs