Android/scrcpy
1
0
Fork 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity
scrcpy/app/src/decoder.c

162 lines
4.3 KiB
C
Raw Normal View History

Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
#include "decoder.h"
Add missing includes Refs #2616 <https://github.com/Genymobile/scrcpy/issues/2616>
2021-09-21 00:27:37 +08:00
#include <libavcodec/avcodec.h>
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
#include <libavformat/avformat.h>
#include "events.h"
Rename "frames" to "video_buffer" It better describes the purpose of the structure.
2019-03-02 22:16:55 +08:00
#include "video_buffer.h"
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
#include "trait/frame_sink.h"
Move utilities to util/
2019-11-24 18:53:00 +08:00
#include "util/log.h"
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
/** Downcast packet_sink to decoder */
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
#define DOWNCAST(SINK) container_of(SINK, struct sc_decoder, packet_sink)
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
static void
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
sc_decoder_close_first_sinks(struct sc_decoder *decoder, unsigned count) {
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
while (count) {
struct sc_frame_sink *sink = decoder->sinks[--count];
sink->ops->close(sink);
}
}
static inline void
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
sc_decoder_close_sinks(struct sc_decoder *decoder) {
sc_decoder_close_first_sinks(decoder, decoder->sink_count);
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
}
static bool
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
sc_decoder_open_sinks(struct sc_decoder *decoder) {
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
for (unsigned i = 0; i < decoder->sink_count; ++i) {
struct sc_frame_sink *sink = decoder->sinks[i];
if (!sink->ops->open(sink)) {
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
sc_decoder_close_first_sinks(decoder, i);
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
return false;
}
}
return true;
}
static bool
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
sc_decoder_open(struct sc_decoder *decoder, const AVCodec *codec) {
Add stream layer The decoder initially read from the socket, decoded the video and sent the decoded frames to the screen: +---------+ +----------+ socket ---> | decoder | ---> | screen | +---------+ +----------+ The design was simple, but the decoder had several responsabilities. Then we added the recording feature, so we added a recorder, which reused the packets received from the socket managed by the decoder: +----------+ ---> | screen | +---------+ / +----------+ socket ---> | decoder | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This lack of separation of concerns now have concrete implications: we could not (properly) disable the decoder/display to only record the video. Therefore, split the decoder to extract the stream: +----------+ +----------+ ---> | decoder | ---> | screen | +---------+ / +----------+ +----------+ socket ---> | stream | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This will allow to record the stream without decoding the video.
2019-03-02 23:43:43 +08:00
decoder->codec_ctx = avcodec_alloc_context3(codec);
if (!decoder->codec_ctx) {
Add OutOfMemory log helper Add a special LOG_OOM() function to log all OutOfMemory errors (i.e. allocations returning NULL).
2021-11-25 05:06:11 +08:00
LOG_OOM();
Replace SDL types by C99 standard types Scrcpy is a C11 project. Use the C99 standard types instead of the SDL-specific types: SDL_bool -> bool SintXX -> intXX_t UintXX -> uintXX_t
2019-03-03 06:52:22 +08:00
return false;
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
}
Set "low delay" decoder flag I don't really know the concrete benefits, but scrcpy definitely wants low delay decoding. Suggested-by: François Cartegnie <fcvlcdev@free.fr>
2021-12-07 20:59:20 +08:00
decoder->codec_ctx->flags |= AV_CODEC_FLAG_LOW_DELAY;
Add stream layer The decoder initially read from the socket, decoded the video and sent the decoded frames to the screen: +---------+ +----------+ socket ---> | decoder | ---> | screen | +---------+ +----------+ The design was simple, but the decoder had several responsabilities. Then we added the recording feature, so we added a recorder, which reused the packets received from the socket managed by the decoder: +----------+ ---> | screen | +---------+ / +----------+ socket ---> | decoder | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This lack of separation of concerns now have concrete implications: we could not (properly) disable the decoder/display to only record the video. Therefore, split the decoder to extract the stream: +----------+ +----------+ ---> | decoder | ---> | screen | +---------+ / +----------+ +----------+ socket ---> | stream | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This will allow to record the stream without decoding the video.
2019-03-02 23:43:43 +08:00
if (avcodec_open2(decoder->codec_ctx, codec, NULL) < 0) {
LOGE("Could not open codec");
avcodec_free_context(&decoder->codec_ctx);
Replace SDL types by C99 standard types Scrcpy is a C11 project. Use the C99 standard types instead of the SDL-specific types: SDL_bool -> bool SintXX -> intXX_t UintXX -> uintXX_t
2019-03-03 06:52:22 +08:00
return false;
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
}
Make video_buffer more generic The video buffer took ownership of the producer frame (so that it could swap frames quickly). In order to support multiple sinks plugged to the decoder, the decoded frame must not be consumed by the display video buffer. Therefore, move the producer and consumer frames out of the video buffer, and use FFmpeg AVFrame refcounting to share ownership while avoiding copies.
2021-04-11 21:01:05 +08:00
decoder->frame = av_frame_alloc();
if (!decoder->frame) {
Add OutOfMemory log helper Add a special LOG_OOM() function to log all OutOfMemory errors (i.e. allocations returning NULL).
2021-11-25 05:06:11 +08:00
LOG_OOM();
Make video_buffer more generic The video buffer took ownership of the producer frame (so that it could swap frames quickly). In order to support multiple sinks plugged to the decoder, the decoded frame must not be consumed by the display video buffer. Therefore, move the producer and consumer frames out of the video buffer, and use FFmpeg AVFrame refcounting to share ownership while avoiding copies.
2021-04-11 21:01:05 +08:00
avcodec_close(decoder->codec_ctx);
avcodec_free_context(&decoder->codec_ctx);
return false;
}
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
if (!sc_decoder_open_sinks(decoder)) {
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
av_frame_free(&decoder->frame);
avcodec_close(decoder->codec_ctx);
avcodec_free_context(&decoder->codec_ctx);
return false;
}
Replace SDL types by C99 standard types Scrcpy is a C11 project. Use the C99 standard types instead of the SDL-specific types: SDL_bool -> bool SintXX -> intXX_t UintXX -> uintXX_t
2019-03-03 06:52:22 +08:00
return true;
Add stream layer The decoder initially read from the socket, decoded the video and sent the decoded frames to the screen: +---------+ +----------+ socket ---> | decoder | ---> | screen | +---------+ +----------+ The design was simple, but the decoder had several responsabilities. Then we added the recording feature, so we added a recorder, which reused the packets received from the socket managed by the decoder: +----------+ ---> | screen | +---------+ / +----------+ socket ---> | decoder | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This lack of separation of concerns now have concrete implications: we could not (properly) disable the decoder/display to only record the video. Therefore, split the decoder to extract the stream: +----------+ +----------+ ---> | decoder | ---> | screen | +---------+ / +----------+ +----------+ socket ---> | stream | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This will allow to record the stream without decoding the video.
2019-03-02 23:43:43 +08:00
}
Enable video output file, with pts set by server
2018-10-11 13:12:36 +08:00
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
static void
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
sc_decoder_close(struct sc_decoder *decoder) {
sc_decoder_close_sinks(decoder);
Make video_buffer more generic The video buffer took ownership of the producer frame (so that it could swap frames quickly). In order to support multiple sinks plugged to the decoder, the decoded frame must not be consumed by the display video buffer. Therefore, move the producer and consumer frames out of the video buffer, and use FFmpeg AVFrame refcounting to share ownership while avoiding copies.
2021-04-11 21:01:05 +08:00
av_frame_free(&decoder->frame);
Add stream layer The decoder initially read from the socket, decoded the video and sent the decoded frames to the screen: +---------+ +----------+ socket ---> | decoder | ---> | screen | +---------+ +----------+ The design was simple, but the decoder had several responsabilities. Then we added the recording feature, so we added a recorder, which reused the packets received from the socket managed by the decoder: +----------+ ---> | screen | +---------+ / +----------+ socket ---> | decoder | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This lack of separation of concerns now have concrete implications: we could not (properly) disable the decoder/display to only record the video. Therefore, split the decoder to extract the stream: +----------+ +----------+ ---> | decoder | ---> | screen | +---------+ / +----------+ +----------+ socket ---> | stream | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This will allow to record the stream without decoding the video.
2019-03-02 23:43:43 +08:00
avcodec_close(decoder->codec_ctx);
avcodec_free_context(&decoder->codec_ctx);
}
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
static bool
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
push_frame_to_sinks(struct sc_decoder *decoder, const AVFrame *frame) {
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
for (unsigned i = 0; i < decoder->sink_count; ++i) {
struct sc_frame_sink *sink = decoder->sinks[i];
if (!sink->ops->push(sink, frame)) {
return false;
}
}
return true;
}
Make stream push packets to sinks Now that decoder and recorder implement the packet sink trait, make stream push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
static bool
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
sc_decoder_push(struct sc_decoder *decoder, const AVPacket *packet) {
Make stream push packets to sinks Now that decoder and recorder implement the packet sink trait, make stream push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
bool is_config = packet->pts == AV_NOPTS_VALUE;
if (is_config) {
// nothing to do
return true;
}
Handle EAGAIN on send_packet in decoder EAGAIN was only handled on receive_frame. In practice, it should not be necessary, since one packet always contains one frame. But just in case.
2021-04-18 23:13:58 +08:00
int ret = avcodec_send_packet(decoder->codec_ctx, packet);
if (ret < 0 && ret != AVERROR(EAGAIN)) {
Add stream layer The decoder initially read from the socket, decoded the video and sent the decoded frames to the screen: +---------+ +----------+ socket ---> | decoder | ---> | screen | +---------+ +----------+ The design was simple, but the decoder had several responsabilities. Then we added the recording feature, so we added a recorder, which reused the packets received from the socket managed by the decoder: +----------+ ---> | screen | +---------+ / +----------+ socket ---> | decoder | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This lack of separation of concerns now have concrete implications: we could not (properly) disable the decoder/display to only record the video. Therefore, split the decoder to extract the stream: +----------+ +----------+ ---> | decoder | ---> | screen | +---------+ / +----------+ +----------+ socket ---> | stream | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This will allow to record the stream without decoding the video.
2019-03-02 23:43:43 +08:00
LOGE("Could not send video packet: %d", ret);
Replace SDL types by C99 standard types Scrcpy is a C11 project. Use the C99 standard types instead of the SDL-specific types: SDL_bool -> bool SintXX -> intXX_t UintXX -> uintXX_t
2019-03-03 06:52:22 +08:00
return false;
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
}
Make video_buffer more generic The video buffer took ownership of the producer frame (so that it could swap frames quickly). In order to support multiple sinks plugged to the decoder, the decoded frame must not be consumed by the display video buffer. Therefore, move the producer and consumer frames out of the video buffer, and use FFmpeg AVFrame refcounting to share ownership while avoiding copies.
2021-04-11 21:01:05 +08:00
ret = avcodec_receive_frame(decoder->codec_ctx, decoder->frame);
Add stream layer The decoder initially read from the socket, decoded the video and sent the decoded frames to the screen: +---------+ +----------+ socket ---> | decoder | ---> | screen | +---------+ +----------+ The design was simple, but the decoder had several responsabilities. Then we added the recording feature, so we added a recorder, which reused the packets received from the socket managed by the decoder: +----------+ ---> | screen | +---------+ / +----------+ socket ---> | decoder | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This lack of separation of concerns now have concrete implications: we could not (properly) disable the decoder/display to only record the video. Therefore, split the decoder to extract the stream: +----------+ +----------+ ---> | decoder | ---> | screen | +---------+ / +----------+ +----------+ socket ---> | stream | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This will allow to record the stream without decoding the video.
2019-03-02 23:43:43 +08:00
if (!ret) {
// a frame was received
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
bool ok = push_frame_to_sinks(decoder, decoder->frame);
Make video_buffer more generic The video buffer took ownership of the producer frame (so that it could swap frames quickly). In order to support multiple sinks plugged to the decoder, the decoded frame must not be consumed by the display video buffer. Therefore, move the producer and consumer frames out of the video buffer, and use FFmpeg AVFrame refcounting to share ownership while avoiding copies.
2021-04-11 21:01:05 +08:00
// A frame lost should not make the whole pipeline fail. The error, if
// any, is already logged.
(void) ok;
Unref decoder AVFrame immediately The frame can be unref immediately after it is pushed to the frame sinks. It was not really a memory leak because the frame was unref every time by avcodec_receive_frame() (and freed on close), but a reference was unnecessarily kept for too long.
2021-04-27 00:05:43 +08:00
av_frame_unref(decoder->frame);
Add stream layer The decoder initially read from the socket, decoded the video and sent the decoded frames to the screen: +---------+ +----------+ socket ---> | decoder | ---> | screen | +---------+ +----------+ The design was simple, but the decoder had several responsabilities. Then we added the recording feature, so we added a recorder, which reused the packets received from the socket managed by the decoder: +----------+ ---> | screen | +---------+ / +----------+ socket ---> | decoder | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This lack of separation of concerns now have concrete implications: we could not (properly) disable the decoder/display to only record the video. Therefore, split the decoder to extract the stream: +----------+ +----------+ ---> | decoder | ---> | screen | +---------+ / +----------+ +----------+ socket ---> | stream | ---- +---------+ \ +----------+ ---> | recorder | +----------+ This will allow to record the stream without decoding the video.
2019-03-02 23:43:43 +08:00
} else if (ret != AVERROR(EAGAIN)) {
LOGE("Could not receive video frame: %d", ret);
Replace SDL types by C99 standard types Scrcpy is a C11 project. Use the C99 standard types instead of the SDL-specific types: SDL_bool -> bool SintXX -> intXX_t UintXX -> uintXX_t
2019-03-03 06:52:22 +08:00
return false;
Enable video output file, with pts set by server
2018-10-11 13:12:36 +08:00
}
Replace SDL types by C99 standard types Scrcpy is a C11 project. Use the C99 standard types instead of the SDL-specific types: SDL_bool -> bool SintXX -> intXX_t UintXX -> uintXX_t
2019-03-03 06:52:22 +08:00
return true;
Initial commit Start a new clean history from here.
2017-12-12 22:12:07 +08:00
}
Reorder decoder functions This will make further commits more readable.
2021-04-11 21:01:05 +08:00
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
static bool
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
sc_decoder_packet_sink_open(struct sc_packet_sink *sink, const AVCodec *codec) {
struct sc_decoder *decoder = DOWNCAST(sink);
return sc_decoder_open(decoder, codec);
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
}
static void
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
sc_decoder_packet_sink_close(struct sc_packet_sink *sink) {
struct sc_decoder *decoder = DOWNCAST(sink);
sc_decoder_close(decoder);
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
}
static bool
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
sc_decoder_packet_sink_push(struct sc_packet_sink *sink,
const AVPacket *packet) {
struct sc_decoder *decoder = DOWNCAST(sink);
return sc_decoder_push(decoder, packet);
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
}
Reorder decoder functions This will make further commits more readable.
2021-04-11 21:01:05 +08:00
void
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
sc_decoder_init(struct sc_decoder *decoder) {
Explicitly initialize decoder sink_count The zero-initialization relied on the fact that the decoder instance is static.
2021-05-29 03:23:10 +08:00
decoder->sink_count = 0;
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
static const struct sc_packet_sink_ops ops = {
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
.open = sc_decoder_packet_sink_open,
.close = sc_decoder_packet_sink_close,
.push = sc_decoder_packet_sink_push,
Expose decoder as packet sink Make decoder implement the packet sink trait. This will allow the stream to push packets without depending on the concrete sink type.
2021-04-11 21:01:05 +08:00
};
decoder->packet_sink.ops = &ops;
Reorder decoder functions This will make further commits more readable.
2021-04-11 21:01:05 +08:00
}
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
void
Use sc_ prefix for decoder
2022-02-03 02:27:41 +08:00
sc_decoder_add_sink(struct sc_decoder *decoder, struct sc_frame_sink *sink) {
assert(decoder->sink_count < SC_DECODER_MAX_SINKS);
Make decoder push frames to sinks Now that screen implements the packet sink trait, make decoder push packets to the sinks without depending on the concrete sink types.
2021-04-11 21:01:05 +08:00
assert(sink);
assert(sink->ops);
decoder->sinks[decoder->sink_count++] = sink;
}
Reference in a new issue Copy permalink