Mouse events coordinates depend on the screen size and location, so the
converter need to access the screen.
The fact that it needs the position or the size is an internal detail,
so pass a pointer to the whole screen structure.
Now, get_window_size() returns the current window size (fullscreen or
not), while get_windowed_window_size() always returned the windowed size
(the size when fullscreen is disabled).
Headers seem to be a bit different in Apple land and you need to include
stddef.h explicitly to the NULL declaration.
This also makes the code a bit more correct, as stddef.h is the header
in the C standard that defines NULL
(https://en.cppreference.com/w/cpp/header/cstddef).
We need several FIFO queues (a queue of packets, a queue of messages,
etc.).
Some of them are implemented using cbuf, a generic circular buffer. But
for recording, we need to store the packets in an unbounded queue until
they are written, so the queue was implemented manually.
Create a generic implementation (using macros) to avoid reimplementing
it every time.
The record file was written from the stream thread. As a consequence,
any blocking I/O to write the file delayed the decoder.
For maximum performance even when recording is enabled, send
(refcounted) packets to a separate recording thread.
To packetize the H.264 raw stream, av_parser_parse2() (called by
av_read_frame()) knows that it has received a full frame only after it
has received some data for the next frame. As a consequence, the client
always waited until the next frame before sending the current frame to
the decoder!
On the device side, we know packets boundaries. To reduce latency,
make the device always transmit the "frame meta" to packetize the stream
manually (it was already implemented to send PTS, but only enabled on
recording).
On the client side, replace av_read_frame() by manual packetizing and
parsing.
<https://stackoverflow.com/questions/50682518/replacing-av-read-frame-to-reduce-delay>
<https://trac.ffmpeg.org/ticket/3354>
If --no-control is set, then the controller is not initialized (both in
the client and the server), so it is not possible to control the device
to turn its screen off.
See <https://github.com/Genymobile/scrcpy/issues/608>.
In portable builds, scrcpy-server.jar was supposed to be present in the
current directory, so in practice it worked only if scrcpy was launched
from its own directory.
Instead, find the absolute path of the executable and build a suitable
path to use scrcpy-server.jar from the same directory.
There was already utf8_to_wide_char(), used to correctly execute
commands on Windows.
Add the reverse converter: utf8_from_wide_char(). We will need it to
build the scrcpy-server path based on the executable directory.
To create a portable build (with scrcpy-server.jar accessible from the
scrcpy directory), replace OVERRIDE_SERVER_PATH by a simple compilation
flag: PORTABLE.
This paves the way to use more complex rules to determine the path of
scrcpy-server.jar in portable builds.
On socket disconnection, on Linux, recv() returns -1 and errno is set.
But on Windows, errno is 0.
In that case, AVERROR(errno) == 0, leading to the warning:
> Invalid return value 0 for stream protocol
To avoid the problem, if errno is 0, return AVERROR_EOF.
Ref: commit 2876463d39
The FPS counter was called only on new frames, so it could not print
values regularly, especially when there are very few FPS (when the
device surface does not change).
To the extreme, it was never able to display 0 fps.
Add a separate thread to print framerate every second.
Only keep "turn device screen off" and POWER button.
After we turn the device screen off (with Ctrl+o), turning it back on
does not always work, and leaves the device in a weird state, where even
the power button may not be sufficient:
<https://github.com/Genymobile/scrcpy/issues/175#issuecomment-497946596>
This is not an acceptable behavior, so disable the shortcut to turn the
physical device screen on. We can use the POWER button (or Ctrl+p)
instead.
The condition "event->type == SDL_KEYDOWN" and the variable
input_manager->controller are used many times. Replace them by local
variables to reduce verbosity.
Add two shortcuts:
- Ctrl+o to turn the device screen off while mirroring
- Ctrl+Shift+o to turn it back on
On power on (either via the POWER key or BACK while screen is off), both
the device screen and the mirror are turned on.
<https://github.com/Genymobile/scrcpy/issues/175>
Use the same variable name in functions declaration and definition.
Signed-off-by: Yu-Chen Lin <npes87184@gmail.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
After the recent refactorings, a "control event" is not necessarily an
"event" (it may be a "command"). Similarly, the unique "device event"
used to send the device clipboard content is more a "reponse" to the
request from the client than an "event".
Rename both to "message", and rename the message types to better
describe their intent.
It was already possible to _paste_ (with Ctrl+v) the content of the
computer clipboard on the device. Technically, it injects a sequence of
events to generate the text.
Add a new feature (Ctrl+Shift+v) to copy to the device clipboard
instead, without injecting the content. Contrary to events injection,
this preserves the UTF-8 content exactly, so the text is not broken by
special characters.
<https://github.com/Genymobile/scrcpy/issues/413>
On Ctrl+C:
- the client sends a GET_CLIPBOARD command to the device;
- the device retrieve its current clipboard text and sends it in a
GET_CLIPBOARD device event;
- the client sets this text as the system clipboard text, so that it
can be pasted in another application.
Fixes <https://github.com/Genymobile/scrcpy/issues/145>
The socket used the device-to-computer direction to stream the video and
the computer-to-device direction to send control events.
Some features, like copy-paste from device to computer, require to send
non-video data from the device to the computer.
To make them possible, use two sockets:
- one for streaming the video from the device to the client;
- one for control/events in both directions.
A string is serialized as a length (2 bytes) followed by the string data
(non nul-terminated).
For now, it is used only once, but we will need to serialize strings in
other events.
Several commands were grouped under the same event type "command", with
a separate field to indicate the actual command.
Move these commands at the same level as other control events. It will
allow to implement commands with arguments.
The cleanup is not linear: for example, the server must be stopped and
its sockets must be shutdown after the stream and controller are stopped
(so that they don't continue processing garbage), but before they are
joined, to avoid a deadlock if they are blocked on a socket read.
Simplify the spaghetti-cleanup by keeping trace of initialization at
runtime.
Compositor bypass is meant for fullscreen games consuming lots of GPU
resources. For a light app that will usually be windowed, this only
causes unnecessary compositor suspends, especially visible (and
annoying) with complying window manager like KWin.
Signed-off-by: Romain Vimont <rom@rom1v.com>
The serialized text is not nul-terminated (its size is explicitely
provided), but the input text in the event is a nul-terminated string.
The test was failing with ASAN enabled.
Replace the "global" control flag in the input_manager by a function
parameter to make explicit that the behavior depends whether
--no-control has been set.
The SDL video subsystem is not necessary if we don't display the video.
Move the sdl_init_and_configure() function from screen.c to scrcpy.c,
because it is not only related to the screen display.
The function video_buffer_offer_decoded_frame() returned a bool to
indicate whether the previous frame had been consumed.
This was confusing, because we could expect the returned bool report
whether the action succeeded.
Make the semantic explicit by using an output parameter.
Also revert the flag (report if the frame has been skipped instead of
consumed) to avoid confusion for the first frame (the previous is
neither skipped nor consumed because there is no previous frame).
The description of scrcpy is "Display and control your Android device".
We want an option to disable display, another one to disable control.
For naming consistency, name it --no-display.
Also change the shortname to -N, so that we can use -n for --no-control
later.
Limit source code to 80 chars, and declare functions return type and
modifiers on a separate line.
This allows to avoid very long lines, and all function names are
aligned.
(We do this on VLC, and I like it.)
The decoder initially read from the socket, decoded the video and sent
the decoded frames to the screen:
+---------+ +----------+
socket ---> | decoder | ---> | screen |
+---------+ +----------+
The design was simple, but the decoder had several responsabilities.
Then we added the recording feature, so we added a recorder, which
reused the packets received from the socket managed by the decoder:
+----------+
---> | screen |
+---------+ / +----------+
socket ---> | decoder | ----
+---------+ \ +----------+
---> | recorder |
+----------+
This lack of separation of concerns now have concrete implications: we
could not (properly) disable the decoder/display to only record the
video.
Therefore, split the decoder to extract the stream:
+----------+ +----------+
---> | decoder | ---> | screen |
+---------+ / +----------+ +----------+
socket ---> | stream | ----
+---------+ \ +----------+
---> | recorder |
+----------+
This will allow to record the stream without decoding the video.
The order of cleanup was not the reverse as the initialization order. As
a consequence, recorder_destroy() could theoretically be called even if
recorder_init() failed.
The deprecated avcodec_decode_video2() should always the whole packet,
so there is no need to loop (cf doc/examples/demuxing_decoding.c in
FFmpeg).
This hack changed the packet size and data pointer. This broke recording
which used the same packet.
The extradata buffer is owned by libav, so it must be allocated with
av_malloc(), not SDL_malloc().
This fixes a crash on Windows during avformat_free_context().
Windows does not support UTF-8, so pushing a file with non-ASCII
characters failed.
Convert the UTF-8 command line to a wide characters string and call
CreateProcessW().
Fixes <https://github.com/Genymobile/scrcpy/issues/422>
Implement recording to Matroska files.
The format to use is determined by the option -F/--record-format if set,
or by the file extension (".mp4" or ".mkv").
Some containers force their own time base. For example, matroska
overwrite time_base to (AVRational) {1, 1000}.
Therefore, rescale our packet timestamps to the output stream time base.
Suggested-by: Steve Lhomme <robux4@ycbcr.xyz>
It is very convenient when I play mobile game and watch video at the
same time.
Tested on Linux mint Cinnamon as well as Windows 10.
Signed-off-by: Yu-Chen Lin <npes87184@gmail.com>
To clean up the device, the client executed "adb shell rm" once the
server was guaranteed to be started (after the connection succeeded).
This implied to track whether the installation state, and failed if an
additional tunnel was used in "forward" mode:
<https://github.com/Genymobile/scrcpy/issues/386#issuecomment-453936034>
Instead, make the server unlink itself on start.
Mouse events position were unsigned (so negative values could not be
handled properly).
To avoid issues with negative values, mouse events outside the device
screen were ignored (commit a7fe9ad779).
But as a consequence, drag&drop were "broken" if the "drop" occurred
outside the device screen.
Instead, use signed 32-bits to store the position, and forward events
outside the device screen.
Fixes <https://github.com/Genymobile/scrcpy/issues/357>.
Never create a "struct point" with a position possibly outside the
device screen (i.e. in the black borders area), and do not transmit such
events.
This fixes an assertion failure on mouse wheel events outside the device
screen area.
The client passes parameters to the server via "adb shell" arguments.
Use "-" instead of "" when no crop is specified to avoid empty
arguments, which are not handled the same way on all devices.
Fixed <https://github.com/Genymobile/scrcpy/issues/337>.
Configuration packets produced by MediaCodec have no valid PTS, and do
not produce frame. Do not queue their (invalid) PTS not to break the
matching between frames and their PTS.
Several frames may be read by read_packet() before they are consumed
(returned by av_read_frame()), so we need to store the PTS of frames in
order, so that the right PTS is assigned to the right frame.
AVStream.codec has been deprecated in favor of AVStream.codecpar.
Due to the FFmpeg/Libav split, this happened in two separate versions:
- 57.33.100 for FFmpeg
- 57.5.0 for Libav
This partially reverts commit f00c6c5b13.
On Ctrl+C, we need to execute cleanup code. For instance, if recording
is enabled, we need to write MP4 file trailer on exit.
Custom SDL signal handlers were disabled because it leaded to process
hanging on Ctrl+C during network calls on initialization, but now it
seems to work correctly, the network calls return immediately on signal.
Since PTS handling has been fixed, the recorder do not associate a PTS
to a wrong frame anymore, so PTS of "configuration packets" (which never
produce a frame), are never read by the recorder. Therefore, there is no
need to ignore them explicitly, so we can remove the MediaCodec flags
completely.
The PTS was read from the socket and set as the current one even before
the frame was consumed, so it could be assigned to the previous frame
"in advance".
Store the PTS for the current frame and the last PTS read from the
packet header of the next frame in separate fields.
As a side-effect, this fixes the warning on quit:
> Application provided invalid, non monotonically increasing dts to
> muxer in stream 0: 17164020 >= 17164020
To handle special chars, text is handled as text input instead of key
events. However, this breaks the separation of DOWN and UP key events.
As a compromise, send letters and space as key events, to preserve
original DOWN/UP events, but send other text input events as text, to be
able to send "special" characters.
Fixes <https://github.com/Genymobile/scrcpy/issues/87>.
Suggested-by: pete1414
Suggested-by: King-Slide <kingslide@gmail.com>
The common command.c handled process errors from system-specific int
values (errno).
Rather, expose a new enum process_result to handle error cause in a
generic way.
There are many user who encounters missing adb.
To stop things happens again, we check it and show
sexy response to user.
Signed-off-by: yuchenlin <npes87184@gmail.com>
The current_process field was never reset after an installation is
complete. As a consequence, installer_stop() attempted to terminate it,
leading to a warning, at best.
In the Android input header file, an enum has a value taking more than
31 bits, leading to the following warning:
ISO C restricts enumerator values to range of ‘int’
Since we don't use it, remove it.
A missing initialization (fixed by the previous commit) leaded to kill
unexpected process.
In order to prevent consequences of similar errors in the future, never
call kill() with a non-positive PID.
See <https://github.com/Genymobile/scrcpy/issues/182>.
The current_process field of struct installer was not initialized.
Since the installer instance is static, its default value was 0.
The call to installer_stop() then called kill(0, SIGTERM) (on Linux),
which sent SIGTERM to every process in the process group. In particular,
the scrcpy process was killed.
As a consequence, the last cleanup steps, like disabling "show touches",
were not executed.
Fixes <https://github.com/Genymobile/scrcpy/issues/183>.
Meson decided to crossbuild for Windows as soon as
meson.is_cross_build() returned true. This made non-Windows crossbuilds
fail.
Instead, add an explicit option "crossbuild_windows".
Fixes <https://github.com/Genymobile/scrcpy/issues/165>.
On Windows, an application is either console or gui, it cannot be both.
Scrcpy should be both: it outputs important information to console, but
we still want to be able to ignore the console and launch it without a
visible cmd.exe window.
Therefore, build two binaries:
- scrcpy.exe
- scrcpy-noconsole.exe
Build the Windows binary from mingw on Linux, using the official
prebuilt binaries for ffmpeg, SDL2 and adb.
MSYS2 and all its packaged dll are not necessary anymore.
In practice, proc_show_touches may not be used uninitialized, since it
checks the flag options->show_touches, but the compiler can't know that,
so initialize it to avoid the warning.
I could not make Ctrl+'+' and Ctrl+'-' work for every keyboard on every
platform.
Instead, use Ctrl+UP and Ctrl+DOWN (like in VLC) to change the volume.
Fixes <https://github.com/Genymobile/scrcpy/issues/103>.
SDL_CreateTexture() is called both during initialization and on frame
size change.
To avoid inconsistent changes to arguments value, factorize them to a
single function create_texture().
Double-clicks were not sent to the device anymore since the
"double-click on black borders" feature.
When a double click occurs inside the device screen, send the event to
the device normally.
Fixes <https://github.com/Genymobile/scrcpy/issues/97>.
Enabling "show touches" involves the execution of an adb command, which
takes some time.
In order to parallelize, execute the command as soon as possible, but
reap the process only once everything is initialized.
If --show-touches is set, then the option must be disabled on quit.
Since it executes an adb command, it takes some time, so close the
window beforehand so that the close window button does not seem
unresponsive.
On H.264 stream EOF, the eof_reached flag is set, but av_read_frame()
still provides a frame, so check the flag only afterwards.
As a side-effect, it also fixes a memory leak (the very last packet was
not unref).
Request SDL not to replace the SIGINT and SIGTERM handlers, so that the
process is immediately terminated on Ctrl+C.
This avoids process hanging on Ctrl+C during network calls on
initialization.
Some of them accepted a timeout, but it was not used since
commit 9b056f5091 anymore.
On Windows and MacOS, resizing blocks the event loop, so resizing events
are not triggered:
- <https://bugzilla.libsdl.org/show_bug.cgi?id=2077>
- <https://stackoverflow.com/a/40693139/1987178>
As a workaround, register an event watcher to render the screen from
another thread.
Since the whole event loop is blocked during resizing, the screen
content is not refreshed (on Windows and MacOS) until resizing ends.
"adb reverse" currently does not work over tcpip (i.e. on a device
connected by "adb connect"):
<https://issuetracker.google.com/issues/37066218>
To work around the problem, if the call to "adb reverse" fails, then
fallback to "adb forward", and reverse the client/server roles.
Keep the "adb reverse" mode as the default because it does not involve
connection retries: when using "adb forward", the client must try to
connect successively until the server listens.
Due to the tunnel, every connect() will succeed, so the client must
attempt to read() to detect a connection failure. For this purpose, when
using the "adb forward" mode, the server initially writes a dummy byte,
read by the client.
Fixes <https://github.com/Genymobile/scrcpy/issues/5>.
The serial is needed for many server actions, but this is an
implementation detail, so the caller should not have to provide it on
every call.
Instead, store the serial in the server instance on server_start().
This paves the way to implement the "adb forward" fallback properly.
The SDL mouse wheel event seems inconsistent between horizontal and
vertical scrolling.
> Movements to the left generate negative x values and to the right
> generate positive x values. Movements down (scroll backward) generate
> negative y values and up (scroll forward) generate positive y values.
<https://wiki.libsdl.org/SDL_MouseWheelEvent#Remarks>
Reverse the horizontal.
Fixes <https://github.com/Genymobile/scrcpy/issues/49>.
The text input control_event was initially designed for mapping
SDL_TextInputEvent, limited to 32 characters.
For simplicity, the copy/paste feature was implemented using the same
control_event: it just sends the text to paste.
However, the pasted text might have a length breaking some assumptions:
- on the client, the event max-size was smaller than the text
max-length,
- on the server, the raw buffer storing the events was smaller than the
max event size.
Fix these inconsistencies, and encode the length on 2 bytes, to accept
more than 256 characters.
Fixes <https://github.com/Genymobile/scrcpy/issues/10>.
Paste computer clipboard to the device on Ctrl+v.
The other direction (pasting the device clipboard to the computer) is
not implemented. It would require a communication channel from the
device to the computer, other than the socket used by the video stream.
The High DPI support is enabled by default, so that the renderer use the
full definition of High DPI screens.
However, there are still mouse coordinates problems on some MacOS having
High DPI support (but not all), so expose a way to disable it.
The decoder sometimes returned a non-zero value on error, but not on
every path.
Since we never use the value, always return 0 at the end (like in the
controller).
SDL_MouseWheelEvent does not provide the mouse location, so we used
SDL_GetMouseState() to retrieve it.
Unfortunately, SDL_GetMouseState() returns a position expressed in the
window coordinate system while the position filled in SDL events are
expressed in the renderer coordinate system. As a consequence, the
scroll was not applied at the right position on the device.
Therefore, convert the coordinate system.
See <https://stackoverflow.com/questions/49111054/how-to-get-mouse-position-on-mouse-wheel-event>.
Use high DPI if available.
Note that on Mac OS X, setting this flag is not sufficient:
> On Apple's OS X you must set the NSHighResolutionCapable Info.plist
> property to YES, otherwise you will not receive a High DPI OpenGL
> display.
<https://wiki.libsdl.org/SDL_CreateWindow#flags>
On user request to quit, two kinds of blocking calls must be interrupted
on the server:
1. the reads from and writes to the socket;
2. the call to MediaCodec.dequeueOutputBuffer().
The former case was handled by calling shutdown() on the socket from the
client, but the latter was not managed.
There is no easy way to wake this call properly, so just terminate the
process from the client (i.e. send SIGTERM on Linux) instead.
The server is copied to /data/local/tmp/scrcpy-server.jar and executed
on the device.
As soon as we are connected, we can unlink (rm) it from /data/local/tmp,
to keep the device clean.
