qutebrowser's primary maintainer, The-Compiler, is currently working part-time on qutebrowser, funded by donations.
To sustain this for a long time, your help is needed! See the GitHub Sponsors page or alternative donation methods for more information. Depending on your sign-up date and how long you keep a certain level, you can get qutebrowser t-shirts, stickers and more!
Contributing to qutebrowser
Important
|
Bandwidth for pull request review is currently quite limited. If you want to contribute where it’s most needed, please consider reviewing or testing open pull requests. |
I <3
[<3
in HTML]
contributors!
This document contains guidelines for contributing to qutebrowser, as well as useful hints when doing so.
If anything mentioned here would prevent you from contributing, please let me know, and contribute anyways! The guidelines are meant to make life easier for me, but if you don’t follow everything in here, I won’t be mad at you. In fact, I will probably change it for you.
If you have any problems, I’m more than happy to help! You can get help in several ways:
-
Send a mail to the mailing list at qutebrowser@lists.qutebrowser.org (optionally subscribe first).
-
Join the IRC channel
#qutebrowser
on Libera Chat (webchat, via Matrix).
Finding something to work on
Chances are you already know something to improve or add when you’re reading this. It might be a good idea to ask on the mailing list or IRC channel to make sure nobody else started working on the same thing already.
If you want to find something useful to do, check the issue tracker. Some pointers:
If you prefer C++ or Javascript to Python, see the relevant issues which involve work in those languages:
-
C++ (mostly work on Qt, the library behind qutebrowser)
There are also some things to do if you don’t want to write code:
-
Help the community, e.g., on the mailinglist and the IRC channel.
-
Improve the documentation.
-
Help on the website and graphics (logo, etc.).
Using git
qutebrowser uses git for its development. You can clone the repo like this:
git clone https://github.com/qutebrowser/qutebrowser.git
If you don’t know git, a git cheatsheet might come in
handy. Of course, if using git is the issue which prevents you from
contributing, feel free to send normal patches instead, e.g., generated via
diff -Nur
.
Getting patches
The preferred way of submitting changes is to fork the repository and to submit a pull request.
If you prefer to send a patch to the mailinglist, you can generate a patch based on your changes like this:
git format-patch origin/main <1>
-
Replace
main
by the branch your work was based on, e.g.,origin/develop
.
Running qutebrowser
After installing qutebrowser in a virtualenv,
you can run .venv/bin/qutebrowser --debug --temp-basedir
to test your changes
with debug logging enabled and without affecting existing running instances.
Alternatively, you can install qutebrowser’s dependencies system-wide and run
python3 -m qutebrowser --debug --temp-basedir
.
Useful utilities
Checkers
qutebrowser uses tox to run its unittests and several linters/checkers.
Currently, the following tox environments are available:
-
Tests using pytest:
-
py38
,py39
, …: Run pytest for python 3.8/3.9/… with the system-wide PyQt. -
py38-pyqt515
, …,py38-pyqt65
: Run pytest with the given PyQt version (py39-*
etc. also works). -
py38-pyqt515-cov
: Run with coverage support (other Python/PyQt versions work too).
-
-
flake8
: Run various linting checks via flake8. -
vulture
: Run vulture to find unused code portions. -
pylint
: Run pylint static code analysis. -
pyroma
: Check packaging practices with pyroma. -
eslint
: Run ESLint javascript checker. -
check-manifest
: Check MANIFEST.in completeness with check-manifest. -
mkvenv
: Bootstrap a virtualenv for testing. -
misc
: Runscripts/misc_checks.py
to check for:-
untracked git files
-
VCS conflict markers
-
common spelling mistakes
-
-
mypy for static type checking:
-
mypy-pyqt5
run mypy with PyQt5 installed -
mypy-pyqt6
run mypy with PyQt6 installed
-
The default test suite is run with tox
; the list of default
environments is obtained with tox -l
.
Please make sure the checks run without any warnings on your new contributions.
There’s always the possibility of false positives; the following techniques are useful to handle these:
-
Use
_foo
for unused parameters, withfoo
being a descriptive name. Using_
is discouraged. -
If you think you have a good reason to suppress a message, then add the following comment:
# pylint: disable=message-name
Note you can add this per line, per function/class, or per file. Please use the smallest scope which makes sense. Most of the time, this will be line scope.
-
If you really think a check shouldn’t be done globally as it yields a lot of false-positives, let me know! I’m still tweaking the parameters.
Running specific tests
While you are developing you often don’t want to run the full test suite each time.
Specific test environments can be run with tox -e <envlist>
.
Additional parameters can be passed to the test scripts by separating
them from tox
arguments with --
.
Examples:
# run only pytest tests which failed in last run:
tox -e py38 -- --lf
# run only the end2end feature tests:
tox -e py38 -- tests/end2end/features
# run everything with undo in the generated name, based on the scenario text
tox -e py38 -- tests/end2end/features/test_tabs_bdd.py -k undo
# run coverage test for specific file (updates htmlcov/index.html)
tox -e py38-cov -- tests/unit/browser/test_webelem.py
Specifying the backend for tests
Tests automatically pick the backend based on what they manage to import. If you have both backends available and you would like the tests to be run with a specific one you can set either of a) the environment variable QUTE_TESTS_BACKEND , or b) the command line argument --qute-backend, to the desired backend (webkit/webengine).
If you need an environment with webkit installed to do testing while we still support it (see #4039) you can re-use the docker container used for the CI test runs which has PyQt5Webkit installed from the archlinux package archives. Examples:
# Get a bash shell in the docker container with
# a) the current directory mounted at /work in the container
# b) the container using the X11 display :27 (for example, a Xephyr instance) from the host
# c) the tox and hypothesis dirs set to somewhere in the container that it can write to
# d) the system site packages available in the tox venv so you can use PyQt
# from the OS without having to run the link_pyqt script
docker run -it -v $PWD:/work:ro -w /work -e QUTE_TESTS_BACKEND=webkit -e DISPLAY=:27 -v /tmp/.X11-unix:/tmp/.X11-unix -e TOX_WORK_DIR="/home/user/.tox" -e HYPOTHESIS_EXAMPLES_DIR="/home/user/.hypothesis/examples" -e VIRTUALENV_SYSTEM_SITE_PACKAGES=True qutebrowser/ci:archlinux-webkit bash
# Start a qutebrowser temporary basedir in the appropriate tox environment to
# play with
tox exec -e py-qt5 -- python3 -m qutebrowser -T --backend webkit
# Run tests, passing positional args through to pytest.
tox -e py-qt5 -- tests/unit
Profiling
In the scripts/dev/ subfolder there’s run_profile.py
which profiles the
code and shows a graphical representation of what takes how much time.
It uses the built-in Python cProfile module. It launches a qutebrowser instance, waits for it to exit and then shows the graph.
Available methods for visualization are:
-
SnakeViz (
--profile-tool=snakeviz
, the default) -
pyprof2calltree and KCacheGrind (
--profile-tool=kcachegrind
) -
gprof2dot (
--profile-tool=gprof2dot
, needsdot
from Graphviz and feh) -
tuna (
--profile-tool=tuna
)
You can also save the binary profile data to a file (--profile-tool=none
).
Debugging
There are some useful functions for debugging in the qutebrowser.utils.debug
module.
When starting qutebrowser with the --debug
flag, you also get useful debug
logs. You can add --logfilter [!]category[,category,…]
to restrict
logging to the given categories.
With --debug
there are also some additional debug-*
commands available,
for example :debug-all-objects
and :debug-all-widgets
which print a list of
all Qt objects/widgets to the debug log — this is very useful for finding
memory leaks.
Useful websites
Some resources which might be handy:
Documentation of used Python libraries:
Related RFCs and standards:
HTTP
-
RFC 7230 - Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing (Errata)
-
RFC 7231 - Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content (Errata)
-
RFC 7232 - Hypertext Transfer Protocol (HTTP/1.1): Conditional Requests (Errata)
-
RFC 7233 - Hypertext Transfer Protocol (HTTP/1.1): Range Requests (Errata)
-
RFC 7234 - Hypertext Transfer Protocol (HTTP/1.1): Caching (Errata)
-
RFC 7235 - Hypertext Transfer Protocol (HTTP/1.1): Authentication (Errata)
-
RFC 5987 - Character Set and Language Encoding for Hypertext Transfer Protocol (HTTP) Header Field Parameters (Errata)
-
RFC 6266 - Use of the Content-Disposition Header Field in the Hypertext Transfer Protocol (HTTP) (Errata)
-
RFC 6265 - HTTP State Management Mechanism (Cookies) (Errata)
Hints
Python and Qt objects
For many tasks, there are solutions available in both Qt and the Python standard library.
In qutebrowser, the policy is usually to use the Python libraries, as they provide exceptions and other benefits.
There are some exceptions to that:
-
QThread
is used instead of Python threads because it provides signals and slots. -
QProcess
is used instead of Python’ssubprocess
. -
QUrl
is used instead of storing URLs as string, see the handling URLs section for details.
When using Qt objects, two issues must be taken care of:
-
Methods of Qt objects report their status with their return values, instead of using exceptions.
If a function gets or returns a Qt object which has an
.isValid()
method such asQUrl
orQModelIndex
, there’s a helper functionensure_valid
inqutebrowser.utils.qtutils
which should get called on all such objects. It will raisequtebrowser.utils.qtutils.QtValueError
if the value is not valid.If a function returns something else on error, the return value should carefully be checked.
-
Methods of Qt objects have certain maximum values based on their underlying C++ types.
To avoid passing too large of a numeric parameter to a Qt function, all numbers should be range-checked using
qutebrowser.qtutils.check_overflow
, or by other means (e.g. by setting a maximum value for a config object).
The object registry
The object registry in qutebrowser.utils.objreg
is a collection of
dictionaries which map object names to the actual long-living objects.
There are currently these object registries, also called scopes:
-
The
global
scope, with objects which are used globally (config
,cookie-jar
, etc.). -
The
tab
scope with objects which are per-tab (hintmanager
,webview
, etc.). Passing this scope toobjreg.get()
selects the object in the currently focused tab by default. A tab can be explicitly selected by passingtab=tab-id, window=win-id
to it.
A new object can be registered by using
objreg.register(name, object[, scope=scope, window=win-id,
tab=tab-id])
. An object should not be registered twice. To update it,
update=True
has to be given.
An object can be retrieved by using objreg.get(name[, scope=scope,
window=win-id, tab=tab-id])
. The default scope is global
.
All objects can be printed by starting with the --debug
flag and using the
:debug-all-objects
command.
The registry is mainly used for command handlers, but it can also be useful in places where using Qt’s signals and slots mechanism would be difficult.
Logging
Logging is used at various places throughout the qutebrowser code. If you add a new feature, you should also add some strategic debug logging.
Unlike other Python projects, qutebrowser doesn’t use a logger per file, instead it uses custom-named loggers.
The existing loggers are defined in qutebrowser.utils.log
. If your feature
doesn’t fit in any of the logging categories, simply add a new line like this:
foo = getLogger('foo')
Then in your source files, do this:
from qutebrowser.utils import log
...
log.foo.debug("Hello World")
The following logging levels are available for every logger:
critical |
Critical issue, qutebrowser can’t continue to run. |
error |
There was an issue and some kind of operation was abandoned. |
warning |
There was an issue but the operation can continue running. |
info |
General informational messages. |
debug |
Verbose debugging information. |
Commands
qutebrowser has the concept of functions which are exposed to the user as commands.
Creating a new command is straightforward:
from qutebrowser.api import cmdutils
...
@cmdutils.register(...)
def foo():
...
The commands arguments are automatically deduced by inspecting your function.
If the function is a method of a class, the @cmdutils.register
decorator
needs to have an instance=...
parameter which points to the (single/main)
instance of the class.
The instance
parameter is the name of an object in the object registry, which
then gets passed as the self
parameter to the handler. The scope
argument
selects which object registry (global, per-tab, etc.) to use. See the
object registry section for details.
There are also other arguments to customize the way the command is
registered; see the class documentation for register
in
qutebrowser.api.cmdutils
for details.
The types of the function arguments are inferred based on their default values,
e.g., an argument foo=True
will be converted to a flag -f
/--foo
in
qutebrowser’s commandline.
The type can be overridden using Python’s function annotations:
@cmdutils.register(...)
def foo(bar: int, baz=True):
...
Possible values:
-
A callable (
int
,float
, etc.): Gets called to validate/convert the value. -
A python enum type: All members of the enum are possible values.
-
A
typing.Union
of multiple types above: Any of these types are valid values, e.g.,typing.Union[str, int]
.
You can customize how an argument is handled using the @cmdutils.argument
decorator after @cmdutils.register
. This can, for example, be used to
customize the flag an argument should get:
@cmdutils.register(...)
@cmdutils.argument('bar', flag='c')
def foo(bar):
...
For a str
argument, you can restrict the allowed strings using choices
:
@cmdutils.register(...)
@cmdutils.argument('bar', choices=['val1', 'val2'])
def foo(bar: str):
...
For typing.Union
types, the given choices
are only checked if other types
(like int
) don’t match.
The following arguments are supported for @cmdutils.argument
:
-
flag
: Customize the short flag (-x
) the argument will get. -
value
: Tell qutebrowser to fill the argument with special values: -
value=cmdutils.Value.count
: Thecount
given by the user to the command. -
value=cmdutils.Value.win_id
: The window ID of the current window. -
value=cmdutils.Value.cur_tab
: The tab object which is currently focused. -
completion
: A completion function (seequtebrowser.completions.models.*
) to use when completing arguments for the given command. -
choices
: The allowed string choices for the argument.
The name of an argument will always be the parameter name, with any trailing underscores stripped and underscores replaced by dashes.
Handling URLs
qutebrowser handles two different types of URLs: URLs as a string, and URLs as
the Qt QUrl
type. As this can get confusing quickly, please follow the
following guidelines:
-
Convert a string to a QUrl object as early as possible, i.e., directly after the user did enter it.
-
Use
utils.urlutils.fuzzy_url
if the URL is entered by the user somewhere. -
Be sure you handle
utils.urlutils.FuzzyError
and display an error message to the user.
-
-
Convert a
QUrl
object to a string as late as possible, i.e., before displaying it to the user.-
If you want to display the URL to the user, use
url.toDisplayString()
so password information is removed. -
If you want to get the URL as string for some other reason, you most likely want to add the
QUrl.EncodeFully
andQUrl.RemovePassword
flags.
-
-
Name a string URL something like
urlstr
, and aQUrl
something likeurl
. -
Mention in the docstring whether your function needs a URL string or a
QUrl
. -
Call
ensure_valid
fromutils.qtutils
whenever getting or creating aQUrl
and take appropriate action if not. Note the URL of the current page always could be an invalid QUrl (if nothing is loaded yet).
Running valgrind on QtWebKit
If you want to run qutebrowser (and thus QtWebKit) with
valgrind, you’ll need to pass --smc-check=all
to it or
recompile QtWebKit with the Javascript JIT disabled.
This is needed so valgrind handles self-modifying code correctly:
This option controls Valgrind’s detection of self-modifying code. If no checking is done and a program executes some code, overwrites it with new code, and then executes the new code, Valgrind will continue to execute the translations it made for the old code. This will likely lead to incorrect behavior and/or crashes.
…
Note that the default option will catch the vast majority of cases. The main case it will not catch is programs such as JIT compilers that dynamically generate code and subsequently overwrite part or all of it. Running with all will slow Valgrind down noticeably.
Setting up a Windows Development Environment
-
Install Python 3.9.
-
Install PyQt via
pip install PyQt5
. -
Install git from the git-scm downloads page. Try not to enable
core.autocrlf
, since that will causeflake8
to complain a lot. Use an editor that can deal with plain line feeds instead. -
Clone your favourite qutebrowser repository.
-
To install tox, open an elevated cmd, enter your working directory and run
pip install -rmisc/requirements/requirements-tox.txt
.
Note that the flake8
tox env might not run due to encoding errors despite having LANG/LC_* set correctly.
Rebuilding the website
If you want to rebuild the website, run ./scripts/asciidoc2html.py --website <outputdir>
.
Chrome URLs
With the QtWebEngine backend, qutebrowser supports several chrome:// urls which can be useful for debugging.
Info pages:
-
chrome://device-log/ (QtWebEngine >= 6.3)
-
chrome://gpu/
-
chrome://sandbox/ (Linux only)
Misc. / Debugging pages:
-
chrome://dino/
-
chrome://histograms/
-
chrome://network-errors/
-
chrome://tracing/ (QtWebEngine >= 5.15.3)
-
chrome://ukm/ (QtWebEngine >= 5.15.3)
-
chrome://user-actions/ (QtWebEngine >= 5.15.3)
-
chrome://webrtc-logs/ (QtWebEngine >= 5.15.3)
Internals pages:
-
chrome://accessibility/
-
chrome://appcache-internals/ (QtWebEngine < 6.4)
-
chrome://attribution-internals/ (QtWebEngine >= 6.4)
-
chrome://blob-internals/
-
chrome://conversion-internals/ (QtWebEngine >= 5.15.3 and < 6.4)
-
chrome://indexeddb-internals/
-
chrome://media-internals/
-
chrome://net-internals/ (QtWebEngine >= 5.15.4)
-
chrome://process-internals/
-
chrome://quota-internals/
-
chrome://serviceworker-internals/
-
chrome://webrtc-internals/
Crash/hang pages:
-
chrome://crash/ (crashes the current renderer process!)
-
chrome://gpuclean/ (crashes the current renderer process!)
-
chrome://gpucrash/ (crashes qutebrowser!)
-
chrome://gpuhang/ (hangs qutebrowser!)
-
chrome://kill/ (kills the current renderer process!)
QtWebEngine internals
This is mostly useful for qutebrowser maintainers to work around issues in Qt - if you don’t understand it, don’t worry, just ignore it.
The hierarchy of widgets when QtWebEngine is involved looks like this:
-
qutebrowser has a
WebEngineTab
object, which is its abstraction over QtWebKit/QtWebEngine. -
The
WebEngineTab
has a_widget
attribute, which is the QWebEngineView -
That view has a QWebEnginePage for everything which doesn’t require rendering.
-
The view also has a layout with exactly one element (which also is its
focusProxy()
). -
Qt 5: That element is the RenderWidgetHostViewQtDelegateWidget (it inherits QQuickWidget) - also often referred to as RWHV or RWHVQDW. It can be obtained via
sip.cast(tab._widget.focusProxy(), QQuickWidget)
. -
Qt 6: That element is the WebEngineQuickWidget (it inherits QQuickWidget). It can be obtained via
tab._widget.focusProxy()
. -
Calling
rootObject()
on that gives us the QQuickItem where Chromium renders into (?). With it, we can do things like.setRotation(20)
.
Style conventions
qutebrowser’s coding conventions are based on PEP8 and the Google Python style guidelines with some additions:
-
The Raise: section is not added to the docstring.
-
Methods overriding Qt methods (obviously!) don’t follow the naming schemes.
-
Everything else does though, even slots.
-
Docstrings should look like described in PEP257 and the google guidelines.
-
Class docstrings have additional Attributes:, Class attributes: and Signals: sections.
-
In docstrings of command handlers (registered via
@cmdutils.register
), the description should be split into two parts by using//
- the first part is the description of the command like it will appear in the documentation, the second part is "internal" documentation only relevant to people reading the sourcecode.Example for a class docstring:
"""Some object. Attributes: blub: The current thing to handle. Signals: valueChanged: Emitted when a value changed. arg: The new value """
Example for a method/function docstring:
"""Do something special. This will do something. // It is based on http://example.com/. Args: foo: ... Return: True if something, False if something else. """
-
The layout of a module should be roughly like this:
-
Shebang (
#!/usr/bin/python
, if needed) -
Copyright
-
GPL boilerplate
-
Module docstring
-
Python standard library imports
-
PyQt imports
-
qutebrowser imports
-
functions
-
classes
-
-
The layout of a class should be like this:
-
docstring
-
__magic__
methods -
other methods
-
overrides of Qt methods
-
-
Type hinting: the qutebrowser codebase uses type hints liberally to enable static type checking and autocompletion in editors.
-
We use mypy in CI jobs to perform static type checking.
-
Not all of the codebase is covered by type hints currently. We encourage including type hints on all new code and even adding type hints to existing code if you find yourself working on some that isn’t already covered. There are some module specific rules in the mypy config file,
.mypy.ini
, to make type hints strictly required in some areas. -
More often than not mypy is correct when it raises issues. But don’t be afraid to add
# type: ignore[...]
statements or casts if you need to. As an optional part of the language not all type information from third parties is always correct. Mypy will raise a new issue if it spots an "ignore" statement which is no longer needed because the underlying issue has been resolved. -
One area where we have to take particular care is in code that deals with differences between PyQt5 and PyQt6. We try to write most code in a way that will work with either backend but when you need to deal with differences you should use a pattern like:
if machinery.IS_QT5: ... # do PyQt5 specific implementation else: # PyQt6 ... # do PyQt6 specific implementation
then you have to tell mypy to treat
machinery.IS_QT5
as a constant value then run mypy twice to cover both branches. There are a handful of variables inqutebrowser/qt/machinery.py
that mypy needs to know about. There are tox jobs (mypy-pyqt5
andmypy-pyqt6
) that take care of telling mypy to use them as constants.
-
Checklists
These are mainly intended for myself, but they also fit in here well.
New Qt release
-
Run all tests and check nothing is broken.
-
Check the Qt bugtracker and make sure all bugs marked as resolved are actually fixed.
-
Update own PKGBUILDs based on upstream Archlinux updates and rebuild.
-
Update recommended Qt version in
README
. -
Grep for
WORKAROUND
in the code and test if fixed stuff works without the workaround. -
Check relevant qutebrowser bugs and check if they’re fixed.
New PyQt release
-
See above.
-
Update
tox.ini
/.github/workflows/ci.yml
to test new versions.
qutebrowser release
-
Make sure there are no unstaged changes and the tests are green.
-
Make sure all issues with the related milestone are closed.
-
Mark the milestone as closed.
-
Consider updating the completions for
content.headers.user_agent
inconfigdata.yml
. -
Minor release: Consider updating some files from main:
-
misc/requirements/
andrequirements.txt
-
scripts/
-
-
Update changelog in main branch and ensure the correct version number has
(unreleased)
-
If necessary: Update changelog in release branch from main.
Automatic release via GitHub Actions (starting with v3.0.0):
-
Double check Python version in
.github/workflows/release.yml
-
Run the
release
workflow on themain
branch, e.g. viagh workflow run release -f release_type=major
(release_type
can bemajor
,minor
orpatch
; you can also overridepython_version
)
Manual release:
-
Make sure Python is up-to-date on build machines.
-
Run
./.venv/bin/python3 scripts/dev/update_version.py {major,minor,patch}
. -
Run the printed instructions accordingly.
Post release:
-
Update
qutebrowser-git
PKGBUILD if dependencies/install changed. -
Add unreleased future versions to changelog
-
Update IRC topic
-
Announce to qutebrowser and qutebrowser-announce mailinglist.
-
Post announcement mail to subreddit
-
Post on the website formerly known as Twitter