Contributors guides =================== Git functionalities ------------------- Clone a remote git repository ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To clone a remote git repository, simply execute: .. code:: bash git clone link/to/repo This will download the git repository and checkout the main branch. There is two ways of cloning a git repository: HTTPS and SSH. We recommend you use SSH, it requires you to set up SSH keys and deploy them on your git server profile. But you won't have to pass your git credentials at every push/pull/fetch command. If you still don't want to use SSH, you can skip the rest of the section. .. note:: make sure you set up the SSH key configuration correctly You need to generate a pair of SSH keys, add the public key to your gitlab SSH keys (or deploy SSH keys), then set up SSH to use the generated private key for connection. Here is an example of ssh config file (put in your home folder ``~/.ssh/config``): :: Host gitlab.imt-atlantique.fr User git Port 22 IdentityFile ~/.ssh/id_rsa # path to your generated public key .. note:: if you use your default keys, you don't have to set up the SSH config file Set-up username and email ~~~~~~~~~~~~~~~~~~~~~~~~~ Git won't let you commit anything while you have not set up the following constants: - ``user.name``: the username you will use on the online git repository (also will show up in the commits you make) - ``user.email``: your e-mail Those can be set up globally using: .. code:: bash git config --global user.name USERNAME git config --global user.email EMAIL Or for a specific project by executing the following from the git directory: .. code:: bash git config --local user.name USERNAME git config --local user.email EMAIL Set up mergetool and difftool ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``git difftool`` is a utility to show and resolve the differences between two commits. ``git mergetool`` is a utility to show and resolve the conflicts between two commits for an ongoing merge. You can set up once the text editor (such as ``meld``) to use for both utilities using the following commands: .. code:: bash git config --global diff.tool EDITOR git config --global merge.tool EDITOR git config --global difftool.prompt false # disable prompt before executing difftool Then those commands can be used. If the commands do not work, you may need to customize the configuration of the editor in the ``.gitconfig`` file (found in your home directory on unix), depending on the editor you chose. Example configuration for the editor ``meld``: .. code:: ini # Add the following to your .gitconfig file. [diff] tool = meld [difftool] prompt = false [difftool "meld"] cmd = meld "$LOCAL" "$REMOTE" [merge] tool = meld [mergetool "meld"] # Choose one of these 2 lines (not both!) explained below. cmd = meld "$LOCAL" "$MERGED" "$REMOTE" --output "$MERGED" cmd = meld "$LOCAL" "$BASE" "$REMOTE" --output "$MERGED" On the last two lines, the difference is about what version of the merged file you want to display in the centre. ``$MERGED`` will contain the partially merged file and the conflict information while ``$BASE`` will contain the version before any merging. In both cases, the centre version is the one that will be used upon commiting the merge. Create feature or hotfix branches ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ As mentionned previously, you shall work routinely on a temporary feature branch. To create it the first time, checkout first the branch you will use as the base (in our case, ``main``), then create your branch spanning from it: .. code:: bash git checkout main # checkout the 'main' branch git pull # update local version of the branch 'main' git checkout -b nnnn-xxxx # create and checkout a new branch 'nnnn-xxxx' from current branch ('main') Same case when working on a hotfix (see :doc:`this ` for more information). Keep local branches up-to-date ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ There are multiple ways to keep your git local repository up-to-date with the online repository ``origin``. The simplest way **if you are well aware of the git structure or are retrieving changes on a branch you have not locally modified** is to execute a pull: .. code:: bash git checkout BRANCH_NAME # checkout `BRANCH_NAME` you want to update git pull # pull the changes from the online repository `origin` If you are unsure, you can first fetch the latest version of every branch: ``git fetch`` This command will fetch the branches on all online repositories configured (in our case only ``origin``). You can then check the state of the branches using a utility like ``gitg``. To apply the changes of the remote version of a branch to your local version, you can merge them: .. code:: bash git fetch # fetch the list of changes from the repositories (without applying them) git checkout BRANCH_NAME # checkout `BRANCH_NAME` you want to update git merge REMOTE_NAME/BRANCH_NAME # merge the changes from the online repository REMOTE_NAME (e.g 'origin') onto the branch BRANCH_NAME Work on feature or hotfix branch ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ When working, you should commit and push your changes as often as possible. To do that, you should select the changes you want to commit using any of the following: - ``git add FILENAME``: add all changes made to ``FILENAME`` since last commit (or add ``FILENAME`` to the tracked files if it wasn't before) - ``git add FOLDER_NAME``: add all changes and all filenames contained in ``FOLDER_NAME`` - ``git add -u``: add all changes made to tracked files - ``git add -p [OTHER_OPTIONS]``: add all changes interactively, you can select precisely which chunks of code are being commited - ``git add --all``: add all changes and files to match the current working tree (**Not advised except on the initial commit of a new repository**) - ``git mv FILENAME NEW_FILENAME``: move/rename the filename ``FILENAME`` to ``NEW_FILENAME`` - ``git rm [--cached] FILENAME``: remove filename ``FILENAME`` from the tracked files (as well as from the current directory if the option ``--cached`` is not used) You can check the status of your git branch at any time: ``git status`` If you want to abort your commit operation, you can execute: ``git reset HEAD``. .. warning:: it will mess up the ``git mv`` operations as the files will keep their new name When you are satisifed with the changes you have selected, you can perform the commit: ``git commit -m COMMIT_MESSAGE [-m COMMIT_DETAILS] [-m ISSUE_REFS]`` (see `this `__ for help in formatting your commits) If you have any second thought about your performed commits, you can easily undo them **while they have not been commited** with: ``git reset HEAD~`` (will cancel the last unpushed commit) Once your commit performed, if you are happy with the changes, push them: ``git push origin LOCAL_BRANCH_NAME`` (for example: ``git push origin 1000-fix-broken-imports``) Rebase of your branch ~~~~~~~~~~~~~~~~~~~~~ **N.B: Do not do the following if you don't have a solid understanding of git in general and the branches structure of the repository you work on!** When you are working on your branch, and changes are being made on the common branch you are based on (e.g ``main``) but you are not ready to merge your changes, or you want your changes to take into account the changes of the common branch retrospectively, you could perform a rebase of your branch on the common branch: .. code:: bash git checkout main # checkout 'main' branch git pull # make sure the local version of 'main' is up-to-date with 'origin' repository git checkout nnnn-xxxx # feature branch we want to rebase git rebase main # modify branch 'nnnn-xxxx' history so its unmerged changes appear after the last `main` commits After performing a rebase, you may have to **force** the next push you will make of this branch as the online repository will have to also modify previous history to cope with the rebase: .. code:: bash git push --force origin nnnn-xxxx`` .. note:: Never force push on a common branch as it will override the remote history of the branch .. warning:: This rebase operation modifies the history of the branch on which we are rebasing, so it must never be performed on any common branch. This would otherwise force all other users with changes spanning from this common branch to update their whole git history, potentially creating conflicts in their previous commits. Merge changes on common branch ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ When you have completed the implementation of a new feature, or made any significant change to the code and you feel confident enough to add them on the common ``main`` branch, you should consider merging the branches. Remember to check you have updated the :ref:`changelog ` file, adding the new feature(s) you developed to it. First, make sure you have commited and pushed your changes on your working branch. Then, it is important to make sure you get the up-to-date version of the branch on which you want to merge (e.g ``main``). .. code:: bash git checkout main # checkout the 'main' branch git pull # make sure the local version of 'main' is up-to-date with 'origin' repository You are ready to merge. You may encounter some conflicts as the ``main`` branch may have had some changes since you last split from it or merged to it. In order to limit those conflicts and keep the git history as clean as possible, you can rebase your branch on the common branch, to take into account any changes into any common branch that has changed since your last merge/split, before merging. In this case, follow the section on `rebase <#rebase-of-your-branch>`__. To perform the actual merge: .. code:: bash git checkout main # checkout branch 'main' where we want to merge 'nnnn-xxxx' git merge [MODE] nnnn-xxxx # merge branch 'nnnn-xxxx' into 'main' There are different modes to perform a merge: - ``--ff`` (default mode): will attempt fast-forward merge if possible, fail back on a 3-way merge if conflicts are detected - ``--ff-only``: will only perform merge if no conflicts are detected - ``--no-ff``: will always perform a 3-way merge If any conflicts appear, the merge will be suspended. You can then resolve the conflicts either by hand by modifying directly the files reported to have conflicts, or by using the command: ``git mergetool`` The latter will open the configured editor to ease the resolution of the conflicts, showing one file at a time, both conflicting versions on the sides, and the merged version in the centre. Once you have resolved all conflicts, you can create a commit for the merge: .. code:: bash git commit -m MESSAGE If you want to abort the merge, you can execute: .. code:: bash git merge --abort When you have finished merging your changes on any common branch, you should push them **as soon as possible**. Any delay before pushing changes increases the chances of conflicts for you and the other persons working on this branch. If you are a developer and not a maintainer, **pushing a merge on a common branch will prompt you to issue a merge request**. This merge request will freeze the push until a maintainer manually validate the changes brought about by this push. To access the merge request associated with your merge operation, you can open the link returned to you by the push in the terminal. Alternatively, you can merge your branch and issue a merge request at the same time on ``gitlab`` website by creating a new merge request, specifying the branches to merge. .. note:: * you can ask the merge request to delete the branch being merged on the common branch (default behaviour) * we do not recommend stashing all changes in one commit when merging When you merge a feature branch, it should mean the feature implementation is over. You may then want to delete the local version of this branch in order to keep your number of local branches as small as possible: .. code:: bash git branch -d BRANCH_NAME # delete local branch 'BRANCH_NAME' git branch -d -r REPOSITORY/BRANCH_NAME # delete remote tracking of 'BRANCH_NAME' from repository 'REPOSITORY' (e.g 'origin') Repurpose an outdated branch ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If for some reasons, you want to work on a branch which is completely outdated but the branch was not attached to an up-to-date branch, and you don't care about the last unmerged commits. You basically want to create a new branch with the same name, then you should delete the existing branch and create a new one from any commit/branch you want: .. code:: bash git branch -d BRANCH_NAME # delete local version of the branch git checkout XXXX # checkout a branch or a specific commit as the new starting point git switch -c BRANCH_NAME # create and switch to a new branch named `BRANCH_NAME` starting from `XXXX` If you need to push changes to this branch, you should first delete the remote version of the branch, then push your local branch on the remote: .. code:: bash git push origin --delete BRANCH_NAME # delete 'origin' remote version of the branch git push origin BRANCH_NAME # push local version of the branch on the remote 'origin' N.B: When deleting a remote version of a branch, the previous commits done on this branch will remain on the remote, they will simply not be associated to the branch name anymore. Release a new version ~~~~~~~~~~~~~~~~~~~~~ .. warning:: This section is obsolete and based on another git workflow .. todo:: update section to current workflow When the development done on ``main`` branch is stable enough, and significant changes have been made, we can release an new version of the code on the git ``main`` branch. It simply requires to merge the ``main`` branch on the ``main`` branch, and then tag this new release with a :ref:`version number ` (do not forget to update the :ref:`changelog ` and :ref:`package requirements ` beforehand): .. code:: bash git checkout main # checkout the 'main' branch git merge --no-ff main # merge the 'main' branch inside in 3-way mode git tag VERSION # tag the new release with VERSION If the release process necessitates other steps, you should create a temporary release branch ``release/VERSION`` (``VERSION`` indicating the :ref:`version ` being released). You will them perform the steps necessary for releasing the new version on this branch. After the release is complete, you will merge it on the ``main`` branch, tag it, then merge it on ``main``, then delete the release branch. .. code:: bash git checkout main # checkout the 'main' branch which you want to release git checkout -b release/VERSION # create a release branch and checkout it ... # work on the release process git checkout main # checkout 'main' branch git merge --no-ff release/VERSION # merge the release branch inside in 3-way mode git tag VERSION # tag the new release with VERSION git checkout main # checkout 'main' branch git merge --no-ff release/VERSION # merge the release branch inside in 3-way mode git branch -d release/VERSION # delete the local release branch git push origin --delete release/VERSION # delete the remote release branch (only if it was pushed) As you can see, the merge for a release is made in 3-way mode. As a matter of fact, **never merge on the** ``main`` **branch in fast-forward as it must only contain the commits of each release**. Also **any changes on the** ``main`` **branch must come from either the** ``main`` **branch or an hotfix branch** (see :doc:`this `). Python environments ------------------- While you can use the default system-wide python environment when working on a single project, although you can have permission issues due to you system install, when working on multiple projects with different requirements (packages, package versions, python version) you can bump into a bunch of compatibility issues. In order to limit the possible conflicts with any other projects, it is recommended you set up `pyenv `__ and a `pyenv-virtualenv `__ or any other mean of containing this project and keep it separate. You can follow the installation instructions of `pyenv `__, then `pyenv-virtualenv `__. This will enable you to install multiple versions of python and set up virtual environments which contain their own python interpreter and their own python packages. This way, projects may coexist side-by-side, each with their own set of installed packages and their python version. Our linters ----------- flake8 ~~~~~~ Usage: ``flake8 [OPTIONS] root_src_folder`` Recommended options: - ``--max-line-length n``: set maximum line length to ``n`` (79 in our case) - ``--ignore=rule_1,rule_2,...``: ignore the given set of rules (recommend E203,W503 for `black <#black>`__ compatibility) isort ~~~~~ Usage: ``isort [OPTIONS] root_source_folder`` Options when applying changes: - ``--atomic``: to avoid syntax errors when applying changes Options for checking for changes to be made: - ``--diff``: print the changes to be made instead of applying them directly - ``--check``: check the files for unsorted/unformatted imports and returns boolean result (can be used atop ``--diff``) Options to use in any situation: - ``--profile black``: to make isort operations compatible with `black <#black>`__ - ``--combine-star``: ensure that if a star import is present, nothing else is imported from that namespace - ``--use-parentheses``: use parenthesis for line continuation instead of slashes - ``-m VERTICAL_HANGING_INDENT``: organize multi-line imports so only one import per line is used - ``-w n``: set maximum line length to ``n`` (as already said, we recommend 79) black ~~~~~ Usage: ``black [OPTIONS] root_src_folder`` Option when checking for changes to be made: - ``--check`` Option to use in any situation: - ``--line-length n``: set maximum line length to ``n`` (as already said, we recommend 79) N.B: For many IDE, black can be directly integrated to auto-format the code.