How much testing is done on each pull request before it is merged?


This is the home for all documentation related to the Gaia UI tests written and maintained by the Web QA Team.

Please see the Gaia UI Testing Task List for a list of daily tasks, in order of priority.

Please see Gaia UI Tests Guidelines on MDN for information on running and writing Gaia UI tests.

Running on Desktop B2G

Please see Testing on Desktop B2G on MDN.

Running on Device

Please see Testing on Devices on MDN.

Note: This is very much a work in progress. Do not take anything below as gospel until it has been finalized.

The first priority, when supporting Gaia UI Testing, is to ensure that our existing tests are accurate and reliable. To that end, we need to monitor the jobs that run in our CI and address any that are failing or are xpassing (passing but marked as expected to fail).

To review the jobs for failures and xpasses, complete the following steps:

Choose the job to review. This involves choosing...
0 0

Continuous integration and pull requests are two important concepts for almost any development team. Besides many other benefits, ensuring code stability and quality, ease of collaboration with other developers and fast release cycles are some of the key aspects. This post describes a solution to combine and Jenkins for automated builds of pull requests and reporting back the status to Stash. Thereby we build not the last commit of the pull request but merge the pull request automatically into the target branch. This gives a much better view if the result of merging the pull request will compile and if other coding guidelines are followed, e.g. static code analysis checks.

[Update 1: 2015-08-02]

[Update 2: 2015-08-13]

Starting with version 1.18 of Pull Request Notifier for Stash the url for the pull request is now provided as a parameter (PULL_REQUEST_URL). The parameter can now be past directly and we don’t need to build the parameter manually. In order to set...
0 0
0 0

Using a distributed version control system has been a fun learning experience, especially the collaborative nature surrounding open source project work. As I stumble upon more light bulb moments, I’m trying to make sure to jot them down here. One of them was the process for validating that someone’s pull request contains valid code by testing it before performing a merge. I figure there’s a couple ways to do this:

Unit testing – See if the code itself is functional Integration testing – See if the code works against a running system and with other bits of code Ad-hoc (human) testing – Run the code manually to see if it works Eyeball testing – Read the code and look for obvious errors or conflicts

Each type of testing has its place. Eyeball testing, as an example, has worked fine for spelling and grammar changes. I don’t need to run those through a battery of tests.

It is important to test using whatever method is appropriate. Anything that enters the master...

0 0

We’ve been working with git for more than a year.
The SCM was migrated from SVN, with all its history.
Our habits were migrated as well.

Our flow is (was) fairly simple:
master branch is were we deploy our code from.
When working on a feature, we create a feature branch. Several people can work on this branch.
Some create private local branch. Some don’t.
Code review is done one-on-one. One member asks another to join and walks through the code.

Recently I introduced to the team, with the help of a teammate the Pull Requests concept.
It takes some time to grasp the methodology and see the benefits.
However, I already start seeing improvements in collaboration, code quality and coding behaviors.

Better collaboration
When a person does a change and calls for a pull request, the entire team can see the change.
Everyone can comment and give remarks.
Discuss changes before they are merged to the main branch. ...
0 0

I’m a big proponent of pre-merge code reviews. From my experience consulting for teams in problematic projects, I can say that (along with daily standup meetings) pre-merge code reviews are one of the most effective and yet fairly easy changes a team can introduce to radically improve the condition of the project. And even if you consider your project to be healthy, there’s always room for improvement.

The Rules

Never Push to Master
Always push to a separate branch per logical unit (story, feature, bug, optimisation, refactor, improvement). Branches are easy to make and easy to merge when you use git (and you apply some of tips further down in this post).

Never Merge Your Own Branch
This helps to ensures that code is in fact reviewed. If you are caught merging into master, you will order pizza for the whole team.

Review Work in Progress First
When you are finished with a task, you notify the other team members that your...

0 0

On 03/20/2013 10:57 AM, Klaas Freitag wrote:

> On 19.03.2013 22:28, Frank Karlitschek wrote:

>> I completely agree


> I agree partly. Of course all code reviews should be done as accurate as

> possible, and that involves patch applying and testing.

I agree with Klaas

> But:
>>> A problem is that I often see: »thumbs up, code looks good but I
>>> didn’t test«
>>> Now I’m sure you’re all code wizards who can parse and render code in
>>> their head … but actually checking out the merge request, open
>>> ownCloud, in an actual browser (or two), and click around a bit if it
>>> really *works* is the crucial part here.
> I don't think that testing of the code changes is always so easy that it
> could be verified by "clicking around a bit". I also don't think that
> code reviews can and should replace a qa step which is an important part

0 0

When developing software, source control makes our lives so much easier. From tracking our changes to introducing code collaboration, it helps increase our productivity. Furthermore, the fact that there are a number of different tools available - Subversion, Perforce, Mercurial, and so on - make it even better by providing us options from which to choose.

In this series, we’re looking specifically at Git and using Bitbucket and how they can help us in our day-to-day work. In this article, we’re going to be specifically focused on using Bitbucket for feature discussion, bug tracking, release tagging, and more.

One of the major features of Git - and, thus, Bitbucket - is the idea of pull requests. In this article, we’re going to be looking at pull requests and how they not only benefit us from a source control standpoint, but from a peer review standpoint, as well.

A Pull Request Primer

When someone issues a pull request into your project, it means...

0 0

Having your first pull request (PR) accepted in an open source project is one of the best feelings there is when it comes to working with open source code. There's nothing like working with other people, contributing to a project you care about, and showing off your crazy coding skills through a great PR. This guide will establish some guidelines on how to make your PRs amazing!

A lot of people wonder how to find a project to contribute and/or make a pull request to. Luckily, this question has an easy answer: contribute to projects you use. You should not be browsing GitHub for random projects and trying to contribute to one of them. You should write code as you normally do, use other peoples' libraries, and when you find a bug or something you'd like to add, then think about contributing.

Contributing to open source projects should be an entirely natural and logical process. Here's some code you use, here's a specific thing you see that would make it better, let's go...

0 0

Pull requests are an excellent tool for fostering code review. If you're using Github for team projects, you should be using these extensively.

Many people don't realise that you can make pull requests between two branches of the same repository (the so-called "shared repository model"). For team projects, this is preferable over the "fork and pull" model because it's simpler: there are less branches and remotes to keep track of.

A good practice is for someone else to merge your code into the mainline, ensuring 2 sets of eyeballs review each feature. This is simple to organise when working in pairs, but in larger teams you may need a system for determining who reviews what.

Sample workflow

Here's a sample workflow demonstrating the use of pull requests.

Work on a story

Create a new branch for the current story you are working on:

(master) $ git checkout -b feature/masquerading

It's important to use a new branch for pull requests...

0 0

Note: The improvements discussed in this post will be rolling out throughout the week.

Lots of stuff this sprint, lets jump in…

Pull requests

The pull request experience has some major updates this release, bringing some really powerful diff capabilities, a new commenting experience, and an entirely refreshed UI.

Redesigned UI

When opening a pull request, the new look and feel is evident immediately. We’ve reorganized the header to summarize all of the critical states and actions, making them accessible from every view in the experience.


The Overview now highlights the pull request description and makes it easier than ever to give feedback. Events and comments are shown with the newest items on top to help reviewers see the latest changes and comments front and center. Policies, work items, and reviewers are all provided in detail and reorganized to be more clear and concise.



0 0

Pull requests are a feature that makes it easier for developers to collaborate using Bitbucket. They provide a user-friendly web interface for discussing proposed changes before integrating them into the official project.

In their simplest form, pull requests are a mechanism for a developer to notify team members that they have completed a feature. Once their feature branch is ready, the developer files a pull request via their Bitbucket account. This lets everybody involved know that they need to review the code and merge it into the master branch.

But, the pull request is more than just a notification—it’s a dedicated forum for discussing the proposed feature. If there are any problems with the changes, teammates can post feedback in the pull request and even tweak the feature by pushing follow-up commits. All of this activity is tracked directly inside of the pull request.

Compared to other collaboration models, this formal solution for sharing commits makes...

0 0

Code review processes can be wildly different across open-source projects. The best place to start is to look for a CONTRIBUTING file in the top level directory for the repository. Large open-source projects usually have one of these. In it, maintainers lay out the rules for engagement when it comes to reporting issues, requesting features, asking for support, or contributing code to the repository.

While reading through a CONTRIBUTING file might sound like a waste of time, it’s actually going to save you time in the long run. If you start contributing code to a repository right away, without going through their guidelines, you may be wasting time away by not conforming to their conventions. If you file an issue with insufficient information, maintainers are going to have to ask for more details, point you to the CONTRIBUTING file so that you actually read it, and generally waste time. At the same time, you’ll be wasting your own time by having to go in repeatedly and update...

0 0

I've used Homu a lot, and I really enjoy the workflow they have, which will automatically try to build + merge once seeing an approval. I'd be delighted to see a similar flow come to GitLab, so I'd like to propose a companion feature.

On this issue of running builds on merged code, I think I agree with the proposal by @sytses, but just to be sure, here's what I would like to see:

A project setting that says: 'Before merging a MR, run builds on a merge with the target' or similar. All commits will still be tested as they are now: builds should run on each commit itself, not on a merge commit. When the merge button is hit, then a merge commit will be created to merge the branch into the target, and builds will run on the merge commit. If the builds succeed, then fast-forward the target branch into the merge commit, merging the MR and closing it successfully. If any builds fail, then abort the merge and report the build failures back to the MR.

(This seems to be mode 2...

0 0
0 0