[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Extended tests are now part of the test queue for Origin



Runtime is ~45m (including setup and running the suite against both
plugins).  There is periodic breakage, mainly around rebases.  A
post-merge job is currently used to catch these kinds of issues, and
members of the networking team use
[testonlyextended][extended:networking] for PRs that they propose.
Ideally the networking tests would run automatically pre-merge to
catch things early without manual intervention being required.

On Tue, Apr 19, 2016 at 9:16 AM, Clayton Coleman <ccoleman redhat com> wrote:
> What's the total runtime of the networking suite now?  Are we seeing
> things merged that break it?
>
>> On Apr 19, 2016, at 1:59 AM, Maru Newby <marun redhat com> wrote:
>>
>> Can we consider adding the networking extended suite to the tests run
>> on every merge?  It would like need to run as a 4th job, though.
>>
>> On Wed, Apr 6, 2016 at 6:56 PM, Clayton Coleman <ccoleman redhat com> wrote:
>>> The test job for Origin now runs a subset of extended tests in a third
>>> worker whenever you add [test] (it'll be the third job when you click
>>> through to Jenkins).  The job won't block merge (yet) but please don't
>>> merge if there are failures.
>>>
>>> The tests that are run are a subset of the full extended suite and are
>>> defined in test/extended/conformance.sh.  It includes all tests that
>>> are flagged with [Conformance] in both Kube and Origin. If you have
>>> extended tests in an area of function that demonstrate a core scenario
>>> and should be required on all merges, please add [Conformance] to the
>>> test description.  Try to focus on the main path - don't just add the
>>> label to all the variations (those tests are run in the
>>> origin_extended job already post merge).
>>>
>>> Treat any flakes in the extended tests like our other tests and file
>>> issues with test/flake label.  The upstream OOM score adjuster bug is
>>> causing deployments with very fast hooks to flake right now, and there
>>> may be a quota flake as well that we're looking into.
>>>
>>> _______________________________________________
>>> dev mailing list
>>> dev lists openshift redhat com
>>> http://lists.openshift.redhat.com/openshiftmm/listinfo/dev


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]