| *** markvoelker has joined #refstack | 00:55 | |
| *** pvaneck has quit IRC | 01:04 | |
| *** markvoelker has quit IRC | 01:36 | |
| *** serverascode has quit IRC | 02:28 | |
| *** serverascode has joined #refstack | 02:31 | |
| openstackgerrit | Merged openstack/refstack: Update documentation to install Node.js v4 https://review.openstack.org/359473 | 06:22 |
|---|---|---|
| pilgrimstack | Hi all | 08:41 |
| pilgrimstack | I have something strange, maybe a mistake from me… | 08:42 |
| pilgrimstack | According to the readme, I use https://refstack.openstack.org/api/v1/guidelines/2016.01/tests?target=compute&type=required&alias=true&flag=false to test my compute infrastructure | 08:43 |
| pilgrimstack | I run the tests and I get this result : https://refstack.openstack.org/#/results/0c1b079b-efcf-42b1-8e7b-2985c62568c7 | 08:43 |
| pilgrimstack | IE The first failed required test is tempest.api.compute.test_authorization.AuthorizationTestJSON.test_create_server_fails_when_tenant_incorrect which is missing in the guidelines. | 08:44 |
| pilgrimstack | So to summarise, it seems there is a delta between 2016.01 guidelines and the 2016.01 results overview. | 08:46 |
| pilgrimstack | Do I miss something ? | 08:46 |
| *** openstackgerrit has quit IRC | 08:48 | |
| *** openstackgerrit has joined #refstack | 08:49 | |
| *** hockeynut has quit IRC | 10:23 | |
| dmellado | pilgrimstack: did you check the UUID | 11:29 |
| dmellado | now and then some tests do change the name, so that might've happened | 11:29 |
| dmellado | in any case from your results I see that you're failing quite some more tests ;) | 11:29 |
| *** MarkBaker has quit IRC | 12:02 | |
| *** luzC has quit IRC | 12:18 | |
| *** luzC has joined #refstack | 12:18 | |
| *** mtreinish has quit IRC | 12:27 | |
| *** mtreinish has joined #refstack | 12:30 | |
| *** edmondsw has joined #refstack | 12:34 | |
| *** markvoelker has joined #refstack | 12:48 | |
| *** mgagne_ is now known as mgagne | 12:54 | |
| pilgrimstack | dmellado: what do you mean by UUID? UUID of the test? | 13:24 |
| mtreinish | dmellado, pilgrimstack: that test was removed from tempest: http://git.openstack.org/cgit/openstack/tempest/commit/?id=a0a9b24de0a2b2c48ce95a988c43cc9affe50652 | 13:39 |
| pilgrimstack | mtreinish: does it means I should use an older version of tempest to run a 2016.01 guidelines? | 13:42 |
| mtreinish | pilgrimstack: that I can't answer, I don't really know | 13:58 |
| *** hockeynut has joined #refstack | 14:11 | |
| *** esker has joined #refstack | 14:13 | |
| *** esker has quit IRC | 14:32 | |
| *** esker has joined #refstack | 14:35 | |
| *** esker has quit IRC | 14:36 | |
| *** esker has joined #refstack | 14:38 | |
| *** esker has quit IRC | 14:38 | |
| *** openstackgerrit has quit IRC | 14:48 | |
| *** openstackgerrit has joined #refstack | 14:49 | |
| *** hockeynut has quit IRC | 14:51 | |
| *** hockeynut has joined #refstack | 15:27 | |
| *** smonov has joined #refstack | 15:33 | |
| *** hockeynu_ has joined #refstack | 15:38 | |
| *** hockeynut has quit IRC | 15:41 | |
| catherineD | pilgrimstack: The result test list from https://refstack.openstack.org/api/v1/guidelines/2016.01/tests?target=compute&type=required&alias=true&flag=false does not include tempest.api.compute.test_authorization.AuthorizationTestJSON.test_create_server_fails_when_tenant_incorrect | 15:44 |
| catherineD | Therefore tempest.api.compute.test_authorization.AuthorizationTestJSON.test_create_server_fails_when_tenant_incorrect was never run or attempt to run (since it no longer exists as mtreinish: has indicated) | 15:45 |
| pilgrimstack | yes but when I use the 2016.01 guideline filter on the result submission, It looks like it's required | 15:46 |
| catherineD | although tempest.api.compute.test_authorization.AuthorizationTestJSON.test_create_server_fails_when_tenant_incorrect listed as the first failed test (since it was never ran) it has a "flag" in-front of it. So it is a flagged test an never counted toward pass/fail status | 15:46 |
| pilgrimstack | mmmh ok | 15:46 |
| catherineD | on the fail list ... please ignore any tests that has a "flag" in-front of them | 15:47 |
| catherineD | DefCore can not remove tests from an approved guideline therefore trouble (tests with testing issues) or removed tests are marked as flagged and they are not counted toward pass/fail status. | 15:49 |
| catherineD | pilgrimstack: just to clarify. RefStack test results only include passed tests. Tests that are not in the pass list will be listed in the "not passed" list. "Not passed" can mean test failed, test skip or not tested ... | 16:00 |
| pilgrimstack | Ok I see | 16:04 |
| pilgrimstack | I just saw the sentence "Excluding flagged tests, this cloud passes 93.8% (91/97) of the required tests." | 16:04 |
| pilgrimstack | Does it means that to have the Compliant on YES, we should have 100% on "Excluding flagged tests" ? | 16:05 |
| *** hockeynu_ has quit IRC | 16:28 | |
| *** hockeynut has joined #refstack | 16:29 | |
| catherineD | pilgrimstack: yes | 16:31 |
| catherineD | so for your result there are 6 more tests needed to pass .. those are the tests need debug | 16:33 |
| *** hockeynut has quit IRC | 17:05 | |
| *** hockeynut has joined #refstack | 17:08 | |
| *** hockeynut has quit IRC | 17:19 | |
| *** hockeynut has joined #refstack | 17:50 | |
| *** hockeynut has quit IRC | 17:51 | |
| *** hockeynut has joined #refstack | 17:53 | |
| *** hockeynut has quit IRC | 18:13 | |
| *** pvaneck has joined #refstack | 18:29 | |
| *** krotscheck has left #refstack | 18:52 | |
| -openstackstatus- NOTICE: The Gerrit service on review.openstack.org is restarting to implement some performance tuning adjustments, and should return to working order momentarily. | 19:47 | |
| *** smonov has quit IRC | 20:47 | |
| *** xinliang has quit IRC | 20:53 | |
| *** xinliang has joined #refstack | 21:09 | |
| openstackgerrit | Merged openstack/refstack: Add missing dependencies for Ubuntu 14/16 https://review.openstack.org/360064 | 21:40 |
| *** hockeynut has joined #refstack | 22:59 | |
| *** hockeynut has quit IRC | 23:10 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!