| *** xinliang has quit IRC | 01:16 | |
| *** mtreinish has quit IRC | 01:28 | |
| *** xinliang has joined #refstack | 01:29 | |
| *** mtreinish has joined #refstack | 01:35 | |
| *** pilgrimstack has joined #refstack | 08:10 | |
| *** andreaf_ has joined #refstack | 08:53 | |
| *** andreaf has quit IRC | 08:54 | |
| *** designbybeck has quit IRC | 08:54 | |
| *** andreaf_ is now known as andreaf | 08:54 | |
| *** designbybeck has joined #refstack | 09:02 | |
| *** edmondsw has joined #refstack | 12:00 | |
| *** pilgrimstack has quit IRC | 12:32 | |
| *** pilgrimstack has joined #refstack | 12:33 | |
| *** pilgrimstack has quit IRC | 14:05 | |
| *** pilgrimstack has joined #refstack | 14:36 | |
| *** openstack has joined #refstack | 18:25 | |
| *** openstack has joined #refstack | 19:07 | |
| *** openstackstatus has joined #refstack | 19:07 | |
| *** ChanServ sets mode: +v openstackstatus | 19:07 | |
| *** rocky_g has joined #refstack | 21:19 | |
| *** edmondsw has quit IRC | 21:19 | |
| *** designbybeck has quit IRC | 21:57 | |
| *** selena_092 has joined #refstack | 22:35 | |
| selena_092 | hi ..something is wrong with refstack, im providing it with this list of 2016.08 tests to run https://refstack.openstack.org/api/v1/guidelines/2016.08/tests?target=platform&type=required&alias=true&flag=false but its running tests that are not even part of this list http://paste.openstack.org/show/585253/ | 22:36 |
|---|---|---|
| selena_092 | for instance its running 15:32:32 {0} tempest.api.compute.admin.test_aggregates.AggregatesAdminTestJSON.test_aggregate_add_host_get_details [0.573120s] ... FAILED | 22:36 |
| selena_092 | which is not refstack 2016.08 test | 22:36 |
| selena_092 | i have tempest tag 13.0.0 or 10.0.0 | 22:36 |
| selena_092 | im running it as ./refstack-client test --insecure -c '.tempest/etc/tempest.conf' -v -- $CUSTOM_TEST | 22:38 |
| selena_092 | actualy this ./refstack-client test --insecure -c '.tempest/etc/tempest.conf' -v --test-list list-of-tests | 22:38 |
| catherineD | selena_092: Hi, yea we find a bug that with the update to use ostestr we actually run test with test id.... | 22:38 |
| catherineD | we have a fix in https://review.openstack.org/#/c/381232/ | 22:38 |
| GheRivero | I should have a clean path tomorrow morning | 22:39 |
| GheRivero | s/path/patch/ | 22:39 |
| catherineD | GheRivero: thx | 22:39 |
| selena_092 | catherineD: thanks | 22:39 |
| selena_092 | will wait till tomorrow then | 22:39 |
| GheRivero | selena_092: the patch is usable if you wanna try | 22:40 |
| catherineD | selena_092: so by running tests with ID , RefStack would run some extra tests .. the test time would be long ... but the important thing is the DefCore test are still being run .. | 22:40 |
| selena_092 | are there any gating jobs for refstack like it tests 2016.08 or 2016.01 tests and verifies it has correct result in 0.json? | 22:40 |
| catherineD | selena_092: test best way to send the data up to resfstack and concentrate debug on the DefCore failed tests ... | 22:41 |
| selena_092 | catherineD: thanks .. | 22:42 |
| selena_092 | just to be clear, if i use GheRivero patch, i will be running tests with their UUID ...correct? | 22:42 |
| catherineD | selena_092: if the test being sent up to RefStack with signature ... you can delete the interim test results to keep our system clean ... that would be greatly appreciated .. | 22:42 |
| catherineD | The reason why sending the result up to RefStack is the best way checking for pass/fail is to make sure that we take into account the tests with aliases ... | 22:43 |
| selena_092 | gotcha..im on it | 22:44 |
| catherineD | GheRivero: we will try to merge your patch as soon as possible once you upload a new patch ... thanks again for submitting the patch ! | 22:46 |
| selena_092 | catherineD: here is result uploaded https://refstack.openstack.org/#/results/588cf6b4-d407-4bda-800d-409ad2d89724 | 22:53 |
| catherineD | selena_092: which guideline do you intented to check this against? | 22:54 |
| selena_092 | 2016.08 | 22:54 |
| selena_092 | so that extra tests that are not part of list dont effect result,, however it just takes so long for them to complete :/ | 22:55 |
| selena_092 | hopefully after the UUID it will be gone !!!! | 22:55 |
| catherineD | selena_092: yea I understand that is the side effect .. this should not have happened ... | 22:56 |
| selena_092 | we used to run refstack in 1 hour..now it takes 3 hours ..thanks for taking look into it catherineD :) | 22:57 |
| catherineD | so I see it take 7551 second for you to run ,,, and it run 527 tests when it supposes to only run about 222 test | 22:57 |
| catherineD | selena_092: I know | 22:58 |
| catherineD | please concentrate your debug on the "Not passed" test that do not have flaggeg in front of it | 22:59 |
| selena_092 | catherineD: yes.i know the issue.its due to floating IP issue...i know the fix and it was a dirty environment due to many tempest runs :) | 23:00 |
| catherineD | for example in the "not passed" test conpute-server-list ... please ignor tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_filtered_by_ip_regex | 23:00 |
| catherineD | because it has a flag in-front of it ... it is a flagged test and is not required to pass | 23:01 |
| catherineD | selena_092: Are you testing against "OpenStack Powered Compute" or "Openstack Pwered Platform" of 2016.08? | 23:03 |
| selena_092 | Powered Compute of 2016.08 | 23:04 |
| selena_092 | we dont intend to test against powered platfom as it is swift-less | 23:05 |
| catherineD | if that is the case use this https://refstack.openstack.org/api/v1/guidelines/2016.08/tests?target=compute&type=required&alias=true&flag=false | 23:06 |
| catherineD | it will not include the swift tests | 23:06 |
| selena_092 | ah ..thank you so much :) | 23:06 |
| selena_092 | this will save more time | 23:06 |
| catherineD | when you look at the data ... take a look at page 13 of https://docs.google.com/presentation/d/1CoegiPzVp8G4VLdrOYTX9ZI_E-PTRx0gWVpuF9ElKRU/edit#slide=id.g85e96ad23_0_20 | 23:08 |
| catherineD | choose "OpenStack Powered Compute" that way your "not passed" will not include the swift tests .. | 23:09 |
| catherineD | the default is "OpenStack Powered Platform" | 23:10 |
| selena_092 | thank you :) you are indeed most helpful in all of IRC i have seen :) | 23:10 |
| *** rocky_g has quit IRC | 23:10 | |
| catherineD | thank you ! one last point, when you have your final result .. try to pin it to 201608 and OpenStack Powered Conpute by following the instruction on slide 14 of the above link | 23:12 |
| catherineD | that way you data always show a PASS results ... | 23:12 |
| selena_092 | catherineD: sometimes i feel like its easy to make refstack pass if someone wants to...like if a user sets run_validation=False, most compute tests are fairly straightforward | 23:12 |
| selena_092 | it just my feeling that refstack should along with 0.json, also check with tempest.conf used by user.. | 23:13 |
| selena_092 | also what tempest was used..that should be something done with refstack-client , hide the tempest repo from user, and only ask him/her to submit a tempest.conf ..and refstack-client clones tempest repo | 23:15 |
| selena_092 | anyway thanks again for help :) i will wait for another run and see it results get better | 23:20 |
| *** selena_092 has quit IRC | 23:21 | |
| catherineD | selena_092: Thanks for the feedback. | 23:34 |
| *** markvoelker has quit IRC | 23:36 | |
| openstackgerrit | Ghe Rivero proposed openstack/refstack-client: Fix regex to find test names https://review.openstack.org/381232 | 23:37 |
| catherineD | selena_092: The result authselena_092: The result authenticity and integrity concern were discussed since day 1 of RefStack. The decision at the time is that the purpose of the testing is to expose interops issues based on honor foundation. And, the real customers of the clouds will be those to judge the interoperability of the claimed interops clouds. | 23:40 |
| catherineD | Of course, we are not there yet either from the Guideline or testing point of view for interops testing. Until the day when RefStack can initiate to run test on a cloud. We will always have this issue. | 23:41 |
| catherineD | RefStack does not require uploading of tempest.conf because of security concern of user/environment credentials may be exposed. | 23:42 |
| catherineD | selena_092: Your feedback shows that you care about RefStack test and for that we really appreciate ... | 23:45 |
| GheRivero | catherineD: I just proposed a new patch. Hope my sleep deprivation didn't play with me | 23:47 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!