| *** pilgrimstack has joined #refstack | 07:57 | |
| *** pilgrimstack has quit IRC | 08:02 | |
| *** pilgrimstack has joined #refstack | 08:09 | |
| *** andrey-mp has joined #refstack | 10:12 | |
| *** edmondsw has joined #refstack | 11:59 | |
| *** pilgrimstack1 has joined #refstack | 14:06 | |
| *** pilgrimstack has quit IRC | 14:08 | |
| *** pilgrimstack1 has quit IRC | 14:20 | |
| *** pilgrimstack has joined #refstack | 14:21 | |
| *** pilgrimstack1 has joined #refstack | 14:24 | |
| *** pilgrimstack has quit IRC | 14:26 | |
| *** andrey-mp has quit IRC | 15:33 | |
| *** davidlenwell has quit IRC | 16:28 | |
| *** davidlenwell has joined #refstack | 16:40 | |
| *** ChanServ sets mode: +o davidlenwell | 16:40 | |
| *** andrey-mp has joined #refstack | 16:59 | |
| luzC | @hogepodge are you around? | 17:34 |
|---|---|---|
| hogepodge | luzC: yes | 17:35 |
| luzC | Hello, I had been looking at the stack trace information from the subunit file/ strict schema validation on Tempest and the best way to patch it... | 17:36 |
| luzC | At the beginning I thought I could get the schema from the error trace, but I can't (not accurately anyway) | 17:37 |
| luzC | I can get the tempest service client(servers, quotas, etc) and the method... | 17:39 |
| luzC | I can patch Tempest at tempest/lib/common/rest_client.py, validate_response, if validation is called by the service_client.method then skip the validation... | 17:43 |
| luzC | This is create a file with [service_client, method], also add a flag to tempest configuration file, change rest_client.py file to verify the configuration flag and go through the file looking for service.method | 17:45 |
| luzC | I can provide a sequence diagram for current and proposed implementation, but I wanted to touch base with you | 17:46 |
| luzC | will this be an acceptable fix? | 17:48 |
| luzC | where you thinking in something different | 17:48 |
| luzC | ? | 17:48 |
| andrey-mp | luzC: what is 'service_client.method' ? | 18:00 |
| *** pvaneck has joined #refstack | 18:01 | |
| luzC | the service clients are under tempest.lib.services.compute.* there are several clients, each have several methods depending on the action/ API call. For instance it can be the "servers_client" and "show_server" method... | 18:03 |
| luzC | anyway all the service clients call the rest_client (from common library) which makes the schema validation | 18:04 |
| andrey-mp | so it's a different patches for different runs, right? | 18:04 |
| luzC | the difference will be the service_client.method... it depends on the error message/ analysis | 18:06 |
| luzC | the mechanism to bypass the validation will be the same | 18:06 |
| andrey-mp | is it suitable to make one patch for tempest? for example - configuration parameter with list of needed methods? | 18:07 |
| andrey-mp | (in the future such changset can be commited to tempest...) | 18:10 |
| luzC | yes, I agree, the QA community reject to change Tempest since it is OK that it fails, vendors shouldn't add stuff in the API response, so that's why this is only a waiver and not included into Tempest branch directly :( | 18:11 |
| andrey-mp | but we can develop a method that will be suitable for QA team | 18:13 |
| luzC | this patch idea will be to add a config option to pass the list of serviceClient.method that should not do the validation... that is because we have a previous failed subunit file... | 18:15 |
| andrey-mp | it's better than patch tempest for each run. each run can copy base tempest.conf to temporary file and run one copy of tempest with this new configuration file. | 18:17 |
| andrey-mp | because there are many negative facts to patch tempest for each run. | 18:17 |
| luzC | yes, that's true | 18:19 |
| luzC | and as you suggested on the spec it would be good to patch tempest during refstack/tempest installation | 18:21 |
| andrey-mp | yeah | 18:25 |
| luzC | I'm updating the spec... | 18:25 |
| luzC | thanks for the idea Andrey :-) | 18:25 |
| andrey-mp | btw, if you will not analyze list of failed tests and skip all strict checking by the flag? | 18:28 |
| andrey-mp | why is it not suitable? | 18:28 |
| hogepodge | luzC: sorry, I got pulled away | 18:58 |
| hogepodge | luzC: Can you link me to the tempest files you would patch? | 19:01 |
| hogepodge | luzC: I'm mainly interested in chasing down exactly which endpoints have been modified | 19:02 |
| hogepodge | andrey-mp: that was my original idea, but it was rejected. However, as part of this solution refstack-client could carry a patch that could do such a thing | 19:04 |
| andrey-mp | hogepodge: this is good news! luzC can describe this idea in the spec and refstack-client will make only one tempest patch in the installation script! | 19:07 |
| luzC | yes, @hogepodge the patch will be on tempest/lib/common/rest_client.py -- validate_response method, bypassing validation when the caller of the function is a given "service.method"... the exact endpoints modified is still on the first Tempest run and matches the service and method from the initial failed subunit file (so, the failed subunit file analysis and gathering of information still applies) | 20:11 |
| luzC | I'll describe the flow, steps, and classes to be modified on the spec | 20:12 |
| andrey-mp | luzC: as I understood first idea was not to analyze results. first idea was to skip all strict checking. | 20:28 |
| luzC | ah ok... | 20:32 |
| luzC | so, at this point both solutions are possible: 1. Skip all validations for all APIs 2. Analyze initial set of results, skip some validations | 20:35 |
| luzC | of course option 1 is the easiest one... | 20:37 |
| andrey-mp | yeah | 20:37 |
| *** andrey-mp has quit IRC | 20:42 | |
| luzC | Andrey, what would be the flow for option 2? | 20:47 |
| luzC | just patch Tempest at installation time to skip all validations when a flag is present... so if a vendor wants to use the waiver, they just have to add the configuration option to their configuration file? | 20:48 |
| luzC | sorry I meant option 1 | 20:49 |
| luzC | @hogepodge what do you think? | 20:56 |
| *** rockyg has joined #refstack | 21:05 | |
| hogepodge | andrey-mp: luzC: turning off all strict checking is not the direction we should be going in. I understand that it's possible to do the difference of strict/non-strict rest results to see what failed because of checking | 21:24 |
| luzC | hogepodge ok, I'll make the proposal of option #2 - skip validations as needed... | 21:27 |
| hogepodge | luzC: I'm thinking to run the tests unmodified, then perhaps only run the ones that failed because of strict checking | 21:28 |
| hogepodge | luzC: are you going to be in San Antonio next week? | 21:28 |
| luzC | yes | 21:29 |
| luzC | I'll be here | 21:29 |
| hogepodge | Let's set aside some time then to work through some of these things in person too. I like option 2. | 21:30 |
| luzC | ok, sound good | 21:32 |
| luzC | I'll get a prototype just to get started from there | 21:32 |
| *** rockyg has quit IRC | 21:49 | |
| *** oomichi has quit IRC | 22:41 | |
| *** oomichi has joined #refstack | 22:50 | |
| *** oomichi has quit IRC | 23:31 | |
| *** oomichi has joined #refstack | 23:40 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!