| *** agopi has quit IRC | 00:01 | |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224 | 00:37 |
|---|---|---|
| *** agopi has joined #oooq | 01:45 | |
| *** agopi has quit IRC | 01:54 | |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224 | 02:38 |
| *** pgadiya has joined #oooq | 03:35 | |
| *** pgadiya has quit IRC | 03:35 | |
| *** yolanda has joined #oooq | 03:44 | |
| *** yolanda_ has quit IRC | 03:46 | |
| *** skramaja has joined #oooq | 04:08 | |
| *** rfolco has joined #oooq | 04:13 | |
| *** links has joined #oooq | 04:14 | |
| *** jbadiapa_ has joined #oooq | 04:20 | |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224 | 04:38 |
| *** jaganathan has joined #oooq | 04:51 | |
| *** jaganathan has quit IRC | 04:59 | |
| *** ykarel has joined #oooq | 05:05 | |
| *** udesale has joined #oooq | 05:09 | |
| *** saneax has joined #oooq | 05:26 | |
| *** quiquell has joined #oooq | 05:28 | |
| quiquell | sshnaidm: Good morning | 05:36 |
| *** udesale_ has joined #oooq | 05:40 | |
| *** udesale has quit IRC | 05:41 | |
| *** marios has quit IRC | 05:46 | |
| *** marios has joined #oooq | 05:47 | |
| *** udesale__ has joined #oooq | 06:01 | |
| *** udesale_ has quit IRC | 06:03 | |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224 | 06:38 |
| *** bogdando has joined #oooq | 06:43 | |
| *** ccamacho has joined #oooq | 06:57 | |
| *** ccamacho has quit IRC | 06:58 | |
| *** ccamacho has joined #oooq | 06:59 | |
| *** holser__ has joined #oooq | 07:08 | |
| *** kopecmartin has joined #oooq | 07:09 | |
| *** matbu has quit IRC | 07:09 | |
| *** jaosorior has joined #oooq | 07:10 | |
| *** florianf has joined #oooq | 07:14 | |
| *** matbu has joined #oooq | 07:14 | |
| *** jtomasek has joined #oooq | 07:14 | |
| bogdando | o/ folks. the reproduce quickstart script is missing in rdo ovb jobs. A known bug? | 07:16 |
| bogdando | rdo ci | 07:16 |
| *** panda|off is now known as panda | 07:17 | |
| quiquell | bogdando: panda Good mornig | 07:19 |
| quiquell | bogdando: Do you have the logs ? | 07:19 |
| bogdando | quiquell: https://review.openstack.org/572533 see the last 3-4 rdo ci runs in comments | 07:19 |
| bogdando | then expand to the job logs link in jenkins logs | 07:20 |
| bogdando | the repro quickstart sh is expected to be there | 07:20 |
| ykarel | bogdando, there is issue with collect logs:- https://logs.rdoproject.org/33/572533/5/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/Z659cc00387184293b4d2ee28ad68cb8c/quickstart_collect_logs.txt.gz | 07:25 |
| *** jaganathan has joined #oooq | 07:25 | |
| ykarel | 'tempest_format' is undefined" | 07:26 |
| quiquell | bogdando, ykarel: Yep, collect logs is failing and the reproducer is not dumped | 07:26 |
| bogdando | ykarel: ack, thanks. so it is a known issue | 07:26 |
| ykarel | no idea which patch caused it and is there already a fix for it. Good to create a bug for it | 07:27 |
| bogdando | ykarel: the fix is make collect logs invokations tolerate failures prolly | 07:27 |
| ykarel | any way there are issues in rdo-cloud currently, so reproducer script will not work | 07:28 |
| ykarel | bogdando, ack, can you create bug, so someone can look at it | 07:28 |
| bogdando | prolly like I do here https://review.openstack.org/572533 | 07:28 |
| *** amoralej|off is now known as amoralej | 07:28 | |
| bogdando | ykarel: ok, and I'll push a patch | 07:28 |
| ykarel | bogdando, Thanks | 07:29 |
| bogdando | https://review.openstack.org/#/c/574152/ | 07:37 |
| sshnaidm | bogdando, not sure, what is purpose of ^^ ? | 07:38 |
| bogdando | sshnaidm: there is a bug in the link | 07:39 |
| sshnaidm | bogdando, it doesn't explain though :) | 07:39 |
| ykarel | bogdando, doesn't seem it's fixing the issue:- 'tempest_format' is undefined, just ignoring | 07:40 |
| bogdando | sshnaidm: "10:26:38 AM GMT+3 - quiquell: bogdando, ykarel: Yep, collect logs is failing and the reproducer is not dumped" | 07:40 |
| *** tosky has joined #oooq | 07:40 | |
| sshnaidm | bogdando, yeah, I know, I'm asking how this patch is related | 07:40 |
| bogdando | ykarel: ok, please feel free to take it over :) | 07:40 |
| ykarel | bogdando, ok will check later if no one else takes it | 07:42 |
| sshnaidm | I'll fix | 07:45 |
| ykarel | sshnaidm, Thanks | 07:45 |
| *** ykarel is now known as ykarel|afk | 07:46 | |
| *** links has quit IRC | 07:50 | |
| ykarel|afk | so it's caused by:- https://review.openstack.org/#/c/566527/ | 07:51 |
| ykarel|afk | updated bug description | 07:52 |
| quiquell | panda: You there ? | 07:55 |
| panda | quiquell: more or less | 07:56 |
| quiquell | panda: Have a zuul question with Depends-On, it's not doing what I was expecting | 07:57 |
| quiquell | panda: https://review.openstack.org/#/c/571435/ | 07:57 |
| quiquell | panda: Job tripleo-ci-centos-7-containerized-undercloud-upgrades | 07:57 |
| panda | quiquell: did you try to ask gozer first ? He's usualy nicer | 07:58 |
| quiquell | panda: The Depends-On has trown's change as parent, but it's not working | 07:58 |
| quiquell | panda: Who is gozer ? | 07:58 |
| panda | quiquell: just kidding | 07:59 |
| panda | internet here today is even slower than the other days. 2 seconds constant lag in IRC | 08:00 |
| quiquell | panda: :-) excus me for my robotic sense of humor | 08:00 |
| quiquell | panda: np, just help me debug the review | 08:00 |
| quiquell | panda: Don't understand the zuul behaviour, maybe is because the change is in queens | 08:00 |
| quiquell | panda: and zuul ignores the other changes ? | 08:00 |
| panda | quiquell: that's probably it, I can be sure only when gertty fetches the change | 08:02 |
| panda | 2 minutes to fetch a change .... | 08:02 |
| panda | I'll try to reboot the router -_- | 08:02 |
| quiquell | panda: Going to debug the logs | 08:04 |
| *** skramaja_ has joined #oooq | 08:22 | |
| *** skramaja has quit IRC | 08:25 | |
| sshnaidm | ykarel|afk, bogdando https://review.openstack.org/#/c/574164/ | 08:27 |
| *** d0ugal has joined #oooq | 08:28 | |
| ykarel|afk | sshnaidm, containerized tempest available in release before master? | 08:29 |
| arxcruz|ruck | ykarel|afk: dude, you're da man! | 08:30 |
| sshnaidm | ykarel|afk, not sure | 08:30 |
| bogdando | sshnaidm: thank you! | 08:30 |
| ykarel|afk | sshnaidm, if no setting default to containers here is wrong | 08:30 |
| sshnaidm | ykarel|afk, why? | 08:30 |
| ykarel|afk | sshnaidm, tempest_container would be included always whenever tempest_format is unset | 08:31 |
| sshnaidm | ykarel|afk, ?? not sure I understand | 08:31 |
| ykarel|afk | "{% if run_tempest|bool and tempest_format|default('containers') == 'containers' -%}tempest_container{%- endif -%}" | 08:31 |
| sshnaidm | ykarel|afk, it's a default for docs generation only | 08:31 |
| ykarel|afk | so if tempest_format is unset, ^^ condition would be true | 08:32 |
| sshnaidm | ykarel|afk, I see your point now.. | 08:33 |
| ykarel|afk | am not sure how this will go with docs | 08:33 |
| ykarel|afk | if we include tempest_format with older releases | 08:33 |
| sshnaidm | ykarel|afk, I'll change the default to default in role | 08:34 |
| panda | quiquell: It should worlk | 08:34 |
| panda | quiquell: the devstack gate script tries to clone master if it doesn't find stable release for the repo | 08:34 |
| quiquell | panda: Look like is not taking trown's change | 08:35 |
| quiquell | panda: The --upgrade-from option is unknown | 08:35 |
| sshnaidm | chandankumar, arxcruz|ruck, any ideas why tempest_format is undefined in logs collection run? | 08:36 |
| arxcruz|ruck | the problem is that tempest_format is in featureset001 to check and create the docs properly | 08:36 |
| arxcruz|ruck | sshnaidm: however, it seems collect_logs doesn't call the validate-tempest, where this tempest_format is defined | 08:37 |
| arxcruz|ruck | i'll add a defaults('packages') | 08:37 |
| arxcruz|ruck | default('packages') | 08:37 |
| sshnaidm | arxcruz|ruck, added already | 08:37 |
| sshnaidm | arxcruz|ruck, it shouldn't call validate-tempest | 08:38 |
| sshnaidm | arxcruz|ruck, it runs as separate playbook, but seems like it doesn't have this var cached | 08:38 |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224 | 08:38 |
| arxcruz|ruck | sshnaidm: well, yeah, but when i wrote the patch i didn't knew it would make problems with the collect-logs | 08:38 |
| arxcruz|ruck | sorry, i'm fixing | 08:38 |
| panda | quiquell: where's john's change in the depends on ? | 08:38 |
| quiquell | panda: It's the parent of the Depends-On | 08:39 |
| ykarel|afk | sshnaidm, yes that would be better infact | 08:39 |
| sshnaidm | arxcruz|ruck, I have a patch already https://review.openstack.org/574164 , but we need to investigate why tempest_format is not cached | 08:39 |
| quiquell | panda: Look http://logs.openstack.org/35/571435/7/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/0277101/job-output.txt.gz#_2018-06-11_06_13_27_328050 | 08:39 |
| *** dtantsur|afk is now known as dtantsur | 08:40 | |
| arxcruz|ruck | sshnaidm: should it cache? it's a separate ansible-playbook call | 08:41 |
| panda | quiquell: I see not john's change up in the chain from the change you showed me | 08:41 |
| quiquell | panda: The parent of the "Depends-On" change no the change itself | 08:42 |
| panda | quiquell: from https://review.openstack.org/571435 I'm clicking on all the Depends-on I see, but none leads me to john's patch | 08:43 |
| quiquell | panda: "Related Changes" are the parents | 08:44 |
| panda | quiquell: ok, so it's not explicit depends on, it's a rebase | 08:46 |
| quiquell | panda: Yep, but it appears at ZUUL_CHANGES in the logs, but later on is like tripleo-ci is not pointing to it | 08:52 |
| sshnaidm | arxcruz|ruck, maybe it's better to set tempest_format for every featureset file when we run tempest | 08:54 |
| arxcruz|ruck | sshnaidm: sure, you're the ansible master :) | 08:55 |
| * arxcruz|ruck didn't finish his ansible course yet :( | 08:55 | |
| panda | quiquell: found the problem | 08:55 |
| panda | quiquell: you forgot a \ | 08:55 |
| panda | quiquell: commented the patch | 08:56 |
| quiquell | panda: Arggg thanks !!! | 08:57 |
| quiquell | :-) | 08:57 |
| panda | quiquell: yeah, it's like the missing ; in C code or unmatched brackets | 08:57 |
| quiquell | panda: Ok, fixed, let's recheck | 08:59 |
| arxcruz|ruck | panda: you know there's also a ; in greek that if you replace it on C code, will make your friends crazy looking for the error :P | 09:01 |
| arxcruz|ruck | http://www.fileformat.info/info/unicode/char/037e/index.htm | 09:01 |
| quiquell | arxcruz|ruck: That's sick !!! | 09:02 |
| panda | is this why you don't see many greek programmers around ? | 09:03 |
| arxcruz|ruck | lol | 09:03 |
| arxcruz|ruck | no idea | 09:03 |
| quiquell | panda, arxcruz|ruck: they only do python and the like :-) | 09:04 |
| arxcruz|ruck | we should ask marios :) | 09:04 |
| marios | o/ arxcruz|ruck hi | 09:05 |
| marios | arxcruz|ruck: wasnt me | 09:05 |
| arxcruz|ruck | marios: the reason to not have greek programmers in C around, is because the ; greek question mark ? | 09:05 |
| marios | oh yeah that is the question mark | 09:05 |
| arxcruz|ruck | marios: so, instead of ? you use ; ? | 09:06 |
| marios | arxcruz|ruck: heh i don't know actually man i'm pretty sure theres quite a few greek coders :) but i have absolutely nothing to backup that assertion with | 09:06 |
| marios | arxcruz|ruck: right | 09:06 |
| arxcruz|ruck | wondering why portuguese didn't adopt that... | 09:06 |
| arxcruz|ruck | we use ; in portuguese for a pause in a phrase that is more slower than a "," and faster than a "." | 09:07 |
| arxcruz|ruck | for punctuation | 09:07 |
| marios | arxcruz|ruck: yeah it is the same in english too | 09:07 |
| marios | like pause in the sentence | 09:07 |
| panda | arxcruz|ruck: because it's modern greek ? | 09:07 |
| arxcruz|ruck | dunno, i never study english grammar | 09:08 |
| arxcruz|ruck | my english came from play world of warcraft | 09:08 |
| panda | arxcruz|ruck: and I think protuguese exported more than it imported, for example the japanese "arigato" | 09:08 |
| arxcruz|ruck | is that a portuguese word? i don't think so | 09:09 |
| marios | arxcruz|ruck: and in greek instead that pause is with 'upper dot' literally a full stop but in the middle of the line not on the bottom. | 09:09 |
| arxcruz|ruck | portuguese import a lot, specially from greek | 09:09 |
| panda | arxcruz|ruck: it's not but it's supposed to come from the protuguese "obrigado" | 09:09 |
| marios | arxcruz|ruck: https://en.wiktionary.org/wiki/%CE%AC%CE%BD%CF%89_%CE%BA%CE%B1%CE%B9_%CE%BA%CE%AC%CF%84%CF%89_%CF%84%CE%B5%CE%BB%CE%B5%CE%AF%CE%B1 | 09:09 |
| arxcruz|ruck | like all the "gias" from greek | 09:09 |
| arxcruz|ruck | biologia, sociologia, pedagogia | 09:10 |
| arxcruz|ruck | panda: yeah, arigato is obrigado in japanese | 09:10 |
| panda | we should open #oooq-languages | 09:10 |
| arxcruz|ruck | we also change a lot over the years | 09:10 |
| arxcruz|ruck | for example, the word "you" in portuguese when we were colony, was Vossa Mercê | 09:11 |
| arxcruz|ruck | then became vosmicê | 09:11 |
| arxcruz|ruck | and now is just você | 09:11 |
| arxcruz|ruck | on the internet language is just vc | 09:11 |
| arxcruz|ruck | hehehehe | 09:11 |
| marios | arxcruz|ruck: and just fyi on that ; in english i'm not lieing ;) https://en.wikipedia.org/wiki/Semicolon | 09:11 |
| arxcruz|ruck | marios: :) | 09:12 |
| *** bogdando has quit IRC | 09:23 | |
| *** zoli is now known as zoli|lunch | 09:28 | |
| *** ykarel|afk is now known as ykarel | 09:30 | |
| quiquell | sshnaidm: I am trying libvirt reproducer but get stuck at nodepool-setup : Create and enable swap on subnode | 09:33 |
| sshnaidm | quiquell, hmm.. just stuck? | 09:35 |
| sshnaidm | quiquell, could you ssh to node to see what happens? I didn't see it before | 09:36 |
| quiquell | sshnaidm: Let me clean up | 09:36 |
| quiquell | sshnaidm: The livirt reproducer can be used over a RHEL ? I don't have centos on the host | 09:37 |
| sshnaidm | quiquell, well, it *should*, but who knows.. | 09:37 |
| quiquell | sshnaidm: Better if I use the beaker and install centos on it | 09:38 |
| sshnaidm | quiquell, yeah, that's definitely better | 09:39 |
| *** udesale_ has joined #oooq | 09:45 | |
| *** udesale__ has quit IRC | 09:47 | |
| *** dtantsur is now known as dtantsur|bbl | 09:49 | |
| *** udesale_ has quit IRC | 09:50 | |
| quiquell | sshnaidm: Can I use beaker to restore this machine ? rdo-ci-fx2-01-s4.v101.rdoci.lab.eng.rdu.redhat.com | 09:53 |
| quiquell | sshnaidm: Or how do we usually do ? | 09:53 |
| sshnaidm | quiquell, yeah, I use beaker to provision it to centos 7.3, and then manually run yum update to update it to 7.5 | 09:54 |
| sshnaidm | quiquell, because 7.3 is last version when provision works.. | 09:54 |
| quiquell | sshnaidm: how do you add the machien to beaker it tell me that it's already there but cannot find it | 09:55 |
| *** ykarel has quit IRC | 09:58 | |
| *** ykarel has joined #oooq | 10:01 | |
| *** holser__ has quit IRC | 10:13 | |
| *** holser__ has joined #oooq | 10:13 | |
| *** florianf has quit IRC | 10:23 | |
| panda | quiquell: your patch has a merge conflict | 10:33 |
| quiquell | panda: Has just rebased and rechecked everything | 10:33 |
| quiquell | Now it's rolling | 10:34 |
| quiquell | panda: Damn... there is a problem at trown's patch arg | 10:36 |
| quiquell | panda: At least now we are testing it | 10:36 |
| quiquell | panda: http://logs.openstack.org/35/571435/8/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/0a57be3/job-output.txt.gz#_2018-06-11_09_53_29_221954 | 10:36 |
| panda | quiquell: yeah, one of my comment was about it | 10:37 |
| *** florianf has joined #oooq | 10:38 | |
| quiquell | panda: bypass with --upgrade-from true will work ? | 10:38 |
| panda | quiquell: I don't thing so, it must be treated as boolean | 10:38 |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224 | 10:38 |
| panda | quiquell: byt the rest of the code | 10:38 |
| quiquell | --upgrade-from True mybe python interpert it as a boolean | 10:38 |
| *** zoli|lunch is now known as zoli | 10:39 | |
| quiquell | panda: I can do changes at my DNM commit | 10:39 |
| chandankumar | arxcruz|ruck: sprint 15 topics https://review.rdoproject.org/etherpad/p/tempest-sprint15 feel free to ++ or -- | 10:40 |
| panda | quiquell: ok, add action="store_true" in the argument parser | 10:42 |
| panda | quiquell: also remove default=False, it's implicit | 10:43 |
| quiquell | panda: Added it | 10:43 |
| quiquell | Let's test it | 10:43 |
| quiquell | panda: n-1 > n is working fine | 10:44 |
| panda | quiquell: at least we are not introducing regressions | 10:45 |
| quiquell | panda: That can allow as merge this without finishing n -> n + 1 ? | 10:46 |
| panda | quiquell: if we really se there's no other choice | 10:47 |
| *** ccamacho has quit IRC | 10:59 | |
| *** holser__ has quit IRC | 10:59 | |
| *** ykarel_ has joined #oooq | 11:00 | |
| *** ccamacho has joined #oooq | 11:01 | |
| *** holser__ has joined #oooq | 11:02 | |
| *** ykarel has quit IRC | 11:02 | |
| *** udesale has joined #oooq | 11:03 | |
| *** holser__ has quit IRC | 11:04 | |
| *** rfolco_ has joined #oooq | 11:07 | |
| *** rfolco has quit IRC | 11:08 | |
| *** udesale has quit IRC | 11:14 | |
| *** lucasagomes is now known as lucas-hungry | 11:15 | |
| *** lucas-hungry is now known as lucasagomes | 11:15 | |
| *** atoth has joined #oooq | 11:17 | |
| *** udesale has joined #oooq | 11:29 | |
| sshnaidm | folks, please review https://review.openstack.org/#/c/574164/ - it enabled reproducer on ovb jobs | 11:30 |
| ykarel_ | sshnaidm, any idea if https://review.openstack.org/#/c/568701/ ever worked, this looks wrong to me | 11:31 |
| *** ykarel_ is now known as ykarel | 11:31 | |
| ykarel | sshnaidm, i am getting below with reproducer:- | 11:32 |
| ykarel | [zuul@subnode-0 ~]$ cat /etc/yum/vars/contentdir | 11:32 |
| ykarel | {"failed": false, "stat": {"exists": false}, "changed": false} | 11:32 |
| *** udesale has quit IRC | 11:35 | |
| *** amoralej is now known as amoralej|lunch | 11:38 | |
| panda | 64 bytes from 8.8.8.8: icmp_seq=132 ttl=56 time=7013 ms | 11:42 |
| * panda sighs | 11:42 | |
| panda | 64 bytes from 8.8.8.8: icmp_seq=2 ttl=56 time=33489 ms | 11:43 |
| panda | \o/ new record | 11:43 |
| quiquell | How may bauds have your modem ? 1600 ? | 11:46 |
| panda | quiquell: what's a modem ? I have an acoustic coupler here. | 11:47 |
| quiquell | panda: You are always prepare for the futre | 11:48 |
| quiquell | future | 11:48 |
| quiquell | panda: btw, maybe we can move a little this https://review.rdoproject.org/r/#/c/13604 | 11:48 |
| panda | every time my daughter cries, the line drops | 11:48 |
| quiquell | panda: Your daughter mood is a telco matter ? | 11:49 |
| panda | quiquell: no, it disturbs the acoustic coupling | 11:51 |
| panda | quiquell: I don't remember anything of that patch | 11:51 |
| quiquell | panda: Related to this, https://bugs.launchpad.net/tripleo/+bug/1768090 | 11:53 |
| openstack | Launchpad bug 1768090 in tripleo "promoter script is not comparing timestamps correctly when folding hashes" [High,In progress] - Assigned to Gabriele Cerami (gcerami) | 11:53 |
| quiquell | panda: Maybe is no longer important | 11:53 |
| *** holser__ has joined #oooq | 11:54 | |
| panda | quiquell: it is | 11:54 |
| panda | quiquell: if it wasn't solved | 11:54 |
| *** dtrainor has quit IRC | 11:54 | |
| quiquell | panda: So we need to merge that ? | 11:55 |
| quiquell | Going for lunch, zuul is the bus that never arrives | 11:56 |
| *** quiquell is now known as quiquell|lunch | 11:56 | |
| *** rfolco_ is now known as rfolco | 11:59 | |
| weshay | need a core https://review.openstack.org/#/c/573457/ | 12:07 |
| weshay | https://review.openstack.org/#/c/573844/ | 12:07 |
| weshay | ykarel++ | 12:10 |
| hubbot | weshay: ykarel's karma is now 3 | 12:10 |
| *** quiquell|lunch is now known as quiquell | 12:12 | |
| weshay | ykarel, I don't see your escalation on rdo-cloud | 12:12 |
| ykarel | weshay, me too not seeing | 12:12 |
| weshay | ykarel, were you hoping that email created it? | 12:12 |
| ykarel | may be some issue with tooling | 12:12 |
| *** trown has joined #oooq | 12:12 | |
| ykarel | weshay, yes i was hoping email creates a card | 12:13 |
| quiquell | weshay: Do I need to abandon this ? https://review.openstack.org/#/c/574218/ | 12:14 |
| weshay | ykarel, For Upstream - | 12:14 |
| weshay | [CIX][<rel tag>][LP:<####>][<CIX phase tag>][<component>] <summary line> | 12:14 |
| *** bogdando has joined #oooq | 12:14 | |
| *** panda is now known as panda|lunch | 12:17 | |
| *** holser__ has quit IRC | 12:17 | |
| ykarel | weshay, i sent [rhos-dev][CIX][tripleoci][proa][rdo-cloud, so that's the reason card not created? | 12:17 |
| *** rlandy has joined #oooq | 12:17 | |
| *** rlandy is now known as rlandy|rover | 12:18 | |
| *** dougbtv has quit IRC | 12:18 | |
| weshay | ykarel, I don't think it went through ykarel.. I don't see it | 12:18 |
| *** dougbtv has joined #oooq | 12:18 | |
| weshay | quiquell, not sure.. your patch looks fine, I'm happy to go w/ yours /me just trying to get to done | 12:19 |
| ykarel | sshnaidm, proposed https://review.openstack.org/574238 for the issue i mentioned about contentdir | 12:19 |
| ykarel | weshay, check ^^ i saw you also faced same issue with original patch | 12:19 |
| weshay | http://post-office.corp.redhat.com/archives/rh-openstack-dev/2018-June/date.html | 12:19 |
| ykarel | this one:- http://post-office.corp.redhat.com/archives/rh-openstack-dev/2018-June/msg00118.html | 12:20 |
| quiquell | weshay: Don't know if it can be in the gates yet | 12:21 |
| weshay | ykarel, so the script that picks those up is from lhh.. lon | 12:21 |
| weshay | ykarel, you may just want to create the card directly on cix | 12:22 |
| weshay | and ping lhh | 12:22 |
| weshay | lon | 12:22 |
| ykarel | weshay, yup i tried to ping lon but he was not available, now he is available | 12:22 |
| ykarel | so will ping | 12:22 |
| weshay | arxcruz|ruck, rlandy|rover btw.. we owe ykarel another high five https://review.openstack.org/#/c/574145/ | 12:23 |
| arxcruz|ruck | weshay: yeah, i have no idea how he figure that out | 12:23 |
| arxcruz|ruck | already told him that he needs to write a blog about it | 12:23 |
| rlandy|rover | nice catch! | 12:25 |
| weshay | panda|lunch, rlandy|rover trown I think I have everyone's comments addressed at this point for https://review.openstack.org/#/c/572217/ and deps | 12:27 |
| ykarel | weshay, created: https://trello.com/c/f1COW32i | 12:29 |
| rlandy|rover | weshay: https://review.openstack.org/#/c/572217/8/toci_gate_test-oooq.sh is going to have the same issue s as single node wrt taking just the rdocloud settings file | 12:29 |
| rlandy|rover | fixing your comments on that patch | 12:30 |
| ykarel | arxcruz|ruck, yes i remember, but not sure what to write there as everytime it's a new issue | 12:35 |
| rlandy|rover | arxcruz|ruck: hey - what's with the 14 hr periodic job? | 12:36 |
| rlandy|rover | sf zuul | 12:36 |
| rlandy|rover | have you checked into it? | 12:36 |
| rlandy|rover | otherwise - will look | 12:36 |
| rlandy|rover | check jobs also running at almost 15hrs | 12:37 |
| ykarel | rlandy|rover, ^^ because of rdocloud issue | 12:37 |
| trown | weshay: the standalone job is failing on https://review.openstack.org/#/c/572217/ | 12:37 |
| weshay | bah of course | 12:38 |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224 | 12:38 |
| *** udesale has joined #oooq | 12:38 | |
| quiquell | trown, panda|lunch, weshay: Adding release to naming gate looks good now https://review.openstack.org/#/c/572736/, it needs some reviewing | 12:39 |
| quiquell | Adding release to the gating repo I mean | 12:39 |
| weshay | trown, hrm.. /me wonders if I need this if/else here https://review.openstack.org/#/c/572217/7..8/toci_gate_test-oooq.sh | 12:41 |
| trown | weshay: the line that is failing is actually in quickstart.sh... seems related to https://github.com/openstack/tripleo-quickstart/commit/b98f1bee5a1183e65040ce99b65fd6e6f7938bad | 12:43 |
| rlandy|rover | ykarel: you get another gold star :) | 12:43 |
| *** tcw has joined #oooq | 12:51 | |
| *** tcw has quit IRC | 12:52 | |
| *** tcw has joined #oooq | 12:52 | |
| *** amoralej|lunch is now known as amoralej | 12:58 | |
| arxcruz|ruck | weshay: hey boss, pike is 4 days without promotion, i cannot find the problem, is that because there weren't any changes in pike ? | 13:00 |
| arxcruz|ruck | weshay: https://trunk.rdoproject.org/centos7-pike/report.html last package update was 4 days ago | 13:00 |
| arxcruz|ruck | just want to double check before the report goes red tomorrow with 5 days without consistent being promoted | 13:01 |
| sshnaidm | weshay, 1:1? | 13:01 |
| weshay | sshnaidm, yes sir.. coming out of another mtg.. need 3min or so | 13:02 |
| sshnaidm | weshay, np | 13:02 |
| *** udesale has quit IRC | 13:07 | |
| weshay | sshnaidm, k.. ready | 13:14 |
| *** udesale has joined #oooq | 13:15 | |
| ykarel | arxcruz|ruck, regarding queens bug | 13:15 |
| arxcruz|ruck | ykarel: yeah ? | 13:15 |
| ykarel | arxcruz|ruck, i can see it's affecting master as well | 13:16 |
| arxcruz|ruck | ykarel: yeah, it doesn't | 13:16 |
| arxcruz|ruck | ykarel: the problem right now is, why it's getting /v2 | 13:16 |
| ykarel | arxcruz|ruck, i think the issue is because of release of novaclient > 10- | 13:16 |
| *** panda|lunch is now known as panda | 13:16 | |
| ykarel | arxcruz|ruck, http://git.openstack.org/cgit/openstack/tripleo-quickstart/tree/roles/tripleo-inventory/templates/get-overcloud-nodes.py.j2#n24 logic is broken | 13:17 |
| ykarel | as if nc_version[0] returns 1 which is less that 6 | 13:17 |
| *** Goneri has joined #oooq | 13:17 | |
| arxcruz|ruck | ykarel: LOL | 13:17 |
| arxcruz|ruck | ykarel: you're right | 13:17 |
| arxcruz|ruck | holly molly | 13:17 |
| arxcruz|ruck | ykarel: i'll create a reproduce env to check that | 13:18 |
| ykarel | arxcruz|ruck, just need to create a virtualenv, install python-novaclient==10.1.0 | 13:18 |
| *** myoung|off is now known as myoung | 13:19 | |
| arxcruz|ruck | ykarel: yup, i think i have queens env up on my virthost | 13:19 |
| ykarel | arxcruz|ruck, ack | 13:19 |
| ykarel | then it's fine, try there | 13:19 |
| ykarel | also as newton is EOL: https://github.com/openstack/python-novaclient/tree/newton-eol | 13:20 |
| ykarel | so that workaround can be removed | 13:20 |
| ykarel | but if "fetch overcloud node names and IPs" is used by any newton deployment that would see issue | 13:21 |
| *** skramaja_ has quit IRC | 13:22 | |
| rlandy|rover | arxcruz|ruck: running the cleanup script on downstream tenant | 13:23 |
| rlandy|rover | something is blocked there | 13:23 |
| arxcruz|ruck | ykarel: so, doesn't seems to be the same problem http://paste.openstack.org/show/723197/ | 13:23 |
| arxcruz|ruck | at least i need to check, also, python-novaclient installed in the system is 9.0 | 13:24 |
| quiquell | panda, trown: n -> n + 1 working https://review.openstack.org/#/c/571435/ | 13:25 |
| quiquell | it integrates trown's work | 13:25 |
| *** dtantsur|bbl is now known as dtantsur | 13:25 | |
| panda | quiquell: champagne! | 13:25 |
| ykarel | arxcruz|ruck, with queens 10.1.0 should be installed : https://trunk.rdoproject.org/centos7-queens/consistent/python2-novaclient-10.1.0-0.20180605113302.ed73aad.el7.noarch.rpm | 13:25 |
| ykarel | for your test.py, have you sorced stackrc | 13:26 |
| arxcruz|ruck | ykarel: yup... | 13:26 |
| arxcruz|ruck | hehehehe | 13:26 |
| arxcruz|ruck | first thing :P | 13:26 |
| arxcruz|ruck | ykarel: but you're right, the nova client is python2-novaclient-10.1.0-0.20180605113302.ed73aad.el7.noarch | 13:27 |
| arxcruz|ruck | https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-queens-upload/e837432/undercloud/var/log/extra/rpm-list.txt.gz | 13:27 |
| ykarel | arxcruz|ruck, in venv you installed which version | 13:28 |
| arxcruz|ruck | 10.1.0 | 13:28 |
| ykarel | ok try 9.1.2 | 13:29 |
| ykarel | to see if there is some difference | 13:29 |
| arxcruz|ruck | ykarel: also, one more thing... | 13:30 |
| arxcruz|ruck | if nc_version[0] <= "6": | 13:30 |
| arxcruz|ruck | this would never work properly | 13:30 |
| arxcruz|ruck | because it's checking strings | 13:30 |
| arxcruz|ruck | lol | 13:30 |
| ykarel | arxcruz|ruck, till now it was working as nc_version[0] used to return 9 | 13:31 |
| ykarel | agree this is not right way to check | 13:31 |
| arxcruz|ruck | ykarel: i think we will have more problems with novaclient | 13:38 |
| arxcruz|ruck | with version 9 is okay the code | 13:38 |
| arxcruz|ruck | but 10 i got keystoneauth1.exceptions.connection.SSLError: SSL exception connecting to https://192.168.24.2:13000/v3/auth/tokens: HTTPSConnectionPool(host='192.168.24.2', port=13000): Max retries exceeded with url: /v3/auth/tokens (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)'),)) | 13:38 |
| ykarel | arxcruz|ruck, hmm seeing the same, but may be a virtualenv issue | 13:39 |
| *** holser__ has joined #oooq | 13:39 | |
| ykarel | as i can see same issue with 9 and 10 | 13:40 |
| *** matbu has quit IRC | 13:40 | |
| quiquell | rfolco: fs037 on queens merged https://review.openstack.org/#/c/570902/ | 13:42 |
| *** matbu has joined #oooq | 13:42 | |
| rfolco | quiquell, :) | 13:43 |
| rfolco | quiquell, this one failed on gate job - https://review.openstack.org/#/c/572306/ | 13:44 |
| quiquell | rfolco: Now the per project commits | 13:44 |
| quiquell | rfolco: checking | 13:44 |
| arxcruz|ruck | ykarel: CURL_CA_BUNDLE="" do the magic | 13:44 |
| ykarel | ack, insecure=true could also be used i think | 13:45 |
| quiquell | rfolco: timed out... :-/ | 13:45 |
| quiquell | arxcruz|ruck: Any issue with tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates ? | 13:45 |
| ykarel | arxcruz|ruck, so test.py works with 9.1.2 and fails with 10.1.0, right? | 13:45 |
| quiquell | arxcruz|ruck: It has timed out in the gates | 13:45 |
| quiquell | arxcruz|ruck: http://logs.openstack.org/06/572306/4/gate/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/d9124ed/job-output.txt.gz | 13:45 |
| arxcruz|ruck | ykarel: yes, 9 pass, 10 fails without CURL_CA_BUNDLE | 13:46 |
| ykarel | and same issue as we see in CI | 13:47 |
| sshnaidm | quiquell still online? | 13:47 |
| quiquell | sshnaidm: Yep | 13:47 |
| quiquell | panda: could you +2 +1w this https://review.openstack.org/#/c/572308/ ? | 13:48 |
| sshnaidm | quiquell, need to sync about dashboard, but can also tomorrow if it's late | 13:48 |
| quiquell | sshnaidm: We have scrum in a few, let's sync tomorrow | 13:48 |
| sshnaidm | quiquell, sure | 13:48 |
| quiquell | sshnaidm: our mornings are quiet | 13:48 |
| panda | quiquell: not sure I'allowed to +2 on that repo, maybe since it's CI stull I can do it | 13:50 |
| quiquell | panda: You are right, let's ask at #tripleo | 13:51 |
| trown | quiquell: i got you :) | 13:51 |
| quiquell | trown: sweet !!! | 13:54 |
| ykarel | arxcruz|ruck, with 10 you see same issue as CI or other | 13:54 |
| quiquell | rfolco: Going to recheck, it doesn't make sense | 13:55 |
| arxcruz|ruck | ykarel: other | 13:57 |
| quiquell | trown: The logs we save are specified in the jobs or in the featuresets ? | 13:57 |
| *** matbu has quit IRC | 13:58 | |
| trown | quiquell: they are specified via config ... it is not featureset though one sec | 14:01 |
| *** matbu has joined #oooq | 14:04 | |
| trown | quiquell: https://github.com/openstack-infra/tripleo-ci/blob/master/toci-quickstart/config/collect-logs.yml | 14:04 |
| ykarel | arxcruz|ruck, i can see the same issue as CI with 10.1.0 | 14:08 |
| ykarel | though i have non tripleo setup | 14:08 |
| ykarel | i mean:- keystoneauth1.exceptions.http.NotFound: (https://localhost:5000/v2.0/tokens): The resource could not be found. (HTTP 404) | 14:08 |
| ykarel | with 9* it works fine | 14:09 |
| arxcruz|ruck | ykarel: i hope this fix https://review.openstack.org/#/c/574270/ | 14:12 |
| ykarel | arxcruz|ruck, can't we just remove the workaround | 14:12 |
| arxcruz|ruck | ykarel: we still have newton jobs | 14:12 |
| ykarel | is newton jobs | 14:12 |
| ykarel | okk | 14:12 |
| ykarel | arxcruz|ruck, a small comment, rest looks good | 14:20 |
| ykarel | also attach bug in commit message | 14:21 |
| ykarel | Thanks, this should work now | 14:26 |
| weshay | rlandy|rover, bad news | 14:28 |
| rlandy|rover | weshay: which exact piece of bad news now? | 14:29 |
| weshay | rlandy|rover, undercloud is still busted on master | 14:30 |
| rlandy|rover | weshay: going nowhere now :( | 14:30 |
| *** dtrainor has joined #oooq | 14:31 | |
| weshay | rlandy|rover, OH wait.. | 14:31 |
| rlandy|rover | failure is not in undercloud | 14:32 |
| weshay | ykarel, you mentioned for the kolla change to come in.. we need to move master to consistent? | 14:32 |
| ykarel | weshay, yes | 14:32 |
| weshay | hrm. k | 14:32 |
| ykarel | should be consistent in few minutes | 14:32 |
| weshay | ykarel++ | 14:32 |
| hubbot | weshay: ykarel's karma is now 4 | 14:32 |
| rfolco | quiquell, tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates on the previous changes.... and -upgrades on the last one.... what am I missing ? | 14:32 |
| ykarel | weshay, we also need arxcruz|ruck patch to promote master/queens:- https://review.openstack.org/#/c/574270/ | 14:33 |
| myoung | *** CI squad: we're triaging tag:quickstart LP issues in half an hour, please kindly update any bugs actively being worked with status as appropriote, will help us whittle down the bug pile :) Thanks! | 14:34 |
| rlandy|rover | arxcruz|ruck: what happened with the queens bug you had fs 2, 20, 35? | 14:34 |
| rlandy|rover | 1 also failing now | 14:34 |
| rlandy|rover | is that still in flight? | 14:34 |
| arxcruz|ruck | rlandy|rover: https://review.openstack.org/#/c/574270/ | 14:35 |
| arxcruz|ruck | need this because meanwhile novaclient releases version 10, which makes the logic in the script fails | 14:35 |
| arxcruz|ruck | ykarel++ | 14:36 |
| hubbot | arxcruz|ruck: ykarel's karma is now 5 | 14:36 |
| rlandy|rover | oh gosh | 14:36 |
| myoung | arxcruz|ruck, ykarel, added reviewers in gerrit and "{patch} += https://trello.com/c/Ble66eRY/431-list-of-upstream-reviews" | 14:37 |
| myoung | trown, rlandy|rover, sshnaidm, panda, weshay ^^ need eyes (promotion blocker) | 14:37 |
| myoung | oi...i'm tired, strike the += above and reverse it | 14:38 |
| * myoung ducks | 14:38 | |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224 | 14:38 |
| quiquell | rfolco: What do you mean ? | 14:38 |
| ykarel | myoung, Thanks | 14:40 |
| *** sanjayu_ has joined #oooq | 14:42 | |
| *** ykarel is now known as ykarel|away | 14:43 | |
| weshay | arxcruz|ruck, not sure how my comment would relate exactly w/ available time and other priorities. It does seem to me that we've spent enough time in get-overcloud-nodes.py that some unit tests around it may helkp | 14:44 |
| weshay | help | 14:44 |
| arxcruz|ruck | weshay: well, the get-overcloud-nodes.py is a template in ansible, not sure how to keep unit tests there | 14:45 |
| weshay | myoung, fyi.. noticing a number of incoming bugs w/o the appropriate tags that we may want to review fyi https://bugs.launchpad.net/tripleo/+bugs?orderby=-datecreated&start=0 | 14:45 |
| *** saneax has quit IRC | 14:45 | |
| ykarel|away | weshay, can we remove alert:- https://bugs.launchpad.net/tripleo/+bug/1775698 | 14:46 |
| openstack | Launchpad bug 1775698 in tripleo "tripleo nova containers restarting in a loop causing - 503 errors - starting container trunk.registry.rdoproject.org/tripleomaster/centos-binary-nova-compute-ironic" [Critical,Triaged] - Assigned to yatin (yatinkarel) | 14:46 |
| weshay | arxcruz|ruck, aye.. I see that too, however it's not really a template either | 14:46 |
| arxcruz|ruck | weshay: yeah, should be files | 14:46 |
| weshay | no jinja there it could be moved to facilitate | 14:46 |
| quiquell | trown: Found the issue with the artifacts | 14:47 |
| quiquell | repo_setup_script: "repo-setup-{{ lookup('env', 'UPGRADE_RELEASE') }}.sh" | 14:47 |
| quiquell | repo_setup_log: "repo-setup-{{ lookup('env', 'UPGRADE_RELEASE') }}.log" | 14:47 |
| quiquell | It's going to be dangerous if we use {{ release }} ? | 14:47 |
| *** agopi has joined #oooq | 14:47 | |
| quiquell | Is part of the refactoring | 14:47 |
| rlandy|rover | arxcruz|ruck: cannot be deleted directly via the port API: has device owner network:ha_router_replicated_interface ... that is killing downstream stack delete | 14:48 |
| arxcruz|ruck | rlandy|rover: shall i open a ticket ? | 14:48 |
| rlandy|rover | arxcruz|ruck: nobody will answer it ;( ... will try fix as much as we can from the tenant | 14:49 |
| rlandy|rover | just fyi - in case you see it again | 14:49 |
| rlandy|rover | I think there is one job that adds that | 14:49 |
| trown | quiquell: hmm... ya | 14:51 |
| trown | quiquell: well actually... using release will be fine, because it is just the name | 14:52 |
| quiquell | trown: Yep for undercloud we are good, we have to check it out for overcloud | 14:52 |
| trown | quiquell: and the other ones are named without any "-release" ... so worst case is just the reverse | 14:52 |
| trown | where some other job has misnamed artifacts | 14:52 |
| *** holser___ has joined #oooq | 14:53 | |
| *** holser__ has quit IRC | 14:53 | |
| quiquell | trown: Also there is one missing artifact install_queens_gating_repo | 14:53 |
| quiquell | Don't know where do I configure those | 14:54 |
| quiquell | Ahhno it's there | 14:54 |
| quiquell | silly me | 14:54 |
| myoung | weshay: ack, let's address in triage, going thru the list now | 14:57 |
| weshay | arxcruz|ruck, rlandy|rover just need one of you for a bug triage | 15:00 |
| weshay | you pick | 15:00 |
| rlandy|rover | weshay: arxcruz|ruck: joining | 15:00 |
| arxcruz|ruck | joining | 15:00 |
| *** bogdando has quit IRC | 15:03 | |
| *** agopi has quit IRC | 15:06 | |
| *** quiquell is now known as quiquell|off | 15:07 | |
| rlandy|rover | ugh ... just missed quiquell|off | 15:31 |
| rlandy|rover | quiquell|off: if you're not really off - your box is reprovisioned | 15:32 |
| rlandy|rover | with the root password you sent | 15:32 |
| rlandy|rover | no centos user | 15:32 |
| rlandy|rover | you can upgrade/add user | 15:32 |
| myoung | chandankumar, weshay, arxcruz|ruck (if you have time) - tempest squad scrum | 15:33 |
| *** sanjay__u has quit IRC | 15:41 | |
| weshay | kopecmartin, hey | 15:59 |
| *** zoli is now known as zoli|gone | 16:09 | |
| *** zoli|gone is now known as zoli | 16:09 | |
| *** rnoriega has quit IRC | 16:13 | |
| *** rnoriega has joined #oooq | 16:20 | |
| weshay | panda, ping | 16:32 |
| *** kopecmartin has quit IRC | 16:33 | |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224 | 16:38 |
| myoung | weshay: rfolco, panda sorry | 16:39 |
| *** trown is now known as trown|lunch | 16:45 | |
| *** udesale has quit IRC | 16:56 | |
| *** jfrancoa has joined #oooq | 17:29 | |
| *** trown|lunch has quit IRC | 17:35 | |
| *** dtantsur is now known as dtantsur|afk | 17:37 | |
| *** agopi has joined #oooq | 17:43 | |
| *** holser___ has quit IRC | 17:49 | |
| *** ykarel|away has quit IRC | 18:00 | |
| rlandy|rover | panda: sshnaidm: could you review pls ... https://review.openstack.org/#/c/573819/ | 18:15 |
| panda | rlandy|rover: is this still "Do not Merge" ? | 18:16 |
| rlandy|rover | panda: I left it as DNM over the weekend so nobody merged it w/o ci core review | 18:17 |
| rlandy|rover | the DNM is removed now | 18:17 |
| rlandy|rover | weshay already left comments which I fixed | 18:18 |
| panda | rlandy|rover: ok | 18:18 |
| rlandy|rover | panda: https://review.openstack.org/#/c/573820/ - also part of the fix | 18:18 |
| rlandy|rover | just deciding what to do about bogdan's -1 | 18:18 |
| rlandy|rover | we generally don;t double tag things | 18:18 |
| *** amoralej is now known as amoralej|off | 18:23 | |
| rlandy|rover | panda: weshay: so this review is not a major deal either way ... https://review.openstack.org/#/c/573820/ - considering bogdan's comment, we could abandon. thoughts? | 18:25 |
| rlandy|rover | and here we go again on periodic | 18:25 |
| *** agopi has quit IRC | 18:29 | |
| weshay | rlandy|rover, def have a new dlrn hash this time | 18:35 |
| weshay | so we should see diff results | 18:35 |
| weshay | "repo_url": "https://trunk.rdoproject.org/centos7-queens/87/66/87669a5034a80e2f04ce8d55d203e89d919f463d_15e3f3fe", | 18:36 |
| rlandy|rover | weshay: great - hoping for green pastures | 18:36 |
| *** trown has joined #oooq | 18:38 | |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224 | 18:38 |
| trown | lost power for a bit if anyone was trying to reach me, my bouncer was down | 18:39 |
| *** agopi has joined #oooq | 18:49 | |
| rlandy|rover | panda: posted comments in https://bugs.launchpad.net/tripleo/+bug/1776186/comments/2 - pls see what you think for scheduling this | 18:50 |
| openstack | Launchpad bug 1776186 in tripleo "Triple-O CI overrides jinja2 rendered network-environment.yaml" [High,Triaged] - Assigned to Ronelle Landy (rlandy) | 18:50 |
| rlandy|rover | probably larger than a ruck/rivertaks - wrt testing all the options | 18:50 |
| weshay | sshnaidm++ | 18:51 |
| hubbot | weshay: sshnaidm's karma is now 2 | 18:51 |
| weshay | sshnaidm, thanks for the fix on standalone | 18:51 |
| *** myoung is now known as myoung|lunch | 18:52 | |
| weshay | rlandy|rover, am I clear to help w/ bm.. https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/tripleo-quickstart-master-rdo_trunk-baremetal-hp_dl360_envE-single_nic_vlans/ | 18:53 |
| weshay | rlandy|rover, re: moving configs over to other env? | 18:53 |
| rlandy|rover | weshay: ok - ... | 18:54 |
| rlandy|rover | my test on friday was a no-op as we still had the file passed in extras | 18:54 |
| *** agopi has quit IRC | 18:54 | |
| rlandy|rover | weshay: I think I can also get back to that now - had some downstream gate clear up to do this morning | 18:55 |
| weshay | %gatestatus | 18:55 |
| weshay | % gatestatus | 18:55 |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224 | 18:55 |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224 | 18:55 |
| weshay | rlandy|rover, k. k.. thank you!! | 18:55 |
| rlandy|rover | weshay: if we can stay with the plan we discussed, I'll set up the poc jobs now | 18:56 |
| weshay | ya.. same plan afaict | 18:56 |
| rlandy|rover | so we actually test what we change | 18:56 |
| rlandy|rover | if you can work on the other env vars, that would be awesome | 18:56 |
| weshay | k | 18:56 |
| weshay | rlandy|rover, trown https://review.openstack.org/#/c/573457/ | 18:57 |
| trown | weshay: it is failed in that patch :P | 18:58 |
| weshay | trown, ya.. we know | 18:59 |
| rlandy|rover | weshay: what trown said - as long as we are ok with a failing job | 18:59 |
| rlandy|rover | it is non-voting | 18:59 |
| * rlandy|rover thinks I asked this last time :( | 18:59 | |
| weshay | trown, maybe if the tripleo-upgrade guys see that.. they will help fix it :) | 19:00 |
| rlandy|rover | weshay: so we know why is was "successfully on experimental pipeline." but not here | 19:01 |
| weshay | rlandy|rover, it's old data | 19:01 |
| weshay | really | 19:01 |
| rlandy|rover | I am against merging more failing jobs | 19:01 |
| rlandy|rover | bad precedent | 19:01 |
| weshay | rlandy|rover, trown imho.. not having check or gating jobs on a repo is worse precedent | 19:02 |
| weshay | I'll recheck it, it should actually work now after sofer's patch | 19:03 |
| rlandy|rover | weshay: thanks - after that I'll happily +2 this | 19:03 |
| weshay | rlandy|rover, note.. this change only runs the job on tripleo-upgrade changes | 19:03 |
| rlandy|rover | doesn't change my objection to failing jobs | 19:04 |
| rlandy|rover | to stand on my soap box more about this ... | 19:05 |
| rlandy|rover | as ruck/rover, do you go through failing jobs to see that it's still the same failure? | 19:05 |
| rlandy|rover | in reality, we should, we don't | 19:06 |
| weshay | rlandy|rover, in reality ruck/rover do not watch check jobs | 19:06 |
| weshay | except for no-op changes | 19:06 |
| rlandy|rover | we get pinged about them from people wanting to know why things failed | 19:07 |
| weshay | rlandy|rover, trown ok.. https://review.openstack.org/#/c/573844/ | 19:07 |
| weshay | that has sofer's patch | 19:07 |
| rlandy|rover | that one has passing jobs - got my +2 | 19:10 |
| rlandy|rover | weshay: fixed your comments on https://review.openstack.org/#/c/573819/ | 19:11 |
| rlandy|rover | we should not merge that until we see test results | 19:11 |
| rlandy|rover | and make sure we didn't mess anything up | 19:11 |
| weshay | rlandy|rover, k https://review.openstack.org/#/c/573819/2/toci-quickstart/config/testenv/ovb-rdocloud.yml | 19:13 |
| weshay | so are the dns changes line 12-13 coming in from a rebase? | 19:13 |
| * weshay looks at head | 19:13 | |
| rlandy|rover | weshay: no - I removed them per your comment | 19:14 |
| rlandy|rover | https://review.openstack.org/#/c/573819/3/toci-quickstart/config/testenv/ovb-rdocloud.yml | 19:14 |
| rlandy|rover | see latest patch version | 19:14 |
| weshay | oh ya.. | 19:14 |
| weshay | sorry was looking at the wrong ver.. | 19:14 |
| weshay | thanks! | 19:14 |
| weshay | good for another patch though | 19:14 |
| rlandy|rover | weshay: ack - one day | 19:15 |
| weshay | rlandy|rover, right | 19:15 |
| weshay | busy busy | 19:15 |
| rlandy|rover | weshay: btw - I am taking the central-ci tenant for rhos-13 gate - test | 19:15 |
| rlandy|rover | central-ci-stage | 19:15 |
| weshay | k | 19:15 |
| rlandy|rover | weshay: ^^ correction - stage tenant | 19:15 |
| *** Goneri has quit IRC | 19:21 | |
| *** holser__ has joined #oooq | 19:22 | |
| *** atoth has quit IRC | 19:23 | |
| rlandy|rover | https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/tq-gate-rhos-13-ci-rhos-ovb-minimal-pacemaker-public-bond/ | 19:23 |
| rlandy|rover | on stage tenant - will see how it does | 19:23 |
| rlandy|rover | weshay: fyi ... I tested this a while back ... https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/poc-rlandy-tq-gate-rhos-12-ci-rhos-ovb-fs/configure - downstream ovb gate running fs001 | 19:24 |
| rlandy|rover | we got a pass once | 19:25 |
| weshay | k.. ya.. that should also run fs001 | 19:25 |
| weshay | agree | 19:25 |
| rlandy|rover | doing 13 test - then can test fs001 | 19:25 |
| rlandy|rover | k - back to bm | 19:26 |
| weshay | fak | 19:35 |
| weshay | trown, how close to ready are we for adding fs51 to use the new release tools.. http://logs.openstack.org/44/573844/3/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades/b93bc99/job-output.txt.gz#_2018-06-11_12_16_37_589110 | 19:37 |
| *** holser___ has joined #oooq | 19:42 | |
| rlandy|rover | ugh | 19:44 |
| rlandy|rover | really? | 19:44 |
| rlandy|rover | https://review.rdoproject.org/jenkins/job/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/299/console | 19:44 |
| rlandy|rover | https://review.rdoproject.org/jenkins/job/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset016-master/567/console - looks better | 19:45 |
| *** holser__ has quit IRC | 19:45 | |
| *** sanjayu_ has quit IRC | 19:46 | |
| weshay | trown, https://review.openstack.org/574417 | 19:47 |
| rlandy|rover | weshay: ^^ looking at fs001 failure in promotion | 19:48 |
| weshay | rlandy|rover, hrm.. in prep-images | 19:48 |
| rlandy|rover | I know right - weird | 19:49 |
| rlandy|rover | waiting for logs | 19:49 |
| rlandy|rover | fs002 look sok | 19:49 |
| rlandy|rover | multinode is progressing | 19:49 |
| *** holser__ has joined #oooq | 19:51 | |
| rlandy|rover | b35fe4ba-d4c2-43f1-9573-7b62eb8d250c. Error: IPMI call failed: power status.'}, {u'result': u'Node 3c3fcde9-30a9-49ec-a631-2834582b0e4d did not reach state "manageable", the state is "enroll", error: Failed to get power state for node | 19:53 |
| rlandy|rover | seen that before | 19:53 |
| rlandy|rover | weshay: that may just be unlucky/sporadic | 19:54 |
| rlandy|rover | let's see what fs035 does | 19:54 |
| weshay | k cool | 19:54 |
| rlandy|rover | fs002 is ok | 19:54 |
| *** holser___ has quit IRC | 19:55 | |
| -openstackstatus- NOTICE: Zuul was restarted for a software upgrade; changes uploaded or approved between 19:30 and 19:50 will need to be rechecked | 19:55 | |
| *** myoung|lunch is now known as myoung | 19:59 | |
| *** holser__ has quit IRC | 20:00 | |
| *** holser__ has joined #oooq | 20:01 | |
| rlandy|rover | weshay: https://review.rdoproject.org/jenkins/job/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master/351/console looks good - I'm going with sporadic on fs001 unless we see other evidence | 20:02 |
| * rlandy|rover wishes for a retry button | 20:02 | |
| weshay | rlandy|rover, depending on the other results and ack from tripelo leads, we could change the critieria | 20:03 |
| weshay | trown, quiquell|off https://review.rdoproject.org/r/14180 | 20:03 |
| rlandy|rover | weshay: eek - didn't know we did that upstream - I guess we're really behind in promotions, it makes sense in this case | 20:06 |
| weshay | rlandy|rover, we usually don't but we've done it a couple times | 20:07 |
| weshay | rlandy|rover, you have to be really careful w/ what you skip as to not impact upstream gate jobs | 20:07 |
| rlandy|rover | weshay: I'll kick a reproducer on my local tenant | 20:07 |
| weshay | rlandy|rover, rock | 20:07 |
| *** ccamacho has quit IRC | 20:14 | |
| *** jfrancoa has quit IRC | 20:16 | |
| weshay | trown, looking for your reassurance / review on our new tooling and fs051. Thanks for the rdo review | 20:17 |
| weshay | guess we'll find out w/ https://review.openstack.org/#/c/574417/ | 20:17 |
| trown | weshay: ya, not sure what will break tbh... might just work | 20:18 |
| trown | weshay: I am a bit surprised it doesnt work without using emit releases... if that is the case, we probably should just take out that featureset restriction | 20:18 |
| trown | I am 100% confident in emit releases when no upgrade is involved... so if upgrades dont work without it, not much to lose by using it | 20:19 |
| trown | maybe they are broken in a different way... but maybe they also "just work" | 20:20 |
| weshay | trown, ya.. | 20:21 |
| *** jtomasek has quit IRC | 20:23 | |
| weshay | rlandy|rover, this needs a blocker bug https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/50abb54/undercloud/home/jenkins/tempest.log.gz | 20:31 |
| weshay | rlandy|rover, probably the same issue arx hit on get_overcloud_nodes.py | 20:31 |
| weshay | arxcruz|ruck, ^ | 20:31 |
| rlandy|rover | looking | 20:32 |
| *** ccamacho has joined #oooq | 20:33 | |
| rlandy|rover | yeah - tempest red all over | 20:33 |
| rlandy|rover | could we what arxcruz|ruck logged before - checking | 20:34 |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224 | 20:39 |
| rlandy|rover | weshay: afaict, this is the bug arxcruz|ruck has open ... https://bugs.launchpad.net/tripleo/+bug/1775365 | 20:43 |
| openstack | Launchpad bug 1775365 in tripleo "periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-queens-upload job is failing to fetch overcloud node names" [Critical,Fix released] - Assigned to Arx Cruz (arxcruz) | 20:43 |
| rlandy|rover | review not yet merged | 20:43 |
| rlandy|rover | looks different | 20:43 |
| weshay | well.. his patch won't fix the problem in tempest.. but wondering if it's the same root | 20:43 |
| rlandy|rover | maybe | 20:44 |
| rlandy|rover | creating another bug in the meant time | 20:44 |
| rlandy|rover | https://review.openstack.org/#/c/574270/ | 20:44 |
| rlandy|rover | but bug refers to https://review.openstack.org/572709 | 20:45 |
| rlandy|rover | I lie - it has merged | 20:45 |
| rlandy|rover | the /574270 has not merged | 20:46 |
| *** holser__ has quit IRC | 20:46 | |
| *** Goneri has joined #oooq | 20:46 | |
| *** trown is now known as trown|outtypewww | 20:56 | |
| rlandy|rover | weshay: https://bugs.launchpad.net/tripleo/+bug/1776301 [master promotion] Tempest is failing with " KeyError: 'resources' "errors - Connection refused | 20:58 |
| openstack | Launchpad bug 1776301 in tripleo "[master promotion] Tempest is failing with " KeyError: 'resources' "errors - Connection refused" [Critical,Triaged] | 20:58 |
| weshay | thanks | 21:04 |
| weshay | bbl | 21:04 |
| *** hamzy has quit IRC | 21:08 | |
| rlandy|rover | well, I guess there is one way to tell | 21:08 |
| rlandy|rover | reproducer - master hash with change | 21:09 |
| *** hamzy has joined #oooq | 21:11 | |
| *** Goneri has quit IRC | 21:37 | |
| rlandy|rover | https://github.com/openstack/python-tempestconf/commit/283de2057402ebe9a5c2eabef9e550359b40e05f | 21:45 |
| rlandy|rover | chandankumar: you around? | 21:46 |
| rlandy|rover | long shot :) | 21:46 |
| *** myoung is now known as myoung|off | 21:50 | |
| rlandy|rover | weshay: back? | 21:50 |
| rlandy|rover | weshay: reproducer gets by perp-images on fs001 | 21:57 |
| rlandy|rover | but obviously we have other problems | 21:58 |
| rlandy|rover | %gatestatus | 22:31 |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224 | 22:31 |
| *** florianf has quit IRC | 22:32 | |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224 | 22:39 |
| *** EmilienM|PTO is now known as EmilienM | 22:41 | |
| *** tosky has quit IRC | 23:32 | |
| rlandy|rover | weshay: testing bm with fs001 ... https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rlandy-poc-tripleo-quickstart-master-baremetal-hp_dl360_envE-single_nic_vlans | 23:51 |
| *** rlandy|rover is now known as rlandy|rover|bbl | 23:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!