| *** agopi__ is now known as agopi | 00:12 | |
| *** dtrainor has joined #oooq | 00:22 | |
| *** rlandy|afk has quit IRC | 00:35 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, stable/pike: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset022-pike @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario009-multinode-oooq, (2 more messages) | 00:51 |
|---|---|---|
| *** jtomasek has joined #oooq | 01:03 | |
| *** jtomasek has quit IRC | 01:07 | |
| *** dtrainor has quit IRC | 01:18 | |
| hubbot1 | FAILING CHECK JOBS on master: legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario009-multinode-oooq, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp- (2 more messages) | 02:51 |
| *** udesale has joined #oooq | 03:14 | |
| *** ykarel|away has joined #oooq | 04:02 | |
| *** saneax has joined #oooq | 04:35 | |
| hubbot1 | FAILING CHECK JOBS on master: legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-3nodes-multinode, tripleo-quickstart-extras-gate-master-delorean-full-featureset052, tripleo-ci-centos-7-scenario009 (2 more messages) | 04:51 |
| *** udesale has quit IRC | 05:09 | |
| *** udesale has joined #oooq | 05:15 | |
| *** ykarel|away is now known as ykarel | 05:23 | |
| *** jfrancoa has joined #oooq | 05:49 | |
| quiquell|off | ssbarnea|bkp2: Don't know man, marios|rover can you look at the bug ? https://bugs.launchpad.net/tripleo/+bug/1786106 ? | 05:49 |
| openstack | Launchpad bug 1786106 in tripleo "containers-prep fails w/ update_containers is undefined due to lack of default" [High,Fix released] - Assigned to Quique Llorente (quiquell) | 05:49 |
| *** quiquell|off is now known as quiquell | 05:49 | |
| marios | o/ quiquell | 05:52 |
| *** udesale has quit IRC | 05:53 | |
| *** udesale has joined #oooq | 05:58 | |
| *** jtomasek has joined #oooq | 06:05 | |
| *** ratailor has joined #oooq | 06:06 | |
| *** holser_ has joined #oooq | 06:21 | |
| hubbot1 | FAILING CHECK JOBS on master: legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-3nodes-multinode, tripleo-quickstart-extras-gate-master-delorean-full-featureset052, tripleo-ci-centos-7-scenario009 (2 more messages) | 06:51 |
| *** bogdando has joined #oooq | 07:11 | |
| *** skramaja has joined #oooq | 07:23 | |
| *** ykarel_ has joined #oooq | 07:24 | |
| *** ykarel has quit IRC | 07:26 | |
| *** ykarel__ has joined #oooq | 07:28 | |
| *** ykarel_ has quit IRC | 07:30 | |
| *** udesale has quit IRC | 07:30 | |
| *** ykarel_ has joined #oooq | 07:41 | |
| *** ykarel__ has quit IRC | 07:44 | |
| *** tosky has joined #oooq | 07:45 | |
| *** ykarel__ has joined #oooq | 07:46 | |
| *** ykarel__ is now known as ykarel | 07:46 | |
| *** ykarel has quit IRC | 07:47 | |
| *** ykarel_ has quit IRC | 07:48 | |
| bogdando | hi folks. I can't recall some recent patch which allows config files opened in the browser, instead of downloading mode | 07:50 |
| bogdando | any help with that? wanted to add logrotated files there as well | 07:50 |
| bogdando | so folks could surf the job logs better :) | 07:51 |
| *** udesale has joined #oooq | 07:53 | |
| *** ykarel has joined #oooq | 07:54 | |
| *** amoralej|off is now known as amoralej | 07:55 | |
| *** chem has joined #oooq | 08:02 | |
| *** ykarel_ has joined #oooq | 08:12 | |
| *** ykarel has quit IRC | 08:15 | |
| *** ykarel__ has joined #oooq | 08:20 | |
| *** ykarel_ has quit IRC | 08:23 | |
| *** ykarel_ has joined #oooq | 08:23 | |
| jfrancoa | good morning! can anybody solve me a doubt about the deployed_server variable in the featuresets? | 08:25 |
| *** ykarel__ has quit IRC | 08:26 | |
| jfrancoa | why do we have it set to true in some featuresets https://github.com/openstack/tripleo-quickstart/blob/master/config/general_config/featureset040.yml#L24 and conditionaly to true or false depending on the release in other featuresets https://github.com/openstack/tripleo-quickstart/blob/master/config/general_config/featureset018.yml#L87 ? | 08:28 |
| *** gkadam has joined #oooq | 08:30 | |
| *** gkadam has quit IRC | 08:31 | |
| kopecmartin|ruck | marios|rover, have you seen something similar like described in these bugs? https://bugs.launchpad.net/tripleo/+bug/1796040 https://bugs.launchpad.net/tripleo/+bug/1793665 | 08:34 |
| openstack | Launchpad bug 1796040 in tripleo "Define stestr facts conditaion check fails in validate-tempest role" [Undecided,New] | 08:34 |
| openstack | Launchpad bug 1793665 in tripleo "Fs016/17/20 periodic jobs fails on validate-tempest: Unexpected response code received" [Critical,Triaged] | 08:34 |
| kopecmartin|ruck | I'm wondering if i can put promotion-blocker there or not | 08:34 |
| marios|rover | kopecmartin|ruck: heh i was just looking at this http://logs.openstack.org/98/604298/20/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/037bd06/job-output.txt.gz#_2018-10-04_04_39_49_234612 | 08:35 |
| marios|rover | i think is the same? | 08:35 |
| marios|rover | 2018-10-04 04:39:49.234612 | primary | fatal: [undercloud]: FAILED! => {"changed": true, "cmd": "tail -10 tempest.log; exit 1", "delta": "0:00:00.012430", "end": | 08:35 |
| marios|rover | rimary | TASK [validate-tempest : Exit with tempest result code if config | 08:35 |
| kopecmartin|ruck | yes, i tihnk it's the same as in the two bugs I shared .. when you scroll up, you can see, that tempest execution failed | 08:36 |
| kopecmartin|ruck | also stestr init failed, which made generation of HTML report failed too | 08:37 |
| kopecmartin|ruck | i think there are two bugs, in the job .. one is the failure of tempest execution and the other failure of stestr init | 08:37 |
| marios|rover | kopecmartin|ruck: added comment to 1796040 | 08:37 |
| marios|rover | kopecmartin|ruck: oh perhaps i chose the wrong one i thought they were duplicates /me reads more carefully | 08:38 |
| marios|rover | kopecmartin|ruck: no i think that testr you point to is invalid since it has ignore on it? | 08:38 |
| marios|rover | kopecmartin|ruck: here https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/6760f43/job-output.txt.gz#_2018-10-01_21_03_01_762184 | 08:39 |
| marios|rover | kopecmartin|ruck: yes it fails but then you see it has 2018-10-01 21:03:01.814479 | tripleo-ovb-centos-7 | ...ignoring | 08:39 |
| marios|rover | kopecmartin|ruck: and if you scroll down https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/6760f43/job-output.txt.gz#_2018-10-01_21_03_22_326496 | 08:39 |
| marios|rover | you see the same error which is the actual issue | 08:40 |
| marios|rover | adding comment | 08:40 |
| kopecmartin|ruck | oh, yeah, that one is probably invalid as you said, ok, i think we can mark it as invalid | 08:40 |
| kopecmartin|ruck | then the comment you added to 1796040 should go to 1793665 :) | 08:41 |
| marios|rover | kopecmartin|ruck: done | 08:42 |
| marios|rover | kopecmartin|ruck: so that took long for the gates to break | 08:42 |
| marios|rover | like 12 hours ! | 08:42 |
| ssbarnea|bkp2 | is anyone aware of a specific pip bug where it downgrades editable packages? i found some weird behavior yesterday where without clear reason pip decided to downgrade tqe from ~dev1000 to something like ~dev850 which obviously broke that build. | 08:43 |
| marios|rover | arxcruz: any ideas on this one please ? because 'tempest' https://bugs.launchpad.net/tripleo/+bug/1793665 | 08:43 |
| openstack | Launchpad bug 1793665 in tripleo "Fs016/17/20 periodic jobs fails on validate-tempest: Unexpected response code received" [Critical,Triaged] | 08:43 |
| *** panda|off is now known as panda | 08:44 | |
| *** ykarel__ has joined #oooq | 08:45 | |
| arxcruz | marios|rover: checking | 08:45 |
| arxcruz | marios|rover: oh, i've seen this error, and i'm already debugging it :) | 08:46 |
| marios|rover | arxcruz: excellent :) | 08:46 |
| arxcruz | i saw it on my patch to fix stackviz | 08:46 |
| *** udesale has quit IRC | 08:46 | |
| *** ykarel_ has quit IRC | 08:48 | |
| bogdando | PTAL https://review.openstack.org/#/c/593103/ | 08:49 |
| *** ykarel_ has joined #oooq | 08:49 | |
| *** udesale has joined #oooq | 08:49 | |
| bogdando | I think all of the concerns had been settled and it's good to go | 08:49 |
| bogdando | and then there is https://review.openstack.org/#/c/576746/ that really needs that one | 08:50 |
| *** ykarel__ has quit IRC | 08:51 | |
| hubbot1 | FAILING CHECK JOBS on master: legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-3nodes-multinode, tripleo-quickstart-extras-gate-master-delorean-full-featureset052, tripleo-ci-centos-7-scenario009 (2 more messages) | 08:51 |
| arxcruz | marios|rover: https://review.openstack.org/605419 should fix it | 08:52 |
| panda | quiquell: how's going ? | 08:53 |
| quiquell | panda: Testing build-containers here https://review.rdoproject.org/r/#/c/16768/ | 08:55 |
| quiquell | panda: fs001 here https://review.rdoproject.org/r/#/c/16390/ gives error | 08:55 |
| quiquell | panda: ^ cannot find the issue | 08:55 |
| panda | quiquell: it has been pretty random | 08:58 |
| quiquell | panda: the "ERROR" ? | 08:58 |
| quiquell | panda: About build-containers, Don't know the implications of running at post, also I don't know if we can override required-projects | 08:59 |
| quiquell | panda: overriding it will override the parent ? | 08:59 |
| quiquell | panda: humm found the issue | 09:00 |
| quiquell | ERROR Unable to find playbook /tmp/tmp0xy6_j82/279a04d46dd147ba97c407d30dcdbbd0/trusted/project_0/review.rdoproject.org/config/playbooks/tripleo-rdo-base/pre.yaml | 09:01 |
| quiquell | damn.. it's yml not yaml | 09:03 |
| quiquell | :-/ | 09:03 |
| quiquell | panda: https://review.rdoproject.org/r/#/c/16771/ | 09:04 |
| quiquell | sshnaidm: Do you have super powers for https://review.rdoproject.org/r/#/c/16771/ ? | 09:05 |
| quiquell | panda: ^ ? | 09:05 |
| sshnaidm | quiquell, yeah | 09:05 |
| panda | quiquell: I think the standard there is to use yaml | 09:05 |
| quiquell | panda: Ok will move the file, sshnaidm wait | 09:06 |
| *** ykarel_ has quit IRC | 09:07 | |
| *** ykarel_ has joined #oooq | 09:07 | |
| quiquell | sshnaidm, panda: Done, please merge https://review.rdoproject.org/r/#/c/16771/ | 09:07 |
| quiquell | So we can continue | 09:07 |
| *** ykarel__ has joined #oooq | 09:10 | |
| quiquell | sshnaidm, panda: Please merge this https://review.rdoproject.org/r/#/c/16771/ we cannot continue without it | 09:12 |
| *** ykarel_ has quit IRC | 09:13 | |
| panda | quiquell: commented 16768 | 09:13 |
| panda | quiquell: I don't have +2 there | 09:13 |
| quiquell | sshnaidm: pleaseee | 09:14 |
| sshnaidm | quiquell, I see it's merging | 09:15 |
| quiquell | sshnaidm: thanks sir | 09:15 |
| quiquell | sshnaidm: We will need your super powers until rlandy is around, so be prepare | 09:15 |
| sshnaidm | always ready! | 09:16 |
| quiquell | panda: fixed | 09:17 |
| *** ykarel has joined #oooq | 09:18 | |
| *** ykarel__ has quit IRC | 09:19 | |
| sshnaidm | quiquell, how are you gonna hide secrets in rdo jobs now? | 09:22 |
| quiquell | sshnaidm: It's a prototype to run without merge, it will fail at secrets, but we will check everything else | 09:23 |
| * sshnaidm remembers a lot of arguing with pabelanger about it | 09:23 | |
| sshnaidm | quiquell, but in general? | 09:23 |
| quiquell | sshnaidm: base jobs at 'config' project it's a trusted one | 09:23 |
| sshnaidm | quiquell, right | 09:23 |
| sshnaidm | quiquell, so all periodic jobs will be there? | 09:24 |
| quiquell | sshnaidm: yep | 09:24 |
| sshnaidm | cool | 09:24 |
| quiquell | sshnaidm: Do we have alternatives with vault or the like to store secrets at untrusted ? | 09:24 |
| sshnaidm | quiquell, not much alternatives.. with ssh I did something like that: https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/playbooks/legacy-tripleo-ci-periodic-base-upload/post.yaml#L6-L21 | 09:26 |
| sshnaidm | quiquell, but not sure if it's possible to do with other passwords | 09:27 |
| sshnaidm | quiquell, although the problem will be that we will not be able run periodic jobs in patch | 09:28 |
| sshnaidm | quiquell, and couldn't test changes with periodic jobs.. | 09:28 |
| sshnaidm | it was pretty nice feature to use | 09:29 |
| quiquell | sshnaidm: where is this uploader_ssh_key.ssh_private_key ? | 09:29 |
| sshnaidm | quiquell, in config repo | 09:29 |
| quiquell | sshnaidm: Ahh so you still depends con cofig repo but you can define jobs at rdo-jobs, that's it ? | 09:30 |
| sshnaidm | https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/secrets.yaml | 09:30 |
| sshnaidm | quiquell, yep | 09:30 |
| quiquell | sshnaidm: how zuul knows how to decrypt the priv key ? | 09:31 |
| quiquell | sshnaidm: where is the password for decypt ? | 09:31 |
| sshnaidm | quiquell, I encrypt it with zuuls key | 09:31 |
| quiquell | sshnaidm: Ahh that's associated with trusted projects, isn't it ? | 09:31 |
| sshnaidm | quiquell, and only zuul can decrypt it with its private key in job | 09:32 |
| *** ykarel has quit IRC | 09:32 | |
| quiquell | sshnaidm: so zuul internally decript it for you before use it | 09:32 |
| *** ykarel has joined #oooq | 09:32 | |
| quiquell | sshnaidm: so we can do the same for normal DLRN passwords and the like | 09:32 |
| sshnaidm | quiquell, if we use passwords in post/pre that are located in trusted, we are good | 09:32 |
| sshnaidm | quiquell, but you shouldn't be able to reveal them during the job | 09:32 |
| quiquell | sshnaidm: then why didn't we use this instead of move jobs to config ? | 09:32 |
| quiquell | sshnaidm: Yep you have to be careful | 09:33 |
| sshnaidm | quiquell, because it's not easy to hid password in job, that's why I used ssh-agent | 09:33 |
| quiquell | sshnaidm: I see | 09:33 |
| sshnaidm | quiquell, if it's just variable, you can create a job in rdo-jobs and post there "echo $MY_SECRET" | 09:33 |
| sshnaidm | quiquell, tbh not sure if we can hide it good enough.. if dlrn_api could provide us a time-based token, it would be easier.. | 09:35 |
| *** ykarel_ has joined #oooq | 09:35 | |
| *** ykarel has quit IRC | 09:38 | |
| panda | quiquell: : "The task includes an option with an undefined variable. The error was: 'toci_jobtype' is undefined" | 09:42 |
| *** ykarel_ is now known as ykarel | 09:43 | |
| panda | quiquell: why are you still running the legacy playbook ? | 09:44 |
| quiquell | panda: where ? | 09:49 |
| panda | quiquell: dnm-test-job | 09:50 |
| panda | quiquell: in your testing tripleo-ci-base-singlenode-containers-build | 09:50 |
| panda | in the container build, we install the undercloud in the host for a series of hitorical reason, the same job was originally scheduled to build the images too, but the idea was changed. The part that is important is the repo setup | 09:51 |
| panda | cloud images | 09:52 |
| quiquell | panda: give me a sec, I am preparing the proto for promote | 09:52 |
| quiquell | panda: so we don't need to run it ? | 09:53 |
| quiquell | panda: Didn't know why we were running it | 09:54 |
| quiquell | panda: Just try to mimic the present stuff | 09:54 |
| quiquell | panda: only repo setup is needed ? | 09:54 |
| panda | quiquell: yes, but don't change the process now, just run the undercloud install | 09:55 |
| quiquell | panda: ack, we can narrow down it later on | 09:55 |
| *** dtantsur|afk is now known as dtantsur | 09:55 | |
| quiquell | panda: btw has try to do an "include: ../zuul-v3.yaml" so we run container build at run not at post, but it's not working | 09:56 |
| quiquell | panda: promote proto https://review.rdoproject.org/r/#/c/16773/ | 09:57 |
| quiquell | panda: so we have to run build-test-package forcing it to build ? | 09:59 |
| *** udesale has quit IRC | 10:04 | |
| panda | quiquell: the periodic base alrady runs the undercloud install | 10:06 |
| panda | quiquell: so you don't have to specify a run | 10:06 |
| panda | quiquell: then we can do the rest in post | 10:06 |
| panda | the problem may be with timeout | 10:06 |
| quiquell | panda: So it's not a problem to build the images at post ? | 10:07 |
| panda | we take a lot to build and upload the images | 10:07 |
| quiquell | panda: It times out we will stil try it | 10:07 |
| quiquell | panda: we can configure post timeouts | 10:07 |
| panda | quiquell: yep, and we better. | 10:07 |
| panda | quiquell: also, if install fails at run, we're still going to try to build images | 10:08 |
| panda | whih may be not ideal | 10:08 |
| quiquell | panda: we can check that with zuul_success variable | 10:08 |
| quiquell | panda: as we do with DLRN reporting at post | 10:08 |
| quiquell | panda: goint to do the timeouts and zuul_success | 10:09 |
| quiquell | panda: Do you know if the variables we have at common.yaml can be used at post ? | 10:09 |
| panda | quiquell: wait | 10:12 |
| panda | why are we using post ? | 10:12 |
| quiquell | panda: because we cannot have more than one playbook at run | 10:12 |
| quiquell | panda: if you set run: it will overwrite parent | 10:13 |
| panda | quiquell: yes we can | 10:13 |
| quiquell | panda: include ? | 10:13 |
| panda | quiquell: well eyah, not directly with zuul | 10:13 |
| panda | quiquell: yeah ? | 10:13 |
| *** udesale has joined #oooq | 10:13 | |
| panda | is it not working ? | 10:13 |
| panda | oh yu tird | 10:13 |
| quiquell | panda: I have trie to move post.yaml to run.yaml and include: ... run-v3.yaml | 10:14 |
| panda | do you have the error ? | 10:14 |
| quiquell | yep, maybe the path is wrong | 10:14 |
| quiquell | panda: is at one of the ps | 10:14 |
| panda | also, use include_playbook | 10:14 |
| panda | nah, doesn't exist | 10:14 |
| quiquell | panda: https://logs.rdoproject.org/68/16768/16/check/periodic-tripleo-centos-7-master-containers-build/66f644b/job-output.txt.gz | 10:14 |
| quiquell | panda: humm wait... | 10:15 |
| quiquell | panda: I know how | 10:15 |
| quiquell | panda: zuul.projects['...tripleo-ci']/playbooks/tripleo-ci/run-v3.yaml | 10:15 |
| quiquell | panda: {{ zuul.projects['...tripleo-ci'].src_dir }}/playbooks/tripleo-ci/run-v3.yaml | 10:15 |
| quiquell | panda: How important is to run build-containers at run and not at post ? | 10:16 |
| panda | quiquell: too many unknowns | 10:17 |
| quiquell | panda: ack, let's do it at run | 10:17 |
| panda | you change the process, you ask for trouble | 10:17 |
| quiquell | panda: ok, changed, let's see if it works with absolute path | 10:21 |
| panda | quiquell: absolutely | 10:23 |
| quiquell | panda: check job failing https://review.rdoproject.org/r/#/c/16390/ | 10:25 |
| quiquell | panda: te-broker client failing | 10:27 |
| quiquell | panda: failing here https://github.com/openstack-infra/tripleo-ci/blob/master/playbooks/tripleo-ci/templates/toci_gate_test.sh.j2#L166 | 10:30 |
| panda | I see it | 10:31 |
| panda | marios|rover: any problem with ovb jobs recently ? | 10:32 |
| marios|rover | panda: yeah seeing lots of overcloud-deploy poking at the moment | 10:32 |
| panda | quiquell: I think it may be just high load , te-broker is working correclty | 10:33 |
| quiquell | panda: ack | 10:34 |
| quiquell | panda: Can i change config/playbooks/legacy-tripleo-ci-periodic-base/post.yaml ? | 10:34 |
| quiquell | panda: Or do I copy paste and change ? | 10:34 |
| panda | quiquell: copy on write | 10:34 |
| panda | quiquell: we may need to revert | 10:34 |
| marios|rover | panda: this what you mean http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/9cda9bb/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 10:34 |
| marios|rover | status ERROR due to "Message: No valid host was found. , Code: 500" | 10:35 |
| panda | it may be easier if we don't change everything | 10:35 |
| quiquell | panda: ok will create playbooks/tripleo-ci-periodic-base for the new stuff | 10:35 |
| *** skramaja_ has joined #oooq | 10:35 | |
| *** skramaja has quit IRC | 10:36 | |
| *** kopecmartin|ruck is now known as kopecmartin|scho | 10:38 | |
| *** udesale has quit IRC | 10:39 | |
| *** udesale has joined #oooq | 10:40 | |
| quiquell | panda, sshnaidm: https://review.rdoproject.org/r/16777 | 10:41 |
| sshnaidm | quiquell, commented | 10:44 |
| panda | quiquell: some fixes needed | 10:49 |
| quiquell | panda: yep, on the way for container-builds | 10:50 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario009 (2 more messages) | 10:51 |
| quiquell | sshnaidm, panda: https://review.rdoproject.org/r/16777 changed | 10:56 |
| marios|rover | kopecmartin|scho: OVB jobs are failing for "No valid host was found. , Code: 500" during the overcloud deployment Edit | 10:57 |
| marios|rover | kopecmartin|scho: https://bugs.launchpad.net/tripleo/+bug/1796069 | 10:57 |
| openstack | Launchpad bug 1796069 in tripleo "OVB jobs are failing for "No valid host was found. , Code: 500" during the overcloud deployment" [High,Triaged] - Assigned to Marios Andreou (marios-b) | 10:57 |
| marios|rover | panda: ^^^ is that what you meant or are you seeing something different? | 10:57 |
| panda | marios|rover: no, testenv client is failing on us, probably heavy load, don't worry for this, and thanks. | 10:59 |
| marios|rover | panda: k | 11:01 |
| panda | quiquell: still indentation | 11:01 |
| panda | quiquell: on the first line | 11:01 |
| panda | quiquell: you didn't fix that | 11:01 |
| quiquell | panda: it's not needed | 11:01 |
| quiquell | panda: identation is ok | 11:02 |
| ssbarnea|bkp2 | panda: what is the meaning of "execute quickstart.sh --clean to ensure the dependencies are installed "? I found this failure at https://ci.centos.org/job/tripleo-quickstart-extras-gate-master-delorean-full-featureset052/834/console and I find the msg bit confusing. | 11:03 |
| panda | quiquell: o_O | 11:03 |
| quiquell | panda: ERROR! 'import_paybook' is not a valid attribute for a Play | 11:03 |
| *** amoralej is now known as amoralej|lunch | 11:03 | |
| quiquell | https://logs.rdoproject.org/68/16768/19/check/periodic-tripleo-centos-7-master-containers-build/c4d3eda/job-output.txt.gz#_2018-10-04_10_55_17_389478 | 11:03 |
| panda | quiquell: yes, import_playbook doesnt' exist | 11:04 |
| quiquell | panda: https://docs.ansible.com/ansible/2.5/modules/import_playbook_module.html | 11:04 |
| quiquell | panda: it exists | 11:04 |
| Tengu | quiquell: well, without the typo, it does exist, yep | 11:04 |
| Tengu | quiquell: an "l" is missing. | 11:04 |
| quiquell | Tengu: typo ? | 11:04 |
| panda | quiquell: oh, I see the indentation ... ok | 11:04 |
| quiquell | argggg !!! | 11:04 |
| Tengu | :) | 11:04 |
| panda | ihihih | 11:05 |
| quiquell | Tengu: book to pay :-) | 11:05 |
| Tengu | call me eagle eye ;) | 11:05 |
| quiquell | Tengu++ | 11:05 |
| hubbot1 | quiquell: Tengu's karma is now 2 | 11:05 |
| quiquell | Tengu++ | 11:05 |
| hubbot1 | quiquell: Tengu's karma is now 3 | 11:05 |
| quiquell | Tengu+++ | 11:05 |
| quiquell | Arggg triple karma | 11:06 |
| Tengu | :D | 11:06 |
| ssbarnea|bkp2 | http://git.openstack.org/cgit/openstack/tripleo-quickstart/tree/quickstart.sh#n538 - not clear to me why it exits with this message if the file does not exist. | 11:07 |
| kopecmartin|scho | marios|rover, checking, thanks | 11:12 |
| quiquell | sshnaidm: what do you mean here ? https://review.rdoproject.org/r/#/c/16777/1/playbooks/tripleo-ci-periodic-base/post.yaml@8 | 11:17 |
| sshnaidm | quiquell, is $workspace defined anywhere? | 11:18 |
| quiquell | sshnaidm: fixed | 11:22 |
| quiquell | panda: Why it didn't do anything at run here ? https://logs.rdoproject.org/68/16768/22/check/periodic-tripleo-centos-7-master-containers-build/28ae198/job-output.txt.gz | 11:24 |
| quiquell | panda: It's not running the imported plabook or the next tasks ? | 11:26 |
| panda | where is that playbook defined ? | 11:26 |
| quiquell | panda: tripleo-ci/playbooks/tripleo-ci/run-v3.yaml | 11:27 |
| panda | review.rdoproject.org/rdo-jobs/playbooks/legacy/tripleo-ci-centos-7-containers-build/run.yam4 | 11:29 |
| quiquell | panda: this is where the import_playbook is | 11:29 |
| quiquell | sshnaidm: Let's merge this https://review.rdoproject.org/r/#/c/16777/ | 11:30 |
| quiquell | sshnaidm: Only affects the new jobs it's a separte post.yaml | 11:30 |
| sshnaidm | quiquell, done | 11:31 |
| quiquell | sshnaidm: I don't know if we are going to have 'featureset' and the other there we will see | 11:31 |
| sshnaidm | quiquell, do you have 10 min for ovb sync? | 11:44 |
| quiquell | sshnaidm: sure | 11:44 |
| sshnaidm | quiquell, https://bluejeans.com/u/sshnaidm/ | 11:44 |
| quiquell | sshnaidm: https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/ovb-jobs.yaml | 11:50 |
| quiquell | sshnaidm: https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/ovb-jobs.yaml#L16 | 11:52 |
| arxcruz | panda: hi, how can i block a task with a dependence of another card ? | 11:56 |
| panda | arxcruz: there is not effective mechanism, you mark the card blocked and add the url of that task, but nothing prevents updates on the card | 11:57 |
| arxcruz | panda: ack | 11:58 |
| weshay | marios|rover, kopecmartin|scho good morning... | 12:02 |
| weshay | I see the gate is once again foooked | 12:02 |
| marios|rover | weshay: yup | 12:03 |
| weshay | I saw a small trend last where qemu jobs timeout and kvm worked fine | 12:03 |
| weshay | marios|rover, http://logs.openstack.org/41/601241/16/gate/tripleo-ci-centos-7-containers-multinode/5a05846/job-output.txt.gz#_2018-10-03_21_37_01_709646 | 12:04 |
| weshay | http://logs.openstack.org/90/605090/10/gate/tripleo-ci-centos-7-scenario001-multinode-oooq-container/6edb958/job-output.txt.gz#_2018-10-03_18_20_58_500872 | 12:04 |
| marios|rover | weshay: https://storyboard.openstack.org/#!/story/2003966 | 12:07 |
| sshnaidm | quiquell, https://github.com/rdo-infra/rdo-jobs/commit/ff0b7d5fb98f85c6585e101765531ef0ea43f67e | 12:08 |
| marios|rover | weshay: this is killing us https://bugs.launchpad.net/tripleo/+bug/1793665 we should just disable tempest | 12:09 |
| openstack | Launchpad bug 1793665 in tripleo "multinode-oooq-container scenarios and Fs016/17/20 periodic jobs fails on validate-tempest: Unexpected response code received" [Critical,Triaged] | 12:09 |
| marios|rover | :) | 12:09 |
| *** trown|outtypewww is now known as trown | 12:13 | |
| *** rfolco has joined #oooq | 12:14 | |
| weshay | ugh.. this one 2018-10-04 06:45:04 | {0} tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern [24.936596s] ... FAILED | 12:14 |
| *** rlandy has joined #oooq | 12:18 | |
| quiquell | rlandy: o/ | 12:20 |
| rlandy | quiquell: hi - how goes it? | 12:20 |
| quiquell | rlandy: we are doing some prototypes for build-containres and promote-current-to-tripleo-ci-testing | 12:21 |
| rlandy | quiquell: k - good - we need those | 12:21 |
| quiquell | rlandy: promote https://review.rdoproject.org/r/#/c/16773/ | 12:21 |
| * rlandy is looking through email for reviews | 12:21 | |
| quiquell | rlandy: build-containers https://review.rdoproject.org/r/#/c/16768/ | 12:22 |
| quiquell | rlandy: forget about 'config' repo one, let's make the repo work first, then we move them to 'config' | 12:22 |
| quiquell | rlandy: I think the promote prototype near to work now | 12:23 |
| *** ssbarnea_ has joined #oooq | 12:23 | |
| rlandy | quiquell: nice - looking | 12:23 |
| *** ssbarnea|bkp2 has quit IRC | 12:25 | |
| *** amoralej|lunch is now known as amoralej | 12:26 | |
| quiquell | rlandy: We are also testing the check job here https://review.rdoproject.org/r/#/c/16390/ | 12:27 |
| quiquell | rlandy: without the Depends-On to see if c'config' stuff is working | 12:27 |
| rlandy | quiquell: cool - same failure on both jobs? | 12:28 |
| rlandy | quiquell: before you log off today, let's juts touch base on where I should pick up | 12:29 |
| quiquell | rlandy: don't know, taking care of the promotion protos | 12:29 |
| rlandy | yeah | 12:29 |
| quiquell | rlandy: sure, btw, I have to bypass the DLRN reporting post for promote job but it's not working | 12:29 |
| rlandy | quiquell: panda around this morning? | 12:30 |
| quiquell | rlandy: yep | 12:30 |
| quiquell | Ok going for lunch now | 12:32 |
| rlandy | quiquell: moving https://tree.taiga.io/project/tripleo-ci-board/task/108?kanban-status=1447274 to progress | 12:32 |
| rlandy | since we are working on it | 12:32 |
| quiquell | rlandy: ack, put the test review there | 12:33 |
| *** quiquell is now known as quiquell|lunch | 12:33 | |
| *** tosky__ has joined #oooq | 12:37 | |
| *** tosky is now known as Guest8399 | 12:37 | |
| *** tosky__ is now known as tosky | 12:37 | |
| *** Guest8399 has quit IRC | 12:39 | |
| rlandy | quiquell|lunch: hmmm - we have a diff failure in the translated job :( | 12:39 |
| quiquell|lunch | I am around | 12:40 |
| quiquell|lunch | rlandy: do you have pointer ? | 12:40 |
| rlandy | yep - have lunch - will investigate | 12:40 |
| quiquell|lunch | rlandy: ack | 12:40 |
| *** agopi has quit IRC | 12:41 | |
| *** agopi has joined #oooq | 12:41 | |
| rlandy | 2018-10-04 10:00:22.235040 | primary | +(/home/zuul/src/git.openstack.org/openstack/tripleo-ci/toci_gate_test.sh:159): sudo kill -9 4725 | 12:43 |
| *** agopi has quit IRC | 12:45 | |
| *** agopi has joined #oooq | 12:45 | |
| *** kopecmartin|scho is now known as kopecmartin|ruck | 12:46 | |
| *** agopi has quit IRC | 12:47 | |
| *** agopi has joined #oooq | 12:47 | |
| *** agopi has quit IRC | 12:48 | |
| *** agopi has joined #oooq | 12:48 | |
| rlandy | quiquell|lunch: think that was just a hitch with te-broker - rerun is going better so far | 12:50 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario008-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario004-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master, tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci-centos-7-scenario007-multinode- (4 more messages) | 12:51 |
| weshay | rfolco, please lead the scrum w/o me | 12:57 |
| *** quiquell|lunch is now known as quiquell | 13:05 | |
| quiquell | rlandy: ok, let's see | 13:05 |
| rlandy | quiquell; I am adding a review for the actual trigger | 13:05 |
| ssbarnea_ | kopecmartin|ruck : i will need some help with https://bugs.launchpad.net/tripleo/+bug/1796048 as it seems to affect some builds. | 13:07 |
| openstack | Launchpad bug 1796048 in tripleo "quiskstart.sh installs downgraded version of tripleo-quickstart-extras" [High,Triaged] | 13:07 |
| kopecmartin|ruck | ssbarnea_, checking | 13:08 |
| kopecmartin|ruck | ssbarnea, a dummy question, are you sure the failure is not related to the changes? | 13:13 |
| kopecmartin|ruck | or maybe it got fixed already, because the job worked here https://review.openstack.org/#/c/601850/ | 13:13 |
| *** vinaykns has joined #oooq | 13:17 | |
| *** ratailor has quit IRC | 13:21 | |
| ssbarnea_ | kopecmartin|ruck weshay helped me yesterday and he said something about a pip caching bug, but I am not sure about that. first, apparently the weird behaviour is while using setup install method and not inside pip. | 13:27 |
| ssbarnea_ | i didn't use eggs too much as I went to wheels as soon it was possible, so i am not sure exact how they behave. | 13:29 |
| panda | ssbarnea_: ping | 13:33 |
| *** holser_ has quit IRC | 13:34 | |
| ssbarnea_ | panda yep. | 13:34 |
| panda | ssbarnea_: can you please rejoin thee meeting ? | 13:35 |
| ssbarnea_ | sure | 13:36 |
| *** ykarel_ has joined #oooq | 13:36 | |
| *** holser_ has joined #oooq | 13:36 | |
| kopecmartin|ruck | ssbarnea, interesting, the bug was seen three months ago https://bugs.launchpad.net/tripleo/+bug/1777261, then it was closed and a new bug was created a month after that https://bugs.launchpad.net/tripleo/+bug/1786106 which looks very similarly | 13:38 |
| openstack | Launchpad bug 1777261 in tripleo "convert-overcloud-undercloud.yml fails on missing update_containers variable" [Critical,Fix released] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 13:38 |
| openstack | Launchpad bug 1786106 in tripleo "containers-prep fails w/ update_containers is undefined due to lack of default" [High,Fix released] - Assigned to Quique Llorente (quiquell) | 13:38 |
| kopecmartin|ruck | weshay, are you sure, the bug https://bugs.launchpad.net/tripleo/+bug/1786106 was fixed, because ssbarnea experience the same now https://bugs.launchpad.net/tripleo/+bug/1796048 | 13:38 |
| openstack | Launchpad bug 1796048 in tripleo "quiskstart.sh installs downgraded version of tripleo-quickstart-extras" [High,Triaged] | 13:38 |
| *** ykarel has quit IRC | 13:38 | |
| weshay | kopecmartin|ruck, ssbarnea is not using the latest code | 13:41 |
| weshay | he's hitting a pip cache issue in ci | 13:41 |
| weshay | kopecmartin|ruck, because jenkins .. weee | 13:41 |
| weshay | would need to see it recreated locally | 13:42 |
| rascasoft | weshay, we should not using the pip cache, that problem hits me every time | 13:42 |
| rascasoft | weshay, would it be possible? | 13:42 |
| weshay | rascasoft, I have not found a way to prevent jenkins from doing it | 13:42 |
| weshay | rascasoft, and this issue is in ci.centos where we are not admins | 13:42 |
| weshay | rascasoft, kopecmartin|ruck I was thinking about NOT using pip to install tq tqe on jenkins ci | 13:43 |
| rascasoft | weshay, what is curious is that I've hit the same problem without using Jenkins | 13:43 |
| weshay | and just git pull and setup the ansible role path | 13:43 |
| rascasoft | weshay+++++++++++++++++++++++++++++ | 13:43 |
| rascasoft | that will also speed up things | 13:43 |
| rascasoft | if we don't need it for specific reasons | 13:44 |
| weshay | rascasoft, ya.. there is no reason why quickstart.sh can't setup the config and roles properly | 13:44 |
| *** agopi has quit IRC | 13:44 | |
| weshay | rascasoft, and more of the deps could be rpm based I suppose | 13:44 |
| weshay | it's worth revisiting | 13:44 |
| rascasoft | weshay, count me in | 13:44 |
| * rascasoft is all for simplifying things | 13:44 | |
| quiquell | rlandy, panda: promote job is 'working' now at the proto https://review.rdoproject.org/r/#/c/16773/ | 13:46 |
| rlandy | cool | 13:46 |
| quiquell | rlandy, panda: Failing as expected at Auth of DLRN, you can review them | 13:46 |
| quiquell | rlandy: has force the release directly in the job | 13:46 |
| rlandy | sec - just adding check job review | 13:48 |
| quiquell | rlandy: ack | 13:48 |
| quiquell | rlandy: going to move the proto into config review | 13:49 |
| ykarel_ | weshay, ssbarnea i think need to adjust that tripleo-quickstart-gate-quick job: https://github.com/rdo-infra/ci-config/blob/master/jenkins/jobs/tripleo-quickstart-templates.yml#L129, add repo-tripleo-quickstart-extras to scm, i don't know how that works, if u know about it, good to try that | 13:49 |
| ykarel_ | looks like tripleo-quickstart-extras is cached somewhere | 13:50 |
| *** ykarel_ is now known as ykarel | 13:50 | |
| *** gkadam has joined #oooq | 14:00 | |
| weshay | marios|rover, kopecmartin|ruck https://review.openstack.org/607904 | 14:01 |
| weshay | oops | 14:01 |
| weshay | sorry | 14:01 |
| weshay | openstack-periodic-24hr | 14:02 |
| *** agopi has joined #oooq | 14:02 | |
| marios|rover | ack cool weshay thanks | 14:02 |
| weshay | that has been running for 130 hrs | 14:03 |
| weshay | let's ask #sf-dfg to kill it | 14:03 |
| quiquell | rlandy, panda: promote proto added to 'config' repo review | 14:06 |
| quiquell | rlandy, panda: About build-containers I don't know why it's not working https://logs.rdoproject.org/68/16768/23/check/periodic-tripleo-centos-7-master-containers-build/64a906b/job-output.txt.gz#_2018-10-04_11_46_28_833529 | 14:07 |
| quiquell | rlandy, panda: Also maybe we can just call build-test-packages instead of parenting | 14:08 |
| quiquell | Do you have need some catch up ? I will drop soon | 14:08 |
| rlandy | in 5 mins | 14:08 |
| quiquell | ok | 14:08 |
| rlandy | sorry - just getting this review in | 14:08 |
| quiquell | rlandy: Very important too | 14:08 |
| *** hamzy has quit IRC | 14:10 | |
| rlandy | quiquell: k - let's meet | 14:14 |
| rlandy | panda: ^^ | 14:14 |
| panda | quiquell: have you tried putting the import_playbook taskunder a host: declaration ? | 14:15 |
| panda | rlandy: where ? | 14:15 |
| rlandy | tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-masterqueued (non-voting) | 14:15 |
| rlandy | tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky-branchqueued (non-voting) | 14:15 |
| rlandy | ha - we have queued check jobs | 14:15 |
| quiquell | panda: nope | 14:15 |
| rlandy | we can use my bj | 14:15 |
| panda | rlandy: id ? | 14:16 |
| panda | sshnaidm: do you want to join ? | 14:16 |
| *** jaosorior has quit IRC | 14:16 | |
| sshnaidm | panda, join what? | 14:16 |
| quiquell | rlandy, panda: connected | 14:17 |
| panda | sshnaidm: the rebellion! | 14:17 |
| panda | rlandy: id ? | 14:17 |
| sshnaidm | panda, where? | 14:17 |
| rlandy | no idea | 14:17 |
| panda | no IDea | 14:18 |
| panda | ok | 14:18 |
| rlandy | blue jeans has gone on strike | 14:18 |
| quiquell | dman | 14:18 |
| sshnaidm | in https://bluejeans.com/u/rlandy/ ? | 14:19 |
| quiquell | I am there now | 14:19 |
| panda | sshnaidm: that's hte idea, but app works only with ids so I'll be late until I find the id | 14:19 |
| rlandy | https://bluejeans.com/1167276556 | 14:22 |
| weshay | arxcruz, thanks for your help today | 14:26 |
| weshay | arxcruz, one last thing if you are still with me.. easy one | 14:26 |
| *** tosky__ has joined #oooq | 14:27 | |
| *** tosky has quit IRC | 14:28 | |
| *** tosky__ is now known as tosky | 14:28 | |
| *** bogdando has quit IRC | 14:29 | |
| *** saneax has quit IRC | 14:30 | |
| *** ykarel is now known as ykarel|afk | 14:32 | |
| *** bogdando has joined #oooq | 14:32 | |
| *** Vorrtex has joined #oooq | 14:34 | |
| *** skramaja_ has quit IRC | 14:36 | |
| *** hamzy has joined #oooq | 14:45 | |
| *** kopecmartin|ruck is now known as kopecmartin|scho | 14:47 | |
| arxcruz | weshay: just my job :) | 14:48 |
| arxcruz | weshay: what do you need boss? | 14:48 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario008-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario004-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master, tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci-centos-7-scenario007-multinode- (4 more messages) | 14:51 |
| weshay | arxcruz, scenario tests added to fs27 | 14:55 |
| weshay | sorry for the delay | 14:55 |
| arxcruz | weshay: so... need to check the list of skipped because tempest is running in undercloud only | 14:56 |
| arxcruz | weshay: if panda authorize me to work on that, i will :P | 14:56 |
| weshay | arxcruz, I'm putting up a patch now.. just asking for review | 14:57 |
| arxcruz | ahhh | 14:57 |
| arxcruz | i was just kidding :) | 14:57 |
| *** toure is now known as toure|biab | 14:59 | |
| marios|rover | dtantsur: ++ thanks for the patch https://review.openstack.org/#/c/607904/ | 14:59 |
| dtantsur | np | 15:00 |
| *** holser_ has quit IRC | 15:11 | |
| *** holser_ has joined #oooq | 15:11 | |
| *** quiquell is now known as quiquell|off | 15:18 | |
| *** toure|biab is now known as toure | 15:19 | |
| weshay | arxcruz, https://review.openstack.org/607987 | 15:19 |
| arxcruz | weshay: k, let's hope it will pass the vm getting booted | 15:20 |
| *** sshnaidm is now known as sshnaidm|afk | 15:20 | |
| weshay | panda, rlandy how we doing on making the change to legacy --> zuulv3 | 15:27 |
| rlandy | panda: sshnaidm|afk: https://review.rdoproject.org/r/#/c/16761 is updated ... https://review.rdoproject.org/r/16783 contains the trugger | 15:29 |
| rlandy | trigger | 15:29 |
| rlandy | weshay: you just missed the whole discussion | 15:29 |
| weshay | still in mtgs | 15:31 |
| weshay | sorry | 15:31 |
| *** agopi is now known as agopi|long_lunch | 15:31 | |
| weshay | just looking for essentially, green,yellow, red.. | 15:31 |
| weshay | happening today, tomorrow.. never | 15:31 |
| panda | rlandy: sshnaidm|afk quiquell|off There is not way to make the import_playbook work. Commented in https://tree.taiga.io/project/tripleo-ci-board/task/119?kanban-status=1447275 | 15:33 |
| panda | weshay: hopefully today, by the end of your day | 15:34 |
| weshay | very nice | 15:34 |
| weshay | nice work everyone | 15:34 |
| *** sshnaidm|afk is now known as sshnaidm | 15:36 | |
| rlandy | panda: let's go for post | 15:36 |
| *** udesale has quit IRC | 15:36 | |
| rlandy | panda: if you see my original view, I had the build in post | 15:36 |
| rlandy | weshay: first check jobs testing on https://review.openstack.org/#/c/607963 | 15:39 |
| *** hamzy_ has joined #oooq | 15:42 | |
| *** bogdando has quit IRC | 15:43 | |
| *** hamzy has quit IRC | 15:44 | |
| *** ykarel|afk is now known as ykarel | 15:52 | |
| *** jfrancoa has quit IRC | 15:55 | |
| *** jaganathan has joined #oooq | 15:58 | |
| *** trown is now known as trown|lunch | 16:04 | |
| *** jaganathan has quit IRC | 16:05 | |
| ssbarnea_ | is this url correct https://images.rdoproject.org/master/rdo_trunk/current-passed-ci/overcloud-full.tar.md5 ? | 16:08 |
| ssbarnea_ | because there is nothing there | 16:08 |
| rlandy | sshnaidm: regarding https://review.rdoproject.org/r/#/c/16777/3/playbooks/tripleo-ci-periodic-base/post.yaml - your comment refers to line 13 - "periodic-" is also required on line 15, right? | 16:09 |
| sshnaidm | rlandy, yep | 16:09 |
| panda | ssbarnea_: where did you see that ? | 16:10 |
| ssbarnea_ | panda https://ci.centos.org/job/tripleo-quickstart-gate-master-full-images/96/console | 16:10 |
| sshnaidm | we don't use current-passed-ci for last 3 years | 16:10 |
| *** dtantsur is now known as dtantsur|afk | 16:10 | |
| rlandy | sshnaidm: https://review.rdoproject.org/r/#/c/16786/ - pls review - thanks | 16:16 |
| *** ykarel_ has joined #oooq | 16:18 | |
| panda | containers build job is installing the undercloud right now | 16:19 |
| *** ykarel_ is now known as ykarel|away | 16:20 | |
| *** ykarel has quit IRC | 16:20 | |
| weshay | rfolco, ssbarnea_ you guys get the email about the infra mtg next week? | 16:22 |
| rfolco | weshay, I did, not sure it will conflict with meetup | 16:22 |
| weshay | rfolco, ya.. it conflict, if you can try to attend the infra mtg.. stepping away from the tc/ua that would be very helpful | 16:23 |
| rfolco | weshay, yes, accepted | 16:23 |
| ssbarnea_ | weshay: yep, accepted right away. | 16:26 |
| ssbarnea_ | rfolco weshay : https://review.openstack.org/#/c/607624/ | 16:28 |
| weshay | ssbarnea_, thinking you can do better than that :) | 16:31 |
| rlandy | panda: anything I can help with? | 16:33 |
| weshay | ssbarnea_, have a minute? I'll show you what is more likely the issue | 16:39 |
| ssbarnea_ | weshay: yep. while I suspect the cause is somewhere else I am confused by the fact that extras-common/tasks/main.yml:24 does not exist, the file is empty. | 16:40 |
| ssbarnea_ | weshay i seen the same task in another role.... | 16:40 |
| weshay | ssbarnea_, ya.. this is a shitty problem, probably a few things going on | 16:41 |
| weshay | let's try to solve on blue? | 16:41 |
| ssbarnea_ | weshay i am on https://bluejeans.com/2655417928 if you have time | 16:41 |
| panda | rlandy: just iterating on the containers build, fixed workspace variable, just rerunning | 16:43 |
| panda | rlandy: undercloud install passes, of course, testing build in post | 16:43 |
| *** holser_ has quit IRC | 16:45 | |
| weshay | ssbarnea_, https://github.com/openstack/tripleo-quickstart-extras/commit/c2a3aac33f175f71575d426115791171e7ad60d3#diff-23af0a8485e71d2131516d07ef64d3d8R24 | 16:49 |
| ssbarnea_ | weshay : https://bugs.launchpad.net/tripleo/+bug/1623933 | 16:50 |
| openstack | Launchpad bug 1623933 in tripleo "Quickstart doesn't allow to use different pip mirrors or cache" [High,Triaged] | 16:50 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario008-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario004-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master, tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci-centos-7-scenario007-multinode- (4 more messages) | 16:51 |
| weshay | ssbarnea_, https://ci.centos.org/view/rdo/view/tripleo-gate/job/tripleo-quickstart-gate-master-delorean-quick-basic/ws/usr/local/share/ansible/roles/extras-common/tasks/main.yml/*view*/ | 16:53 |
| ssbarnea_ | weshay : the one i created today https://bugs.launchpad.net/tripleo/+bug/1796048 | 16:53 |
| openstack | Launchpad bug 1796048 in tripleo "quiskstart.sh installs downgraded version of tripleo-quickstart-extras" [High,Triaged] | 16:53 |
| rlandy | panda: - anyone worked on commenting out the promoter? | 16:54 |
| rlandy | otherwise will pick that up | 16:54 |
| *** ykarel|away has quit IRC | 16:58 | |
| *** trown|lunch is now known as trown | 17:04 | |
| *** gkadam has quit IRC | 17:08 | |
| panda | rlandy: you want to comment to prevent false positives ? | 17:10 |
| rlandy | panda: ack | 17:23 |
| *** amoralej is now known as amoralej|off | 17:37 | |
| *** vinaykns has quit IRC | 17:38 | |
| rlandy | panda: https://review.rdoproject.org/r/16793 - for when we are ready to test promotions | 17:49 |
| weshay | rlandy, so that will not attempt to promote because ci-testing -> current-tripleo is remarked out | 17:51 |
| weshay | right? | 17:51 |
| rlandy | weshay: that is the hope | 17:51 |
| weshay | rlandy, was thinking we could add a fake criteria | 17:51 |
| weshay | like featureset099 | 17:51 |
| weshay | but either is fine by me | 17:52 |
| rlandy | https://review.rdoproject.org/r/#/c/16783 - switches the jobs | 17:52 |
| weshay | unless we're not confident | 17:52 |
| weshay | \0/ | 17:52 |
| rlandy | I think we can start uncommenting some lines when we see one successful run | 17:52 |
| weshay | rlandy++ | 17:52 |
| hubbot1 | weshay: rlandy's karma is now 24 | 17:52 |
| weshay | rlandy, is your confidence 90%+ on https://review.rdoproject.org/r/#/c/16793/ | 17:53 |
| weshay | or should we merge say.. pike.ini | 17:53 |
| rlandy | weshay: panda is working on the promote/container build stuff - we need to wait on that job | 17:53 |
| weshay | and watch the promoter | 17:53 |
| rlandy | if panda is confident on that piece, I am semi confident on the rest | 17:54 |
| rlandy | weshay: the check jobs could move forward though | 17:55 |
| rlandy | https://review.rdoproject.org/r/#/c/16781/ | 17:56 |
| weshay | rlandy, that's just an addition right | 17:57 |
| weshay | so we'll see legacy + the new job | 17:57 |
| weshay | together atm | 17:57 |
| rlandy | weshay: yep for the moment | 17:57 |
| weshay | k cool | 17:57 |
| weshay | I only have +1 | 17:57 |
| weshay | there | 17:57 |
| rlandy | if the current run passes, we can switch the legacy out - your choice | 17:57 |
| rlandy | still in test rim | 17:58 |
| rlandy | run | 17:58 |
| weshay | if they work sure.. move forward.org | 17:58 |
| weshay | :) | 17:58 |
| rlandy | lol - quite the movement for the moment | 17:58 |
| *** ykarel|away has joined #oooq | 17:59 | |
| *** ykarel_ has joined #oooq | 18:04 | |
| *** holser_ has joined #oooq | 18:05 | |
| *** holser_ has quit IRC | 18:05 | |
| *** holser_ has joined #oooq | 18:06 | |
| *** ykarel|away has quit IRC | 18:07 | |
| weshay | rfolco, any thoughts on how to resolve https://bugs.launchpad.net/tripleo/+bug/1795718 | 18:12 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 18:12 |
| panda | spank the server | 18:14 |
| rfolco | hahaha | 18:14 |
| panda | bad boy, bad. | 18:14 |
| rfolco | take videogame from him | 18:15 |
| weshay | rfolco, it's in our zuul config | 18:16 |
| weshay | panda, you added it :) | 18:16 |
| rfolco | weshay, did we test lower ttl ? | 18:16 |
| weshay | https://github.com/openstack-infra/tripleo-ci/blame/master/zuul.d/base.yaml#L15 | 18:18 |
| * rfolco looks to panda | 18:18 | |
| weshay | http://codesearch.openstack.org/?q=unbound_cache_min_ttl&i=nope&files=&repos= | 18:18 |
| weshay | maybe remove it? | 18:18 |
| weshay | panda, did paul or david mention anything about that setting? | 18:19 |
| weshay | I do like the jokes btw :) | 18:19 |
| weshay | I laughed | 18:19 |
| rfolco | weshay, I can test it while we don't find an explanation | 18:20 |
| rfolco | weshay, like overriding ttl (removing it from the job) | 18:20 |
| weshay | rfolco, k.. thanks.. just curious how one would test that? | 18:20 |
| weshay | just in ci? | 18:20 |
| rfolco | weshay, yes, patch to tripleo-ci, remove that line, and use default (0) | 18:20 |
| rfolco | weshay, also can hide other jobs to speed up tetsing | 18:21 |
| weshay | rfolco, fine thanks.. panda please let me know if it was added based on some advice or something | 18:21 |
| rfolco | wip | 18:21 |
| rfolco | weshay, https://review.openstack.org/#/c/608051 meanwhile can look at the bug but I don't feel very familiar with whats going on there | 18:28 |
| panda | rfolco: the build starts but we cannot access the rdo registry, so it's not going too far | 18:29 |
| panda | rlandy: ^ | 18:29 |
| panda | weshay: rfolco I added that variable only because I copied from somewhere else. | 18:29 |
| weshay | panda, where? | 18:29 |
| panda | mindless ccopying | 18:29 |
| weshay | I don't see it in the rest of openstack | 18:29 |
| weshay | ack k | 18:29 |
| panda | legacy job of something similar | 18:30 |
| panda | or | 18:30 |
| weshay | legacy had zuul_vars? | 18:30 |
| weshay | panda, are you pto tomorrow? | 18:31 |
| weshay | ssbarnea_, what is openstack coats? | 18:35 |
| weshay | https://wiki.openstack.org/wiki/Jacket | 18:35 |
| weshay | ? | 18:35 |
| weshay | deltacloud much ^ rlandy lolz | 18:35 |
| rlandy | oh my gosh ... flashback | 18:36 |
| rlandy | well new jobs fail the same way the old jobs did | 18:40 |
| rlandy | not sure how happy we should be about that | 18:40 |
| ssbarnea_ | weshay: coats = some (greasemonkey) scripts to improve the browsing experience, wait few days to get mpeterson code merged into it and I will send an email. | 18:48 |
| weshay | ah neat | 18:48 |
| weshay | totally diff than jackets :) | 18:48 |
| weshay | lolz | 18:48 |
| ssbarnea_ | weshay: has nothing to do with jacket, didn't know about jacket when I picked that name. | 18:49 |
| ssbarnea_ | initially i was considering opentack-monkeys but infra guys told me that some people may find the term controversial | 18:50 |
| ssbarnea_ | anyway, is not something we will release, just a place to keep internal goodies that makes our day to day life easier. | 18:50 |
| weshay | heh | 18:50 |
| panda | weshay: no pto tomorrow, leaving a bit early to fly | 18:51 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci-centos-7-scenario007 (3 more messages) | 18:51 |
| panda | rlandy: ? | 18:51 |
| rlandy | panda: ? | 18:52 |
| panda | rlandy: the new jobs failing like the old ? | 18:53 |
| rlandy | fs001 master v3 and legacy both fail overcloud deploy with 'no valid host' error | 18:53 |
| panda | that's great ? | 18:54 |
| rlandy | panda: well in a weird way, yes | 18:54 |
| panda | I agree | 18:54 |
| panda | rlandy: ok , so the container build runs, fails on the credentials | 18:54 |
| rlandy | I just reset the test job with another change | 18:54 |
| *** agopi|long_lunch is now known as agopi | 18:54 | |
| panda | I think we can migrate at this point. We need the real deal | 18:54 |
| rlandy | panda: legit fails on creds? | 18:54 |
| panda | rlandy: yes, it doesn't have credentials, so can't access the rdo registry to check the images | 18:55 |
| panda | rlandy: https://logs.rdoproject.org/68/16768/29/check/periodic-tripleo-centos-7-master-containers-build/20c9b6d/job-output.txt.gz#_2018-10-04_18_18_12_684392 | 18:55 |
| rlandy | panda: ok - if you merge your job, I can merge https://review.rdoproject.org/r/#/c/16793/ (no promote) and https://review.rdoproject.org/r/#/c/16783/ | 18:56 |
| rlandy | panda: weshay: can we kill the current promotion | 18:57 |
| rlandy | fs016 still queued | 18:57 |
| weshay | rlandy, which? | 18:57 |
| weshay | rlandy, oh.. I already asked | 18:57 |
| rlandy | 134 hours | 18:57 |
| weshay | in sf-dfg | 18:57 |
| rlandy | that will kill our testing | 18:57 |
| weshay | actually have asked a few poeple | 18:57 |
| weshay | jebus | 18:58 |
| weshay | rlandy, btw.. rocky bm works :) https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/all%20top%20level%20multijobs/job/rdo-promote-rocky-rdo_trunk/13/ | 18:59 |
| rlandy | 135hr - wow | 18:59 |
| rlandy | weshay: woohoo - how nice | 18:59 |
| * rlandy hasn't looked at downstream in ages | 19:00 | |
| weshay | rlandy, no worries | 19:00 |
| weshay | ignore it ;) | 19:00 |
| weshay | I'm not worried about it at all.. just wanted to let you know it was working | 19:00 |
| rlandy | worry is part of genetic make up | 19:00 |
| panda | rlandy: modyfying https://review.rdoproject.org/api/16761 to include the containers build part, we can merge that, then we can start the switch | 19:01 |
| rlandy | panda++ | 19:01 |
| hubbot1 | rlandy: panda's karma is now 3 | 19:01 |
| rlandy | panda: we should change the periodic trigger to start the test run as soon as we are ready ... | 19:07 |
| panda | rlandy: yes | 19:08 |
| rlandy | panda: ok- adding a separate review so we can revert it easily | 19:09 |
| panda | rlandy: I guess we'll change it a lot more in the next few days | 19:09 |
| rlandy | if they don;t kill the current promote, that will harm us | 19:10 |
| *** ykarel__ has joined #oooq | 19:13 | |
| *** ykarel__ has quit IRC | 19:14 | |
| *** ykarel__ has joined #oooq | 19:14 | |
| rlandy | waiting on when we are ready to switch before modifying time | 19:15 |
| *** ykarel_ has quit IRC | 19:15 | |
| *** ykarel_ has joined #oooq | 19:17 | |
| *** ykarel__ has quit IRC | 19:20 | |
| *** ykarel__ has joined #oooq | 19:21 | |
| *** ykarel_ has quit IRC | 19:23 | |
| panda | rlandy: ok, first shot https://review.rdoproject.org/api/16761 | 19:24 |
| * rlandy checks | 19:24 | |
| *** ykarel__ has quit IRC | 19:26 | |
| rlandy | panda: ok - here we go | 19:27 |
| rlandy | let's merge | 19:27 |
| rlandy | can't do damage in its own | 19:27 |
| panda | ka-boom | 19:28 |
| rlandy | panda: pls review https://review.rdoproject.org/r/#/c/16793/ so we don;t damage anything yet | 19:28 |
| rlandy | done and done | 19:31 |
| panda | rlandy: commented | 19:32 |
| rlandy | yeah - good point but idk what that would do to phase1 promotions | 19:33 |
| rlandy | panda: ok - merging https://review.rdoproject.org/r/#/c/16793/ | 19:34 |
| panda | there's no phase1 if there's not tripleo promotion <add black man touching his head with the finger meme> | 19:34 |
| rlandy | well, I didn;t check if phase 1 promoted yet off the last tripleo promotion | 19:35 |
| rlandy | hmmm - zuul ci? gate | 19:36 |
| rlandy | ah running now | 19:36 |
| panda | rlandy: is this done ? https://tree.taiga.io/project/tripleo-ci-board/task/108?kanban-status=1447275 | 19:37 |
| rlandy | panda: we have not merged the review yet ... https://review.rdoproject.org/r/#/c/16781/4/zuul.d/tripleo.yaml - so no | 19:38 |
| rlandy | but this will be soon - can reassign to me and unblock ... | 19:38 |
| rlandy | oh it is unblocked | 19:39 |
| rlandy | https://tree.taiga.io/project/tripleo-ci-board/task/105?kanban-status=1447275 | 19:39 |
| panda | I unblocked it | 19:41 |
| rlandy | panda: ^^ what's the right process- do I assign that card to me or leave it with quique? | 19:41 |
| panda | It was blocked only by the notice | 19:41 |
| panda | rlandy: at this point we're collaborating so closely it really doesn't matter | 19:41 |
| rlandy | ok | 19:42 |
| rlandy | now for the real damage ... | 19:42 |
| rlandy | https://review.rdoproject.org/r/#/c/16783/1/zuul.d/tripleo.yaml | 19:42 |
| rlandy | please vote | 19:42 |
| rlandy | so we have some legitimacy | 19:42 |
| panda | can I vote -2 just because I'm a coward ? | 19:42 |
| rlandy | you could but we would be here forever | 19:43 |
| panda | ugh, I can't vote -2 | 19:43 |
| rlandy | at some point I would go and retire in the south of spain and it would be all yours | 19:43 |
| * panda wonder where quiquell|off will run then | 19:44 | |
| panda | he's already in the south of spain (more or less) | 19:44 |
| rlandy | lol | 19:44 |
| rlandy | he can come to ny | 19:44 |
| rlandy | we'll switch places | 19:44 |
| rlandy | how do you get any work done if you live in such a nice, sunny place??? | 19:45 |
| rlandy | panda: it's not going to get any easier :( | 19:46 |
| panda | no spain no gain. | 19:46 |
| rlandy | lol | 19:46 |
| panda | rlandy: did I mention I'll be in Brno all next week ? | 19:46 |
| rlandy | that was actually very funny | 19:46 |
| rlandy | panda: I deserve that - after I left for two days every week | 19:47 |
| rlandy | ugh - I had to rebase it anyways | 19:47 |
| rlandy | whatever | 19:48 |
| rlandy | panda; ok - now the question is which pipeline do you want to trigger and at what time? | 19:49 |
| rlandy | heaven help us ... here we go | 19:50 |
| panda | rlandy: frankly, I'll be in and out for the next two hours then I'll disappear, better that you decide your faith | 19:51 |
| rlandy | panda: are you in at all tomorrow? | 19:51 |
| panda | rlandy: yes, but I'll leave earlier to catch my flight for Brno | 19:52 |
| panda | rlandy: at around 15 UTC | 19:52 |
| panda | I'l fight all morning with quique | 19:52 |
| panda | and sagi | 19:52 |
| rlandy | alrighty then | 19:53 |
| panda | oh such bravery! | 19:53 |
| rlandy | we've done it now | 19:53 |
| * panda hears a strange sound of gears grinding | 19:54 | |
| rlandy | timer: | 19:55 |
| rlandy | - time: '45 4,9,14,22 * * *' | 19:55 |
| rlandy | time now is ... | 19:55 |
| rlandy | 19:55 | 19:55 |
| * rlandy wonders if it's worth changing the time | 19:55 | |
| rlandy | changing 22 to 20 | 19:56 |
| rlandy | panda: before you disappear | 19:58 |
| rlandy | https://review.rdoproject.org/r/16797 | 19:58 |
| rlandy | correct that that will kick in 10 mins? | 19:59 |
| rlandy | hmmm ... making that 20 - we may need merge time | 19:59 |
| rlandy | weshay; ^^ ok? | 20:04 |
| rlandy | will revert as soon as job kicks | 20:04 |
| panda | rlandy: should be ok | 20:05 |
| panda | rlandy: why do you want to revert ? I think We'll need to change the trigger time a lot | 20:05 |
| rlandy | ok | 20:06 |
| panda | I would almost suggest to run it only manually at this point | 20:06 |
| rlandy | how do we kick that manually? | 20:06 |
| panda | well, manually I mean, put only a single time to trigger there | 20:06 |
| rlandy | that is a much better suggestion | 20:06 |
| panda | then move it wehn needed to trigger | 20:07 |
| panda | again | 20:07 |
| rlandy | panda: ack - and we should comment out the second pipeline | 20:07 |
| panda | agree, let's deal with master and rocky first | 20:08 |
| rlandy | panda: ok - putting in another review | 20:08 |
| rlandy | will merge after first job kicks | 20:09 |
| rlandy | hmmmm - how do you disable a pipeline? | 20:12 |
| rlandy | OMG - we have triggering | 20:21 |
| rlandy | panda: still there? | 20:24 |
| rlandy | well that was quick | 20:25 |
| panda | a quick failure ? | 20:26 |
| panda | usually the failure are mapped with retries in a fibonacci way : first retry fails in 1 minute, second in 1 minute, third in 3 minutes, and so on | 20:27 |
| rlandy | panda: yep - could not find file | 20:27 |
| rlandy | but I ma not sure where I get those logs | 20:27 |
| rlandy | I know where the promoter logs are but not the container buildlogs | 20:27 |
| rlandy | promote to tripleo-ci testing seemed to work | 20:28 |
| rlandy | also ... https://review.rdoproject.org/r/16798 Temp disable 24hr pipeline and trigger periodic pipeline once | 20:28 |
| panda | rlandy: hash_info.sh ? | 20:32 |
| rlandy | panda: ack | 20:32 |
| rlandy | how did you know? | 20:33 |
| rlandy | logs? | 20:33 |
| weshay | rlandy, ok.. the job was killed | 20:34 |
| rlandy | weshay: ack - container build failed | 20:34 |
| weshay | ah.. hrm.. | 20:34 |
| weshay | wonder if we could point it at a known good hash | 20:34 |
| rlandy | weshay: stupid question - but where are the logs? | 20:35 |
| weshay | for which hash was good? | 20:35 |
| weshay | or jobs? | 20:35 |
| rlandy | container build jobs | 20:35 |
| rlandy | job run, failed, no trace | 20:35 |
| panda | weshay: https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/ | 20:35 |
| weshay | rlandy, https://review.rdoproject.org/zuul/builds.html | 20:35 |
| panda | oh, uhm | 20:36 |
| weshay | pipeline = openstack-periodic | 20:36 |
| weshay | post-failure | 20:36 |
| rlandy | thank you | 20:36 |
| weshay | https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-rocky-containers-build/98a6239/ | 20:36 |
| weshay | mary | "msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'change'\n\nThe error appears to have been in '/tmp/tmpaklh4emf/98a623969dd44c69ae739bd06ccebd59/untrusted/project_3/git.openstack.org/openstack-infra/tripleo-ci/playbooks/tripleo-ci/run-v3.yaml': line 92, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to | 20:37 |
| weshay | be:\n\n\n - shell:\n ^ here\n" | 20:37 |
| weshay | 2018-10-04 20:24:50.539569 | primary | } | 20:37 |
| rlandy | well let's start here ... https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-promote-consistent-to-tripleo-ci-testing/49cf39a/job-output.txt.gz - really a success? | 20:37 |
| weshay | https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-rocky-containers-build/98a6239/job-output.txt.gz#_2018-10-04_20_24_50_539517 | 20:37 |
| *** holser_ has quit IRC | 20:39 | |
| *** jtomasek has quit IRC | 20:39 | |
| panda | mmhhh | 20:41 |
| panda | we'll need to change run-v3 | 20:42 |
| rlandy | on the upside ... | 20:42 |
| rlandy | tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens-branch success (non-voting) | 20:42 |
| weshay | panda, rlandy anything I can do to help? | 20:42 |
| *** tosky has quit IRC | 20:42 | |
| *** tosky has joined #oooq | 20:43 | |
| panda | yeee | 20:43 |
| panda | weshay: from now it's iteration and quick fixes | 20:43 |
| panda | so the first problem is that to form LOG_PATH we need zuul.chanes | 20:43 |
| panda | which is not defined in periodic pipeline | 20:43 |
| weshay | panda, ok.. the gate quueue is short | 20:44 |
| panda | I can't even find where we are using this LOG_PATH | 20:44 |
| weshay | anything need review? | 20:44 |
| *** hamzy_ has quit IRC | 20:44 | |
| weshay | the LOG_PATH var is defined from zuul.changes wtf? | 20:44 |
| * weshay looks | 20:44 | |
| panda | it's used in tempest stuff, tempestmail to post results | 20:44 |
| weshay | that's what caused the post failure? | 20:45 |
| weshay | panda, we can turn that off in the periodic jobs | 20:45 |
| weshay | no need for tempest mail notifications from that | 20:45 |
| panda | ok | 20:45 |
| panda | let's add a conditional then | 20:45 |
| panda | we set LOG_PATH only if zuul.change is defined | 20:46 |
| weshay | http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/playbooks/tripleo-ci/run-v3.yaml#n108 | 20:47 |
| panda | weshay: yes | 20:48 |
| weshay | http://codesearch.openstack.org/?q=LOG_PATH&i=nope&files=&repos=tripleo-ci,tripleo-quickstart,tripleo-quickstart-extras | 20:48 |
| rlandy | side point ... https://review.rdoproject.org/r/#/c/16781/ - I think this can merge | 20:48 |
| weshay | odd | 20:48 |
| rlandy | picking up the change | 20:49 |
| rlandy | failures were the same as legacy | 20:49 |
| rlandy | in master | 20:49 |
| rlandy | queens passes | 20:49 |
| panda | https://review.openstack.org/608080 | 20:50 |
| panda | I love those change id | 20:50 |
| panda | that | 20:50 |
| rlandy | cool | 20:51 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci-centos-7-scenario007 (4 more messages) | 20:51 |
| rlandy | panda: we'll have the same problem in ... https://github.com/openstack-infra/zuul-jobs/blob/master/roles/set-zuul-log-path-fact/tasks/main.yaml#L4 | 20:53 |
| rlandy | in playbooks/tripleo-ci-periodic-base/containers-build.yaml | 20:54 |
| *** trown is now known as trown|outtypewww | 20:54 | |
| panda | rlandy: something tells me that role is prepared for problem like this | 20:55 |
| rlandy | we | 20:55 |
| rlandy | will find out | 20:55 |
| panda | rlandy: yep it's handled already | 20:56 |
| rlandy | nice | 20:56 |
| panda | rlandy: wait there's more | 20:56 |
| rlandy | oh what now? | 20:56 |
| rlandy | https://review.openstack.org/#/c/607963/ | 20:57 |
| rlandy | see resultes | 20:57 |
| rlandy | results | 20:57 |
| panda | rlandy: it's actually handled a lot better than we do | 20:57 |
| rlandy | copy, copy | 20:57 |
| weshay | ? | 20:57 |
| rlandy | when: zuul.change is defined | 20:57 |
| rlandy | weshay, https://github.com/openstack-infra/zuul-jobs/blob/master/roles/set-zuul-log-path-fact/tasks/main.yaml#L4 | 20:58 |
| rlandy | ^^ handle the problem | 20:58 |
| panda | yes | 20:58 |
| weshay | ah | 20:58 |
| panda | I'll add the role to run-v3 playbook | 20:58 |
| weshay | so that can change to {{ zuul_log_path }} | 20:58 |
| weshay | nice find rlandy | 20:58 |
| weshay | rlandy, so my last comment on https://review.openstack.org/#/c/608080/1 | 20:59 |
| weshay | is that correct? | 20:59 |
| weshay | https://review.openstack.org/#/c/604487 | 21:01 |
| rlandy | we would have to call the role to have zuul_log_path defined | 21:01 |
| panda | uploaded patch set 2 | 21:01 |
| weshay | right | 21:01 |
| weshay | patch needs a depends on | 21:02 |
| weshay | or patch set 2 | 21:02 |
| weshay | cool | 21:02 |
| rlandy | looks good to me ... | 21:02 |
| rlandy | so ... to trigger again | 21:02 |
| rlandy | https://review.rdoproject.org/r/#/c/16798/1/zuul.d/upstream.yaml | 21:03 |
| rlandy | to retrigger | 21:03 |
| rlandy | change the time | 21:03 |
| rlandy | weshay: panda: ok - since we are living dangerously, - can I merge https://review.rdoproject.org/r/#/c/16781/ | 21:05 |
| weshay | +1 from me | 21:05 |
| rlandy | here we go | 21:05 |
| weshay | rlandy, when you have a minute https://review.openstack.org/#/c/604487/ | 21:06 |
| rlandy | will look in a bit | 21:07 |
| rlandy | could we add a depends-on to this review https://review.rdoproject.org/r/#/c/16798/ and trigger? | 21:08 |
| rlandy | that wouldn't work | 21:09 |
| panda | nope | 21:09 |
| rlandy | yeah | 21:10 |
| panda | to make it work we would need all the jobs to clone the change before running | 21:10 |
| rlandy | panda: unless we made a temp run-v3 playbook | 21:12 |
| *** agopi has quit IRC | 21:12 | |
| rlandy | and merged that | 21:12 |
| rlandy | in rdo-jobs | 21:12 |
| panda | rlandy: but you have to change all the base jobs to run it | 21:18 |
| rlandy | at least we can merge there | 21:19 |
| rlandy | depends how long it will take us to get this change to merge | 21:19 |
| rlandy | getting through gates etc. | 21:19 |
| panda | it makes sense in the short run | 21:20 |
| panda | it may not be the only change we have to make to it ... | 21:20 |
| rlandy | really only ovb base, multinode base, singlenode base | 21:20 |
| rlandy | correct | 21:20 |
| rlandy | panda could even be a depends on change to the trigger | 21:20 |
| rlandy | forget that | 21:21 |
| * rlandy keeps trying to limit the damage | 21:21 | |
| rlandy | but we're all in now | 21:21 |
| *** toure is now known as toure|gone | 21:22 | |
| rlandy | panda: are you making that change or should I? | 21:28 |
| panda | rlandy: I'm about to call it a day | 21:32 |
| rlandy | panda: sure - it's late for you | 21:32 |
| rlandy | I'll look into the change | 21:32 |
| rlandy | even just to have it ready for you | 21:33 |
| rlandy | maybe your review will go through | 21:33 |
| rlandy | I'll watch it | 21:33 |
| panda | rlandy: thanks. See you tomorrow. | 21:36 |
| *** panda is now known as panda|off | 21:36 | |
| *** Vorrtex has quit IRC | 21:49 | |
| *** weshay is now known as weshay_pto | 21:51 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario008-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario004-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master, tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci-centos-7-scenario007-multinode- (4 more messages) | 22:51 |
| *** tosky has quit IRC | 23:17 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!