| *** agopi|out has joined #oooq | 02:22 | |
| *** agopi|out is now known as agopi | 02:24 | |
| *** matbu has quit IRC | 02:37 | |
| *** rlandy|bbl is now known as rlandy | 02:41 | |
| *** rlandy has quit IRC | 02:42 | |
| *** gkadam_ has joined #oooq | 03:24 | |
| *** ssbarnea has joined #oooq | 03:28 | |
| *** udesale has joined #oooq | 03:47 | |
| *** ssbarnea has quit IRC | 04:12 | |
| *** ssbarnea has joined #oooq | 04:21 | |
| *** skramaja has joined #oooq | 04:22 | |
| *** gkadam_ has quit IRC | 04:30 | |
| *** apetrich has joined #oooq | 05:05 | |
| *** ratailor has joined #oooq | 05:16 | |
| *** saneax has joined #oooq | 06:06 | |
| *** jfrancoa has joined #oooq | 06:22 | |
| *** bogdando has joined #oooq | 07:09 | |
| *** amoralej|off is now known as amoralej | 07:11 | |
| *** ykarel has joined #oooq | 07:31 | |
| ykarel | is this issue known:- resources.ServiceChain: Property error: resources[68].properties: Property DockerKeepalivedConfigImage not assigned | 07:33 |
|---|---|---|
| ykarel | seeing here:- https://logs.rdoproject.org/82/591982/2/openstack-experimental/legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/2bf5069/logs/undercloud/home/zuul/failed_deployment_list.log.txt.gz | 07:34 |
| marios | https://bugs.launchpad.net/tripleo/+bug/1787268 ykarel maybe this | 07:37 |
| openstack | Launchpad bug 1787268 in tripleo "Property DockerCinderConfigImage not assigned" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 07:37 |
| ykarel | marios, ok so same fix for both cinder and keepalived ? | 07:42 |
| marios | ykarel: yeah i suspect so because the fix in https://review.openstack.org/#/c/592275/ is about all the container image params | 07:44 |
| ykarel | marios, yes , Thanks | 07:45 |
| *** ykarel is now known as ykarel|lunch | 07:46 | |
| marios | welcome ykarel|lunch | 07:46 |
| *** jaosorior has quit IRC | 07:47 | |
| jfrancoa | amoralej: ykarel|lunch: marios|rover: hey, could I get any reviews for https://review.rdoproject.org/r/#/c/15557/ ,please? it's supposed to fix all the container-to-container legacy upgrades jobs | 07:55 |
| marios | ack jfrancoa | 08:00 |
| jfrancoa | marios: thanks | 08:00 |
| *** ykarel|lunch is now known as ykarel | 08:56 | |
| *** panda|off is now known as panda | 09:08 | |
| *** udesale has quit IRC | 09:31 | |
| bogdando | o/ folks, I forgot why we don't want that https://review.openstack.org/#/c/576746 ? | 09:50 |
| bogdando | still hitting a lot of issues with deploying on my box w/o ssh access to itself | 09:50 |
| *** udesale has joined #oooq | 10:31 | |
| bogdando | ^^ PTAL updated. It makes oooq better supporting non-disposable devboxes | 10:43 |
| *** udesale has quit IRC | 11:26 | |
| weshay|ruck | oh man.. where is rlandy | 12:21 |
| weshay|ruck | https://bugzilla.redhat.com/show_bug.cgi?id=1616347 | 12:21 |
| openstack | bugzilla.redhat.com bug 1616347 in openstack-glance "Undercloud deployment gets stuck after starting glance_api_db_sync" [Unspecified,Closed: notabug] - Assigned to cyril | 12:21 |
| *** ratailor has quit IRC | 12:23 | |
| weshay|ruck | bogdando, I'm not against it | 12:27 |
| bogdando | weshay|ruck: thanks, that's already something! :) | 12:35 |
| *** trown|outtypewww is now known as trown | 12:35 | |
| *** rlandy has joined #oooq | 12:39 | |
| rlandy | weshay|ruck: hi - ticket says ci-rhos should be back - trying again | 12:41 |
| weshay|ruck | rlandy, I kicked a job | 12:44 |
| rlandy | yep - I see | 12:44 |
| weshay|ruck | https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/tq-gate-rhos-13-ci-rhos-ovb-featureset001/135/console | 12:44 |
| rlandy | I removed stack delete from that job | 12:45 |
| rlandy | so we could debug | 12:45 |
| rlandy | so we need to manually clean up | 12:45 |
| rlandy | just fyi | 12:45 |
| *** amoralej is now known as amoralej|lunch | 12:46 | |
| *** skramaja has quit IRC | 12:53 | |
| rlandy | rfolco: hi | 13:08 |
| rlandy | rfolco: need to chat with you re: vars in zuul jobs | 13:09 |
| rfolco | rlandy, bj ? | 13:10 |
| rlandy | rfolco: yep - 5 mins - just checking something - will ping you | 13:11 |
| rfolco | rlandy, ok, its will be good to discuss different approaches to solve the problem | 13:12 |
| rfolco | rlandy, feel free to chat here if you prefer | 13:14 |
| rlandy | rfolco: k - let's bj | 13:16 |
| rfolco | grabbing coffee | 13:17 |
| rlandy | rfolco: https://bluejeans.com/u/rlandy/ - when you are ready | 13:17 |
| apetrich | rlandy, I'm working on https://bugs.launchpad.net/tripleo/+bug/1736950 but I'm unable to make it fail on my env. Do you have a reproducer script that always fails (or fails a lot at least>) | 13:23 |
| openstack | Launchpad bug 1736950 in tripleo "CI: mistral testmistral_tempest_tests.tests.api.v2.test_actions.ActionTestsV2.test_get_list_actions_not_in_list_filter fails in gate scenario003 containers" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 13:23 |
| rlandy | apetrich: sec in meeting | 13:30 |
| apetrich | no worries | 13:30 |
| *** amoralej|lunch is now known as amoralej | 13:43 | |
| *** jfrancoa has quit IRC | 13:46 | |
| rlandy | apetrich: you tried this reproducer on your personal tenant? http://logs.openstack.org/16/592216/2/check/tripleo-ci-centos-7-scenario003-multinode-oooq-container/fc62c9d/logs/reproducer-quickstart.sh | 13:50 |
| *** jaganathan has quit IRC | 13:52 | |
| apetrich | rlandy, That's the one linked in the bug. I runned yesterday but it didn't finish for me. | 13:53 |
| apetrich | let me run again to see what was wrong with it | 13:54 |
| *** jfrancoa has joined #oooq | 13:58 | |
| rfolco | rlandy, hmm actually ovb legacy has its own hierarchy so if we remove vars from environment_type/ovb.yaml, we have to add to rdo-jobs zuul config | 13:59 |
| rfolco | ovb jobs parent to --> legacy-tripleo-ci-periodic-multinode --> legacy-tripleo-ci-periodic-base | 13:59 |
| rlandy | apetrich: would be good to know why it did not finish | 13:59 |
| rfolco | rlandy, https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/jobs.yaml#L97 | 13:59 |
| apetrich | it was late, so probably my fault | 13:59 |
| rlandy | rfolco: those jobs never parented off tripleo-ci-base | 13:59 |
| rlandy | I am leaving legacy as they are | 14:00 |
| rlandy | only concerned about the one job fe053 we are reparenting to tripleo-ci-base-ovb | 14:00 |
| rfolco | rlandy, ok it depends on how you are changing the env_type files to load those vars | 14:00 |
| rlandy | rfolco: yeah - the old type never touched those vars | 14:01 |
| rlandy | rfolco: what about pre-run? https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/base.yaml#L32 | 14:11 |
| rlandy | ^^ only needed for multinode | 14:11 |
| rlandy | worth moving? | 14:11 |
| rlandy | not sure about ordering then | 14:11 |
| rfolco | rlandy, I would keep this for now as I am quite sure if this needs to run before the other pre-runs | 14:12 |
| rfolco | rlandy, worth adding a NOTE :) | 14:12 |
| rlandy | rfolco: k - will do - we can discuss | 14:12 |
| rlandy | rfolco: is this correct - https://github.com/openstack-infra/tripleo-ci/blob/master/playbooks/tripleo-ci/vars/environment_type/singlenode.yaml#L3 | 14:19 |
| rlandy | should playbooks be in []? | 14:19 |
| rlandy | like tags | 14:19 |
| rfolco | yep I got the same question | 14:20 |
| rfolco | rlandy, let me do a quick test | 14:21 |
| rfolco | rlandy, it works | 14:23 |
| rfolco | although not consistent :) | 14:23 |
| rfolco | rlandy, https://review.rdoproject.org/paste/show/133/ | 14:25 |
| rlandy | rfolco: https://review.openstack.org/#/c/593063 - note commit message - incomplete item to do | 14:27 |
| rlandy | panda: ^^ | 14:27 |
| rlandy | just want a quick review on this and then will work on incomplete items | 14:28 |
| weshay|ruck | 2018-08-17 10:12:21 | 2018-08-17 10:12:21,378 INFO: Notice: /Stage[main]/Glance::Db::Metadefs/Exec[glance-manage db_load_metadefs]: Triggered 'refresh' from 3 events │·············2018-08-17 10:27:21 | 2018-08-17 10:27:21,388 INFO: Error: Command exceeded timeout │······ | 14:28 |
| weshay|ruck | ·······2018-08-17 10:27:21 | 2018-08-17 10:27:21,390 INFO: Error: /Stage[main]/Nova::Db::Sync/Exec[nova-db-sync]/returns: change from notrun to 0 failed: Command exceeded timeout | 14:28 |
| weshay|ruck | rlandy, ^ | 14:28 |
| rlandy | weshay|ruck: https://bugs.launchpad.net/tripleo/+bug/1661396 | 14:29 |
| openstack | Launchpad bug 1661396 in OpenStack Compute (nova) "undercloud install fails (nova-db-sync timeout) on VM on an SATA disk hypervisor" [Undecided,Incomplete] | 14:29 |
| rlandy | ]was looking at that | 14:29 |
| rlandy | osp 13 does include the 900 sec change | 14:29 |
| rlandy | but maybe we should increase even that | 14:29 |
| rlandy | weshay|ruck, https://bugzilla.redhat.com/show_bug.cgi?id=1418782#c7 | 14:32 |
| openstack | bugzilla.redhat.com bug 1418782 in puppet-tripleo "rhosp-director: Undercloud deployment fails with: Error: /Stage[main]/Nova::Db::Sync/Exec[nova-db-sync]/returns: change from notrun to 0 failed: Command exceeded timeout" [High,Closed: currentrelease] - Assigned to athomas | 14:32 |
| rlandy | could ping alex on his opinion | 14:32 |
| rlandy | weshay|ruck: also ... | 14:34 |
| rlandy | <jschlueter> puddle announcement for OSP 13 z2 just sent out ... finally passed phase 2 ci for 2018-08-16.1 | 14:34 |
| rlandy | maybe we should switch to z2 | 14:34 |
| weshay|ruck | rlandy, ok | 14:46 |
| weshay|ruck | rlandy, one more shot on z2 ovb | 14:46 |
| rlandy | weshay|ruck: I think the nova dbsync timeout may still exist | 14:46 |
| rlandy | weshay|ruck: what do you think about increasing that? | 14:47 |
| * rlandy doesn;t know how to commit a change to osp13 to test | 14:47 | |
| rlandy | queens branch? | 14:47 |
| rlandy | will that take? | 14:47 |
| rlandy | hmmm ... why didn't we get offered a RDO rubber ducky? https://twitter.com/RDOcommunity/status/1030120768770854913 | 14:50 |
| weshay|ruck | rlandy, you cool w/ https://review.openstack.org/#/c/587880/ | 14:52 |
| rlandy | weshay|ruck: yeah - should not be needed but no harm | 14:53 |
| rlandy | weshay|ruck: pls see question above (re: nova-db-sync timeout) | 14:54 |
| weshay|ruck | rlandy, is that something we can pass as a param? | 14:55 |
| weshay|ruck | the timeout? | 14:56 |
| rlandy | weshay|ruck: I don;t think so - can ask on #tripleo | 14:56 |
| rlandy | hence question about how to add a change osp will take | 14:56 |
| weshay|ruck | rlandy, maybe in hiera | 14:56 |
| rlandy | weshay|ruck: I think we will see the same problem on z2 - I could be wrong though | 14:57 |
| rlandy | but I'm guessing the hw is not good enough | 14:57 |
| weshay|ruck | https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/undercloud-deploy/templates/hieradata-overrides-t-h-t-undercloud.yaml.j2 | 14:57 |
| ykarel | weshay|ruck, need to push https://review.openstack.org/#/c/592365/ as well to clear rocky promotions, if it's already being tracked somewhere then it's ok | 14:58 |
| rlandy | weshay|ruck: I can try add it to that template | 14:59 |
| rlandy | let's see | 15:00 |
| * rlandy edits job | 15:01 | |
| ykarel | weshay|ruck, current failure are real one, the job is looking for centos7-rocky/current-tripleo, so recheck will not work | 15:01 |
| ykarel | it seems a chicken egg problem | 15:01 |
| weshay|ruck | ykarel, w/ https://review.openstack.org/#/c/592365/ ? | 15:02 |
| ykarel | weshay|ruck, yes | 15:02 |
| ykarel | weshay|ruck, see http://logs.openstack.org/65/592365/2/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/c8d2d56/job-output.txt.gz#_2018-08-17_11_13_05_893272 | 15:02 |
| ykarel | so it looks for stable_release -1, that is rocky after that patch. and since there is no promotion centos7-rocky/current-tripleo do not exist | 15:03 |
| weshay|ruck | ykarel, ya.. it should not be using rocky | 15:03 |
| weshay|ruck | k | 15:03 |
| weshay|ruck | ykarel, we'll fix it | 15:04 |
| ykarel | weshay|ruck, ack Thanks | 15:04 |
| *** ykarel is now known as ykarel|away | 15:04 | |
| rlandy | weshay|ruck: I am deleting that stack | 15:08 |
| *** ykarel|away has quit IRC | 15:12 | |
| *** jfrancoa has quit IRC | 15:16 | |
| rlandy | weshay|ruck: https://code.engineering.redhat.com/gerrit/#/c/147394/ | 15:25 |
| weshay|ruck | rlandy, merged | 15:34 |
| rlandy | weshay|ruck: thank - here we go again ... https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/tq-gate-rhos-13-ci-rhos-ovb-featureset001/136/console | 15:36 |
| rlandy | weshay|ruck: https://review.openstack.org/#/c/593077/1/roles/undercloud-deploy/templates/hieradata-overrides-t-h-t-undercloud.yaml.j2 | 15:36 |
| rlandy | ^^ tried that | 15:36 |
| weshay|ruck | rfolco, http://codesearch.openstack.org/ | 15:43 |
| *** saneax has quit IRC | 15:45 | |
| weshay|ruck | https://github.com/openstack/tripleo-heat-templates/tree/master/zuul.d | 16:07 |
| weshay|ruck | rfolco, ^ | 16:07 |
| weshay|ruck | rfolco, https://github.com/openstack/tripleo-heat-templates/blob/master/zuul.d/layout.yaml | 16:07 |
| weshay|ruck | https://github.com/openstack/tripleo-heat-templates/blob/stable/pike/zuul.d/layout.yaml | 16:08 |
| weshay|ruck | rfolco, ^ | 16:08 |
| *** ykarel has joined #oooq | 16:21 | |
| *** ykarel has quit IRC | 16:32 | |
| *** amoralej is now known as amoralej|off | 16:44 | |
| weshay|ruck | rfolco, https://docs.openstack.org/tripleo-docs/latest/contributor/index.html | 16:46 |
| *** bogdando has quit IRC | 16:52 | |
| *** bogdando has joined #oooq | 16:53 | |
| rlandy | weshay|ruck: hmm ... did I make the change in the right file? https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/undercloud-deploy/templates/hieradata-overrides-t-h-t-undercloud.yaml.j2 vs https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/undercloud-deploy/templates/hieradata-overrides-classic-undercloud.yaml.j2? | 16:55 |
| rlandy | only the second file is included in /home/stack | 16:55 |
| weshay|ruck | rlandy, ha.. tbh.. I would have to check and try | 17:05 |
| weshay|ruck | that is VERY confusing | 17:05 |
| weshay|ruck | it was originally named undercloud-blah.. but was hierra config for the overcloud | 17:05 |
| weshay|ruck | so when the ability came and we could do the same on the undercloud | 17:06 |
| weshay|ruck | what do you name it???? | 17:06 |
| *** myoung is now known as myoung|bbl | 17:08 | |
| apetrich | rlandy, I see why it failed to error that bug. the skip list was merged by the time it ran | 17:11 |
| rlandy | weshay|ruck: I think I changed the wrong file but if it fails will run again with the other file changed | 17:23 |
| *** trown is now known as trown|lunch | 17:24 | |
| rlandy | apetrich: cool | 17:29 |
| rlandy | panda: thanks for the review - updated - now will add the missing pieces | 17:30 |
| *** bogdando has quit IRC | 17:48 | |
| rlandy | weshay|ruck: hmmm ... for better or worse we have a different error this time ... https://paste.fedoraproject.org/paste/dnVoNTGlVuMV6JX36uYddg | 18:08 |
| rlandy | opening undercloud on the old tmate | 18:09 |
| rlandy | return glareclient.Client | 18:13 |
| rlandy | 2018-08-17 14:02:06.440 31775 ERROR mistral.actions.openstack.action_generator.base return glareclient.Client | 18:17 |
| rlandy | 2018-08-17 14:02:06.440 31775 ERROR mistral.actions.openstack.action_generator.base AttributeError: 'NoneType' object has no attribute 'Client' | 18:17 |
| rlandy | rfolco: I think we are doing duplicate work | 18:29 |
| rlandy | https://review.openstack.org/#/c/593129/ | 18:29 |
| rlandy | vs | 18:29 |
| rlandy | https://review.openstack.org/#/c/593063/ | 18:29 |
| rfolco | rlandy, I know. This is about getting up to speed more than what patch will go further | 18:29 |
| rlandy | rfolco:ok then | 18:30 |
| rfolco | rlandy, I want to study possible solutions for the problem | 18:30 |
| rlandy | rfolco: np | 18:30 |
| rfolco | rlandy, I tried to add as much as i could to the patch so I can work on any task this sprint | 18:31 |
| rfolco | and be a good qe for others as i did my own poc :) | 18:32 |
| rlandy | rfolco: that's fine | 18:40 |
| rfolco | rlandy, I guess nobody is working on reproducer side of it. How we load these vars... | 18:40 |
| rfolco | woring --> poc | 18:40 |
| rfolco | working | 18:40 |
| rlandy | rfolco: when we assign tasks, I can do that | 18:41 |
| rfolco | hmm perhaps you have this in your mind already | 18:41 |
| rfolco | I have no clues | 18:41 |
| rlandy | need to step out for a but dr appointment. van look at it when I return | 18:41 |
| rlandy | can | 18:41 |
| rfolco | rlandy, cool, thank you | 18:42 |
| *** rlandy is now known as rlandy|afk | 18:48 | |
| *** trown|lunch is now known as trown | 18:55 | |
| *** apetrich has quit IRC | 19:26 | |
| *** apetrich has joined #oooq | 20:33 | |
| *** rlandy|afk is now known as rlandy | 20:36 | |
| *** trown is now known as trown|outtypewww | 21:00 | |
| *** apetrich has quit IRC | 21:10 | |
| rlandy | weshay|ruck: around? | 21:45 |
| rlandy | ha - not bad ... https://review.openstack.org/#/c/593063/ | 21:49 |
| rlandy | weshay|ruck: 2018-08-17 19:16:00 | Undercloud install complete. | 23:18 |
| rlandy | party ... we go through undercloud install ... https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/tq-gate-rhos-13-ci-rhos-ovb-featureset001/137/console | 23:19 |
| *** rlandy has quit IRC | 23:27 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!