| *** ecerquei has joined #tripleo | 00:09 | |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
|---|---|---|
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 00:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] | 00:10 |
| *** axisys_ has quit IRC | 00:13 | |
| *** axisys has joined #tripleo | 00:14 | |
| *** itlinux has quit IRC | 00:15 | |
| *** tosky has quit IRC | 00:27 | |
| *** rhallisey has quit IRC | 00:33 | |
| *** rhallisey has joined #tripleo | 00:39 | |
| *** bcafarel has quit IRC | 00:44 | |
| *** bcafarel has joined #tripleo | 00:51 | |
| openstackgerrit | Dan Sneddon proposed openstack/tripleo-heat-templates master: Fixes heat resource name for Internal API Network https://review.openstack.org/506345 | 00:51 |
| *** gkadam has joined #tripleo | 00:59 | |
| *** ecerquei has quit IRC | 01:04 | |
| *** ecerquei has joined #tripleo | 01:04 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 01:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] | 01:10 |
| *** gvrangan has joined #tripleo | 01:22 | |
| *** bnemec has quit IRC | 01:34 | |
| *** bnemec has joined #tripleo | 01:39 | |
| *** gvrangan has quit IRC | 01:45 | |
| *** gvrangan has joined #tripleo | 01:45 | |
| *** bcafarel has quit IRC | 01:51 | |
| *** itlinux has joined #tripleo | 01:51 | |
| *** achadha has joined #tripleo | 01:56 | |
| *** gvrangan has quit IRC | 01:56 | |
| *** gvrangan has joined #tripleo | 01:56 | |
| *** bcafarel has joined #tripleo | 01:58 | |
| *** achadha has quit IRC | 02:00 | |
| *** itlinux has quit IRC | 02:07 | |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 02:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] | 02:10 |
| *** tzumainn has quit IRC | 02:10 | |
| *** lblanchard has quit IRC | 02:10 | |
| *** cdearborn has quit IRC | 02:11 | |
| *** itlinux has joined #tripleo | 02:14 | |
| *** itlinux has quit IRC | 02:15 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements https://review.openstack.org/503679 | 02:16 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Use "become: true" in deploy steps tasks https://review.openstack.org/504103 | 02:18 |
| *** gvrangan has quit IRC | 02:20 | |
| *** achadha has joined #tripleo | 02:24 | |
| *** achadha has quit IRC | 02:28 | |
| *** ramishra has joined #tripleo | 02:30 | |
| *** itlinux has joined #tripleo | 02:33 | |
| *** daidv has joined #tripleo | 02:36 | |
| *** achadha has joined #tripleo | 02:38 | |
| *** achadha has quit IRC | 02:42 | |
| *** gvrangan has joined #tripleo | 02:43 | |
| openstackgerrit | Merged openstack/puppet-tripleo master: Disables port status updates with ODL in HA https://review.openstack.org/505788 | 02:44 |
| openstackgerrit | Merged openstack/tripleo-common stable/newton: Remove machine-id from image https://review.openstack.org/505310 | 02:44 |
| *** achadha has joined #tripleo | 02:47 | |
| *** mbeierl has quit IRC | 02:51 | |
| *** achadha has quit IRC | 02:52 | |
| *** itlinux has quit IRC | 02:52 | |
| openstackgerrit | Tim Rozet proposed openstack/puppet-tripleo stable/pike: Disables port status updates with ODL in HA https://review.openstack.org/508034 | 02:53 |
| *** achadha has joined #tripleo | 02:57 | |
| openstackgerrit | Merged openstack/puppet-tripleo master: Add new MySQL server option to mysql_bundle https://review.openstack.org/507113 | 02:57 |
| *** gvrangan has quit IRC | 02:58 | |
| *** gvrangan has joined #tripleo | 03:00 | |
| *** achadha has quit IRC | 03:01 | |
| *** mbeierl has joined #tripleo | 03:06 | |
| *** achadha has joined #tripleo | 03:06 | |
| *** janki has joined #tripleo | 03:09 | |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 03:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] | 03:10 |
| *** achadha has quit IRC | 03:10 | |
| *** mbeierl has quit IRC | 03:10 | |
| *** pdeore has joined #tripleo | 03:12 | |
| *** gvrangan has quit IRC | 03:12 | |
| *** gvrangan has joined #tripleo | 03:14 | |
| *** moshele has quit IRC | 03:14 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Use different variables for deploy and upgrade scenarios https://review.openstack.org/500552 | 03:17 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Use different variables for deploy and upgrade scenarios https://review.openstack.org/500552 | 03:17 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: ocata2pike: add missing current-pike repo https://review.openstack.org/500671 | 03:18 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Add all services to container scenarios https://review.openstack.org/501872 | 03:18 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Add all services to multinode-containers.yaml https://review.openstack.org/507471 | 03:18 |
| *** aditya_r has joined #tripleo | 03:19 | |
| *** aditya_r has quit IRC | 03:20 | |
| *** aditya_r has joined #tripleo | 03:20 | |
| *** gvrangan has quit IRC | 03:20 | |
| *** achadha has joined #tripleo | 03:24 | |
| *** gvrangan has joined #tripleo | 03:26 | |
| *** bnemec has quit IRC | 03:26 | |
| *** mbeierl has joined #tripleo | 03:26 | |
| *** rlandy|afk is now known as rlandy | 03:30 | |
| *** rlandy has quit IRC | 03:31 | |
| *** Tengu has quit IRC | 03:33 | |
| *** gvrangan has quit IRC | 03:34 | |
| *** Tengu has joined #tripleo | 03:39 | |
| *** achadha has quit IRC | 03:41 | |
| *** udesale has joined #tripleo | 03:42 | |
| *** links has joined #tripleo | 03:44 | |
| *** abregman|afk has quit IRC | 03:46 | |
| *** gvrangan has joined #tripleo | 03:52 | |
| *** tdasilva has quit IRC | 03:53 | |
| *** dpawar has joined #tripleo | 04:00 | |
| *** shreshtha has joined #tripleo | 04:00 | |
| *** achadha has joined #tripleo | 04:02 | |
| *** agurenko has quit IRC | 04:05 | |
| *** achadha has quit IRC | 04:06 | |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 04:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] | 04:10 |
| *** tdasilva has joined #tripleo | 04:11 | |
| *** agurenko has joined #tripleo | 04:14 | |
| *** gvrangan has quit IRC | 04:17 | |
| *** ykarel has joined #tripleo | 04:20 | |
| *** itlinux has joined #tripleo | 04:28 | |
| *** aditya_r has quit IRC | 04:32 | |
| *** aditya_r has joined #tripleo | 04:32 | |
| *** agurenko has quit IRC | 04:41 | |
| *** ratailor has joined #tripleo | 04:45 | |
| *** tongl has joined #tripleo | 04:46 | |
| *** itlinux has quit IRC | 04:48 | |
| *** achadha has joined #tripleo | 04:48 | |
| *** nyechiel has joined #tripleo | 04:52 | |
| *** psachin has joined #tripleo | 04:53 | |
| *** achadha has quit IRC | 04:57 | |
| *** mdnadeem has joined #tripleo | 04:58 | |
| *** tongl has quit IRC | 04:58 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Move fstrim to block device layer https://review.openstack.org/504579 | 04:58 |
| *** achadha has joined #tripleo | 05:02 | |
| *** gkadam is now known as gkadam-brb | 05:03 | |
| *** abregman has joined #tripleo | 05:05 | |
| *** achadha has quit IRC | 05:07 | |
| *** iranzo has joined #tripleo | 05:07 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 05:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] | 05:10 |
| *** achadha has joined #tripleo | 05:21 | |
| *** achadha has quit IRC | 05:25 | |
| *** abregman has quit IRC | 05:28 | |
| *** abregman has joined #tripleo | 05:29 | |
| *** dtrainor has quit IRC | 05:33 | |
| *** aditya_ra has joined #tripleo | 05:34 | |
| *** iranzo has quit IRC | 05:35 | |
| *** aditya_r has quit IRC | 05:37 | |
| *** aditya_ra has quit IRC | 05:37 | |
| *** aditya_r has joined #tripleo | 05:38 | |
| *** leitan has quit IRC | 05:38 | |
| *** iranzo has joined #tripleo | 05:39 | |
| *** leitan has joined #tripleo | 05:39 | |
| *** marios has joined #tripleo | 05:42 | |
| *** psahoo has joined #tripleo | 05:43 | |
| *** achadha has joined #tripleo | 05:44 | |
| *** achadha has quit IRC | 05:48 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates stable/pike: Make CephConfigOverrides append to ceph.conf[global] https://review.openstack.org/508059 | 05:55 |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/472607 | 06:00 |
| *** mrunge_ is now known as mrunge | 06:00 | |
| *** dparkes has joined #tripleo | 06:01 | |
| *** jaganathan has joined #tripleo | 06:02 | |
| *** jaganathan has quit IRC | 06:02 | |
| *** dparkes has quit IRC | 06:05 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements https://review.openstack.org/503679 | 06:06 |
| *** achadha has joined #tripleo | 06:07 | |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 06:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] | 06:10 |
| *** achadha has quit IRC | 06:12 | |
| *** ecerquei has quit IRC | 06:19 | |
| *** pcaruana has joined #tripleo | 06:19 | |
| *** achadha has joined #tripleo | 06:21 | |
| *** jlinkes has joined #tripleo | 06:22 | |
| *** mrunge has quit IRC | 06:22 | |
| *** waleedm has joined #tripleo | 06:24 | |
| *** achadha has quit IRC | 06:26 | |
| *** mrunge has joined #tripleo | 06:27 | |
| *** achadha has joined #tripleo | 06:30 | |
| *** achadha has quit IRC | 06:35 | |
| *** dpawar has quit IRC | 06:35 | |
| *** jfrancoa has joined #tripleo | 06:39 | |
| *** aditya_r has quit IRC | 06:40 | |
| *** aditya_r has joined #tripleo | 06:40 | |
| *** moshele has joined #tripleo | 06:46 | |
| *** pgadiya has joined #tripleo | 06:48 | |
| *** florianf has joined #tripleo | 06:51 | |
| openstackgerrit | Damien Ciabrini proposed openstack/puppet-tripleo master: WIP Galera: add support for encrypted SST https://review.openstack.org/507852 | 06:51 |
| *** achadha has joined #tripleo | 06:54 | |
| *** dciabrin has quit IRC | 06:58 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Move fstrim to block device layer https://review.openstack.org/504579 | 06:58 |
| *** achadha has quit IRC | 06:58 | |
| *** rcernin has joined #tripleo | 07:00 | |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-image-elements stable/pike: Update ironic-agent-multipath element https://review.openstack.org/508068 | 07:04 |
| *** oidgar has joined #tripleo | 07:06 | |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-image-elements stable/ocata: Update ironic-agent-multipath element https://review.openstack.org/508069 | 07:07 |
| *** ebarrera has joined #tripleo | 07:07 | |
| *** fzdarsky has joined #tripleo | 07:08 | |
| *** achadha has joined #tripleo | 07:08 | |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart-extras master: Add compute_feature_enabled.console_output to true https://review.openstack.org/508071 | 07:09 |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 07:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] | 07:10 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-image-elements stable/newton: Update ironic-agent-multipath element https://review.openstack.org/508072 | 07:10 |
| *** hjensas has joined #tripleo | 07:10 | |
| *** hjensas has quit IRC | 07:10 | |
| *** hjensas has joined #tripleo | 07:10 | |
| *** ffiore has joined #tripleo | 07:12 | |
| *** achadha has quit IRC | 07:12 | |
| *** aditya_r is now known as aditya_r|lunch | 07:14 | |
| *** achadha has joined #tripleo | 07:26 | |
| *** pdeore has quit IRC | 07:28 | |
| *** shardy has joined #tripleo | 07:29 | |
| *** ebarrera has quit IRC | 07:30 | |
| *** ebarrera has joined #tripleo | 07:30 | |
| oidgar | Did someone encounter an overcloud deployment problem in which controller step3 is failed due to mariadb start failed? ""Error: Systemd start for mariadb failed!" | 07:30 |
| *** achadha has quit IRC | 07:31 | |
| oidgar | I'm deploying containerized overcloud and haven't seen such thing before | 07:31 |
| *** gfidente|afk is now known as gfidente | 07:34 | |
| *** pdeore has joined #tripleo | 07:34 | |
| *** jpena|off is now known as jpena | 07:34 | |
| *** achadha has joined #tripleo | 07:36 | |
| *** dpawar has joined #tripleo | 07:36 | |
| *** pdeore has quit IRC | 07:39 | |
| *** aditya_r|lunch is now known as aditya_r | 07:40 | |
| *** ffiore has quit IRC | 07:40 | |
| *** pdeore has joined #tripleo | 07:43 | |
| *** jpich has joined #tripleo | 07:44 | |
| *** achadha has quit IRC | 07:45 | |
| *** ccamacho has quit IRC | 07:45 | |
| *** ccamacho has joined #tripleo | 07:45 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Move fstrim to block device layer https://review.openstack.org/504579 | 07:49 |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart-extras master: Create tempest role creator for barbican tempest tests https://review.openstack.org/507829 | 07:50 |
| *** egonzalez has joined #tripleo | 07:50 | |
| *** aditya_r has quit IRC | 07:50 | |
| *** ffiore has joined #tripleo | 07:51 | |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart master: Add compute-feature-enabled.attach_encrypted_volume to True https://review.openstack.org/499928 | 07:53 |
| chandankumar | arxcruz: done, let see how it goes | 07:53 |
| arxcruz | chandankumar: no you don't creator role still there... :/ | 07:54 |
| chandankumar | arxcruz: you mean to keep only __member_ role? | 07:54 |
| arxcruz | chandankumar: yes, and sorry, it's Member, not _member_ just checked on tempest :) | 07:55 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates master: WIP Handle using encryption by default for pacemaker in upgrade https://review.openstack.org/489132 | 07:55 |
| chandankumar | arxcruz: so we need to create Member role in tripleo-quickstart extras and use the same role in scenario2 job? | 07:56 |
| *** ffiore has quit IRC | 07:56 | |
| *** threestrands has quit IRC | 07:58 | |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart-extras master: Create tempest role Member for barbican tempest tests https://review.openstack.org/507829 | 07:58 |
| *** achadha has joined #tripleo | 07:59 | |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-quickstart master: DO NOT REVIEW validate new pacemaker builds for containers in HA deployments https://review.openstack.org/469998 | 08:00 |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart master: Add compute-feature-enabled.attach_encrypted_volume to True https://review.openstack.org/499928 | 08:00 |
| *** dparkes has joined #tripleo | 08:01 | |
| chandankumar | arxcruz: now correct? | 08:02 |
| arxcruz | chandankumar: perfect! let's see if it will work, if so, we don't need to create a role :) | 08:03 |
| *** achadha has quit IRC | 08:03 | |
| *** hjensas has quit IRC | 08:06 | |
| *** hjensas has joined #tripleo | 08:08 | |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 08:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 08:10 |
| *** moshele has quit IRC | 08:11 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: Keep floating ip reachability during pacemaker migration. https://review.openstack.org/474967 | 08:12 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: Keep floating ip reachability during pacemaker migration. https://review.openstack.org/474967 | 08:12 |
| *** achadha has joined #tripleo | 08:13 | |
| lvdombrk | folks, my overcloud deploy stucks and in both nodes (compute and controllers) i see this information : https://pasteboard.co/GMqDU1s.jpg https://pasteboard.co/GMqE4nB.jpg | 08:15 |
| lvdombrk | any ideas what does it mean? | 08:15 |
| *** ffiore has joined #tripleo | 08:15 | |
| *** bogdando has joined #tripleo | 08:17 | |
| *** achadha has quit IRC | 08:17 | |
| *** snecklifter has joined #tripleo | 08:17 | |
| *** aditya_r has joined #tripleo | 08:22 | |
| *** derekh has joined #tripleo | 08:26 | |
| *** dciabrin has joined #tripleo | 08:26 | |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-ui master: App config and I18n selectors refactor https://review.openstack.org/494141 | 08:30 |
| *** cylopez has joined #tripleo | 08:31 | |
| *** yamahata has joined #tripleo | 08:36 | |
| chandankumar | arxcruz: https://review.openstack.org/#/c/508071/ | 08:44 |
| arxcruz | chandankumar: the jobs didn't finish yet :) | 08:45 |
| lvdombrk | folks, my overcloud deploy stucks and in both nodes (compute and controllers) i see this information : https://pasteboard.co/GMqDU1s.jpg https://pasteboard.co/GMqE4nB.jpg | 08:45 |
| lvdombrk | any ideas what does it mean? | 08:46 |
| *** dbecker has joined #tripleo | 08:46 | |
| *** yamahata has quit IRC | 08:46 | |
| *** yamahata has joined #tripleo | 08:47 | |
| shardy | lvdombrk: can you log onto the nodes via ssh? | 08:49 |
| lvdombrk | shardy: yes can into both | 08:49 |
| shardy | lvdombrk: Ok so I'd be looking at /var/log/messages and trying to figure out where/why it got stuck | 08:50 |
| shardy | vs the primary console logging | 08:50 |
| *** iranzo has quit IRC | 08:50 | |
| *** achadha has joined #tripleo | 08:50 | |
| *** iranzo has joined #tripleo | 08:51 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates master: Remove package if service stopped and disabled https://review.openstack.org/479886 | 08:53 |
| *** achadha has quit IRC | 08:54 | |
| lvdombrk | shardy: this last what i see in messages : http://paste.openstack.org/raw/622117/ | 08:55 |
| shardy | lvdombrk: Ok so something may be wrong with the VIP configuration but the actual place where it got stuck is earlier, you need to search for the most recent os-collect-config run where presumably a heat-config hook got run and either got stuck or failed | 08:58 |
| shardy | lvdombrk: is this the setup with only two nics, where you're trying to switch from the provisioning nic to a bonded pair? | 08:58 |
| lvdombrk | shardy: yes its this setup | 08:59 |
| shardy | lvdombrk: Ok and what heat resource is stuck IN_PROGRESS, e.g from the events output from overcloud deploy? | 08:59 |
| shardy | or heat resource-list -n5 overcloud | grep IN_PROGRESS | 08:59 |
| shardy | lvdombrk: I suspect it's the NetworkDeployment, and that is probably because the network goes down when you reconfigure the ctlplane nic? | 09:01 |
| *** shardy has quit IRC | 09:03 | |
| lvdombrk | shardy: probably yes, but is happened only when i try deploy controller + compute, i tried just now deploy controller +compute (all in one node) with same setup and is was fine | 09:04 |
| lvdombrk | shardy: and when i try more than one node same result | 09:05 |
| *** aditya_r has quit IRC | 09:07 | |
| *** aditya_r has joined #tripleo | 09:09 | |
| *** mrch has joined #tripleo | 09:10 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates master: WIP: Adds pacemaker update_tasks for Pike minor update workflow https://review.openstack.org/486567 | 09:10 |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 09:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 09:10 |
| therve | Urg that highlight is annoying :) | 09:10 |
| *** stendulker has joined #tripleo | 09:11 | |
| *** salmankhan has joined #tripleo | 09:20 | |
| *** marios has quit IRC | 09:21 | |
| *** artom has joined #tripleo | 09:24 | |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Add all services to container scenarios https://review.openstack.org/501872 | 09:26 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Switch scenario004-containers to use ceph-ansible https://review.openstack.org/507526 | 09:26 |
| *** achadha has joined #tripleo | 09:32 | |
| *** waleedm has quit IRC | 09:33 | |
| *** achadha has quit IRC | 09:36 | |
| *** dparkes has quit IRC | 09:37 | |
| *** salmankhan has quit IRC | 09:39 | |
| *** yamahata has quit IRC | 09:40 | |
| *** salmankhan has joined #tripleo | 09:40 | |
| *** gkadam-brb has quit IRC | 09:42 | |
| *** etingof has quit IRC | 09:44 | |
| *** agurenko has joined #tripleo | 09:45 | |
| *** akrivoka has joined #tripleo | 09:54 | |
| *** anshul has joined #tripleo | 09:55 | |
| *** achadha has joined #tripleo | 09:55 | |
| *** ratailor is now known as rtailor | 09:56 | |
| rtailor | mandre, you around ? | 09:57 |
| rtailor | mandre, Have you deploy TLS enabled tripleo deployment using oooq ? | 09:57 |
| rtailor | mandre, I am getting this error, while trying it. https://bugs.launchpad.net/tripleo/+bug/1718712 | 09:58 |
| openstack | Launchpad bug 1718712 in tripleo "cloud-init resets eth0 of supplemental node breaking networking" [High,In progress] - Assigned to Harry Rybacki (hrybacki-h) | 09:58 |
| *** chem has joined #tripleo | 09:58 | |
| *** eck`gone has quit IRC | 09:58 | |
| *** radez has quit IRC | 09:58 | |
| *** jpena has quit IRC | 09:59 | |
| *** marios has joined #tripleo | 09:59 | |
| *** dalvarez has quit IRC | 09:59 | |
| *** amoralej has quit IRC | 09:59 | |
| mandre | rtailor: hello, yes I did use oooq to deploy with TLS, it was about a month ago so things may have changed since then | 09:59 |
| *** achadha has quit IRC | 10:00 | |
| rtailor | mandre, do you have any suggestion/workaround ? | 10:00 |
| mandre | rtailor: oh wait, I remember I used a workaround for dns issue | 10:00 |
| *** waleedm has joined #tripleo | 10:01 | |
| *** etingof has joined #tripleo | 10:01 | |
| mandre | rtailor: i'm looking for it, moment :) | 10:02 |
| *** artom has quit IRC | 10:02 | |
| rtailor | mandre, sure | 10:02 |
| *** abregman has quit IRC | 10:03 | |
| *** radez has joined #tripleo | 10:04 | |
| *** dalvarez has joined #tripleo | 10:04 | |
| *** eck`gone has joined #tripleo | 10:04 | |
| *** amoralej has joined #tripleo | 10:04 | |
| *** jpena has joined #tripleo | 10:05 | |
| mandre | rtailor: my issue was that the supplemental node had a DNS that was not available outside of the VPN, so I had to change it to a public one in order to get the VM through provisioning | 10:06 |
| *** ffiore has quit IRC | 10:06 | |
| mandre | rtailor: http://paste.openstack.org/show/622123/, related issue https://bugs.launchpad.net/tripleo/+bug/1712081 | 10:07 |
| openstack | Launchpad bug 1712081 in tripleo "SSH connection to supplemental ipa VM takes minutes" [Medium,In progress] - Assigned to Harry Rybacki (hrybacki-h) | 10:07 |
| mandre | rtailor: so hrybacki has a pending patch at https://review.openstack.org/#/c/495899/ that should help, but then you still have to change the setting yourself through quickstart | 10:07 |
| rtailor | mandre, will try it.. Thanks! | 10:08 |
| mandre | rtailor: not sure it's the same issue but it's certainly worth a try | 10:08 |
| rtailor | mandre, sure, | 10:09 |
| *** etingof has quit IRC | 10:10 | |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 10:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 10:10 |
| derekh | Did this not used to be 1G ? -rw-r--r--. 1 stack stack 2.9G Sep 26 13:48 /home/stack/overcloud-full.qcow2 | 10:10 |
| *** marios has quit IRC | 10:12 | |
| *** ffiore has joined #tripleo | 10:12 | |
| *** pgadiya has quit IRC | 10:13 | |
| jaosorior_sick | dciabrin: ping | 10:18 |
| *** janki has quit IRC | 10:20 | |
| jaosorior_sick | mandre, rtailor what's the issue? | 10:21 |
| chandankumar | jaosorior_sick: we are testing the stuff https://review.openstack.org/#/c/499928/ if with member if it not works we will revert back | 10:22 |
| *** etingof has joined #tripleo | 10:25 | |
| *** agurenko has quit IRC | 10:26 | |
| *** etingof has quit IRC | 10:27 | |
| *** aditya_r has quit IRC | 10:27 | |
| *** thrash|g0ne is now known as thrash | 10:29 | |
| *** janki has joined #tripleo | 10:30 | |
| therve | Can I get +W on https://review.openstack.org/506067 ? | 10:31 |
| chandankumar | arxcruz: it is not working we need to revert back to previous changes http://logs.openstack.org/28/499928/13/check/gate-tripleo-ci-centos-7-scenario002-multinode-oooq/d7921f1/logs/undercloud/home/jenkins/tempest_output.log.txt.gz | 10:34 |
| *** jkilpatr has quit IRC | 10:35 | |
| lvdombrk | folks someone has tested controller HA (pacamaker) with two controller nodes not 3 as mentioned in documentation? | 10:36 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack-infra/tripleo-ci master: Make wait_for print the failed ouput. https://review.openstack.org/508107 | 10:39 |
| *** udesale has quit IRC | 10:39 | |
| jaosorior_sick | chandankumar: so, alee and me were checking what was wrong with the barbican deployment. And it was actually a completely unrelated issue: https://bugs.launchpad.net/tripleo/+bug/1719929 | 10:40 |
| openstack | Launchpad bug 1719929 in tripleo "Debug paramter using wrong type" [High,In progress] - Assigned to Ade Lee (alee-3) | 10:40 |
| jaosorior_sick | chandankumar: one of the fixes had landed, so with that it could pass hopefully. | 10:40 |
| chandankumar | jaosorior_sick: so you mean to say that creator role will be automatically created? | 10:41 |
| jaosorior_sick | chandankumar: no | 10:41 |
| jaosorior_sick | chandankumar: I mean to say that the job was breaking cause of that bug. | 10:41 |
| jaosorior_sick | chandankumar: now, we still need the creator role in the roles_list that were set before. which is now not there in the current patch. | 10:42 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack-infra/tripleo-ci master: Make wait_for print the failed ouput. https://review.openstack.org/508107 | 10:42 |
| chandankumar | jaosorior_sick: with patchset 12 things are working fine | 10:42 |
| chandankumar | patchset 11 | 10:42 |
| chandankumar | jaosorior_sick: https://review.openstack.org/#/c/499928/11 | 10:43 |
| jaosorior_sick | yep, that looked just fine (adding the creator role explicitly) | 10:43 |
| chandankumar | and https://review.openstack.org/#/c/507829/2 | 10:43 |
| jaosorior_sick | that shouldn't be necessary | 10:44 |
| chandankumar | jaosorior_sick: you mean https://review.openstack.org/#/c/507829/2 this one? | 10:44 |
| *** ykarel is now known as ykarel|afk | 10:44 | |
| jaosorior_sick | yeah | 10:44 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/pike: Make CephConfigOverrides append to ceph.conf[global] https://review.openstack.org/508059 | 10:45 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack-infra/tripleo-ci master: Make wait_for print the failed ouput. https://review.openstack.org/508107 | 10:45 |
| chandankumar | let me update the patch then | 10:45 |
| jaosorior_sick | adding the roles is the job of auth.tempest_roles. So, you shouldn't need to do it twice. | 10:45 |
| chandankumar | auth.tempest_roles will just set the value in tempest.conf | 10:46 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack-infra/tripleo-ci master: Make wait_for print the failed ouput. https://review.openstack.org/508107 | 10:46 |
| chandankumar | it will not create any role which leads to failure if role is not found | 10:46 |
| chandankumar | that's why i have added depends-on | 10:46 |
| *** stendulker_ has joined #tripleo | 10:47 | |
| *** raildo has joined #tripleo | 10:47 | |
| *** ffiore has quit IRC | 10:50 | |
| *** stendulker has quit IRC | 10:50 | |
| *** jkilpatr has joined #tripleo | 10:52 | |
| *** stendulker_ has quit IRC | 10:52 | |
| lvdombrk | folks someone has tested controller HA (pacamaker) with two controller nodes not 3 as mentioned in documentation? | 10:52 |
| *** cylopez has quit IRC | 10:52 | |
| jaosorior_sick | chandankumar: I thought the whole purpose of tempest_roles was to create those roles | 10:53 |
| chandankumar | jaosorior_sick: we will be fixing those stuffs in tempestconf | 10:54 |
| jaosorior_sick | "It is worth pointing out that depending on your cloud configuration you might need to assign a role to each of the users created by Tempest’s dynamic credentials. This can be set using the tempest_roles option. It takes in a list of role names each of which will be assigned to each of the users created by dynamic credentials. This option will not have any effect when Tempest is not configured to use | 10:54 |
| jaosorior_sick | dynamic credentials." | 10:54 |
| jaosorior_sick | chandankumar: from the docs ^^ | 10:54 |
| *** yolanda has quit IRC | 10:54 | |
| *** tosky has joined #tripleo | 10:55 | |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-heat-templates master: Add Mistral event engine https://review.openstack.org/462975 | 10:56 |
| jaosorior_sick | that's how we got it to work in puppet-openstack-integration | 10:56 |
| rtailor | jaosorior_sick, my deployment is getting failed at http://paste.openstack.org/show/622132/ while trying TLS enabled tripleo deployment using oooq. | 10:56 |
| rtailor | jaosorior_sick, hrybacki is working on this ^^ https://bugs.launchpad.net/tripleo/+bug/1712081 | 10:57 |
| openstack | Launchpad bug 1712081 in tripleo "SSH connection to supplemental ipa VM takes minutes" [Medium,In progress] - Assigned to Harry Rybacki (hrybacki-h) | 10:57 |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart-extras master: Create tempest role Member and creator https://review.openstack.org/507829 | 10:57 |
| jaosorior_sick | rtailor: oh that :/ yeah | 10:57 |
| rtailor | jaosorior_sick, I discussed that with you earlier, you suggested to talk with hrybacki | 10:57 |
| jaosorior_sick | rtailor: yep | 10:58 |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart master: Add compute-feature-enabled.attach_encrypted_volume to True https://review.openstack.org/499928 | 10:58 |
| chandankumar | jaosorior_sick: things will work, i will update the docs | 10:58 |
| *** amoralej is now known as amoralej|lunch | 10:59 | |
| chandankumar | on eanling dynamic credentials tempest creates __member_ role | 10:59 |
| *** ykarel|afk is now known as ykarel | 11:02 | |
| *** MVenesio has joined #tripleo | 11:03 | |
| *** ffiore has joined #tripleo | 11:05 | |
| *** marios has joined #tripleo | 11:05 | |
| *** abregman has joined #tripleo | 11:05 | |
| *** yolanda has joined #tripleo | 11:08 | |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 11:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 11:10 |
| *** abregman has quit IRC | 11:11 | |
| *** abregman has joined #tripleo | 11:12 | |
| *** MVenesio has quit IRC | 11:17 | |
| *** MVenesio has joined #tripleo | 11:18 | |
| *** shardy has joined #tripleo | 11:21 | |
| openstackgerrit | Tom Barron proposed openstack/tripleo-heat-templates stable/newton: manila: set "host" to "hostgroup" https://review.openstack.org/508117 | 11:21 |
| openstackgerrit | Numan Siddique proposed openstack/tripleo-heat-templates master: ovn: Provide the option to configure the integration bridge https://review.openstack.org/508118 | 11:22 |
| rook | shardy: hey ran into this yesterday - https://gist.github.com/jtaleric/153fd30e5a31d47dee95362c231e7b46 - this all worked prior to my chnage to ceph-ansible, not sure why that matters... but prior to ceph-ansible I didn't have issues with this | 11:26 |
| *** shreshtha has quit IRC | 11:27 | |
| *** ipsecguy_ is now known as ipsecguy | 11:28 | |
| shardy | rook: hrm, could you please mkdir plan_tmp && cd plan_tmp && openstack container save overcloud and send me a tarball of the results? | 11:31 |
| shardy | actually there's a plan export command now which may do the same thing, but I'd like to grep the rendered templates to see where the problem is | 11:32 |
| lvdombrk | shardy: you asked on what deployment stucks, it stucks on: 2017-09-28 10:14:57Z [overcloud.AllNodesDeploySteps.ControllerDeployment_Step4]: CREATE_COMPLETE state changed | 11:32 |
| rook | shardy: yeah, once this deploy of just ceph and controller completes... seeing if more ceph nodes impacts timings. | 11:32 |
| shardy | lvdombrk: the thing it's stuck on is the last thing that's CREATE_IN_PROGRESS | 11:32 |
| shardy | CREATE_COMPLETE means it's complete | 11:32 |
| rook | shardy: ill get that too you soon, thanks | 11:32 |
| *** MVenesio has quit IRC | 11:32 | |
| shardy | rook: there was this fix for a similar issue https://review.openstack.org/#/q/I014e7d6679c5733b17243d647eaad228c276585a | 11:33 |
| shardy | rook: but I think your error looks different | 11:33 |
| rook | shardy: okay. | 11:34 |
| rook | shardy I am really at a loss on what to add to fix this, but ill generate that plan and send -- can i generate the plan during the deploy and you get the info necessary? | 11:34 |
| shardy | rook: yeah or just save it from the failed deployment | 11:35 |
| rook | ah failed deployment | 11:35 |
| shardy | I just need to see the templates that got passed to heat | 11:35 |
| rook | ok | 11:35 |
| rook | right | 11:35 |
| shardy | which should be in swift | 11:35 |
| * rook waits then | 11:35 | |
| rook | hm, nm ill just kill this | 11:36 |
| rook | also shardy while I am thinking of it again, how hard would it be to have openstack stack delete include the time it takes for ironic to clean nodes, if ironic cleaning is enabled./ | 11:40 |
| rook | I have rushed the gun too many times, simply based on the heat stack going away | 11:40 |
| *** jlabarre has joined #tripleo | 11:41 | |
| *** jaosorior_sick is now known as jaosorior | 11:41 | |
| *** jaganathan has joined #tripleo | 11:41 | |
| jaosorior | shardy: could you check this out https://review.openstack.org/#/c/507516/ ? | 11:41 |
| *** ansmith has quit IRC | 11:42 | |
| numans | jaosorior, hi, can you please review this when you get some time - https://review.openstack.org/#/c/506694/. This patch is similar to the ODL patch - https://review.openstack.org/#/c/498383/ | 11:42 |
| shardy | rook: not that difficult, but IMO if we're going to add that, we should refactor this CLI stuff into a mistral workflow first | 11:43 |
| shardy | https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_delete.py | 11:43 |
| shardy | rook: then the UI delete will be able to take advantage of the same logic | 11:43 |
| shardy | rook: I'd raise a bz and perhaps chat with thrash about it | 11:43 |
| shardy | https://github.com/openstack/python-tripleoclient/commit/d6a2db92fcedbf759de51f07d23e3fa1401533f1#diff-ae38f7b279aae2c21c7fa6e20942fc4e | 11:44 |
| shardy | rook: it looks like the stack delete got converted to a workflow, but we need another workflow which wraps the stack delete, plan delete and wait for node cleaning | 11:45 |
| *** bfournie has quit IRC | 11:45 | |
| lvdombrk | shardy: yes, but there is now any process wich CREATE_IN_PROGRESS, last is : 2017-09-28 10:14:57Z [overcloud.AllNodesDeploySteps.ControllerDeployment_Step4]: CREATE_COMPLETE state changed | 11:45 |
| numans | jaosorior, thanks. | 11:45 |
| *** bfournie has joined #tripleo | 11:45 | |
| shardy | lvdombrk: Ok so heat resource-list -n5 overcloud | grep IN_PROGRESS shows no resources at all? | 11:45 |
| shardy | jaosorior: ack, done | 11:47 |
| *** iranzo has quit IRC | 11:48 | |
| jaosorior | shardy: thanks | 11:49 |
| rook | shardy: what was the command for the plan download | 11:49 |
| *** bfournie has quit IRC | 11:49 | |
| shardy | rook: openstack container save overcloud | 11:49 |
| shardy | assuming you're using the default stack/plan name | 11:50 |
| shardy | you need to be in a tmpdir though or it writes files all over your home directory | 11:50 |
| shardy | don't ask me how I know that :D | 11:50 |
| lvdombrk | shardy: http://paste.openstack.org/raw/622138/ | 11:51 |
| *** abishop has joined #tripleo | 11:51 | |
| shardy | rook: I think openstack overcloud plan export gives you a tarball actually, but I've not used it myself yet | 11:51 |
| rook | http://perf1.perf.lab.eng.bos.redhat.com/jtaleric/Archive/shardy-2.tar.gz shardy | 11:51 |
| shardy | rook: thanks, looking | 11:52 |
| *** ffiore has quit IRC | 11:52 | |
| *** iranzo has joined #tripleo | 11:52 | |
| openstackgerrit | Numan Siddique proposed openstack/tripleo-quickstart master: Enable OVN HA profile in fset28 https://review.openstack.org/499570 | 11:53 |
| *** dpawar has quit IRC | 11:54 | |
| *** lvdombrk has quit IRC | 11:54 | |
| *** lvdombrkr has joined #tripleo | 11:54 | |
| *** trown has joined #tripleo | 11:55 | |
| shardy | rook: the problem is in ./puppet/r620compute-role.yaml and ./puppet/r930compute-role.yaml - they expect NovaComputeExtraConfig which is wrong | 11:57 |
| shardy | rook: was this an update after removing deprecated_param_extraconfig from the roles_data for those roles? | 11:57 |
| openstackgerrit | Derek Higgins proposed openstack/tripleo-common master: Add ironic api/pxe health checks during kolla build https://review.openstack.org/506125 | 11:58 |
| shardy | those roles shouldn't have any of the deprecated_param* and they shouldn't have uses_deprecated_params: True | 11:58 |
| rook | ok | 11:58 |
| rook | let me see | 11:58 |
| shardy | so I'd fix that in the roles data, delete the plan, then start again | 11:58 |
| rcernin | tripleo: quick question if the heat tries to save event resource but it is too big. Is there a way to actually truncate them? | 11:58 |
| rook | shardy: in the roles_data? | 11:59 |
| rook | i removed those in roles_data | 11:59 |
| shardy | rook: well they're in the plan roles_data, sec let me paste | 11:59 |
| rook | the only occurance is for the compute | 11:59 |
| rook | Compute role | 11:59 |
| shardy | rook: but all the other deprecated params are still in the extra compute roles | 12:00 |
| shardy | they should all be removed, as they's special and only for backwards compat with the default Compute role | 12:00 |
| rook | ok | 12:01 |
| rook | all deprecated keys | 12:01 |
| shardy | rook: http://paste.openstack.org/show/622141/ | 12:01 |
| rook | uses_deprecated_params: True │···················································· | 12:01 |
| shardy | rook: yeah, we'll have to add a comment or validation as I can see this may be confusing | 12:01 |
| rook | should ^ be false | 12:01 |
| shardy | rook: yeah see the paste, basically delete all reference to deprecated things for custom roles | 12:01 |
| shardy | it's just workarounds for old inconsistent names in the default roles | 12:02 |
| shardy | rook: I'm not quite sure why this is causing your NovaComputeExtraConfig error though, as those templates should have been re-rendered | 12:02 |
| shardy | rook: but I'd fix that then start with a deleted plan, and we can debug further if needed | 12:03 |
| rook | so keep that true shardy | 12:03 |
| *** dsariel has quit IRC | 12:04 | |
| *** brian83 has joined #tripleo | 12:04 | |
| rook | shardy: also, plan delete? what is the command :) | 12:04 |
| shardy | rook: openstack plan delete ;) | 12:05 |
| shardy | rook: if you do openstack overcloud delete it should already be gone, you can check via openstack container list | 12:05 |
| *** pradk has joined #tripleo | 12:05 | |
| rook | ah ha ok | 12:06 |
| rook | easy | 12:06 |
| *** ecerquei has joined #tripleo | 12:06 | |
| rook | sorry for the dumb questions, I assume i know nothig now | 12:06 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo master: Set meta container-attribute-target=host attribute https://review.openstack.org/497766 | 12:06 |
| *** dpawar has joined #tripleo | 12:07 | |
| shardy | rcernin: in old heat versions there was code to truncate event data, but I think recently it just references the resource property object stored in the DB | 12:07 |
| shardy | rcernin: I'd maybe drop into #heat if there are issues there | 12:08 |
| rcernin | shardy: having role_data too big to store in DB collumn is not an issue during deployment right? | 12:08 |
| shardy | rcernin: it was an issue we fixed like around juno/liberty timeframe - what release are we talking about here? :) | 12:09 |
| *** morazi has joined #tripleo | 12:09 | |
| rcernin | shardy: newton | 12:09 |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 12:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 12:10 |
| therve | rcernin, What's the issue? | 12:10 |
| *** cdearborn has joined #tripleo | 12:11 | |
| *** pdeore has quit IRC | 12:11 | |
| *** pdeore has joined #tripleo | 12:11 | |
| rcernin | therve: i doubt its the issue tho, just verifying here, while storing the event in the database, the resource_properties role_data is too big to be stored in the DB collumn. | 12:11 |
| therve | rcernin, Yeah we fixed that quite a few times. The latest occurences ought to be harmless indeed (though fixed) | 12:11 |
| rcernin | therve: thats what I thought it might be. Just role_data property end up with being "Deleted, too large" | 12:12 |
| therve | rcernin, Oh you mean you don't have an error? | 12:12 |
| rcernin | therve: I do, I just mean there might be another error and we can ignore this one. | 12:13 |
| rcernin | Looking at the resource it will report "Error": "Resource properties are too large to store fully" and role_data being one of the "Deleted, too large" resource properties. | 12:13 |
| therve | rcernin, Oh, resource properties? | 12:14 |
| rcernin | question was if the deployment will actually continue despite this error. | 12:14 |
| rook | shardy: kicked it again | 12:14 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Request appropriate EKU for haproxy certificate explictly https://review.openstack.org/507514 | 12:15 |
| therve | That might be a new one | 12:15 |
| * shardy looks for the code on newton | 12:15 | |
| therve | rcernin, If you had logs or something that would help | 12:15 |
| shardy | I thought that error was harmless, and happened when we had to not store the resource_properties | 12:15 |
| *** aputtur has joined #tripleo | 12:16 | |
| therve | Yeah if done on events indeed | 12:16 |
| rcernin | therve: thats Heat Event. | 12:17 |
| therve | rcernin, So that's not an issue | 12:17 |
| *** marrusl has quit IRC | 12:17 | |
| shardy | https://github.com/openstack/heat/blob/stable/newton/heat/engine/event.py#L96 | 12:18 |
| rook | shardy: :( | 12:18 |
| rook | u'message': u"Failed to run action [action_ex_id=d356e448-e3f8-4b33-b91b-c97f5ac42c9b, action_cls='<class 'mistral.actions.action_factory.DeployStackAction'>', attributes='{}', params='{u'skip_deploy_identifier': False, u'container': u'overcloud', u'timeout': 240}']\n ERROR: Parameter Groups error: : | 12:18 |
| therve | Yeah that's the correct handling code | 12:18 |
| shardy | yeah it's just hit this path because we're passing around a giant map of roles_data | 12:18 |
| rook | resources.R620Compute<nested_stack>.resources.0<http://192.168.24.1:8080/v1/AUTH_4d7f64c5ca094637abd0bc0363603240/overcloud/puppet/r620compute-role.yaml>.parameter_groups.deprecated: : The parameters must be provided for each parameter group.", | 12:18 |
| rook | u'status': u'FAILED'} | 12:18 |
| rook | gar | 12:18 |
| rook | shardy: want the plan again? | 12:18 |
| rcernin | shardy: therve: thanks for quick help. | 12:18 |
| shardy | rook: yeah, please | 12:18 |
| *** pdeore has quit IRC | 12:19 | |
| *** bfournie has joined #tripleo | 12:19 | |
| shardy | therve: it'd be cool to add an option to completely disable storing properties_data with events, except for the top level stack | 12:19 |
| *** dsneddon has quit IRC | 12:19 | |
| shardy | I imagine that would save quite some DB space on the undercloud | 12:19 |
| *** bfournie has quit IRC | 12:19 | |
| *** dougbtv_ has quit IRC | 12:20 | |
| therve | shardy, Now that we have a separate table I don't think it's an issue anymore | 12:20 |
| shardy | therve: ah, yeah I guess that's true | 12:20 |
| therve | We still need to store it for resources for real, and now we link to that | 12:20 |
| *** trown has quit IRC | 12:21 | |
| rook | http://perf1.perf.lab.eng.bos.redhat.com/jtaleric/Archive/shardy-3.tar.gz shardy | 12:22 |
| shardy | therve: If we create $many nested stacks and pass a large identical roles_data map into all of them, I imagine we still create a record per resource vs being smart enough to de-duplicate where the value is the same? | 12:23 |
| *** dpawar has quit IRC | 12:23 | |
| shardy | when you're passing an attribute it'd be nice to store that, and reference it instead of having the function resolve then copy the data | 12:23 |
| therve | shardy, Yeah we probably copy | 12:24 |
| *** amoralej|lunch is now known as amoralej | 12:25 | |
| shardy | therve: that may actually explain why ResourceChain is so slow creating a large number of mostly empty stacks | 12:25 |
| * shardy adds it to the list of things to look into at some point | 12:25 | |
| janki | shardy, hey. got a minute? | 12:28 |
| shardy | rook: https://paste.fedoraproject.org/paste/wJWwrV8q-9TfjtdDU7SRQw/ | 12:29 |
| shardy | rook: the custom roles were still referencing deprecated things and setting uses_deprecated_params: True | 12:29 |
| *** dprince has joined #tripleo | 12:29 | |
| *** pdeore has joined #tripleo | 12:29 | |
| *** rbrady-afk has quit IRC | 12:30 | |
| rook | oh, i thought i left that | 12:30 |
| rook | ok | 12:30 |
| rook | so remove all occurances not just for the custom computes | 12:30 |
| shardy | rook: try the one I just pasted | 12:30 |
| *** rlandy has joined #tripleo | 12:30 | |
| shardy | rook: remove for all the custom computes | 12:30 |
| shardy | but you can leave them for the default roles (unless you don't care about backwards compat, then you can remove them for all roles) | 12:31 |
| *** pcaruana has quit IRC | 12:31 | |
| *** cdearborn has quit IRC | 12:31 | |
| shardy | janki: hi | 12:31 |
| janki | shardy, I see what you meant by paunch restarting containers. I tried that. I do have question though | 12:32 |
| shardy | janki: sure | 12:32 |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-validations master: Fix repo-availability validation with Ansible 2.3 https://review.openstack.org/483393 | 12:32 |
| janki | there is no TRIPLEO_HASH_CONFIG for opendaylight-api container. I don't think that should be the case | 12:33 |
| janki | shardy, ^ | 12:33 |
| *** ffiore has joined #tripleo | 12:33 | |
| shardy | janki: TRIPLEO_CONFIG_HASH is calculated by docker-puppet.py when the config is generated | 12:33 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Config download support for standalone deployments https://review.openstack.org/505827 | 12:34 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/docker/docker-puppet.py#L377 | 12:34 |
| janki | shardy, yes. so it should have been generated for odl-api too. so my setup is buggy? | 12:35 |
| janki | shardy, in fact, there are few other services without the "HASH". | 12:35 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/docker/docker-puppet.py#L100 | 12:36 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Better support for redis in Zaqar https://review.openstack.org/506067 | 12:36 |
| shardy | janki: is there an md5sum for those services in /var/lib/config-data? | 12:36 |
| shardy | perhaps it's a naming mismatch or something? | 12:36 |
| *** abregman is now known as abregman|mtg | 12:36 | |
| janki | shardy, let me check. give me a minute | 12:36 |
| openstackgerrit | Thomas Herve proposed openstack/tripleo-heat-templates master: Allow configuration Zaqar with Redis https://review.openstack.org/506071 | 12:38 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Config download support for standalone deployments https://review.openstack.org/505827 | 12:38 |
| janki | shardy, there is md5sum for "opendaylight". container/service name is "opendaylight-api". could this be an issue? | 12:39 |
| shardy | janki: yup probably | 12:39 |
| shardy | lets take a look at the templates | 12:39 |
| janki | heat template right? shardy | 12:40 |
| shardy | yeah - check out the code I just linked and you'll see how it works | 12:40 |
| shardy | we iterate over the keys in the json file, then look for a md5sum file with the same name | 12:40 |
| shardy | janki: https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/opendaylight-api.yaml#L69 | 12:41 |
| shardy | try changing the config_volume so it matches the service_name | 12:41 |
| shardy | we should probably add a validation to check that in CI | 12:42 |
| janki | shardy, will do and submit a patch soon. | 12:42 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/opendaylight-api.yaml#L101 | 12:42 |
| shardy | janki: that probably also has to change, I'd look to align all the paths with the service name if possible | 12:43 |
| fultonj | mandre: does tripleo modify /etc/sysconfig/docker on overcloud nodes during deploy so they can pull from the appropriate docker repos? e.g. add an --insecure-registry? | 12:43 |
| fultonj | s/repos/registries/ | 12:44 |
| shardy | fultonj: yes, there's a puppet maniefest in puppet-tripleo which does it | 12:44 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/docker.yaml#L52 | 12:44 |
| *** liverpooler has joined #tripleo | 12:44 | |
| *** ykarel has quit IRC | 12:44 | |
| fultonj | that's what i was looking for thanks shardy | 12:45 |
| shardy | np | 12:45 |
| *** agurenko has joined #tripleo | 12:45 | |
| *** trown has joined #tripleo | 12:45 | |
| fultonj | rook: ^ | 12:46 |
| *** rbrady-afk has joined #tripleo | 12:46 | |
| *** jpena is now known as jpena|lunch | 12:46 | |
| janki | shardy, 1 more - I changed "HASH" for neutron-opendaylight-server and all neutron-* containers updated. this might cause a problem where say we want neutron-opl to update but not neutron-lbaas | 12:46 |
| openstackgerrit | Ana Krivokapic proposed openstack/tripleo-ui master: Validate extension for tarballs on plan create/update https://review.openstack.org/508145 | 12:47 |
| mandre | shardy: you're so quick to respond, thanks :) | 12:47 |
| shardy | mandre: heh, no worries - do you know if we have any intra service dependencies defined anywhere or only the steps? | 12:48 |
| shardy | mandre: ref the question from janki above, I didn't expect changing the config hash for one service in the container environment to update any other containers | 12:48 |
| mandre | shardy: that's only in the steps that we do orchestration AFAIK | 12:48 |
| shardy | janki: Ok we may need to dig into that a bit more as it's not expected - if you can perhaps look into it and raise a bug with as much detail as possible please? | 12:49 |
| *** dsariel has joined #tripleo | 12:49 | |
| * mandre reads backlog | 12:49 | |
| rook | fultonj: I am missing some context | 12:49 |
| rook | shardy: the deploy seems to be progressing | 12:49 |
| *** dmarlin has joined #tripleo | 12:50 | |
| shardy | rook: \o/ | 12:50 |
| rook | shardy: thanks looks like removing all occurances and updating that var fixed it | 12:50 |
| shardy | sounds good | 12:50 |
| * shardy goes to get coffee | 12:50 | |
| fultonj | rook: for the other problem with downloading the ceph docker image | 12:50 |
| rook | ahhh yeah | 12:50 |
| rook | ok | 12:50 |
| *** ansmith has joined #tripleo | 12:50 | |
| rook | thanks fultonj - I switched to the upstream again | 12:50 |
| rook | :/ | 12:50 |
| janki | shardy, sure. will do. I have pasted the output in pastebin Can you verify if my understanding about other serivces upgrading is correct https://pastebin.com/vJDiR4fa | 12:50 |
| janki | mandre, ^ | 12:50 |
| *** shreshtha has joined #tripleo | 12:52 | |
| fultonj | rook: when you go back to downstream you just need to set DockerInsecureRegistryAddress to include the downstream registry address | 12:52 |
| *** aditya_r has joined #tripleo | 12:52 | |
| fultonj | i was spoiled by the defaults "just working" | 12:53 |
| *** rmascena has joined #tripleo | 12:53 | |
| rook | "just working" (tm) | 12:53 |
| rook | : | 12:53 |
| janki | shardy, mandre I udnerstand there is only 1 "HASH" created for "all" of neutron and hence all of them are updating | 12:53 |
| openstackgerrit | Merged openstack/tripleo-validations master: Fix repo-availability validation with Ansible 2.3 https://review.openstack.org/483393 | 12:54 |
| *** Goneri has joined #tripleo | 12:55 | |
| *** raildo has quit IRC | 12:55 | |
| *** artom has joined #tripleo | 12:55 | |
| *** artom has quit IRC | 12:56 | |
| *** artom has joined #tripleo | 12:56 | |
| *** artom has quit IRC | 12:57 | |
| *** artom has joined #tripleo | 12:57 | |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Add all services to container scenarios https://review.openstack.org/501872 | 12:58 |
| *** artom has quit IRC | 12:58 | |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Switch scenario004-containers to use ceph-ansible https://review.openstack.org/507526 | 12:58 |
| *** artom has joined #tripleo | 12:58 | |
| *** artom has quit IRC | 12:59 | |
| mandre | janki: correct, the hash is created per config_volume, which may be shared among multiple containers | 12:59 |
| *** artom has joined #tripleo | 12:59 | |
| *** gbarros has joined #tripleo | 12:59 | |
| *** artom has quit IRC | 13:00 | |
| janki | mandre, shouldn't all services have have thier own config volume names? I noticed the same for nova. everything nova related is just nova.md5sum | 13:00 |
| *** tzumainn has joined #tripleo | 13:00 | |
| *** artom has joined #tripleo | 13:00 | |
| *** pcaruana has joined #tripleo | 13:00 | |
| janki | mandre, did you look at the pastebin? just to confirm that I am raising a valid concern | 13:01 |
| *** artom has quit IRC | 13:01 | |
| shardy | janki: ah, yeah well it's tricky for services which support plugins etc but don't have a config file per plugin | 13:01 |
| *** raildo has joined #tripleo | 13:01 | |
| *** ukalifon has joined #tripleo | 13:01 | |
| *** artom has joined #tripleo | 13:01 | |
| *** dpawar has joined #tripleo | 13:01 | |
| *** rmascena has quit IRC | 13:02 | |
| *** artom has quit IRC | 13:02 | |
| janki | shardy, whats the best way to solve this issue then? or is this expected behavior? | 13:02 |
| *** atoth has joined #tripleo | 13:02 | |
| *** artom has joined #tripleo | 13:02 | |
| arxcruz | EmilienM: can I have your bless in https://review.openstack.org/#/c/508071 ? | 13:03 |
| *** bfournie has joined #tripleo | 13:03 | |
| arxcruz | EmilienM: oh, nevermind, trown was faster :D | 13:03 |
| arxcruz | chandankumar: ^ | 13:04 |
| *** cdearborn has joined #tripleo | 13:04 | |
| trown | i win | 13:04 |
| chandankumar | arxcruz: thanks :-) | 13:04 |
| *** ykarel has joined #tripleo | 13:04 | |
| shardy | janki: first can I clarify which service did you change the CONFIG_HASH for? or did you change one of the md5sum files? | 13:05 |
| shardy | that part isn't included in your paste | 13:05 |
| *** dsariel has quit IRC | 13:05 | |
| shardy | janki: but as mandre says, if a config volume, e.g neutron, is shared between containers right now we have no way to really know what should be restarted, other than to restart all those things that reference the config volume that changed | 13:05 |
| *** rbrady-afk has quit IRC | 13:06 | |
| janki | shardy, I changed hash from json file "neutron_db_sync": {"image": "192.168.24.1:8787/tripleoupstream/centos-binary-neutron-server-opendaylight:custom", "environment": ["TRIPLEO_CONFIG_HASH=5f1234577673be | 13:07 |
| janki | a38860123456b64c53"], | 13:07 |
| *** eck`gone is now known as eck` | 13:07 | |
| *** abregman|mtg has quit IRC | 13:07 | |
| *** jcoufal has joined #tripleo | 13:07 | |
| *** jmelvin has joined #tripleo | 13:08 | |
| chandankumar | EmilienM: arxcruz please have a look on https://review.openstack.org/#/q/topic:tempest-scenario002+(status:open+OR+status:merged) | 13:09 |
| chandankumar | things are started working | 13:09 |
| *** dsariel has joined #tripleo | 13:09 | |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 13:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 13:10 |
| *** pchavva has joined #tripleo | 13:11 | |
| arxcruz | chandankumar: awesome! | 13:12 |
| *** rtailor has quit IRC | 13:12 | |
| arxcruz | chandankumar: weird thing is that the Member role should be created by tempest because we have the auth.use_dynamic_credentials set to true | 13:13 |
| * chandankumar is confused is creator role is hardcoded somewhere. | 13:13 | |
| arxcruz | chandankumar: the creator role is hardcoded ? | 13:13 |
| *** trown has quit IRC | 13:13 | |
| chandankumar | arxcruz: no idea, the test should suppose to work with any role | 13:13 |
| *** chlong has quit IRC | 13:14 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements https://review.openstack.org/503679 | 13:14 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Save console files with timestamps https://review.openstack.org/487362 | 13:14 |
| chandankumar | arxcruz: anyway, let's get it merged we can invistigate later on | 13:14 |
| *** trown has joined #tripleo | 13:14 | |
| *** catintheroof has joined #tripleo | 13:15 | |
| arxcruz | chandankumar: then please add a #TODO comment | 13:16 |
| hrybacki | rtailor: So I was unable to resolve the issue yesterday. Working with trown yesterday the root issue was not readily apparent. I've created an additional bug for this issue: https://bugs.launchpad.net/tripleo/+bug/1720137 | 13:17 |
| openstack | Launchpad bug 1720137 in tripleo "Overcloud Deployment fails: No valid host was found. " [Undecided,New] | 13:17 |
| *** atoth has quit IRC | 13:19 | |
| *** leitan has quit IRC | 13:21 | |
| *** sshnaidm is now known as sshnaidm|afk | 13:23 | |
| *** abregman|mtg has joined #tripleo | 13:23 | |
| *** MVenesio has joined #tripleo | 13:28 | |
| openstackgerrit | James Slagle proposed openstack/python-tripleoclient master: Config download support for standalone deployments https://review.openstack.org/505828 | 13:29 |
| openstackgerrit | Andy Smith proposed openstack/tripleo-heat-templates master: WIP Separate rpc and notify messaging backends https://review.openstack.org/507963 | 13:30 |
| *** shreshtha has quit IRC | 13:30 | |
| *** mbeierl has left #tripleo | 13:30 | |
| *** bfournie1 has joined #tripleo | 13:31 | |
| *** jrist has quit IRC | 13:31 | |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart master: Remove cloud-init from IPA supplemental node https://review.openstack.org/508158 | 13:32 |
| *** jfrancoa has quit IRC | 13:32 | |
| *** bfournie has quit IRC | 13:32 | |
| *** yamahata has joined #tripleo | 13:32 | |
| *** waleedm has quit IRC | 13:32 | |
| *** lblanchard has joined #tripleo | 13:33 | |
| openstackgerrit | Merged openstack/tripleo-docs master: docs: installation: remove unnecessary blockquotes https://review.openstack.org/504517 | 13:33 |
| *** atoth has joined #tripleo | 13:34 | |
| *** jaosorior has quit IRC | 13:34 | |
| *** jrist has joined #tripleo | 13:35 | |
| *** trown|brb has joined #tripleo | 13:37 | |
| *** leitan has joined #tripleo | 13:37 | |
| *** trown has quit IRC | 13:37 | |
| *** dsariel has quit IRC | 13:38 | |
| *** pdeore has quit IRC | 13:38 | |
| *** trown|brb is now known as trown | 13:38 | |
| *** rbowen has joined #tripleo | 13:39 | |
| *** gbarros has quit IRC | 13:41 | |
| *** jpena|lunch is now known as jpena | 13:46 | |
| *** abregman|mtg has quit IRC | 13:48 | |
| EmilienM | hi | 13:49 |
| EmilienM | chandankumar: ok | 13:49 |
| *** jfrancoa has joined #tripleo | 13:49 | |
| *** morazi has quit IRC | 13:50 | |
| openstackgerrit | Merged openstack/tripleo-common master: Add OVN container images to TripleO-common https://review.openstack.org/506694 | 13:52 |
| openstackgerrit | Merged openstack/instack-undercloud master: Remove certificate resubmit from haproxy script https://review.openstack.org/507516 | 13:52 |
| *** bnemec has joined #tripleo | 13:53 | |
| *** sshnaidm|afk is now known as sshnaidm | 13:54 | |
| chandankumar | EmilienM: https://review.openstack.org/#/q/topic:tempest-scenario002+(status:open+OR+status:merged) both review passed need your power on that | 13:56 |
| *** dbecker has quit IRC | 13:57 | |
| EmilienM | matbu_, marios, shardy: you win, I removed -2 | 13:58 |
| EmilienM | but I remain sad | 13:58 |
| shardy | EmilienM: ack, thanks - I agree we need the coverage, lets try to get it asap | 13:59 |
| *** morazi has joined #tripleo | 13:59 | |
| matbu_ | EmilienM: hey thx, i added some stuffs regarding update/upgrade in the tripleo-ci-squad etherpad | 14:00 |
| *** yamahata has quit IRC | 14:01 | |
| *** links has quit IRC | 14:02 | |
| marios | EmilienM: on a call right now but thanks ... as in the mail i think we all agree that it is wrong we don't have that coverage but right now it isn't feasible in time i think | 14:02 |
| *** gbarros has joined #tripleo | 14:04 | |
| *** abregman|mtg has joined #tripleo | 14:05 | |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common master: Bring the keystone utils up-to-date with Mistral https://review.openstack.org/508165 | 14:05 |
| thrash | honza: ^^^ | 14:05 |
| thrash | honza: that should solve the swift log upload problem. | 14:06 |
| honza | thrash: /me looks! | 14:06 |
| *** ebarrera has quit IRC | 14:06 | |
| *** ebarrera has joined #tripleo | 14:06 | |
| honza | thrash: perfect, i'll test it out shortly | 14:07 |
| *** artom has quit IRC | 14:07 | |
| *** yolanda has quit IRC | 14:08 | |
| *** jaosorior has joined #tripleo | 14:08 | |
| *** rbowen has quit IRC | 14:08 | |
| *** yolanda has joined #tripleo | 14:08 | |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 14:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 14:10 |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common master: Bring the keystone utils up-to-date with Mistral https://review.openstack.org/508165 | 14:10 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements https://review.openstack.org/503679 | 14:11 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: Keep floating ip reachability during pacemaker migration. https://review.openstack.org/474967 | 14:11 |
| lvdombrkr | shardy: any ideas why all in one deployment success but when i ty deploy compute and controller separate nodes its stucks on compute deployment? | 14:12 |
| shardy | lvdombrkr: what do the logs say went wrong? As I said earlier you'll probably see why if you tail /var/log/messages on the nodes | 14:14 |
| shardy | without more info it's impossible to say why it's not working | 14:15 |
| *** psahoo has quit IRC | 14:16 | |
| *** jrist has quit IRC | 14:17 | |
| *** dpawar has quit IRC | 14:24 | |
| *** jfrancoa has quit IRC | 14:24 | |
| *** dsariel has joined #tripleo | 14:25 | |
| *** oidgar has quit IRC | 14:25 | |
| EmilienM | marios, matbu_, shardy: I created https://bugs.launchpad.net/tripleo/+bug/1720153 with "tech-debt" tag, so we can track the work being done on that front | 14:27 |
| openstack | Launchpad bug 1720153 in tripleo "No CI coverage for minor updates" [Critical,Triaged] | 14:27 |
| openstackgerrit | Merged openstack/puppet-tripleo stable/pike: Disables port status updates with ODL in HA https://review.openstack.org/508034 | 14:27 |
| marios | EmilienM: ack | 14:29 |
| *** abregman|mtg has quit IRC | 14:33 | |
| *** mrch has quit IRC | 14:34 | |
| *** ffiore has quit IRC | 14:36 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-common master: Config download support for all deployments https://review.openstack.org/508188 | 14:37 |
| EmilienM | shardy, marios, matbu_ : do we have a document that describe the steps for minor updates? I want to start looking at quickstart roles and see how can I help to bring this new ci coverage | 14:37 |
| openstackgerrit | James Slagle proposed openstack/tripleo-common master: Config download support for all deployments https://review.openstack.org/508189 | 14:38 |
| *** chlong has joined #tripleo | 14:38 | |
| *** ffiore has joined #tripleo | 14:40 | |
| *** oidgar has joined #tripleo | 14:41 | |
| *** jfrancoa has joined #tripleo | 14:41 | |
| weshay | EmilienM, I'm attempting to put together a list of items that the DF-Upgrades and CI team need to coordinate on. Not sure if it's ready for an in depth review but I wanted to make you aware of it | 14:43 |
| weshay | EmilienM, https://etherpad.openstack.org/p/tripleo-ci-squad-meeting lines 25-52 | 14:43 |
| weshay | morazi, ^ | 14:43 |
| -openstackstatus- NOTICE: The infra team is now taking Zuul v2 offline and bringing Zuul v3 online. Please see https://docs.openstack.org/infra/manual/zuulv3.html for more information, and ask us in #openstack-infra if you have any questions. | 14:44 | |
| *** mdnadeem has quit IRC | 14:44 | |
| shardy | EmilienM: I think when https://review.openstack.org/#/c/487488/ lands it'll work the same as before containers/ansible, except I'm not sure if interactive mode will work the same | 14:45 |
| EmilienM | chandankumar: why setUpClass (tempest.scenario.test_encrypted_cinder_volumes is still ignored? | 14:46 |
| EmilienM | http://logs.openstack.org/28/499928/14/check/gate-tripleo-ci-centos-7-scenario002-multinode-oooq-container/f805d0e/logs/tempest.html.gz | 14:46 |
| EmilienM | chandankumar: on https://review.openstack.org/#/c/499928/ | 14:46 |
| marios | EmilienM: i know yurii had an etherpad, but we don't have full docs yet. i added a note https://review.openstack.org/#/c/496223/5 (see my latest comment) which should give you some idea | 14:47 |
| EmilienM | chandankumar: ah you missed the container featureset | 14:47 |
| shardy | EmilienM: so basically we need to deploy multiple controller w/HA and containers via quickstart, then re-run the container image prepare to switch the versions of the containers, then run openstack overcloud update stack | 14:47 |
| shardy | EmilienM: we'll also want some way to validate e.g run tempest/pingtest before after the update, and ideally check the state of the services during the update, as they should remain available | 14:48 |
| morazi | jfrancoa, ^^ you might be interested in the therpad weshay mentioned above | 14:48 |
| shardy | it'd also be nice to check the containers we expected to update actually did | 14:48 |
| shardy | e.g check before/after docker ps output | 14:48 |
| EmilienM | marios: ah nice | 14:48 |
| shardy | matbu_/marios may have other ideas also | 14:48 |
| shardy | marios: that's about upgrades though right? | 14:49 |
| *** ffiore has quit IRC | 14:50 | |
| marios | shardy: i was pointing at https://review.openstack.org/#/c/496223/5 my last comment on there is about running the minor update using matbu patches | 14:50 |
| EmilienM | marios, shardy: it would be great to have one of you working on the doc asap, even WIP but at least something we can use as reference for now | 14:50 |
| marios | EmilienM: yeah i will try and update matbu review tomorrow with that | 14:50 |
| shardy | marios: aha, thanks I was looking at the patch itself | 14:50 |
| shardy | +1 to adding it to the docs and quickstart | 14:50 |
| chandankumar | EmilienM: yes i was not looking at container featureset | 14:52 |
| EmilienM | marios, shardy: wdyt if I patch project-config with a new CI job, experimental / non voting on THT that would run minor updates | 14:52 |
| EmilienM | chandankumar: please do | 14:52 |
| openstackgerrit | Or Idgar proposed openstack/tripleo-heat-templates master: WIP Calling Octavia post deploy from overcloud stack https://review.openstack.org/508195 | 14:52 |
| shardy | EmilienM: sounds good - perhaps the 3nodes job would be a good starting point? | 14:53 |
| marios | pabelanger: o/ hi sorry to disturb, when you next get some review time please check this? https://review.openstack.org/#/c/505574/ it is for https://bugs.launchpad.net/tripleo/+bug/1718387 thank you! | 14:53 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 14:53 |
| jfrancoa | morazi: thanks, matbu_ brought that etherpad up during the scrum meeting. I can try giving a hand with it, as some of those task are a priority | 14:53 |
| EmilienM | shardy: why 3nodes, because of HA? | 14:53 |
| EmilienM | shardy: 3nodes sounds heavy imho | 14:53 |
| shardy | EmilienM: yeah | 14:53 |
| shardy | EmilienM: well as I said on the ML, it's not really a good test unless we test the zero downtime aspect | 14:53 |
| shardy | which means running 3 nodes (although I think the 3nodes job is 1UC plus two nodes?) | 14:54 |
| EmilienM | shardy: I'm trying to do baby steps but ok | 14:54 |
| EmilienM | yes, it's 1UC / 2OC | 14:54 |
| shardy | EmilienM: ack, sure running a single node update would be better than nothing for sure | 14:54 |
| EmilienM | shardy: we could start small for now, having updates on one node overcloud | 14:55 |
| EmilienM | shardy: and iterate | 14:55 |
| shardy | I just know we've been bitten by the bouncing services during minor updates more than once | 14:55 |
| shardy | EmilienM: sure, sounds good | 14:55 |
| mwhahaha | we currently don't have a method to detect the multiple restarts of services other than manually going and looking at the puppet steps in the logs | 14:55 |
| mwhahaha | it would be nice to be able to automate that checking | 14:56 |
| shardy | mwhahaha: yeah, that's why above I was saying it'd be nice to have some way to check the expected containers get replaced before/after | 14:56 |
| chandankumar | EmilienM: does container scenario jobs come from different featureset? | 14:56 |
| EmilienM | chandankumar: yes | 14:56 |
| EmilienM | chandankumar: 1 job == 1 FS | 14:56 |
| mwhahaha | with containerization is becomes easier because we could check the container timestamps | 14:56 |
| shardy | we could probably compare the hashes in the env file from overcloud image prepare with the docker os output | 14:57 |
| shardy | mwhahaha: yeah the timestamps and hashes should help | 14:57 |
| EmilienM | chandankumar: FS17 | 14:57 |
| chandankumar | yes | 14:58 |
| chandankumar | updating | 14:58 |
| lvdombrkr | shardy: can you look into quickly if you have some sec. http://paste.openstack.org/raw/622164/ | 14:58 |
| *** MVenesio has quit IRC | 14:58 | |
| *** jrist has joined #tripleo | 14:59 | |
| *** jrist has joined #tripleo | 14:59 | |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart master: Add compute-feature-enabled.attach_encrypted_volume to True https://review.openstack.org/499928 | 15:00 |
| chandankumar | EmilienM: ^^ | 15:00 |
| *** iranzo has quit IRC | 15:00 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/ocata: Addition of Nuage as mechanism driver for ML2 https://review.openstack.org/492224 | 15:00 |
| *** iranzo has joined #tripleo | 15:01 | |
| shardy | lvdombrkr: [DEBUG] Running /usr/libexec/heat-config/hooks/puppet < /var/lib/heat-config/deployed/936f8437-0f5f-4a5e-b58d-c6f74912caa1.json shows that it's probably running puppet | 15:01 |
| shardy | lvdombrkr: does ps ax | grep puppet show a puppet apply running? | 15:02 |
| EmilienM | chandankumar: thx | 15:02 |
| shardy | or ps ax | grep heat should show the heat hook running | 15:02 |
| *** jlinkes has quit IRC | 15:02 | |
| janki | shardy, hey. you there? | 15:03 |
| shardy | janki: hi | 15:03 |
| lvdombrkr | shardy: run commands on compute node, yes? | 15:03 |
| shardy | lvdombrkr: yes | 15:03 |
| janki | shardy, I made changes to ODL THT. md5sum is created now but still no hash for odl image | 15:03 |
| janki | "opendaylight_api": {"start_order": 0, "image": "192.168.24.1:8787/tripleoupstream/centos-binary-opendaylight:custom", "environment": ["KOLLA_CONFIG_STRATEGY=COPY_ALWAYS"], "user": "odl", "volumes": ["/etc/hosts:/etc/hosts:ro", "/etc/localtime:/etc/localtime:ro", "/etc/puppet:/etc/puppet:ro", "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro", "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro", "/etc/pki/tls/certs/ca-bundle.trus | 15:04 |
| janki | t.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", "/dev/log:/dev/log", "/etc/ssh/ssh_known_hosts:/etc/ssh/ssh_known_hosts:ro", "/sys/fs/selinux:/sys/fs/selinux", "/var/lib/kolla/config_files/opendaylight_api.json:/var/lib/kolla/config_files/config.json:ro", "/var/lib/config-data/puppet-generated/opendaylight_api/:/var/lib/kolla/config_files/src:ro", "/var/log/containers/opendaylight:/opt/opendaylight/data/log | 15:04 |
| janki | ", "/var/lib/opendaylight/journal:/opt/opendaylight/journal", "/var/lib/opendaylight/snapshots:/opt/opendaylight/snapshots"], "net": "host", "detach": true, "privileged": false, "restart": "always"}, | 15:04 |
| lvdombrkr | shardy: http://paste.openstack.org/raw/622168/ | 15:04 |
| shardy | janki: please use paste.openstack.org | 15:05 |
| janki | shardy, soory. I didnot think it owuld take so many lines | 15:05 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: add pike release https://review.openstack.org/508198 | 15:05 |
| shardy | janki: also please add ls /var/lib/config-data, thanks | 15:05 |
| *** jaganathan has quit IRC | 15:05 | |
| shardy | lvdombrkr: try sudo before the ps? The log says puppet should be running | 15:07 |
| *** ffiore has joined #tripleo | 15:08 | |
| lvdombrkr | shardy: same result... on compute node screen i see something like : https://pasteboard.co/GMtnmIo.jpg | 15:10 |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 15:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 15:10 |
| *** MVenesio has joined #tripleo | 15:12 | |
| EmilienM | shardy, marios, matbu_ : also FYI project-config is in freeze right now due to zuul v3 migration - so we won't make much progress today - I'll still push and try to get bits ready | 15:12 |
| *** jaosorior has quit IRC | 15:12 | |
| shardy | EmilienM: ack, I guess we can start the quickstart support anyway and wait for it to unfreeze | 15:12 |
| shardy | EmilienM: thanks for the help :) | 15:12 |
| janki | shardy, https://pastebin.com/32jwk1pc | 15:13 |
| *** jfrancoa has quit IRC | 15:13 | |
| *** dbecker has joined #tripleo | 15:13 | |
| lvdombrkr | shardy: tried one more time now i see : http://paste.openstack.org/raw/622170/ | 15:14 |
| shardy | lvdombrkr: Ok so puppet is running - what makes you think the deployment is stuck, how long has it been running for? | 15:15 |
| *** atoth has quit IRC | 15:15 | |
| *** atoth has joined #tripleo | 15:16 | |
| *** jaganathan has joined #tripleo | 15:18 | |
| shardy | janki: Ok I'm not sure why it's not working, you may have to add some debug to docker-puppet.py and run it manually | 15:19 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/docker/docker-puppet.py#L372 | 15:19 |
| *** gbarros has quit IRC | 15:19 | |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/common/deploy-steps-tasks.yaml#L31 | 15:19 |
| shardy | janki: see there you can export the same environment variables then run it on the CLI, and add as much logging as you need to figure out why the hash isn't updated | 15:20 |
| *** aditya_r has quit IRC | 15:20 | |
| *** aditya_r has joined #tripleo | 15:21 | |
| *** gbarros has joined #tripleo | 15:21 | |
| janki | shardy, i was just going thorugh docker-puppet.py. logically it seems to have worked. I am trying to run that part in local python compiler. lets see what happens | 15:21 |
| *** psachin has quit IRC | 15:21 | |
| janki | shardy, do you how can I run that withoout having to deploy overcloud everytime? | 15:22 |
| shardy | janki: yes I just linked how it's run via ansible, you can just do the same on the CLI | 15:22 |
| shardy | janki: export the environment variables, then run the script - you could optionally copy/modify the json to only contain the odl stuff | 15:23 |
| lvdombrkr | shardy: i was waiting for 2 hours aprox. when deployed first time. now i run deploy second time and stuck on overcloud.AllNodesDeploySteps.ControllerDeployment_Step4] same as first time: http://paste.openstack.org/raw/622173/ | 15:23 |
| janki | shardy, sorry. didtnt see that link before. thanks. will try this | 15:23 |
| *** ykarel has quit IRC | 15:24 | |
| sri_ | @EmilienM: hi, it seems beagles isn't around | 15:24 |
| shardy | janki: np, more here https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/tips_tricks.html#debugging-docker-puppet-py | 15:24 |
| beagles | sri_, I'm here | 15:24 |
| lvdombrkr | shardy: and if i ssh compute node i see that there is no vlan, bridges created etc. as i understood on this steps bridges and vlans must be created already | 15:24 |
| shardy | lvdombrkr: yes that probably indicates an issue with your network templates, check /etc/os-net-config/config.json looks as you expect | 15:25 |
| beagles | sri_ you had questions regarding l3 ha and dvr? | 15:25 |
| *** shardy is now known as shardy_afk | 15:25 | |
| *** alee has quit IRC | 15:26 | |
| sri_ | beagles: hi, sorry , yes , when we enable the DVR l3 was disable right ? | 15:26 |
| beagles | sri_, by default - yes that is correct | 15:26 |
| beagles | sri_, you *can* enable both however if you wanted | 15:26 |
| *** shreshtha has joined #tripleo | 15:27 | |
| beagles | sri_, there were some issues with L3 HA and DVR so we set the defaults up that way | 15:27 |
| sri_ | beagles: ok , do i need pass any env to enable both | 15:27 |
| beagles | sri_, yep | 15:27 |
| *** shreshtha has quit IRC | 15:27 | |
| *** shreshtha has joined #tripleo | 15:27 | |
| sri_ | beagles: is there known issue we need to look for when we enable both | 15:28 |
| beagles | sri_, I've been trying to "recall" (i.e. search old emails) what the HA/DVR issues were | 15:29 |
| *** oidgar has quit IRC | 15:29 | |
| beagles | sri_, there used to be - I'll let you know as soon as I sort it out | 15:29 |
| beagles | (should have kept notes on that) | 15:29 |
| sri_ | beagles: cool , how can enable l3 ha with dvr ? any clue | 15:30 |
| beagles | sri_, are you familiar with how dvr is enabled? | 15:31 |
| sri_ | yes | 15:31 |
| *** jaganathan has quit IRC | 15:31 | |
| sri_ | beagles: environment/neutron-ovs-dvr.yaml | 15:32 |
| beagles | sri_, add to an environment file included on the command line NeutronL3HA: true | 15:32 |
| sri_ | beagles: ok, got it | 15:33 |
| *** jkilpatr has quit IRC | 15:34 | |
| sri_ | beagles: thanks :) | 15:34 |
| beagles | sri_, np | 15:34 |
| *** marios has quit IRC | 15:35 | |
| *** jrist has quit IRC | 15:36 | |
| *** ukalifon has quit IRC | 15:37 | |
| *** anshul has quit IRC | 15:39 | |
| lvdombrkr | shardy: /os-net-config/config.json are empty in compute and controller, but in controller network configuration configured as expected | 15:41 |
| lvdombrkr | in compute not as i mentioned | 15:41 |
| *** iranzo has quit IRC | 15:41 | |
| openstackgerrit | Ana Krivokapic proposed openstack/tripleo-ui master: Validate extension for tarballs on plan create/update https://review.openstack.org/508145 | 15:44 |
| *** atoth has quit IRC | 15:44 | |
| *** rcernin has quit IRC | 15:46 | |
| *** MVenesio has quit IRC | 15:50 | |
| *** ramishra has quit IRC | 15:50 | |
| openstackgerrit | Ana Krivokapic proposed openstack/tripleo-ui master: Validate extension for tarballs on plan create/update https://review.openstack.org/508145 | 15:51 |
| weshay | EmilienM, mwhahaha, matbu_ we need to run upgrade ci in rdo sf to avoid timeouts https://review.rdoproject.org/r/9823 | 15:55 |
| weshay | rlandy, fyi.. please review my jjb https://review.rdoproject.org/r/9823 | 15:55 |
| mwhahaha | weshay: for master? | 15:55 |
| *** abregman has joined #tripleo | 15:57 | |
| weshay | mwhahaha, so looking at the upstream jobs... they are running on master and stable/pike | 15:57 |
| *** ffiore has quit IRC | 15:57 | |
| weshay | stable/pike would be the preference I assume | 15:58 |
| mwhahaha | weshay: ok, i asked because it might be nice to also move the ocata ones (since we're basically blocked on those) | 15:58 |
| weshay | mwhahaha, I can update this to run stable/ocata stable/pike | 15:58 |
| mwhahaha | and i'm half tempted to just switch them to NV cause it's been months and no one has done anything | 15:58 |
| *** egonzalez has quit IRC | 15:59 | |
| weshay | mwhahaha, I'm tempted to move them in whole to sf | 15:59 |
| weshay | due to timeouts | 15:59 |
| mwhahaha | would make upstream happy | 15:59 |
| mwhahaha | less jobs in infra :D | 15:59 |
| *** cdearborn_ has joined #tripleo | 16:00 | |
| *** ffiore has joined #tripleo | 16:00 | |
| *** ffiore has quit IRC | 16:01 | |
| *** cdearborn__ has joined #tripleo | 16:01 | |
| EmilienM | weshay: cool | 16:02 |
| EmilienM | thanks! | 16:02 |
| mwhahaha | if we do that we can get that minor updates job in | 16:02 |
| *** tongl has joined #tripleo | 16:02 | |
| EmilienM | mwhahaha: most probably, I'm afraid they'll timeout as well in upstream infra | 16:03 |
| *** tongl has quit IRC | 16:03 | |
| *** cdearborn has quit IRC | 16:04 | |
| *** cdearborn_ is now known as cdearborn | 16:04 | |
| EmilienM | mwhahaha: the container prepare thing takes forever | 16:04 |
| *** itlinux has joined #tripleo | 16:04 | |
| mwhahaha | oh right | 16:04 |
| weshay | EmilienM, mwhahaha ok.. updated the job for stable/pike and stable/ocata | 16:04 |
| *** tosky is now known as ri | 16:04 | |
| *** ri is now known as tosky | 16:04 | |
| EmilienM | weshay: I commented | 16:08 |
| weshay | EmilienM, as always.. good comments | 16:10 |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 16:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 16:10 |
| *** jrist has joined #tripleo | 16:10 | |
| *** kbyrne has quit IRC | 16:10 | |
| weshay | dmsimard, question for you.. I need to add upgrade jobs to rdo sf. Should we jump right in and have them running on every patch or would you rather see them proved out w/ check-experimental? | 16:11 |
| weshay | https://review.rdoproject.org/r/#/c/9823/ | 16:11 |
| *** kbyrne has joined #tripleo | 16:12 | |
| *** oidgar has joined #tripleo | 16:13 | |
| weshay | EmilienM, let me run them once or twice w/ experimental | 16:13 |
| *** gbarros has quit IRC | 16:13 | |
| EmilienM | weshay: ok, wfm | 16:14 |
| *** nyechiel_ has joined #tripleo | 16:14 | |
| weshay | EmilienM, this is where we should experiment w/ new upgrade jobs.. FYI.. matbu_ | 16:14 |
| EmilienM | yeah | 16:14 |
| EmilienM | weshay: even for minor updates | 16:15 |
| weshay | ya | 16:15 |
| *** itlinux has quit IRC | 16:15 | |
| *** nyechiel has quit IRC | 16:15 | |
| *** gbarros has joined #tripleo | 16:17 | |
| weshay | jistr, FYI ^ | 16:17 |
| *** gvrangan has joined #tripleo | 16:19 | |
| *** alee has joined #tripleo | 16:21 | |
| EmilienM | matbu_: it would have been nice to have a release note on https://review.openstack.org/#/c/487496 | 16:23 |
| EmilienM | but my commented was ignored :( | 16:23 |
| *** aditya_r has quit IRC | 16:24 | |
| *** jaganathan has joined #tripleo | 16:27 | |
| matbu_ | EmilienM: i'll add a patch with the release note on top of this patch | 16:28 |
| EmilienM | matbu_: thank you! | 16:29 |
| *** itlinux has joined #tripleo | 16:29 | |
| *** ebarrera has quit IRC | 16:29 | |
| *** nyechiel_ has quit IRC | 16:29 | |
| *** nyechiel_ has joined #tripleo | 16:30 | |
| *** salmankhan has quit IRC | 16:37 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack-infra/tripleo-ci master: Make sure we always have /etc/machine-id available. https://review.openstack.org/508226 | 16:38 |
| *** panda is now known as panda|off | 16:38 | |
| janki | shardy_afk, hey | 16:43 |
| *** karts has joined #tripleo | 16:44 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Config download support for standalone deployments https://review.openstack.org/505827 | 16:45 |
| *** trown is now known as trown|lunch | 16:46 | |
| *** tbonds has joined #tripleo | 16:46 | |
| *** numans has quit IRC | 16:46 | |
| *** karthiks has quit IRC | 16:47 | |
| *** rwsu has joined #tripleo | 16:47 | |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-common master: Add release note for minor update workflow https://review.openstack.org/508233 | 16:48 |
| matbu_ | EmilienM: https://review.openstack.org/#/c/508233/ | 16:48 |
| matbu_ | EmilienM: so you can now A+ the patch :p | 16:49 |
| *** shreshtha has quit IRC | 16:49 | |
| *** numans has joined #tripleo | 16:49 | |
| weshay | matbu_, is the code for minor update in the upgrade role? | 16:50 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: Keep floating ip reachability during pacemaker migration. https://review.openstack.org/474967 | 16:51 |
| *** ffiore has joined #tripleo | 16:51 | |
| *** salmankhan has joined #tripleo | 16:52 | |
| *** jlinkes has joined #tripleo | 16:54 | |
| *** jaganathan has quit IRC | 16:55 | |
| *** derekh has quit IRC | 16:56 | |
| *** bogdando has quit IRC | 16:56 | |
| *** abregman has quit IRC | 16:57 | |
| *** jpich has quit IRC | 16:59 | |
| *** MVenesio has joined #tripleo | 17:02 | |
| *** shreshtha has joined #tripleo | 17:02 | |
| *** oidgar has quit IRC | 17:05 | |
| *** ffiore has quit IRC | 17:06 | |
| matbu_ | weshay: i don't think | 17:06 |
| *** salmankhan has quit IRC | 17:07 | |
| weshay | matbu_, is it in the new role? | 17:07 |
| matbu_ | weshay: the minor update workflow ? it need to check but i don't think | 17:08 |
| *** jaganathan has joined #tripleo | 17:08 | |
| matbu_ | weshay: it was hard to automate until it hasn't merged because the patch of ooo-common wasnt easy | 17:08 |
| *** dbecker has quit IRC | 17:08 | |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 17:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 17:10 |
| *** dpawar has joined #tripleo | 17:10 | |
| sri_ | beagles: I've deployed overcloud with dvr and l3 ha enabled , as for as i can see i didn't found any issue in my environment :) | 17:15 |
| *** lblanchard has quit IRC | 17:16 | |
| sri_ | beagles: i am testing stable/ocata release | 17:17 |
| *** dsneddon has joined #tripleo | 17:18 | |
| *** dsneddon has quit IRC | 17:18 | |
| *** dsneddon has joined #tripleo | 17:18 | |
| *** karts has quit IRC | 17:25 | |
| *** numans has quit IRC | 17:26 | |
| openstackgerrit | Ian Main proposed openstack/tripleo-heat-templates master: Explicitly set healthcheck command. https://review.openstack.org/502843 | 17:26 |
| *** numans has joined #tripleo | 17:31 | |
| *** aditya_r has joined #tripleo | 17:31 | |
| dmsimard | weshay: whatever makes it easier for you | 17:32 |
| dmsimard | check experimental is just a way to easily iterate without running "legit" things, it's not something I require :p | 17:32 |
| *** vpickard has quit IRC | 17:35 | |
| *** karts has joined #tripleo | 17:38 | |
| mwhahaha | EmilienM: is https://bugs.launchpad.net/tripleo/+bug/1714905 done? | 17:43 |
| openstack | Launchpad bug 1714905 in tripleo "Composable scenarios BM -> Containers Upgrade jobs never deploy on BM" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 17:43 |
| *** jpena is now known as jpena|off | 17:47 | |
| beagles | sri_, cool! | 17:47 |
| *** itlinux has quit IRC | 17:51 | |
| *** achadha has joined #tripleo | 17:53 | |
| *** itlinux has joined #tripleo | 17:53 | |
| *** vpickard has joined #tripleo | 17:55 | |
| EmilienM | mwhahaha: it's wip, https://etherpad.openstack.org/p/tripleo-containers-ha-ci | 17:56 |
| EmilienM | mwhahaha: almost done | 17:56 |
| *** jlinkes has quit IRC | 17:56 | |
| mwhahaha | k | 17:56 |
| sri_ | beagles: i think before mitaka release we can use l3 ha with dvr it's not possible on neutron leval , form newton we can use l3 ha with (snat ha) dvr . | 17:56 |
| sri_ | beagles: when we enable dvr in overcloud we also keep the l3 ha by default. | 17:57 |
| mwhahaha | myoung: is Bug 1718000 done? Looks like the rdo stuf merged | 17:58 |
| openstack | bug 1718000 in tripleo "tracking bug for RDO packaging issue blocking rdophase2 promotion: python-cheetah dependencies" [Critical,Triaged] https://launchpad.net/bugs/1718000 - Assigned to Alan Pevec (apevec) | 17:58 |
| *** lblanchard has joined #tripleo | 17:59 | |
| *** akrivoka has quit IRC | 18:03 | |
| myoung | mwhahaha, weshay, jschlueter: ^^ tracking bug for python-cheetah issue, afaik we're using this until we see these changes in a promoted RDO hash. at that point we'll disable the side repos in CI presently being used for RDO on RHEL in ospphase0. | 18:03 |
| myoung | mwhahaha, weshay, jschlueter: I'm a kindertaxi for 1 hour, back at ~3 EDT and can chase down if this has been promoted yet | 18:03 |
| mwhahaha | myoung: thanks | 18:03 |
| myoung | mwhahaha, weshay, jschlueter, apevec: I haven't used a tracking bug in tripleo for an issue like this before, I'm not sure what precedent is, or when we should close the LP. | 18:04 |
| mwhahaha | myoung: if all the work has been completed then it should be switched to fix released | 18:05 |
| *** dprince has quit IRC | 18:05 | |
| *** dprince has joined #tripleo | 18:07 | |
| myoung | mwhahaha, weshay: makes sense, sounds like it's time for the LP to change state then if work is done from RDO/packaging side | 18:07 |
| mwhahaha | myoung: but if there are still elements to be done to ensure it's in ci then we should wait for that. I don't know enough about the current state of the issue which is why i asked :D | 18:08 |
| myoung | weshay, jschlueter ^^ could you please update? | 18:08 |
| *** cdearborn has quit IRC | 18:10 | |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 18:10 |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 18:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 18:10 |
| openstackgerrit | Andy Smith proposed openstack/tripleo-common master: WIP Update to separate messaging backends https://review.openstack.org/508259 | 18:12 |
| *** dprince has quit IRC | 18:13 | |
| *** trown|lunch is now known as trown | 18:13 | |
| *** dprince has joined #tripleo | 18:14 | |
| *** jkilpatr has joined #tripleo | 18:14 | |
| *** iranzo has joined #tripleo | 18:15 | |
| *** iranzo has joined #tripleo | 18:15 | |
| *** amoralej is now known as amoralej|off | 18:15 | |
| *** aditya_r is now known as aditya_r|gn | 18:17 | |
| openstackgerrit | Andy Smith proposed openstack/tripleo-heat-templates master: WIP Separate rpc and notify messaging backends https://review.openstack.org/507963 | 18:18 |
| *** bogdando has joined #tripleo | 18:19 | |
| *** itlinux has quit IRC | 18:20 | |
| *** nyechiel_ has quit IRC | 18:20 | |
| *** sshnaidm is now known as sshnaidm|off | 18:21 | |
| *** sri_ has quit IRC | 18:22 | |
| *** aditya_r|gn has quit IRC | 18:22 | |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: Exclude remove-machine-id from bootstrap https://review.openstack.org/508263 | 18:22 |
| *** iranzo has quit IRC | 18:26 | |
| *** dprince has quit IRC | 18:31 | |
| *** dpawar has quit IRC | 18:35 | |
| *** itlinux has joined #tripleo | 18:36 | |
| *** bogdando has quit IRC | 18:36 | |
| *** atoth has joined #tripleo | 18:38 | |
| Goneri | could I get a review on this: https://review.openstack.org/#/c/507889/ it's a trivial patch, but it's a blocker for us (DCI). | 18:44 |
| *** janki has quit IRC | 18:45 | |
| *** janki has joined #tripleo | 18:48 | |
| *** salmankhan has joined #tripleo | 18:50 | |
| *** salmankhan has quit IRC | 18:55 | |
| *** ansmith has quit IRC | 18:59 | |
| *** MVenesio has quit IRC | 18:59 | |
| *** dprince has joined #tripleo | 19:01 | |
| *** sri_ has joined #tripleo | 19:01 | |
| *** itlinux has quit IRC | 19:02 | |
| *** gbarros has quit IRC | 19:03 | |
| *** shreshtha has quit IRC | 19:04 | |
| *** bnemec has quit IRC | 19:07 | |
| *** gvrangan has quit IRC | 19:08 | |
| *** assassin has quit IRC | 19:09 | |
| *** assassin has joined #tripleo | 19:09 | |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 19:10 |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 19:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 19:10 |
| *** bnemec has joined #tripleo | 19:13 | |
| mwhahaha | zuulv3'd | 19:14 |
| * mwhahaha cries | 19:14 | |
| *** ansmith has joined #tripleo | 19:14 | |
| EmilienM | lol | 19:15 |
| *** aputtur has quit IRC | 19:16 | |
| *** bnemec has quit IRC | 19:16 | |
| *** assassin has quit IRC | 19:16 | |
| *** assassin has joined #tripleo | 19:17 | |
| *** mwhahaha changes topic to "CI Status: RED - zuul v3'd (see alerts) | http://tripleo.org/ | https://docs.openstack.org/tripleo-docs/latest/" | 19:19 | |
| dmsimard | mwhahaha, EmilienM: let me know if you need v3 help | 19:21 |
| mwhahaha | dmsimard: you broke the world | 19:22 |
| mwhahaha | plz fix | 19:22 |
| dmsimard | please be more specific | 19:22 |
| dmsimard | what world | 19:22 |
| dmsimard | :) | 19:22 |
| mwhahaha | all tripleo legacy jobs | 19:22 |
| mwhahaha | dmsimard: http://logs.openstack.org/89/507889/2/gate/legacy-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/a8829f3/job-output.txt.gz#_2017-09-28_19_06_33_406425 | 19:22 |
| mwhahaha | fails on all | 19:22 |
| dmsimard | ok, one thing at a time. Let's look. | 19:22 |
| mwhahaha | see https://review.openstack.org/#/c/507889/ | 19:22 |
| mwhahaha | all failed with same thing | 19:22 |
| dmsimard | that's a devstack-gate thing | 19:23 |
| dmsimard | is anyone working on that ? | 19:23 |
| dmsimard | I can take a stab at it | 19:23 |
| mwhahaha | i asked in infra | 19:23 |
| mwhahaha | not so much on the answer | 19:23 |
| * mwhahaha creates bug for tracking purposes | 19:24 | |
| EmilienM | they're probably busy | 19:24 |
| EmilienM | they're usually responsive | 19:24 |
| *** gbarros has joined #tripleo | 19:24 | |
| mwhahaha | yea i'm sure | 19:24 |
| mwhahaha | well they said something not sure if anyone is actually looking due to other things | 19:24 |
| *** itlinux has joined #tripleo | 19:25 | |
| mwhahaha | https://bugs.launchpad.net/tripleo/+bug/1720220 | 19:26 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs faiing on gate_hook: not found" [Critical,Triaged] | 19:26 |
| mwhahaha | for the tracking | 19:26 |
| dmsimard | mwhahaha: looks like they're busy with other things | 19:28 |
| dmsimard | I'll try and see if I can understand what's going on | 19:28 |
| * EmilienM wonders if mwhahaha has "l" on his keyboard | 19:28 | |
| * mwhahaha ooks | 19:28 | |
| * EmilienM hides | 19:28 | |
| *** jaganathan has quit IRC | 19:28 | |
| mwhahaha | nope i don't have it | 19:28 |
| dmsimard | mwhahaha: so that job is https://github.com/openstack-infra/openstack-zuul-jobs/blob/master/zuul.d/zuul-legacy-jobs.yaml#L16755 | 19:29 |
| EmilienM | hey Aex! | 19:29 |
| mwhahaha | dmsimard: yea i'm looking at teh ansible bits | 19:29 |
| dmsimard | mwhahaha: and that job runs this https://github.com/openstack-infra/openstack-zuul-jobs/blob/master/playbooks/legacy/tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/run.yaml | 19:30 |
| mwhahaha | yea gate book is https://github.com/openstack-infra/openstack-zuul-jobs/blob/master/playbooks/legacy/tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/run.yaml#L90 | 19:30 |
| mwhahaha | but then it's lost | 19:30 |
| mwhahaha | so something is not properly exporting it | 19:30 |
| *** dbecker has joined #tripleo | 19:31 | |
| *** pcaruana has quit IRC | 19:31 | |
| dmsimard | I'll try and reproduce locally | 19:34 |
| *** bnemec has joined #tripleo | 19:35 | |
| mwhahaha | dmsimard: so in the logs http://logs.openstack.org/89/507889/2/gate/legacy-tripleo-ci-centos-7-scenario004-multinode-oooq-puppet/efd5b46/job-output.txt.gz#_2017-09-28_19_04_28_151301 | 19:36 |
| mwhahaha | dmsimard: i don't see teh bits before it http://logs.openstack.org/89/507889/2/gate/legacy-tripleo-ci-centos-7-scenario004-multinode-oooq-puppet/efd5b46/job-output.txt.gz#_2017-09-28_19_04_28_151301 | 19:36 |
| mwhahaha | er https://github.com/openstack-infra/openstack-zuul-jobs/blob/master/playbooks/legacy/tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/run.yaml#L90 | 19:36 |
| dmsimard | yeah I see waht you mean | 19:36 |
| mwhahaha | does the function declaration not show up in set+x | 19:36 |
| *** etingof has joined #tripleo | 19:39 | |
| dmsimard | mwhahaha: I used this task: http://paste.openstack.org/show/622198/ and it parses properly http://paste.openstack.org/show/622197/ | 19:39 |
| mwhahaha | dmsimard: so that points to the declaration won't show up in the logs (kinda annoying for troubleshooting this) | 19:40 |
| mwhahaha | so we have to assume it did declare the function and exported it correctly there | 19:40 |
| mwhahaha | but then it's not available in the devstack script | 19:40 |
| *** rhallisey has quit IRC | 19:41 | |
| dmsimard | in my local reproducer, I see the function declaration and export | 19:41 |
| jrist | honza: seen this one? https://paste.fedoraproject.org/paste/oagP0Lo3tVzIIMlCBrxtmw | 19:41 |
| jrist | can't start tripleo-ui on master | 19:41 |
| mwhahaha | dmsimard: it's not in the stderr lines | 19:41 |
| mwhahaha | dmsimard: of http://paste.openstack.org/show/622197/ | 19:42 |
| dmsimard | ah, you're right | 19:42 |
| mwhahaha | so it occurs but doesn't show up in the std output | 19:42 |
| dmsimard | mwhahaha: wait, those are two different tasks | 19:43 |
| mwhahaha | dmsimard: orly which task is being run | 19:43 |
| dmsimard | mwhahaha: ugh, I understand what's going on | 19:44 |
| mwhahaha | dmsimard: k at least you do :D | 19:44 |
| dmsimard | mwhahaha: it's a bit more obvious if you look here instead http://logs.openstack.org/89/507889/2/gate/legacy-tripleo-ci-centos-7-scenario004-multinode-oooq-puppet/efd5b46/ara/result/b67d1480-0bef-460c-b229-462692e55528/ | 19:44 |
| dmsimard | the gate hook not found error comes from the devstack-gate ansible | 19:44 |
| dmsimard | not the zuul ansible | 19:44 |
| mwhahaha | i was wondering if that was the problem | 19:45 |
| dmsimard | so the devstack gate ansible doesn't see that function | 19:45 |
| mwhahaha | or is it failing because /etc/nodepool/primary_node_private is missing | 19:45 |
| mwhahaha | and that's the error of the next step | 19:45 |
| dmsimard | those files are probably there | 19:45 |
| mwhahaha | i know when we had seen that previously it wasn't the problem, it was the host setup failing | 19:45 |
| dmsimard | oh wait they're not anymore in v3 | 19:46 |
| mwhahaha | dmsimard: http://logs.openstack.org/89/507889/2/gate/legacy-tripleo-ci-centos-7-scenario004-multinode-oooq-puppet/efd5b46/job-output.txt.gz#_2017-09-28_19_05_42_399263 | 19:46 |
| mwhahaha | is that what it's failing on? | 19:46 |
| *** eck` is now known as eck`gone | 19:46 | |
| mwhahaha | (if it is, that's not very obvious) | 19:46 |
| dmsimard | mwhahaha: yeah I noticed that just now | 19:46 |
| dmsimard | it doesn't seem too fatal to me | 19:46 |
| mwhahaha | set -e makes that fatal | 19:46 |
| dmsimard | but that's from devstack-gate | 19:47 |
| dmsimard | eh, there's a couple nested layers in there... | 19:47 |
| mwhahaha | need moar complexity | 19:47 |
| * mwhahaha applies ansible | 19:47 | |
| dmsimard | yeah we'll need to move ooo jobs to native ansible asap | 19:47 |
| dmsimard | and out of that legacy thing | 19:48 |
| dmsimard | ok so........ | 19:48 |
| dmsimard | according to http://docs.ansible.com/ansible/latest/shell_module.html | 19:48 |
| dmsimard | it says that the default shell is /bin/sh | 19:48 |
| dmsimard | yet, we use /bin/bash in the legacy playbooks | 19:49 |
| dmsimard | so I suspect that ansible things out of devstack-gate do not set the /bin/bash executable so it doesn't get the env variables from the parent shell | 19:49 |
| dmsimard | I'll try and reproduce locally | 19:49 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-quickstart-extras master: Make UI accessable by default on OVB https://review.openstack.org/507969 | 19:52 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-quickstart-extras master: Sudo fix overcloud-custom-tht-script.sh.j2 https://review.openstack.org/499018 | 19:52 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-quickstart-extras master: Use os_keypair for key upload https://review.openstack.org/445224 | 19:52 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-quickstart-extras master: Allow OVB to attach undercloud to existing floating IP https://review.openstack.org/506397 | 19:52 |
| fultonj | zuul! | 19:54 |
| dmsimard | mwhahaha: ok, that was wrong but I have another hint | 19:55 |
| *** sri_ has quit IRC | 19:56 | |
| *** ccamacho has quit IRC | 19:59 | |
| *** janki has quit IRC | 20:02 | |
| *** chlong has quit IRC | 20:03 | |
| *** ansmith has quit IRC | 20:03 | |
| *** liverpooler has quit IRC | 20:07 | |
| *** catintheroof has quit IRC | 20:09 | |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 20:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 20:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 20:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 20:10 |
| *** weshay has quit IRC | 20:10 | |
| *** weshay has joined #tripleo | 20:11 | |
| *** gfidente has quit IRC | 20:14 | |
| *** ansmith has joined #tripleo | 20:15 | |
| *** eck`gone is now known as eck` | 20:15 | |
| *** toure is now known as toure_biab | 20:16 | |
| dmsimard | mwhahaha: waiting on https://review.openstack.org/#/c/508292/ to merge and then I'll do a recheck to see what happens | 20:23 |
| *** jcoufal has quit IRC | 20:24 | |
| *** leitan has quit IRC | 20:26 | |
| EmilienM | dmsimard: nice spot | 20:27 |
| dmsimard | EmilienM: do you know if there's any non-multinode jobs failing ? | 20:27 |
| * dmsimard opens a few reviews | 20:27 | |
| EmilienM | dmsimard: I haven't checked CI today | 20:27 |
| *** itlinux has quit IRC | 20:30 | |
| Slower | slagle: down to 3 minutes for heat to render templates using a tmpfs sqlite db! | 20:31 |
| mwhahaha | dmsimard: not sure, most of our stuff is multinode | 20:32 |
| mwhahaha | actually i think it's all multinode except the undercloud jobs | 20:33 |
| dmsimard | mwhahaha: for example I found centos-7-ovb-ha-oooq | 20:33 |
| dmsimard | which is single node | 20:33 |
| *** itlinux has joined #tripleo | 20:33 | |
| mwhahaha | dmsimard: but that's on ovb | 20:33 |
| mwhahaha | does that still run the same stuff? | 20:34 |
| *** ecerquei has quit IRC | 20:34 | |
| *** florianf has quit IRC | 20:34 | |
| dprince | slagle: working w/ Slower on the undercloud w/ your patches. we went from 7 minutes to 3 minutes by running heat-all's database (sqlite) in a ramdisk | 20:35 |
| dprince | slagle: we'll push a patch to make this the defualt | 20:35 |
| *** nyechiel_ has joined #tripleo | 20:35 | |
| dprince | slagle: for the undercloud that is quite a difference... | 20:35 |
| dprince | thanks Slower | 20:35 |
| Slower | oh man, gotta keep it cryptic longer :) | 20:36 |
| Slower | hold the suspense | 20:36 |
| slagle | one step closer to the ephemeral Heat :) | 20:38 |
| *** agurenko has quit IRC | 20:39 | |
| *** ansmith has quit IRC | 20:41 | |
| *** raildo has quit IRC | 20:42 | |
| *** rcernin has joined #tripleo | 20:42 | |
| *** ecerquei has joined #tripleo | 20:43 | |
| Slower | it's exciting is what it is | 20:45 |
| *** gbarros has quit IRC | 20:47 | |
| mwhahaha | dmsimard: so is there something we can recheck to see if the first patch fixed it? | 20:50 |
| dmsimard | mwhahaha: should probably wait for https://review.openstack.org/#/c/508301/ to land as well | 20:50 |
| mwhahaha | dmsimard: k | 20:50 |
| dmsimard | first patch was multinode, that is ovb | 20:50 |
| *** ebarrera has joined #tripleo | 20:50 | |
| dmsimard | once the second is landed, a recheck could be good | 20:50 |
| dmsimard | I just don't know what patch to recheck | 20:51 |
| dmsimard | I guess you would know | 20:51 |
| mwhahaha | we could use that one i found this on | 20:51 |
| dmsimard | ideally a patch that isn't supposed to fail :) | 20:51 |
| dmsimard | ok | 20:51 |
| mwhahaha | that was in the gate :D | 20:51 |
| dmsimard | which one was it ? | 20:51 |
| mwhahaha | https://review.openstack.org/#/c/507889/ | 20:51 |
| dmsimard | mwhahaha: hm, no ovb in that one | 20:52 |
| mwhahaha | dmsimard: yea there is | 20:52 |
| dmsimard | not in the gate | 20:52 |
| dmsimard | is there ? | 20:52 |
| mwhahaha | dmsimard: we don't gate on ovb | 20:52 |
| EmilienM | matbu_: probably too late for you but I commented https://review.openstack.org/#/c/487496/ | 20:53 |
| dmsimard | mwhahaha: oh but a recheck would trigger the check queue, so we're fine, ok | 20:53 |
| EmilienM | it's in the gate so probably too late as well | 20:53 |
| mwhahaha | EmilienM: it'll fail :D | 20:53 |
| mwhahaha | oh maybe not | 20:53 |
| mwhahaha | it still has the v2 jobs | 20:54 |
| *** mcornea has joined #tripleo | 20:54 | |
| *** yamahata has joined #tripleo | 20:57 | |
| *** jmelvin has quit IRC | 20:59 | |
| *** lblanchard has quit IRC | 21:00 | |
| *** trown is now known as trown|outtypewww | 21:01 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-common master: Config download support for all deployments https://review.openstack.org/508189 | 21:01 |
| *** ebarrera has quit IRC | 21:02 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: overcloud-deploy: add config-download + ansible run feature https://review.openstack.org/508306 | 21:06 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: fs10: deploy steps with ansible https://review.openstack.org/508307 | 21:09 |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 21:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 21:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 21:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 21:10 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: overcloud-deploy: add config-download + ansible run feature https://review.openstack.org/508306 | 21:10 |
| *** abishop has quit IRC | 21:13 | |
| *** jlinkes has joined #tripleo | 21:14 | |
| dmsimard | mwhahaha: still not merged T_T | 21:19 |
| mwhahaha | booos | 21:19 |
| * dmsimard totally read boobs | 21:19 | |
| mwhahaha | :o | 21:19 |
| openstackgerrit | Gonéri Le Bouder proposed openstack/tripleo-quickstart-extras master: reconfig gitreview to internal git https://review.openstack.org/508315 | 21:20 |
| openstackgerrit | Gonéri Le Bouder proposed openstack/tripleo-quickstart-extras master: validate-tempest: create swiftoperator if missing https://review.openstack.org/508316 | 21:20 |
| mwhahaha | dmsimard: so why doesn't https://review.openstack.org/#/c/508301/ show up on status.o.o/zuul | 21:20 |
| *** nyechiel_ has quit IRC | 21:22 | |
| dmsimard | mwhahaha: I dunno but it's on http://zuulv3.openstack.org/ though | 21:22 |
| mwhahaha | i see | 21:22 |
| mwhahaha | new status page | 21:22 |
| * EmilienM bookmakrs | 21:23 | |
| * mwhahaha prefers the ugly older one | 21:23 | |
| dmsimard | mwhahaha: they probably haven't moved it yet or something | 21:23 |
| dmsimard | I dunno | 21:23 |
| EmilienM | oh wow | 21:23 |
| EmilienM | there is now a console http://zuulv3.openstack.org/static/stream.html?uuid=957e33e5c8154f76b06aeb6a3add77e6&logfile=console.log | 21:23 |
| EmilienM | I'll miss telnet :D | 21:23 |
| dmsimard | yeah, the new webconsole is awesome | 21:23 |
| mwhahaha | dmsimard: when do trusty jobs die? newton eol? | 21:25 |
| dmsimard | I dunno | 21:25 |
| dmsimard | mwhahaha: people can use trusty if they want ? | 21:26 |
| mwhahaha | as we're waiting for the trusty job :D | 21:26 |
| dmsimard | oh | 21:26 |
| dmsimard | hmmm | 21:26 |
| *** itlinux has quit IRC | 21:26 | |
| dmsimard | that particular job will stay there for as long as there's a trusty image in use | 21:26 |
| dmsimard | I wrote it so feel free to complain :D | 21:27 |
| mwhahaha | yea that was my question on when they'll kill trusty fully | 21:27 |
| mwhahaha | i didn't think anyone still used trusty but i figure it's something related to newton | 21:27 |
| *** pchavva has quit IRC | 21:28 | |
| dmsimard | mwhahaha: looks like a bunch of things http://codesearch.openstack.org/?q=trusty&i=nope&files=&repos= | 21:28 |
| mwhahaha | EmilienM: when do incomplete bugs expire? https://bugs.launchpad.net/tripleo/+bug/1405021 | 21:28 |
| openstack | Launchpad bug 1405021 in tripleo "openvswitch sysctl SELinux denials" [High,Incomplete] | 21:28 |
| dmsimard | this link sucks less: http://codesearch.openstack.org/?q=ubuntu-trusty&i=nope&files=&repos= | 21:28 |
| *** Goneri has quit IRC | 21:29 | |
| EmilienM | mwhahaha: I don't remember now | 21:29 |
| mwhahaha | we should cleanup the puppet one, i'm pretty sure we ditched trusty with mitaka | 21:29 |
| mwhahaha | cruft-tastic | 21:29 |
| dmsimard | yeah migration to v3 is a good opportunity to do clean ups | 21:30 |
| *** thrash is now known as thrash|g0ne | 21:30 | |
| *** bfournie1 has quit IRC | 21:31 | |
| *** ecerquei has quit IRC | 21:32 | |
| *** jlinkes has quit IRC | 21:32 | |
| *** itlinux has joined #tripleo | 21:35 | |
| *** leitan has joined #tripleo | 21:36 | |
| *** itlinux has quit IRC | 21:37 | |
| openstackgerrit | Ian Main proposed openstack/python-tripleoclient master: Make it so heat won't exit even if the stack succeeds. https://review.openstack.org/508319 | 21:38 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-docs master: Document simpler undercloud major upgrade https://review.openstack.org/508320 | 21:38 |
| *** ansmith has joined #tripleo | 21:39 | |
| *** chlong has joined #tripleo | 21:41 | |
| *** leitan has quit IRC | 21:41 | |
| dmsimard | mwhahaha: it merged so I did a recheck on https://review.openstack.org/#/c/507889/ | 21:43 |
| mwhahaha | k | 21:43 |
| dmsimard | mwhahaha: but from looking at -infra, nodepool is currently busted | 21:43 |
| dmsimard | so I'm not optimistic | 21:44 |
| mwhahaha | k | 21:44 |
| *** rbowen has joined #tripleo | 21:45 | |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-docs master: Remove source admonition https://review.openstack.org/508323 | 21:49 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-docs master: Remove source admonition https://review.openstack.org/508323 | 21:50 |
| *** zoli has quit IRC | 21:53 | |
| *** gchamoul has quit IRC | 21:53 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: add team structure doc https://review.openstack.org/508325 | 21:55 |
| *** zoli has joined #tripleo | 21:56 | |
| *** gchamoul has joined #tripleo | 21:56 | |
| honza | jrist: no, never :( are your node_modules up to date? | 21:57 |
| *** cdearborn__ has quit IRC | 21:57 | |
| *** mcornea has quit IRC | 21:59 | |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-docs master: Document use of undercloud_hostname https://review.openstack.org/508328 | 22:00 |
| *** morazi has quit IRC | 22:02 | |
| *** bnemec has quit IRC | 22:05 | |
| *** bogdando has joined #tripleo | 22:05 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-quickstart master: Allow customizing the hard-drive Serial and/or WWN https://review.openstack.org/404156 | 22:07 |
| *** alee has quit IRC | 22:07 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 22:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 22:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 22:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 22:10 |
| *** bfournie has joined #tripleo | 22:21 | |
| *** catintheroof has joined #tripleo | 22:24 | |
| *** bogdando has quit IRC | 22:26 | |
| jrist | honza: yes | 22:28 |
| jrist | honza: nuked and reinstalled | 22:28 |
| *** bfournie has quit IRC | 22:48 | |
| *** bfournie has joined #tripleo | 22:49 | |
| *** alee has joined #tripleo | 23:01 | |
| *** achadha has quit IRC | 23:01 | |
| *** achadha has joined #tripleo | 23:02 | |
| *** Goneri has joined #tripleo | 23:04 | |
| *** achadha_ has joined #tripleo | 23:05 | |
| openstackgerrit | Ian Main proposed openstack/tripleo-heat-templates master: Add a name to the post deployment. https://review.openstack.org/508351 | 23:06 |
| *** achadha has quit IRC | 23:06 | |
| *** achadha_ has quit IRC | 23:09 | |
| *** achadha has joined #tripleo | 23:09 | |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 23:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 23:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 23:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 23:10 |
| *** achadha has quit IRC | 23:10 | |
| *** achadha has joined #tripleo | 23:11 | |
| *** achadha has quit IRC | 23:12 | |
| *** achadha has joined #tripleo | 23:12 | |
| *** achadha has quit IRC | 23:16 | |
| dmsimard | mwhahaha: multinode http://zuulv3.openstack.org/static/stream.html?uuid=78e1ff2fbd7640b4af2b19557adb68dd&logfile=console.log | 23:19 |
| dmsimard | mwhahaha: ovb http://zuulv3.openstack.org/static/stream.html?uuid=dd044082616c447ea86bde92207d57a8&logfile=console.log | 23:19 |
| dmsimard | I have to step away for now | 23:20 |
| *** tosky has quit IRC | 23:21 | |
| mwhahaha | K | 23:24 |
| mwhahaha | Failed | 23:25 |
| *** itlinux has joined #tripleo | 23:25 | |
| openstackgerrit | Ian Main proposed openstack/tripleo-heat-templates master: Add a name to the post deployment. https://review.openstack.org/508351 | 23:26 |
| *** itlinux has quit IRC | 23:31 | |
| *** Goneri has quit IRC | 23:31 | |
| *** amoralej|off has quit IRC | 23:41 | |
| *** jpena|off has quit IRC | 23:43 | |
| *** amoralej has joined #tripleo | 23:46 | |
| *** jpena|off has joined #tripleo | 23:46 | |
| *** Goneri has joined #tripleo | 23:52 | |
| *** yamahata has quit IRC | 23:56 | |
| *** Goneri has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
