| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates stable/ocata: GATE CHECK for TripleO https://review.openstack.org/564291 | 00:00 |
|---|---|---|
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates stable/rocky: GATE CHECK for TripleO https://review.openstack.org/604293 | 00:00 |
| *** dtrainor_ has quit IRC | 00:04 | |
| *** jcoufal has joined #tripleo | 00:05 | |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 00:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795411 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795418 | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 00:10 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 00:10 |
| openstack | Launchpad bug 1795418 in tripleo "The containers-multinode scenario jobs fail overcloud deploy on docker-puppet step3 or starting containers" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 00:10 |
| *** jcoufal has quit IRC | 00:11 | |
| *** boazel has joined #tripleo | 00:14 | |
| *** moshele has quit IRC | 00:19 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-common master: tripleo-container-image-prepare add --log-file https://review.openstack.org/603944 | 00:53 |
| *** phuongnh has joined #tripleo | 01:02 | |
| *** saneax has joined #tripleo | 01:08 | |
| *** saneax has quit IRC | 01:08 | |
| *** HuyNQ has joined #tripleo | 01:09 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795411 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795418 | 01:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 01:10 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 01:10 |
| openstack | Launchpad bug 1795418 in tripleo "The containers-multinode scenario jobs fail overcloud deploy on docker-puppet step3 or starting containers" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 01:10 |
| *** saneax has joined #tripleo | 01:11 | |
| weshay | mwhahaha, you have chosen WISELY | 01:21 |
| mwhahaha | :D | 01:21 |
| *** mrsoul has quit IRC | 01:22 | |
| *** mschuppert has quit IRC | 01:24 | |
| *** dxiri has quit IRC | 01:24 | |
| *** med_ has joined #tripleo | 01:27 | |
| *** dxiri has joined #tripleo | 01:28 | |
| *** owalsh_ has joined #tripleo | 01:29 | |
| *** threestrands_ has joined #tripleo | 01:30 | |
| *** dxiri has quit IRC | 01:32 | |
| *** owalsh has quit IRC | 01:33 | |
| *** threestrands has quit IRC | 01:33 | |
| *** thrash is now known as thrash|g0ne | 01:36 | |
| *** threestrands_ has quit IRC | 01:43 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: Introduce undercloud_container_cli parameter https://review.openstack.org/600512 | 01:45 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: Add podman support to validate-tempest role https://review.openstack.org/605356 | 01:46 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: Add podman support for log collection https://review.openstack.org/605090 | 01:46 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: Add podman support to validate-tempest role https://review.openstack.org/605356 | 01:46 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Set proper setype for service directories https://review.openstack.org/600534 | 01:46 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Allow to deactivate SELinux separation for selected containers https://review.openstack.org/600535 | 01:47 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Support podman when tagging container for Pacemaker https://review.openstack.org/604180 | 01:47 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Introduce OS::TripleO::Services::Podman https://review.openstack.org/604235 | 01:47 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: undercloud: deploy podman https://review.openstack.org/605221 | 01:47 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Add tempest workspace volume for tempest https://review.openstack.org/605980 | 01:48 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Switch fs027 to deploy with podman https://review.openstack.org/600517 | 01:49 |
| *** tzumainn has quit IRC | 01:59 | |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795411 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795418 | 02:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 02:10 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 02:10 |
| openstack | Launchpad bug 1795418 in tripleo "The containers-multinode scenario jobs fail overcloud deploy on docker-puppet step3 or starting containers" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 02:10 |
| *** mcornea has quit IRC | 02:10 | |
| *** dmacpher_ has joined #tripleo | 02:13 | |
| *** pvc has joined #tripleo | 02:15 | |
| pvc | hi | 02:15 |
| pvc | anyone | 02:15 |
| pvc | Determining IP information for ens4f0... failed; no link present. Check cable? | 02:15 |
| *** dxiri has joined #tripleo | 02:22 | |
| *** chem has quit IRC | 02:24 | |
| *** abishop_ has quit IRC | 02:27 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Ping default gateways before controllers https://review.openstack.org/604229 | 02:57 |
| openstackgerrit | Michael Johnson proposed openstack/diskimage-builder master: Fix DIB ubuntu-minimal running on bionic (18.04) https://review.openstack.org/604478 | 03:01 |
| *** med_ has quit IRC | 03:01 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add networks to IronicConductor role. https://review.openstack.org/604089 | 03:07 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Remove non-voting job from gate https://review.openstack.org/606390 | 03:07 |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 03:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795411 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795418 | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 03:10 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 03:10 |
| openstack | Launchpad bug 1795418 in tripleo "The containers-multinode scenario jobs fail overcloud deploy on docker-puppet step3 or starting containers" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 03:10 |
| openstackgerrit | Merged openstack/puppet-pacemaker master: Remove the duplicated "the" https://review.openstack.org/576347 | 03:14 |
| openstackgerrit | Merged openstack/puppet-pacemaker master: Update overriden to overridden https://review.openstack.org/571415 | 03:16 |
| *** med_ has joined #tripleo | 03:21 | |
| *** med_ has quit IRC | 03:38 | |
| openstackgerrit | Merged openstack/tripleo-common stable/rocky: Fix validation runs in a containerized environment https://review.openstack.org/603031 | 03:44 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Revert "Fix postrotate to notify holders of rotated logs" https://review.openstack.org/607052 | 03:44 |
| *** dxiri has quit IRC | 03:57 | |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795411 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795418 | 04:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 04:10 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 04:10 |
| openstack | Launchpad bug 1795418 in tripleo "The containers-multinode scenario jobs fail overcloud deploy on docker-puppet step3 or starting containers" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 04:10 |
| *** pcaruana has joined #tripleo | 04:36 | |
| *** dxiri has joined #tripleo | 04:42 | |
| *** pcaruana has quit IRC | 04:43 | |
| *** boazel has quit IRC | 04:51 | |
| *** dxiri has quit IRC | 04:54 | |
| *** dciabrin has quit IRC | 04:57 | |
| *** pvc has quit IRC | 05:03 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795411 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795418 | 05:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 05:10 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 05:10 |
| openstack | Launchpad bug 1795418 in tripleo "The containers-multinode scenario jobs fail overcloud deploy on docker-puppet step3 or starting containers" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 05:10 |
| Tengu | hello there | 05:33 |
| *** agurenko has joined #tripleo | 05:36 | |
| *** quiquell|off is now known as quiquell | 05:38 | |
| quiquell | Tengu o/ | 05:38 |
| Tengu | quiquell: hello :) | 05:40 |
| quiquell | Tengu: No workflow possible ? | 05:44 |
| *** zaneb has quit IRC | 05:45 | |
| Tengu | quiquell: I can't workflow at least. | 05:46 |
| Tengu | and apparently the CI still has issues. | 05:46 |
| *** zaneb has joined #tripleo | 05:50 | |
| *** iranzo has joined #tripleo | 05:54 | |
| *** ksambor has joined #tripleo | 06:09 | |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795411 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795418 | 06:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 06:10 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 06:10 |
| openstack | Launchpad bug 1795418 in tripleo "The containers-multinode scenario jobs fail overcloud deploy on docker-puppet step3 or starting containers" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 06:10 |
| *** mschuppert has joined #tripleo | 06:12 | |
| *** jfrancoa has joined #tripleo | 06:18 | |
| *** yprokule has joined #tripleo | 06:26 | |
| *** d0ugal has joined #tripleo | 06:32 | |
| *** cylopez has joined #tripleo | 06:41 | |
| Tengu | damn.. jinja2 is a templating language from hell. | 06:43 |
| *** pvc has joined #tripleo | 07:00 | |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates master: Fix privileged typo in docker-puppet.py https://review.openstack.org/607106 | 07:00 |
| pvc | hi guys anyone encounter an error on overcloud instance "Determining IP information for ens4f0... failed; no link present. Check cable?" so it will fail on the NetworkDeployment part | 07:00 |
| *** rcernin has quit IRC | 07:01 | |
| *** pcaruana has joined #tripleo | 07:01 | |
| *** moshele has joined #tripleo | 07:01 | |
| *** cylopez has left #tripleo | 07:02 | |
| *** shardy has joined #tripleo | 07:04 | |
| *** moshele has quit IRC | 07:04 | |
| *** slaweq has joined #tripleo | 07:07 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795411 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795418 | 07:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 07:10 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 07:10 |
| openstack | Launchpad bug 1795418 in tripleo "The containers-multinode scenario jobs fail overcloud deploy on docker-puppet step3 or starting containers" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 07:10 |
| *** kopecmartin|off is now known as kopecmartin|ruck | 07:17 | |
| *** moshele has joined #tripleo | 07:23 | |
| pvc | hi mandre | 07:30 |
| mandre | hello pvc | 07:31 |
| pvc | im failing on NetworkDeployment with this error Determining IP information for ens4f0... failed; no link present. Check cable? | 07:32 |
| pvc | but sometimes not | 07:32 |
| openstackgerrit | Damien Ciabrini proposed openstack/tripleo-heat-templates master: mysql: do not overwrite password file during docker-puppet https://review.openstack.org/602969 | 07:33 |
| *** holser_ has joined #tripleo | 07:35 | |
| *** psachin has joined #tripleo | 07:36 | |
| *** slaweq has quit IRC | 07:39 | |
| *** aufi has joined #tripleo | 07:39 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart master: Create CentOS-7 specific release files https://review.openstack.org/606943 | 07:46 |
| *** tosky has joined #tripleo | 07:47 | |
| *** slaweq has joined #tripleo | 07:49 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart master: Swap soft linked config/release files with targets https://review.openstack.org/606944 | 07:49 |
| *** jpena|off is now known as jpena | 07:50 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart master: Remove old config/release files https://review.openstack.org/606944 | 07:53 |
| *** d0ugal has quit IRC | 07:53 | |
| *** holser_ has quit IRC | 07:55 | |
| *** holser_ has joined #tripleo | 07:58 | |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: fedora28 standalone job definition https://review.openstack.org/595374 | 07:59 |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO https://review.openstack.org/604298 | 08:00 |
| mandre | pvc: maybe you have an issue with your cable? try with another cable? | 08:00 |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: add support for distro and major release delineation https://review.openstack.org/605843 | 08:00 |
| *** jpich has joined #tripleo | 08:01 | |
| mandre | sorry, there is not much I can do if this is a physical issue with your setup | 08:01 |
| *** jtomasek has joined #tripleo | 08:04 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-heat-templates stable/rocky: Add networks to IronicConductor role. https://review.openstack.org/607120 | 08:04 |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-heat-templates stable/queens: Add networks to IronicConductor role. https://review.openstack.org/607121 | 08:05 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: libvirt standalone deployment https://review.openstack.org/591077 | 08:05 |
| *** chem has joined #tripleo | 08:05 | |
| *** owalsh_ is now known as owalsh | 08:06 | |
| *** bfournie has quit IRC | 08:07 | |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart-extras master: WIP - Fix stackviz https://review.openstack.org/605419 | 08:08 |
| *** amoralej|off is now known as amoralej | 08:08 | |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 08:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795411 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795418 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 08:10 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 08:10 |
| openstack | Launchpad bug 1795418 in tripleo "The containers-multinode scenario jobs fail overcloud deploy on docker-puppet step3 or starting containers" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 08:10 |
| pvc | can i set it on the heat tempalte mandre? | 08:11 |
| *** zaneb has quit IRC | 08:12 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Set proper setype for service directories https://review.openstack.org/600534 | 08:12 |
| *** zaneb has joined #tripleo | 08:12 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Allow to deactivate SELinux separation for selected containers https://review.openstack.org/600535 | 08:12 |
| pvc | hi mandre is that possible sir? | 08:13 |
| *** bogdando has joined #tripleo | 08:13 | |
| mandre | pvc: supposing your issue is with the interface name, you could try to use a custom template for your NICs, there are some docs at https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/network_isolation.html explaining how to do it | 08:23 |
| mandre | pvc: I also suggest you try with a different network cable, just in case | 08:24 |
| *** akrivoka has joined #tripleo | 08:24 | |
| *** pvc_ has joined #tripleo | 08:26 | |
| pvc_ | hi im sorry mandre | 08:26 |
| pvc_ | im DC | 08:27 |
| pvc_ | may i know if i can edit that on heat template changing the interface name mandre | 08:27 |
| mandre | pvc_: supposing your issue is with the interface name, you could try to use a custom template for your NICs, there are some docs at https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/network_isolation.html explaining how to do it | 08:27 |
| mandre | pvc_: I also suggest you try with a different network cable, just in case | 08:27 |
| pvc_ | i mean cable mandre | 08:28 |
| pvc_ | but i think it is not getting on the part that it will create a directory in /etc/os-net-config | 08:28 |
| pvc_ | the problem is the firstboot of the instance, it cannot start the network service | 08:28 |
| *** pvc has quit IRC | 08:29 | |
| *** sshnaidm|off is now known as sshnaidm | 08:30 | |
| *** jidar_ has joined #tripleo | 08:31 | |
| *** jidar has quit IRC | 08:32 | |
| *** jidar_ is now known as jidar | 08:32 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: libvirt standalone deployment https://review.openstack.org/591077 | 08:33 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: f28 support for quickstart https://review.openstack.org/591652 | 08:33 |
| *** pvc_ has quit IRC | 08:35 | |
| *** pvc has joined #tripleo | 08:36 | |
| pvc | hi mandre how can i use another cable? | 08:36 |
| *** derekh has joined #tripleo | 08:37 | |
| mandre | pvc: is this a physical or a virtual setup? | 08:42 |
| pvc | im on physical server | 08:42 |
| pvc | IBM server rather | 08:42 |
| mandre | pvc: all your nodes are bare metal? or are they VMs running on your server? | 08:43 |
| *** fhubik has joined #tripleo | 08:43 | |
| pvc | yes all my nodes are bare metal | 08:43 |
| pvc | Undercloud then 1 baremetal overcloud compute plus 1 controller baremetal | 08:43 |
| Tengu | hmm. is there a location for reusable ansible roles for tripleo-heat-templates? shardy maybe? | 08:44 |
| mandre | pvc: in that case, these nodes are connected with each other via physical network cables. Maybe one of them is malfunctioning and needs to be replaced | 08:44 |
| pvc | but i can assign a static IP on each of my cable. Also if i up the failing interface it will have and have an IP. | 08:45 |
| pvc | also there is a scenario that the instance will sucessfully boot | 08:45 |
| mandre | pvc: if you're sure it's not a problem with the cable then good, you can try to use fixed interface names for your testbed. There are some docs at https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/network_isolation.html explaining how to do it | 08:47 |
| *** gkadam has joined #tripleo | 08:48 | |
| pvc | but there are big chances that overcloud instance first boot doesnt have an IP address because of the failed network service. with Determining IP information for ens4f0... failed; no link present. Check cable? | 08:48 |
| openstackgerrit | Yurii Prokulevych proposed openstack/tripleo-upgrade master: Use ceph-ansible environments during Q->R upgrade. https://review.openstack.org/607124 | 08:48 |
| Tengu | aha! in https://github.com/openstack/tripleo-common apparently. good. let's add things. | 08:48 |
| jistr | quiquell: can i +A the q/r release change patch? | 08:49 |
| *** paramite has quit IRC | 08:50 | |
| quiquell | jistr: sure but I think the gates are broken we cannot workflow stuff | 08:52 |
| quiquell | jistr: All the Depends-On are merged already | 08:52 |
| quiquell | jistr: But's it's an important bug to fix | 08:52 |
| *** ykarel has joined #tripleo | 08:53 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-common master: Load and persist modules from the host. https://review.openstack.org/607130 | 08:57 |
| *** sri_ has joined #tripleo | 09:01 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Load isci_tcp module from the host. https://review.openstack.org/605450 | 09:04 |
| jistr | quiquell: ack, we may need to hold off with +A for now then | 09:07 |
| quiquell | jistr: or wait for jaosorio and see if he give us the blessing | 09:09 |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 09:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795411 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795418 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 09:10 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 09:10 |
| openstack | Launchpad bug 1795418 in tripleo "The containers-multinode scenario jobs fail overcloud deploy on docker-puppet step3 or starting containers" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 09:10 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Load ip_vs module from the host https://review.openstack.org/605446 | 09:18 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Load dm-multipath module from the host. https://review.openstack.org/605452 | 09:18 |
| *** slaweq has quit IRC | 09:19 | |
| hjensas | marios: bandini: what about the ERROR:scss.compiler:Maximum number of supported selectors in Internet Explorer (4095) exceeded! ? | 09:20 |
| bandini | hjensas: iirc that is horizon complaining since forever (i.e. harmless) | 09:20 |
| *** salmankhan has joined #tripleo | 09:21 | |
| bandini | hjensas: are you unconvinced about the volumes thing theory? | 09:21 |
| *** slaweq has joined #tripleo | 09:21 | |
| hjensas | bandini: yes. :) | 09:21 |
| bandini | hjensas: I should have an env we can play with shortly and we can do some live tests around that? | 09:21 |
| *** ykarel is now known as ykarel|away | 09:22 | |
| *** paramite has joined #tripleo | 09:25 | |
| hjensas | bandini: ok, cool. Looks like the 604298 job's on the bug failed because ntp puppet failed. And it was some network glitch: http://logs.openstack.org/98/604298/14/check/tripleo-ci-centos-7-scenario004-multinode-oooq-container/218b2c4/logs/subnode-2/var/log/journal.txt.gz#_Oct_01_08_05_35 (But that does'nt explain the other's.) | 09:26 |
| *** rdopiera has joined #tripleo | 09:26 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-common master: Add libseccomp for podman wrappers in neutron/ovn https://review.openstack.org/606946 | 09:29 |
| *** hamzy has quit IRC | 09:32 | |
| *** hamzy has joined #tripleo | 09:32 | |
| *** dtantsur|afk is now known as dtantsur | 09:33 | |
| openstackgerrit | Jose Luis Franco proposed openstack-infra/tripleo-ci master: Switch previous release of master from 'queens' to 'rocky' https://review.openstack.org/590774 | 09:34 |
| openstackgerrit | Jose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Don't use mixed_upgrade special release files https://review.openstack.org/604032 | 09:34 |
| openstackgerrit | Jose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Link subnodes-setup into multinode jobs. https://review.openstack.org/604121 | 09:34 |
| openstackgerrit | Yurii Prokulevych proposed openstack/tripleo-upgrade master: [WIP] Add rocky specific parameters to custom NICs. https://review.openstack.org/605407 | 09:38 |
| openstackgerrit | Yurii Prokulevych proposed openstack/tripleo-upgrade master: [WIP] Add rocky specific parameters to custom NICs. https://review.openstack.org/605407 | 09:41 |
| openstackgerrit | Derek Higgins proposed openstack/tripleo-heat-templates master: Add scenario 012 - overlcoud baremetal+ansible-ml2 https://review.openstack.org/579603 | 09:44 |
| *** moshele has quit IRC | 09:52 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/puppet-tripleo master: Fix wrapper containers for podman w/o sockets https://review.openstack.org/606095 | 09:53 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/puppet-tripleo master: Run wrappers as a non-root and SYSADMIN capability https://review.openstack.org/607139 | 09:53 |
| *** moshele has joined #tripleo | 09:55 | |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates master: Collectd documentation refactor https://review.openstack.org/606956 | 09:57 |
| *** moshele has quit IRC | 09:58 | |
| *** fhubik has left #tripleo | 09:58 | |
| *** fhubik has quit IRC | 09:58 | |
| *** Aelia has quit IRC | 09:59 | |
| *** psachin has quit IRC | 09:59 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Rework neutron/own agent wrapper tools for podman https://review.openstack.org/606975 | 10:01 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Use cap sysadmin for Neutron/OVN agents https://review.openstack.org/606045 | 10:01 |
| openstackgerrit | Jose Luis Franco proposed openstack-infra/tripleo-ci master: Switch previous release of master from 'queens' to 'rocky' https://review.openstack.org/590774 | 10:03 |
| openstackgerrit | Jose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Don't use mixed_upgrade special release files https://review.openstack.org/604032 | 10:03 |
| openstackgerrit | Jose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Link subnodes-setup into multinode jobs. https://review.openstack.org/604121 | 10:03 |
| *** moshele has joined #tripleo | 10:09 | |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 10:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795411 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795418 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 10:10 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 10:10 |
| openstack | Launchpad bug 1795418 in tripleo "The containers-multinode scenario jobs fail overcloud deploy on docker-puppet step3 or starting containers" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 10:10 |
| *** moshele has quit IRC | 10:12 | |
| bandini | hjensas: I think I agree, I am also a bit unconvinced now. will look some more at the other logs | 10:14 |
| hjensas | bandini: damn, I was hoping it was just me being silly. | 10:15 |
| holser_ | Tengu I think https://review.openstack.org/#/c/607130/ is half measure | 10:19 |
| holser_ | There is a thread in RHOS-DEV started by bogdando | 10:20 |
| *** pvc has quit IRC | 10:20 | |
| Tengu | holser_: it at least allows to load *known* module currently loaded from kolla containers. | 10:20 |
| Tengu | holser_: 3rd party modules are probably to be managed differently. | 10:21 |
| *** phuongnh has quit IRC | 10:21 | |
| holser_ | Tengu, have a look at problems I raised | 10:21 |
| holser_ | Co-Exist, Upgrade, Fleet problems | 10:21 |
| Tengu | bogdando: holser_ care to link the ML thingy? | 10:22 |
| Tengu | holser_: issue is: the current way is not good. that's a realy issue. the proposed patch does what was done with the kolla thingy, but using host modules directly, instead of mounting the host "modules" directory into a container and running the "modprobe" from there. | 10:23 |
| Tengu | holser_: among the issues I saw in said thread, there were the way modules might or might not be upgraded - this patch solves the issue for the nodes present on the host itself, managed by rpm and thus updated in a correct, uniform way. | 10:24 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/puppet-tripleo master: Revert "Revert "Fix postrotate to notify holders of rotated logs"" https://review.openstack.org/607146 | 10:24 |
| Tengu | imho the "3rd party modules" are another case that need a better handling if they are shipped within a container. | 10:25 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Load ip_vs module from the host https://review.openstack.org/605446 | 10:25 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Load dm-multipath module from the host. https://review.openstack.org/605452 | 10:25 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Load openvswitch module gfrom the host https://review.openstack.org/607147 | 10:25 |
| Tengu | woops. typo. | 10:26 |
| *** ykarel|away has quit IRC | 10:26 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Load openvswitch module from the host https://review.openstack.org/607147 | 10:26 |
| Tengu | holser_: more over: my patch allows to NOT use modules in the containers :). so that solves most of your concerns (just found out the thread)... at least, for non-3rd party thingy - and I don't address that part with the said patch. | 10:29 |
| holser_ | Tengu - Can you answer on Day2 questions | 10:29 |
| holser_ | How are we going to upgrade kernel? | 10:30 |
| holser_ | Will it break containers? | 10:30 |
| Tengu | holser_: I don't understand how my patch creates issue on that field.... ? container aren't involved anymore for modprobe things. | 10:30 |
| holser_ | but apps in containers may require some specific kernel modules | 10:30 |
| Tengu | holser_: (third time: it does NOT take care of 3rd party modules shipped by partners/others - this is another case) | 10:31 |
| bogdando | Tengu, holser_: http://post-office.corp.redhat.com/archives/rhos-df/2018-September/msg00184.html | 10:31 |
| Tengu | holser_: re-re-re-resaying it: this is for the *standard* modules that are already present on the system, currently loaded from a container when they should be from the host directly, Nothing about the said 3rd party modules that are the real issue with updates. | 10:31 |
| holser_ | Tengu - even if kernel modules are kolla, that doesn't answer to question how to load/unload kernel modules, how to update, how to find a host where these modules are present | 10:32 |
| Tengu | as they might be compiled against wrong kernel, and so on. | 10:32 |
| holser_ | Tengu - think wider :D | 10:32 |
| Tengu | holser_: I'm just mixing meat with fruits. | 10:32 |
| Tengu | thats all. | 10:32 |
| Tengu | you don't use the same tool for both, usually. | 10:33 |
| holser_ | I understand your work, but it's half of measure... as all day2 operations are manual | 10:33 |
| Tengu | I currently have a first BIG issue with the kolla-embeded modprobes, my patch correct that, without creating any new issue. | 10:33 |
| holser_ | Tengu - Runtime Engine should handle that... in the same way as it handles file systems for containers | 10:33 |
| *** hamzy has quit IRC | 10:33 | |
| Tengu | holser_: kernel update means reboot, means new modules are loaded. as all the 4 kolla embedded modules are managed by standard RPMs on the host, we don't have any issue. | 10:34 |
| *** hamzy has joined #tripleo | 10:34 | |
| holser_ | if we update them at the same time | 10:34 |
| Tengu | what's the point about "day2 issues" when, for that case (again: NO 3rd party!!!!) there isn't any issue ? | 10:34 |
| holser_ | we might not update them | 10:34 |
| holser_ | we won't even know that kolla containers will be broken | 10:35 |
| Tengu | update are host-by-host. | 10:35 |
| Tengu | so no problem. | 10:35 |
| Tengu | you update host1 (containers + kernel+modules), reboot, check it, go_next() | 10:35 |
| holser_ | I can run | 10:35 |
| holser_ | yum upgrade kernel | 10:35 |
| holser_ | ONLY | 10:35 |
| *** ykarel|away has joined #tripleo | 10:35 | |
| holser_ | without any kolla rpms | 10:35 |
| holser_ | nothing stops me to do that | 10:36 |
| Tengu | well, yes, there is plenty of reason NOT to do that. | 10:36 |
| Tengu | among them: what happens of, let's say, kvm modules are new, and need nova update? | 10:36 |
| Tengu | I didn't see nova containers loading kvm modules | 10:36 |
| Tengu | meaning it's the one provided by the host. | 10:37 |
| Tengu | same issue. | 10:37 |
| holser_ | modules don't care about nova | 10:37 |
| holser_ | container engine won't start nova | 10:37 |
| holser_ | that all | 10:37 |
| Tengu | cool, so no way ti manage instances. | 10:37 |
| Tengu | that's the same issue, and apparently it doesn't make all the fuz. | 10:37 |
| *** gfidente has joined #tripleo | 10:37 | |
| Tengu | as kvm modules aren't loaded from kolla, thus not fixed in version. | 10:37 |
| holser_ | in my case I see the issue and it's visible... so I will upgrade nova and will restart container | 10:37 |
| holser_ | in your case the issue is completely invisible | 10:38 |
| Tengu | why so? | 10:38 |
| holser_ | happy debugging | 10:38 |
| Tengu | keepalived will crash as well if module isn't good. | 10:38 |
| Tengu | neutron will more than probably spit issues if OVS module isn't good either. | 10:38 |
| holser_ | yeah it will crash without letting me knowing why | 10:38 |
| Tengu | well, like nova. | 10:38 |
| Tengu | in your checks after an update, you will ensure containers are back to normal, won't you? | 10:39 |
| holser_ | so, if container requires something from host it should be in specification | 10:39 |
| holser_ | that's it | 10:39 |
| Tengu | i.e. ensure you don't have some Exited (1) containers | 10:39 |
| holser_ | mismatch - not start container | 10:39 |
| holser_ | letting operator know | 10:39 |
| Tengu | so that requires patches in kolla in order to fetch kernel/module version and spit something. | 10:40 |
| Tengu | iff those spec are even known in fact. | 10:40 |
| holser_ | yeah | 10:40 |
| holser_ | so it's chicken and egg problem | 10:40 |
| *** two_oes has joined #tripleo | 10:40 | |
| holser_ | we can add kolla containers as a dependancy for kernel rpm :D | 10:41 |
| Tengu | in any case, running modprobe from containers is a bad idea, except if they are flagged as "Infra" and are a one-time shot with elevated privileges. | 10:41 |
| holser_ | Tengu I am with you | 10:41 |
| holser_ | I was against the specification of running kernel modules in containers | 10:42 |
| Tengu | I understood there were "a few ppl" against it. | 10:42 |
| Tengu | a pity I wasn't there earlier. I would have added some more things against that case ;). anyway. It was accepted, and we need to find a way to correct that while not break all the existing. | 10:42 |
| holser_ | k, lunch time :D | 10:43 |
| Tengu | we can't "docker copy <module>" of course, as the kernel mismatch will breack ;). | 10:43 |
| Tengu | holser_: we might continue this discussion after lunch then ;). | 10:43 |
| holser_ | :D | 10:43 |
| Tengu | btw: The Undercloud has been successfully installed. using my patches :). | 10:43 |
| Tengu | there's just the openvswitch module, I wasn't sure where I should load it. | 10:44 |
| Tengu | (i.e. what service) | 10:44 |
| *** d0ugal has joined #tripleo | 11:01 | |
| *** aedc has quit IRC | 11:01 | |
| *** aedc has joined #tripleo | 11:02 | |
| *** amoralej is now known as amoralej|lunch | 11:05 | |
| *** aedc has quit IRC | 11:07 | |
| *** HuyNQ has quit IRC | 11:07 | |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 11:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795411 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795418 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 11:10 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 11:10 |
| openstack | Launchpad bug 1795418 in tripleo "The containers-multinode scenario jobs fail overcloud deploy on docker-puppet step3 or starting containers" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 11:10 |
| *** med_ has joined #tripleo | 11:13 | |
| jtomasek | slagle, shardy: I've just read the edge squad meeting log. GUI currently calls Swift API directly in certain cases. This can (and will) be removed as part of CLI/UI parity work. There is one exception which is files upload part of plan creation. Mistral is not able to handle file uploads, so we're calling Swift directly to upload either a tarball or a directory of files. We would have to introduce a replacement for uploading files | 11:16 |
| *** jtomasek has quit IRC | 11:16 | |
| *** jtomasek has joined #tripleo | 11:16 | |
| *** jaosorior has joined #tripleo | 11:16 | |
| *** jtomasek has quit IRC | 11:17 | |
| *** jtomasek has joined #tripleo | 11:18 | |
| *** nawar has joined #tripleo | 11:19 | |
| d0ugal | jtomasek: I don't have any good ideas for solving that. Do you? | 11:20 |
| nawar | hi | 11:21 |
| d0ugal | jtomasek: I had previously thought we could add support in Mistral, but that would be quite complex | 11:22 |
| jtomasek | d0ugal: I think we'd have to introduce a new server side app to handle file uploads for clients to complement Mistral | 11:22 |
| d0ugal | jtomasek: that sounds like writing our own swift :) | 11:22 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Load openvswitch module from the host https://review.openstack.org/607147 | 11:24 |
| jtomasek | d0ugal: yeah, maybe we'll end up using Swift just to get files to undercloud and Mistral workflow to move the files from swift into a new place where plan is actually stored and accessed | 11:24 |
| dtantsur | tripleo team, could you make sure this does not break anything: https://review.rdoproject.org/r/#/c/16637/ | 11:24 |
| d0ugal | "anything" :) | 11:24 |
| dtantsur | shardy, slagle, EmilienM, mwhahaha, jaosorior, sorry for mass pinging, but this is important ^^^ | 11:24 |
| *** dprince has joined #tripleo | 11:29 | |
| *** hamzy has quit IRC | 11:32 | |
| *** hamzy has joined #tripleo | 11:32 | |
| *** aufi_ has joined #tripleo | 11:37 | |
| fultonj | jistr: Can you recommend a spot in the upgrade process to address https://bugs.launchpad.net/tripleo/+bug/1795513 ? | 11:37 |
| openstack | Launchpad bug 1795513 in tripleo "During upgrade from Q to R, rename ceph-ansible fetch directory tarball in Swift" [Medium,Triaged] - Assigned to John Fulton (jfulton-org) | 11:37 |
| fultonj | gfidente: ^ | 11:38 |
| *** aufi has quit IRC | 11:39 | |
| *** boazel has joined #tripleo | 11:39 | |
| nawar | i'm installing undercloud and getting "TypeError: get_session_client() got an unexpected keyword argument 'constructor'" | 11:41 |
| nawar | this installation after a cleaning of pervious installation | 11:42 |
| jistr | fultonj, gfidente: when does that need to happen? i'm thinking https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/ceph-ansible/ceph-base.yaml#L493 but there are two concerns: | 11:42 |
| jistr | 1) isn't it too late | 11:42 |
| jistr | 2) will we have the necessary credentials available | 11:42 |
| fultonj | jistr: right, we don't have the necessary credentials | 11:43 |
| fultonj | jistr: i was hoping to get it via mistral | 11:43 |
| fultonj | is the upgrade process done 100% in ansible? | 11:43 |
| fultonj | what do you mean by too late? | 11:43 |
| fultonj | any queens user upgrading to rocky will hit the bug | 11:44 |
| jistr | fultonj: yes it's 100% ansible, the Mistral wrapper is generic (same for upgrades/updates) | 11:44 |
| fultonj | but if a queens user gets the file renamed, they should be fine | 11:44 |
| fultonj | jistr: in order to get the user the credentials to interact w/ swift, i create PUT/GET URLs in mistral | 11:44 |
| jistr | fultonj: we may need to insert an idempotent action somewhere, which will always do this rename then | 11:44 |
| * jistr searches | 11:45 | |
| fultonj | jistr: e.g. so when i generate the GET/PUT URLs i check there? | 11:45 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Introduce OS::TripleO::Services::Podman https://review.openstack.org/604235 | 11:45 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: undercloud: deploy podman https://review.openstack.org/605221 | 11:45 |
| fultonj | jistr: if name like "foo-$timestamp" rename | 11:45 |
| jistr | fultonj: yea that would be best i guess | 11:45 |
| *** moshele has joined #tripleo | 11:47 | |
| *** mcornea has joined #tripleo | 11:47 | |
| *** jpena is now known as jpena|lunch | 11:47 | |
| slagle | dtantsur: the only way for someone to confirm that dib change doesn't break anything is to test it as have no coverage of packaging changes in tripleo ci | 11:47 |
| dtantsur | sigh | 11:48 |
| fultonj | jistr: looks to me like /plan_management.yaml | 11:48 |
| fultonj | jistr: https://github.com/openstack/tripleo-common/blob/master/workbooks/plan_management.yaml | 11:48 |
| fultonj | would be where the change would be needed | 11:48 |
| fultonj | jistr: under update_deployment_plan | 11:49 |
| fultonj | https://github.com/openstack/tripleo-common/blob/master/workbooks/plan_management.yaml#L244 | 11:49 |
| *** ykarel|away has quit IRC | 11:49 | |
| *** shardy has quit IRC | 11:49 | |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart-extras master: WIP - Fix stackviz https://review.openstack.org/605419 | 11:49 |
| fultonj | jistr: do you think that's the right place? | 11:50 |
| *** shardy has joined #tripleo | 11:50 | |
| slagle | dtantsur: that I know of anyway :). we used to at one point | 11:50 |
| *** ykarel has joined #tripleo | 11:50 | |
| fultonj | jistr: i.e. if i follow the upgrade procedure, is the plan management workbook run? | 11:51 |
| *** maufart__ has joined #tripleo | 11:51 | |
| jistr | fultonj: i'm checking now, i don't see that being run | 11:51 |
| *** abishop_ has joined #tripleo | 11:53 | |
| jistr | fultonj: we essentially run the `overcloud deploy --update-plan-only`, but i don't see the workflow you pointed at being executed there | 11:53 |
| *** aufi_ has quit IRC | 11:54 | |
| fultonj | jistr: https://github.com/openstack/tripleo-common/commit/bfabb3f5755628f707be58b1ead41b2f799ca099 | 11:54 |
| *** dtrainor has joined #tripleo | 11:54 | |
| fultonj | jistr: ^ i bring that up because i figured i could add the check during the ^ above | 11:54 |
| fultonj | i.e. when those bits of code are getting executed | 11:54 |
| jistr | fultonj: maybe it's called indirectly... | 11:54 |
| jistr | fultonj: i found https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L380 | 11:55 |
| jistr | fultonj: ok yea i traced it further down the call stack, the workflow you found is the right spot | 11:56 |
| jistr | fultonj: it is being executed | 11:56 |
| *** michchap has joined #tripleo | 11:56 | |
| jistr | https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/workflows/plan_management.py#L231 | 11:56 |
| jistr | https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/workflows/plan_management.py#L117 | 11:56 |
| jistr | fultonj: ^ | 11:56 |
| fultonj | jistr: so this workflow https://github.com/openstack/tripleo-common/blob/master/workbooks/plan_management.yaml#L244 | 11:58 |
| fultonj | i could add a new task there to check swift and do the rename if necessary | 11:58 |
| jistr | fultonj: yea +1 | 11:58 |
| fultonj | jistr++ thanks very much for confirming that | 11:58 |
| jistr | yw | 11:58 |
| openstackgerrit | Nicolas Hicher proposed openstack-infra/tripleo-ci master: provider: Add vexxhost https://review.openstack.org/596432 | 11:59 |
| *** morazi has joined #tripleo | 11:59 | |
| *** rh-jelabarre has joined #tripleo | 12:03 | |
| *** ssbarnea|bkp has joined #tripleo | 12:04 | |
| *** thrash|g0ne is now known as thrash | 12:06 | |
| *** mcornea has quit IRC | 12:07 | |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add rdo multinode vars at rdo-cloud provider https://review.openstack.org/607165 | 12:07 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-quickstart-extras master: Use TLS environment files from environments/ssl/ https://review.openstack.org/602171 | 12:08 |
| *** ykarel is now known as ykarel|away | 12:08 | |
| *** aedc has joined #tripleo | 12:10 | |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 12:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795411 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795418 | 12:10 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: Introduce undercloud_container_cli parameter https://review.openstack.org/600512 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 12:10 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 12:10 |
| openstack | Launchpad bug 1795418 in tripleo "The containers-multinode scenario jobs fail overcloud deploy on docker-puppet step3 or starting containers" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 12:10 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: Add podman support for log collection https://review.openstack.org/605090 | 12:10 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: Add podman support to validate-tempest role https://review.openstack.org/605356 | 12:10 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart master: DNM: testin ovb extra node 2 https://review.openstack.org/601850 | 12:12 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: DNM: extra node for OVB stack role https://review.openstack.org/542368 | 12:12 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Add ability to deploy FreeIPA on OVB https://review.openstack.org/534435 | 12:12 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Switch fs027 to deploy with podman https://review.openstack.org/600517 | 12:13 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: DNM: test ovb extra node 1 https://review.openstack.org/601839 | 12:16 |
| *** trown|outtypewww is now known as trown|brb | 12:16 | |
| *** trown|brb is now known as trown | 12:16 | |
| *** beagle is now known as beagles | 12:16 | |
| jaosorior | sshnaidm: around? | 12:18 |
| sshnaidm | jaosorior, yep | 12:18 |
| *** ykarel|away has quit IRC | 12:19 | |
| jaosorior | sshnaidm: hey, saw you uploaded a new patch for the OVB/FreeIPA work. Did you figure out the issues we were talking about regarding the DNS? | 12:19 |
| *** amoralej|lunch is now known as amoralej | 12:19 | |
| sshnaidm | jaosorior, I hope this patch will solve it | 12:19 |
| jaosorior | awesome! | 12:19 |
| jaosorior | sshnaidm: will it just run in the gate? or do we have to test it manually somewhere? | 12:20 |
| sshnaidm | jaosorior, after it works manually we can start creating the gate job | 12:21 |
| *** mcornea has joined #tripleo | 12:21 | |
| jaosorior | sshnaidm: are you gonna do the manual deployment? | 12:21 |
| sshnaidm | jaosorior, yeah, I test it locally | 12:22 |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/tripleo-heat-templates stable/rocky: Remove NeutronServicePlugins from octavia environment files https://review.openstack.org/607168 | 12:22 |
| sshnaidm | jaosorior, it's not really manual, I run it with reproducer | 12:23 |
| sshnaidm | jaosorior, so actually as fake gate job | 12:23 |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/tripleo-heat-templates stable/queens: Remove NeutronServicePlugins from octavia environment files https://review.openstack.org/607169 | 12:23 |
| *** lucasagomes has joined #tripleo | 12:23 | |
| *** haleyb has quit IRC | 12:23 | |
| jaosorior | sshnaidm: how do you run it? | 12:24 |
| *** quiquell is now known as quiquell|lunch | 12:24 | |
| sshnaidm | jaosorior, https://paste.fedoraproject.org/paste/LWQybvi8NwW-NOMQxngehA | 12:25 |
| jaosorior | nice | 12:25 |
| jaosorior | sshnaidm: thanks! | 12:25 |
| *** rbrady-afk is now known as rbrady | 12:26 | |
| *** pvc_ has joined #tripleo | 12:26 | |
| pvc_ | hi mandre | 12:26 |
| *** ssbarnea|bkp has quit IRC | 12:26 | |
| *** ssbarnea|bkp2 has joined #tripleo | 12:26 | |
| pvc_ | are you familiar with this "The current boot mode is bios" and ironic_api_client Failures static status code 404. | 12:26 |
| pvc_ | hi guys when i run introspection i have this error "The current boot mode is bios" and ironic_api_client Failures static status code 404. | 12:27 |
| *** haleyb has joined #tripleo | 12:27 | |
| *** nawar has quit IRC | 12:31 | |
| *** raildo has joined #tripleo | 12:32 | |
| *** jcoufal has joined #tripleo | 12:32 | |
| *** raildo_ has joined #tripleo | 12:34 | |
| *** ade_lee has quit IRC | 12:34 | |
| *** agopi|brb is now known as agopi | 12:35 | |
| *** raildo has quit IRC | 12:37 | |
| *** nawar has joined #tripleo | 12:40 | |
| *** tzumainn has joined #tripleo | 12:41 | |
| *** ansmith has joined #tripleo | 12:41 | |
| *** jpena|lunch is now known as jpena | 12:42 | |
| openstackgerrit | Bob Fournier proposed openstack/tripleo-heat-templates stable/rocky: Ping default gateways before controllers https://review.openstack.org/607173 | 12:43 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM: Enable verbose output from toci_quickstart https://review.openstack.org/562327 | 12:44 |
| *** yolanda has quit IRC | 12:45 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-quickstart master: WIP: Move overcloud_templates_* vars into featuresets for mixed_upgrade jobs. https://review.openstack.org/607175 | 12:45 |
| *** waleedm has joined #tripleo | 12:48 | |
| weshay | bogdando, thanks for https://review.openstack.org/#/c/606921/2 | 12:53 |
| openstackgerrit | Jose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Link subnodes-setup into multinode jobs. https://review.openstack.org/604121 | 12:53 |
| openstackgerrit | Emilien Macchi proposed openstack/paunch master: podman: create/delete systemd unit files when restart policy is used https://review.openstack.org/600849 | 12:53 |
| openstackgerrit | Martin André proposed openstack/tripleo-quickstart-extras master: Remove unused ansible.cfg https://review.openstack.org/607177 | 12:56 |
| *** pcaruana has quit IRC | 12:57 | |
| bandini | mwhahaha, EmilienM: is there a heat parameter on the containerized undercloud that tells me that I am deploying an undercloud? (IsUndercloud or the likes?) If it does not exist, do you have anything against adding one? | 12:58 |
| *** pvc_ has quit IRC | 12:58 | |
| EmilienM | bandini: why that | 12:59 |
| EmilienM | it's the default | 12:59 |
| EmilienM | and we don't support instack-undercloud anymore | 12:59 |
| bandini | EmilienM: because in a specific service I need to add an iptables rule *only* if I am in the undercloud | 12:59 |
| bandini | EmilienM: am not using instack-undercloud am using containerized UC, but I need to do a special host thing if I am deploying the undercloud (and must not do it when deploying the overcloud) | 13:00 |
| EmilienM | bandini: no, there is no such a parameter. I would just create a service specific parameter | 13:00 |
| *** d0ugal has quit IRC | 13:00 | |
| EmilienM | like "ManageFirewallRuleMyService" | 13:00 |
| EmilienM | or something | 13:00 |
| jaosorior | EmilienM: well, we do have an environment file for the undercloud | 13:00 |
| bandini | and then set ManageFirewallRuleMyService: true in the undercloud deployment | 13:01 |
| EmilienM | bandini: yes | 13:01 |
| jaosorior | bandini: would using this environment file be useful https://github.com/openstack/tripleo-heat-templates/blob/master/environments/undercloud.yaml ? | 13:02 |
| bandini | jaosorior: yeah that is where we will add this new parameter | 13:02 |
| bandini | we initially thought of adding some 'IsUndercloud' boolean that can then be used in any service yaml file that needs it | 13:03 |
| bandini | but I think also a specialiced parameter like 'AddTheDamnIptablesNATRuleUndercloud' also works for us I think | 13:03 |
| bandini | mschuppert: ^ ? | 13:03 |
| mschuppert | yes should work as well | 13:03 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates master: Fix misleading Ansible log message https://review.openstack.org/607179 | 13:04 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Don't disable keepalived in nonha-arch.yaml https://review.openstack.org/607180 | 13:06 |
| *** moshele has quit IRC | 13:07 | |
| *** lblanchard has joined #tripleo | 13:08 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-common master: Set workflow status to success when manipulating deployment status https://review.openstack.org/607181 | 13:08 |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: add support for distro and major release delineation https://review.openstack.org/605843 | 13:08 |
| openstackgerrit | James Slagle proposed openstack/python-tripleoclient master: Pass stack name to enable_ssh_admin workflow https://review.openstack.org/606964 | 13:09 |
| openstackgerrit | James Slagle proposed openstack/python-tripleoclient master: Update deployment status when deploying or failed https://review.openstack.org/607182 | 13:09 |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 13:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795411 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795418 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 13:10 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 13:10 |
| openstack | Launchpad bug 1795418 in tripleo "The containers-multinode scenario jobs fail overcloud deploy on docker-puppet step3 or starting containers" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 13:10 |
| *** egonzalez has joined #tripleo | 13:16 | |
| openstackgerrit | James Slagle proposed openstack/python-tripleoclient master: Pass stack name to enable_ssh_admin workflow https://review.openstack.org/606964 | 13:16 |
| openstackgerrit | James Slagle proposed openstack/python-tripleoclient master: Update deployment status when deploying or failed https://review.openstack.org/607182 | 13:16 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: WIP: Check periodic with bash https://review.openstack.org/607183 | 13:20 |
| *** d0ugal has joined #tripleo | 13:21 | |
| bogdando | folks, who can recall the recent fixes for mistral to handle stalled rpc on sighup? | 13:22 |
| *** nawar has quit IRC | 13:22 | |
| bogdando | I think that the same issue may be the root cause for https://bugs.launchpad.net/tripleo/+bug/1795411 | 13:22 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 13:22 |
| d0ugal | bogdando: This? https://review.openstack.org/#/q/Ic98e2db02abd8483591756d73e06784cc2e9cbe3 | 13:23 |
| bogdando | d0ugal: likely | 13:23 |
| d0ugal | therve: ^ | 13:23 |
| *** moshele has joined #tripleo | 13:23 | |
| *** nawar has joined #tripleo | 13:23 | |
| *** quiquell|lunch is now known as quiquell|off | 13:23 | |
| bogdando | at least, I can see the logrotation happend around the deployment failure | 13:23 |
| bogdando | and engine/executor logs got wrapped | 13:24 |
| bogdando | and 5 min later, deployment failed | 13:24 |
| bogdando | I see no rpc timeout messages tho | 13:24 |
| d0ugal | hmm | 13:24 |
| bogdando | correction, it's a 3 min, not 5 | 13:24 |
| d0ugal | I'm in a meeting but can take a look in a bit | 13:24 |
| bogdando | http://logs.openstack.org/45/560445/151/check/tripleo-ci-centos-7-scenario002-multinode-oooq-container/cd439d5/logs/undercloud/var/log/containers/mistral/executor.log.txt.gz | 13:24 |
| bogdando | d0ugal, thanks | 13:25 |
| bogdando | and the failure message http://logs.openstack.org/45/560445/151/check/tripleo-ci-centos-7-scenario002-multinode-oooq-container/cd439d5/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-10-01_10_03_05, d0ugal | 13:25 |
| bogdando | d0ugal: would be nice to see if that job already has https://review.openstack.org/#/c/605633/ promoted | 13:26 |
| bogdando | into mistral packages with that fix, in mistral containers... | 13:26 |
| openstackgerrit | Sam Doran proposed openstack/tripleo-ansible master: Use generic names for container platform https://review.openstack.org/600847 | 13:27 |
| *** pcaruana has joined #tripleo | 13:27 | |
| *** nawar has left #tripleo | 13:27 | |
| weshay | marios|rover, I put up a patch just to turn off ara installs until we're certain we have the right version of ansible getting intalled | 13:28 |
| jaosorior | weshay: so we still ahve the ara versioning issues? | 13:28 |
| weshay | marios|rover, is that the path we should take or just try to fix one branch at a time | 13:28 |
| weshay | the bug is still open | 13:28 |
| weshay | jaosorior, gate jobs are not failing w/ it atm, but check might be.. marios|rover would know | 13:29 |
| weshay | jaosorior, things are just a little better this morning :) | 13:29 |
| jaosorior | yay | 13:30 |
| therve | bogdando: Doesn't look like the same issue | 13:32 |
| therve | Not that it's clear what the issue is :) | 13:32 |
| bogdando | I see... but timing of rotation and failure events still looks close | 13:32 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Deploy openshift all in one in scenario009 https://review.openstack.org/603780 | 13:33 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Use glusterfs for registry when deploying with CNS https://review.openstack.org/605825 | 13:33 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Use openshift-ansible container instead of RPMs https://review.openstack.org/583868 | 13:33 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Remove unused networks from OpenShift roles https://review.openstack.org/604727 | 13:33 |
| therve | bogdando: Right. That fix only fixes the API server though | 13:33 |
| mandre | trown: ^^ reordered the patch to land the all-in-one more easily | 13:33 |
| therve | But I'm debugging another issue around that area | 13:33 |
| bogdando | therve: please keep https://bugs.launchpad.net/tripleo/+bug/1795411 posted :) | 13:34 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 13:34 |
| openstackgerrit | James Slagle proposed openstack/python-tripleoclient master: Update deployment status when deploying or failed https://review.openstack.org/607182 | 13:34 |
| *** d0ugal has quit IRC | 13:36 | |
| trown | mandre: awesome... I was having trouble getting it to work with any patches removed, I will give that a go | 13:37 |
| *** moshele has quit IRC | 13:37 | |
| openstackgerrit | Sam Doran proposed openstack/tripleo-common master: Use known_hosts module for tripleo-ssh-known-hosts role https://review.openstack.org/550278 | 13:37 |
| mandre | trown: there is still a ton of (ready) patches under it... I simply reordered the one that was giving me trouble in my env | 13:38 |
| *** moshele has joined #tripleo | 13:39 | |
| *** raildo_ is now known as raildo | 13:39 | |
| mandre | trown: I'll start proposing the rocky backports for everything that merged already | 13:39 |
| trown | mandre: oh right we have to backport all of this :( | 13:39 |
| mandre | trown: yeah... i hope I won't miss a patch in the mix | 13:40 |
| *** dprince has quit IRC | 13:44 | |
| openstackgerrit | Sam Doran proposed openstack/tripleo-heat-templates master: Use ternary filter for configs https://review.openstack.org/587506 | 13:45 |
| openstackgerrit | Merged openstack/tripleo-upgrade stable/rocky: Configure undercloud for containerized upgrade. https://review.openstack.org/605663 | 13:48 |
| jaosorior | mwhahaha, EmilienM, weshay: Is there automation for deploying a standalone deployment already? | 13:50 |
| EmilienM | in oooqe | 13:50 |
| EmilienM | there is a role | 13:50 |
| marios | jaosorior: the things in bold at https://review.rdoproject.org/etherpad/p/ruckrover-sprint20 (i.e. bugs https://bugs.launchpad.net/tripleo/+bug/1795009 https://bugs.launchpad.net/tripleo/+bug/1795411 https://bugs.launchpad.net/tripleo/+bug/1795418 once those fixes merge our check scenario jobs should be in much better shape | 13:50 |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 13:50 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 13:50 |
| openstack | Launchpad bug 1795418 in tripleo "The containers-multinode scenario jobs fail overcloud deploy on docker-puppet step3 or starting containers" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 13:50 |
| marios | weshay: ^ | 13:50 |
| *** moshele has quit IRC | 13:50 | |
| *** waleedm has quit IRC | 13:52 | |
| mwhahaha | bandini: can't you just add the firewall rule to environments/undercloud.yaml maybe? | 13:53 |
| *** d0ugal has joined #tripleo | 13:55 | |
| openstackgerrit | Yurii Prokulevych proposed openstack/tripleo-upgrade stable/rocky: Assert services/containers are started after reboot. https://review.openstack.org/605632 | 13:55 |
| openstackgerrit | Martin André proposed openstack/tripleo-common stable/rocky: Switch to openshift 3.10 https://review.openstack.org/607193 | 13:57 |
| openstackgerrit | Martin André proposed openstack/tripleo-common stable/rocky: Switch to origin-docker-build https://review.openstack.org/607194 | 13:57 |
| openstackgerrit | Martin André proposed openstack/tripleo-common stable/rocky: Add container-registry image to openshift master role https://review.openstack.org/607195 | 13:57 |
| openstackgerrit | Martin André proposed openstack/tripleo-common stable/rocky: Tag openshift images for Infra service https://review.openstack.org/607196 | 13:57 |
| jaosorior | #startmeeting tripleo | 14:00 |
| openstack | Meeting started Tue Oct 2 14:00:43 2018 UTC and is due to finish in 60 minutes. The chair is jaosorior. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| *** openstack changes topic to " (Meeting topic: tripleo)" | 14:00 | |
| openstack | The meeting name has been set to 'tripleo' | 14:00 |
| Tengu | «o/ more or less - another mtg in // | 14:00 |
| *** thrash is now known as thrash|bbl | 14:01 | |
| jaosorior | #topic agenda | 14:01 |
| *** openstack changes topic to "agenda (Meeting topic: tripleo)" | 14:01 | |
| mwhahaha | if you're not currently in 4 meetings at once, you're doing it wrong? :D | 14:01 |
| jaosorior | * Review past action items | 14:01 |
| jaosorior | * One off agenda items | 14:01 |
| jaosorior | * Squad status | 14:01 |
| jaosorior | * Bugs & Blueprints | 14:01 |
| jaosorior | * Projects releases or stable backports | 14:01 |
| jaosorior | * Specs | 14:01 |
| jaosorior | * open discussion | 14:01 |
| jaosorior | Anyone can use the #link, #action and #info commands, not just the moderatorǃ | 14:01 |
| jaosorior | Hey folks! who's around? | 14:01 |
| Tengu | «o/ more or less - another mtg in // | 14:01 |
| jfrancoa | o/ | 14:01 |
| bandini | o/ | 14:01 |
| rfolco | o/ | 14:01 |
| matbu | o/ | 14:01 |
| jbadiapa | o/ | 14:01 |
| mschuppert | o/ | 14:01 |
| beagles | o/ | 14:01 |
| d0ugal | o/ | 14:02 |
| ksambor | o/ | 14:02 |
| marios|rover | o/ | 14:02 |
| dpeacock | o7 | 14:02 |
| mandre | o/ | 14:02 |
| jrist | o/ | 14:03 |
| jaosorior | #topic review past action items | 14:03 |
| *** openstack changes topic to "review past action items (Meeting topic: tripleo)" | 14:03 | |
| jaosorior | We had one: mwhahaha to move rocky-rc2 bugs out since it's been released | 14:03 |
| mwhahaha | done | 14:03 |
| jaosorior | thanks mwhahaha | 14:03 |
| mwhahaha | rocky-rc2 has also been closed out in launchpad | 14:04 |
| jaosorior | #topic one off agenda items | 14:04 |
| *** openstack changes topic to "one off agenda items (Meeting topic: tripleo)" | 14:04 | |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-meeting-items | 14:04 |
| jaosorior | No one off agenda items, it seems | 14:04 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Load iscsi_tcp module from the host. https://review.openstack.org/605450 | 14:05 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Load ip_vs module from the host https://review.openstack.org/605446 | 14:05 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Load dm-multipath module from the host. https://review.openstack.org/605452 | 14:05 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Load openvswitch module from the host https://review.openstack.org/607147 | 14:05 |
| jaosorior | Ah! | 14:05 |
| jaosorior | friendly reminder from the CI folks: | 14:05 |
| jaosorior | CI Community Meeting starts immediately upon this meeting closing in #oooq. This is our team's weekly "open office hours." All are welcome! Ask/discuss anything, we don't bite. Agenda (add items freely) --> https://etherpad.openstack.org/p/tripleo-ci-squad-meeting. | 14:05 |
| weshay | 0/ | 14:05 |
| rfolco | thanks :) | 14:05 |
| jaosorior | \o | 14:05 |
| jaosorior | #topic Squad status | 14:06 |
| *** openstack changes topic to "Squad status (Meeting topic: tripleo)" | 14:06 | |
| jaosorior | ci | 14:06 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting | 14:06 |
| jaosorior | upgrade | 14:06 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status | 14:06 |
| jaosorior | containers | 14:06 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-containers-squad-status | 14:06 |
| jaosorior | integration | 14:06 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-integration-squad-status | 14:06 |
| jaosorior | ui/cli | 14:06 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status | 14:06 |
| jaosorior | validations | 14:06 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-validations-squad-status | 14:06 |
| jaosorior | networking | 14:06 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-networking-squad-status | 14:06 |
| jaosorior | workflows | 14:06 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-workflows-squad-status | 14:06 |
| jaosorior | security | 14:06 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-security-squad | 14:06 |
| jaosorior | edge | 14:06 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-edge-squad-status | 14:06 |
| jaosorior | Anything specific the squads want to report in this meeting? | 14:06 |
| Tengu | hmm maybe a small word for validation? | 14:08 |
| jaosorior | sure | 14:08 |
| Tengu | the validation framework will be pushed as a tech-prev for next release at most. | 14:08 |
| Tengu | as the spec was accepted end of last week, we're now good for basic implem. | 14:08 |
| holser_ | o/ | 14:08 |
| owalsh | o/ | 14:09 |
| chem | o/ | 14:09 |
| Tengu | #link http://specs.openstack.org/openstack/tripleo-specs/specs/stein/validation-framework.html in case. | 14:09 |
| dpeacock | Tengu: next release as in 16? | 14:09 |
| Tengu | dpeacock: Stein, yep. We won't be able to do more than basic implem, with tech-prev as the "maximum goal". | 14:09 |
| dpeacock | ack | 14:09 |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 14:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795411 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795418 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 14:10 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 14:10 |
| jaosorior | Tengu: so, some basic building blocks for it in this release, and an MVP in the T release? | 14:10 |
| openstack | Launchpad bug 1795418 in tripleo "The containers-multinode scenario jobs fail overcloud deploy on docker-puppet step3 or starting containers" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 14:10 |
| Tengu | jaosorior: probably yep. | 14:10 |
| Tengu | jaosorior: first meeting with the validation team next Tuesday morning (CET) | 14:10 |
| Tengu | I'll post update on a regular basis. | 14:11 |
| jaosorior | Tengu: at what time? | 14:11 |
| openstackgerrit | Sam Doran proposed openstack/tripleo-ansible master: Use generic names for container platform https://review.openstack.org/600847 | 14:11 |
| openstackgerrit | Sam Doran proposed openstack/tripleo-ansible master: Create Ansible module for gathering podman images facts https://review.openstack.org/605229 | 14:11 |
| dpeacock | Tengu: we should start meeting within NA friendly hours regularly too then | 14:11 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Switch fs027 to deploy with podman https://review.openstack.org/600517 | 14:11 |
| Tengu | dpeacock: will keep you in touch :). | 14:12 |
| jaosorior | Tengu thanks for the update | 14:12 |
| jaosorior | #topic bugs & blueprints | 14:14 |
| *** openstack changes topic to "bugs & blueprints (Meeting topic: tripleo)" | 14:14 | |
| jaosorior | #link https://launchpad.net/tripleo/+milestone/stein-1 | 14:14 |
| jaosorior | For Stein we currently have 31 blueprints open in Launchpad. | 14:14 |
| jaosorior | Bugs: 778 (-5) stein-1. 102 (-1) open Storyboard bugs. | 14:14 |
| openstack | bug 778 in helix-player (Ubuntu) "installscript fails to create a symbolic link for plugin registration (mozilla-firefox)" [Medium,Invalid] https://launchpad.net/bugs/778 | 14:14 |
| jaosorior | #link https://storyboard.openstack.org/#!/project_group/76 | 14:14 |
| *** med_ has quit IRC | 14:15 | |
| jaosorior | #topic projects releases or stable backports | 14:16 |
| *** openstack changes topic to "projects releases or stable backports (Meeting topic: tripleo)" | 14:16 | |
| jaosorior | #topic specs | 14:18 |
| *** openstack changes topic to "specs (Meeting topic: tripleo)" | 14:18 | |
| jaosorior | #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open | 14:18 |
| jaosorior | anybody has a spec that might need attention? | 14:18 |
| *** bogdando has quit IRC | 14:19 | |
| jaosorior | #topic open discussion | 14:20 |
| *** openstack changes topic to "open discussion (Meeting topic: tripleo)" | 14:20 | |
| jaosorior | anything else folks want to bring up to the meeting? | 14:20 |
| jrist | can we get more eyes on validations reviews? | 14:21 |
| jrist | please | 14:21 |
| jrist | https://review.openstack.org/#/q/project:openstack/tripleo-validations | 14:21 |
| *** two_oes has quit IRC | 14:21 | |
| jaosorior | jrist: added to my queue. Thanks for the poke. | 14:22 |
| jrist | thanks jaosorior | 14:22 |
| jaosorior | jrist: who's core in validations? | 14:22 |
| jrist | jaosorior: I believe gchamoul, florianf, and akrivoka | 14:23 |
| jrist | maybe more? | 14:23 |
| *** Vorrtex has joined #tripleo | 14:24 | |
| jaosorior | Alright! That was a fast one :D | 14:27 |
| jaosorior | thanks for joining | 14:27 |
| jaosorior | #endmeeting | 14:27 |
| *** openstack changes topic to "Welcome to Rocky | CI Status: RED, DO NOT WORKFLOW OR RECHECK (unless explicitly for CI fixing) https://docs.openstack.org/tripleo-docs/latest/" | 14:27 | |
| openstack | Meeting ended Tue Oct 2 14:27:19 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:27 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/tripleo/2018/tripleo.2018-10-02-14.00.html | 14:27 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/tripleo/2018/tripleo.2018-10-02-14.00.txt | 14:27 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/tripleo/2018/tripleo.2018-10-02-14.00.log.html | 14:27 |
| weshay | ci meeting nowish | 14:27 |
| jaosorior | weshay: nowish? | 14:27 |
| rfolco | #oooq https://bluejeans.com/5878458097 | 14:27 |
| weshay | jaosorior, :) | 14:28 |
| weshay | jaosorior, is | 14:28 |
| weshay | it's open door | 14:28 |
| weshay | the door remains open for $time | 14:28 |
| gfidente | weshay I am not sure if I will be able to join the call | 14:28 |
| gfidente | had a topic | 14:28 |
| gfidente | marios knows about it as well | 14:28 |
| *** lblanchard has quit IRC | 14:30 | |
| marios|rover | gfidente: joining sure will bring it up | 14:30 |
| gfidente | marios++ | 14:31 |
| weshay | mwhahaha, you need anything | 14:36 |
| weshay | ? | 14:36 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates stable/rocky: Set container images for CNS https://review.openstack.org/607204 | 14:36 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates stable/rocky: Define keepalived service in environments/docker.yaml https://review.openstack.org/607205 | 14:36 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates stable/rocky: Move to openshift-ansible 3.10 https://review.openstack.org/607206 | 14:36 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates stable/rocky: Use openshift_cockpit_deployer_image directly https://review.openstack.org/607207 | 14:36 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates stable/rocky: Add simple validation that OpenShift is deployed https://review.openstack.org/607208 | 14:36 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates stable/rocky: Refactor openshift services for composable roles https://review.openstack.org/607209 | 14:36 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates stable/rocky: Fix openshift new node detection https://review.openstack.org/607210 | 14:36 |
| mwhahaha | weshay: maybe? in meeting | 14:36 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates stable/rocky: Clean up previous osa inventory dir before deployment https://review.openstack.org/607211 | 14:36 |
| openstackgerrit | Merged openstack/tripleo-ansible master: Use generic names for container platform https://review.openstack.org/600847 | 14:39 |
| weshay | jaosorior, http://dashboard-ci.tripleo.org/d/cEEjGFFmz/cockpit?orgId=1&panelId=61&fullscreen | 14:40 |
| *** med_ has joined #tripleo | 14:42 | |
| *** bfournie has joined #tripleo | 14:43 | |
| bandini | marios|rover: we are a bit at loss around https://bugs.launchpad.net/tripleo/+bug/1795418 fyi | 14:43 |
| openstack | Launchpad bug 1795411 in tripleo "duplicate for #1795418 The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Fix released] - Assigned to Marios Andreou (marios-b) | 14:43 |
| *** dciabrin has joined #tripleo | 14:44 | |
| bandini | oh alex updated it | 14:44 |
| * bandini looks | 14:44 | |
| weshay | jaosorior, http://grafana.openstack.org/?orgId=1 | 14:45 |
| marios|rover | https://etherpad.openstack.org/p/tripleo-ci-squad-meeting jaosorior | 14:46 |
| jaosorior | marios|rover: ack | 14:46 |
| *** chem has quit IRC | 14:47 | |
| *** iranzo has quit IRC | 14:49 | |
| Tengu | jaosorior: fyi, I have a complete undercloud running with podman, and full selinux enabled: https://gist.github.com/cjeanner/1f6a7ad930f3b4cdff9d86000a28f327 | 14:49 |
| jaosorior | niiiice :D!!! | 14:50 |
| Tengu | jaosorior: it embeds my modprobe modifications as well, so that I don't hit any serious issue. | 14:50 |
| Tengu | still, I'll need to ensure all is working on that undercloud instance. | 14:50 |
| Tengu | but, well, that's it: it's deployed. | 14:51 |
| Tengu | but apparently: I can run openstack commands, as I could upload the overcloud images, list them and so on. wow. | 14:51 |
| *** agurenko has quit IRC | 14:56 | |
| mwhahaha | weshay: so i don't think i have anything today, CI is in a better shape now right? | 14:57 |
| *** dprince has joined #tripleo | 14:58 | |
| weshay | mwhahaha, yes.. thanks to you | 14:59 |
| marios|rover | mwhahaha: thanks for jumping on that issue yesterday | 15:01 |
| marios|rover | mwhahaha: ++ | 15:01 |
| marios|rover | mwhahaha: the things in bold at https://review.rdoproject.org/etherpad/p/ruckrover-sprint20 (i.e. bugs https://bugs.launchpad.net/tripleo/+bug/1795009 https://bugs.launchpad.net/tripleo/+bug/1795411 https://bugs.launchpad.net/tripleo/+bug/1795418 once those fixes merge our check scenario jobs should be in much better shape | 15:01 |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 15:01 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Fix released] - Assigned to Marios Andreou (marios-b) | 15:01 |
| openstack | Launchpad bug 1795411 in tripleo "duplicate for #1795418 The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Fix released] - Assigned to Marios Andreou (marios-b) | 15:01 |
| mwhahaha | k i rechecked my patch for 1795009 so hopefully that gets resolved | 15:01 |
| *** med_ has quit IRC | 15:03 | |
| *** ksambor has quit IRC | 15:08 | |
| mwhahaha | jaosorior: when you have a second, https://review.openstack.org/#/c/605156/ | 15:09 |
| *** boazel has quit IRC | 15:09 | |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 15:10 |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 15:10 |
| jaosorior | mwhahaha: what's the difference between standalone-overcloud and standalone-tripleo? | 15:11 |
| *** d0ugal has quit IRC | 15:12 | |
| mwhahaha | jaosorior: one is for use with an undercloud, one is not | 15:12 |
| mwhahaha | jaosorior: it's in the description | 15:12 |
| *** d0ugal has joined #tripleo | 15:12 | |
| jaosorior | mwhahaha: yeah, read the description. Still don't get it. | 15:13 |
| openstackgerrit | Yurii Prokulevych proposed openstack/tripleo-upgrade stable/queens: [QUEENS-only] Use ceph-ansible during P->Q upgrade. https://review.openstack.org/607286 | 15:13 |
| dalvarez | may i kindly ask for love to this simple patch? https://review.openstack.org/#/c/605406/ thanks a lot! | 15:14 |
| mwhahaha | jaosorior: standalone-tripleo.yaml is for 'openstack tripleo deploy' and standalone-overcloud.yaml is for 'openstack overcloud deploy' | 15:14 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: Render playbooks minimal, move everything to roles https://review.openstack.org/607288 | 15:15 |
| jaosorior | mwhahaha: lol yeah, that's what I'm getting confused with now. (I'm only using openstack overcloud deploy). What's the difference now between those two commands? | 15:18 |
| mwhahaha | jaosorior: openstack tripleo deploy is what the standalone/undercloud use (no undercloud) | 15:19 |
| mwhahaha | jaosorior: openstack overcloud deploy requires an undercloud | 15:19 |
| jaosorior | gotcha | 15:19 |
| mwhahaha | so one is a single node action and the other requires at least 2 nodes | 15:19 |
| *** mwhahaha changes topic to "Welcome to Rocky | CI Status: ORANGEish (seems to be stable, please be careful) | https://docs.openstack.org/tripleo-docs/latest/" | 15:22 | |
| *** chem has joined #tripleo | 15:23 | |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: fedora28 standalone job definition https://review.openstack.org/595374 | 15:25 |
| *** dprince has quit IRC | 15:27 | |
| *** aufi_ has joined #tripleo | 15:32 | |
| *** maufart__ has quit IRC | 15:34 | |
| *** jabadia has joined #tripleo | 15:36 | |
| *** jfrancoa has quit IRC | 15:38 | |
| *** hamzy has quit IRC | 15:40 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: Add rdoinfo base job and playbook https://review.openstack.org/607290 | 15:44 |
| *** yprokule has quit IRC | 15:45 | |
| jaosorior | panda: could you checkt his out https://review.openstack.org/#/c/602171/ ? | 15:50 |
| Tengu | btw, can any of your review this small, tiny doc change? https://review.openstack.org/#/c/589156/ thank you :) | 15:50 |
| Tengu | thanks jaosorior :) | 15:51 |
| Tengu | holser_: still here? | 15:52 |
| holser_ | yeah | 15:52 |
| holser_ | Tengu, what's up? | 15:52 |
| Tengu | holser_: well, we didn't finish the discussion about modprobe I think ;) | 15:53 |
| holser_ | Tengu, I like your patches | 15:53 |
| Tengu | holser_: just want to ensure we get a correct way of doing tings, and maybe future steps for a better integration | 15:54 |
| holser_ | but I see bigger issue | 15:54 |
| Tengu | holser_: that's a nice start :). | 15:54 |
| gfidente | weshay holser_ whers is the trello card we talked about? :D | 15:54 |
| Tengu | holser_: like the day-2 for updates right? | 15:54 |
| holser_ | Tengu, right | 15:54 |
| Tengu | holser_: dunno if you saw my (long) answer to the thread. I really hope the ideas inside might trigger some nice dev and talks. | 15:55 |
| holser_ | Sean Mooney agreed that should be in runtime OCI | 15:55 |
| holser_ | but that's verrrryyyyyy long process | 15:55 |
| holser_ | as standards are always slow | 15:56 |
| Tengu | :) | 15:56 |
| weshay | gfidente, panda https://tree.taiga.io/project/tripleo-ci-board/epic/97 | 15:56 |
| Tengu | holser_: so you're OK if those patches are merged as a first (tiny) step toward a better solution, and provide maybe advices about how to go next? I'm not a "container-guru", so I probably don't have the big picture. | 15:57 |
| holser_ | so, there are 2 fronts of work. 1st - your patches, 2nd - OCI runtime changes | 15:57 |
| gfidente | what is taiga.io | 15:57 |
| gfidente | this is | 15:57 |
| holser_ | Tengu - I wasn't against your patches | 15:57 |
| gfidente | weshay or how do I add people to the card/story? | 15:57 |
| holser_ | I wanted to bring questions to start any movements on container sides | 15:58 |
| holser_ | otherwise, containers will remain for userspace only | 15:58 |
| Tengu | holser_: ok :). well, at least it started on tripleo and kolla side. but we need to port that further. | 15:58 |
| weshay | gfidente, unfortunately you/they need to be added to taiga | 15:59 |
| *** jcoufal has quit IRC | 15:59 | |
| panda | jaosorior: /602171/ ssl dir works with all branches ? | 15:59 |
| panda | gfidente: sent invitation | 15:59 |
| panda | gfidente: to taiga | 15:59 |
| weshay | gfidente, can you register, panda can you add him to the project? | 15:59 |
| holser_ | Tengu, it started on kolla/tripleo because we consume containers. So we provide feedback as operators | 15:59 |
| weshay | panda, thanks | 15:59 |
| holser_ | and issues we face into | 15:59 |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO https://review.openstack.org/604298 | 16:00 |
| openstackgerrit | Merged openstack/tripleo-specs master: Remove useless placeholders in spec directories https://review.openstack.org/589156 | 16:00 |
| *** aufi_ has quit IRC | 16:00 | |
| holser_ | Tengu, I understand we create a lot of limitations, some bumpy procedures for upgrade/update, migration, securing | 16:00 |
| holser_ | but I believe, we should influence container folks to adjust runtime not just to setup cgroups, filesystem and pid1 process | 16:01 |
| Tengu | holser_: who is "we"? container team, or tripleo, or.. ? | 16:02 |
| holser_ | we - ooo | 16:02 |
| Tengu | ok | 16:02 |
| holser_ | they - container team | 16:02 |
| Tengu | jut want to get it right. | 16:02 |
| holser_ | that's fine | 16:02 |
| holser_ | Imagine DPDK case | 16:02 |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: WIP: Add support for multi-distros release config https://review.openstack.org/607294 | 16:02 |
| holser_ | or huge memory case | 16:02 |
| jaosorior | panda: it should, yeah. They're the same files (pretty much) than the other ones. | 16:03 |
| holser_ | when container runtime may require to allocate huge pages to process inside of container | 16:03 |
| holser_ | privileged mode is not silver bullet | 16:03 |
| Tengu | (elasticsearch :D) | 16:03 |
| Tengu | right | 16:03 |
| *** jcoufal has joined #tripleo | 16:03 | |
| Tengu | more than aggree | 16:03 |
| jaosorior | panda: those files exist since pike | 16:03 |
| panda | jaosorior: and we don't care about ocata anymore | 16:04 |
| panda | ok | 16:04 |
| holser_ | so, kubelet + k8s have good scheduler but there should be some changes on runtime level to allow engines to setup containers | 16:04 |
| *** dciabrin has quit IRC | 16:06 | |
| holser_ | Tengu - so, let's do in the way we do right now (your patches). However, at the same time we might ask container folks to think about specification update to cover more customer use cases. | 16:06 |
| Tengu | holser_: fine for me | 16:07 |
| *** dciabrin has joined #tripleo | 16:07 | |
| Tengu | we already pushed work for them as we stumbled on "some" issues while trying the podman integration in tripleo. | 16:07 |
| Tengu | + features | 16:07 |
| holser_ | yeah | 16:08 |
| holser_ | I know | 16:08 |
| holser_ | read EmilienM saga's email about status of Podman in tripleo | 16:08 |
| Tengu | ;) | 16:08 |
| holser_ | I really love reading it | 16:08 |
| gfidente | panda weshay so how is best to add comments to the epic, "we need to figure how to trigger the correct job" "we need to figure how to checkout a PR from github" ? | 16:08 |
| EmilienM | holser_: it's like Dallas | 16:08 |
| Tengu | hehe | 16:09 |
| EmilienM | we don't know what happens tomorrow | 16:09 |
| Tengu | unless I do hope there will be an end. | 16:09 |
| Tengu | eventually. | 16:09 |
| EmilienM | no end | 16:09 |
| holser_ | EmilienM hope is near :D | 16:10 |
| weshay | gfidente, i'll add an example | 16:10 |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 16:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 16:10 |
| weshay | gfidente, sorry for the one off tool.. you can blame the sf folks :) | 16:10 |
| *** agopi has quit IRC | 16:10 | |
| weshay | gfidente, https://tree.taiga.io/project/tripleo-ci-board/us/98 | 16:11 |
| weshay | gfidente, add it here https://tree.taiga.io/project/tripleo-ci-board/epic/97 | 16:11 |
| weshay | Related user stories | 16:11 |
| gfidente | thanks | 16:13 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates master: Chunk up ceph-ansible output to prevent clogging the CLI https://review.openstack.org/607302 | 16:17 |
| openstackgerrit | Merged openstack/tripleo-common stable/rocky: Add config_download_timeout input https://review.openstack.org/606942 | 16:18 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Honor timeout with config-download https://review.openstack.org/586358 | 16:18 |
| gfidente | jistr wait why no_log ? | 16:19 |
| gfidente | an that is the ansible task only, it's still logging | 16:19 |
| jistr | gfidente: because otherwise it's going to kill the Zaqar transport of logs | 16:19 |
| gfidente | right mistral doesn't store but it's still flying | 16:19 |
| jistr | gfidente: yea that's the intention. See the next task, that's where i print the actual logs. | 16:19 |
| gfidente | why do you want to print it though? | 16:20 |
| gfidente | who cares/ | 16:20 |
| gfidente | I'd thrash it | 16:20 |
| jistr | gfidente: we have been doing that already i think? | 16:20 |
| gfidente | in the workflow we tell mistral to trash it | 16:20 |
| jistr | gfidente: it's for https://bugzilla.redhat.com/show_bug.cgi?id=1631382 -- essentially the ceph-ansible output is printed there, but it comes as an escaped mess, a giant single-line text | 16:21 |
| openstack | bugzilla.redhat.com bug 1631382 in python-tripleoclient "[UPGRADES][14] external-upgrade failed during ceph upgrade" [High,Assigned] - Assigned to jstransk | 16:21 |
| jistr | and that seems to kill something on the way Mistral-Zaqar-CLI, didn't find out exactly what it was | 16:21 |
| jistr | but i'm hoping chunking up the output into actual lines could help it | 16:22 |
| *** holser__ has joined #tripleo | 16:22 | |
| *** ade_lee has joined #tripleo | 16:22 | |
| jistr | and it would also be readable, which previously it wasn't much :) | 16:22 |
| jistr | gfidente: ^ | 16:22 |
| gfidente | jistr beause we used to point people to the actual ansible log | 16:22 |
| *** holser_ has quit IRC | 16:22 | |
| gfidente | l463 | 16:23 |
| gfidente | i'd happily throw it away and keep the .log file only | 16:23 |
| jistr | gfidente: yea i recall that... i'm actually not sure what controls how much detail we print... Whether we run verbose by default. I guess we do? | 16:24 |
| * jistr searches | 16:24 | |
| gfidente | for ceph-ansible it's a param I think it defaults to 1 -v | 16:24 |
| jistr | the action default is here | 16:25 |
| jistr | https://github.com/openstack/tripleo-common/blob/master/tripleo_common/actions/ansible.py#L285 | 16:25 |
| * jistr looks for overrides | 16:25 | |
| *** rfolco has quit IRC | 16:26 | |
| jistr | gfidente: https://github.com/openstack/tripleo-common/blob/17fb3b1dea6a354a5a2622d42c29be0708c36923/workbooks/package_update.yaml#L100 | 16:26 |
| gfidente | jistr l473 | 16:26 |
| gfidente | in ceph-base.yaml | 16:26 |
| jistr | gfidente: ack. It matters what both processes run (outer tripleo ansible + inner ceph-ansible) | 16:27 |
| jistr | gfidente: looks like both use 1 -v | 16:27 |
| gfidente | oh I didn't know they'd interact there | 16:27 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Enable TLS in the internal network for Octavia API https://review.openstack.org/597038 | 16:28 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Pass parameters for TLS proxy in front of Octavia-API https://review.openstack.org/597039 | 16:28 |
| jistr | gfidente: if we didn't use -v on outer layer, we'd not show the output in the main playbook log no matter how many -v would ceph-ansible use | 16:28 |
| gfidente | jistr but the -v affects what is written in the ANSIBLE_LOG no? | 16:28 |
| gfidente | which ceph-ansible redefines for its own run | 16:29 |
| jistr | gfidente: yea but even if we don't pass any -v to ceph-ansible, it will still produce many lines of output, which translates into an incomprehensible single-line mess in the outer log currently | 16:29 |
| jistr | gfidente: i'm checking whether -v only affects ANSIBLE_LOG or also stdout, but i don't think it matters much | 16:31 |
| jistr | gfidente: based on what yprokule uploaded to that BZ, the -v probably affects stdout too. But even a non-verbose output would be a mess :) | 16:32 |
| *** dprince has joined #tripleo | 16:32 | |
| gfidente | jistr so let's put things differently | 16:33 |
| gfidente | +1 on the existing submission | 16:33 |
| gfidente | +2 if you remove the print | 16:33 |
| gfidente | unless I miss why the print is also useful | 16:34 |
| jistr | gfidente: hmm i'm not sure about the print, it's both good and bad :) It produces a lot of output which is not good if all's running fine, good when things break. | 16:36 |
| nhicher | weshay, sshnaidm: do you have any idea why tripleo-ci-centos-7-undercloud-containers failed on https://review.openstack.org/#/c/596432/ | 16:37 |
| jistr | gfidente: i could remove it but i'm worried that we'll be answering complaints that "it broke and i don't know why" | 16:37 |
| weshay | if I had a nickel | 16:37 |
| jistr | gfidente: and we'll be sending users to fish for logs in /var/lib/mistral/... instead of having the breakage reason as the last thing in the CLI output of a command they just ran | 16:37 |
| jistr | gfidente: so IDK, i'm a bit on a fence here, so far leaning a bit more towards printing it | 16:38 |
| jistr | gfidente: i could also print it only if it fails? | 16:38 |
| *** jpich has quit IRC | 16:39 | |
| weshay | nhicher, http://logs.openstack.org/32/596432/44/check/tripleo-ci-centos-7-undercloud-containers/c08168f/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2018-10-02_12_39_02 | 16:39 |
| jistr | gfidente: maybe there's discrepancy between deploy and upgrade though. I think maybe on deploy we don't run the outer layer with -v, but on upgrade we do. That's probably why Yurii was able to deploy Ceph but not upgrade it -- it only overflowed on upgrade. | 16:40 |
| sshnaidm | nhicher, "failure: repodata/3d88a8e82ac381f195de096c52e79387ad9510cb724e6c48f37760de7efa2a05-primary.sqlite.bz2 from quickstart-centos-extras: [Errno 256] No more mirrors to try. | 16:41 |
| jistr | gfidente: i'll WIP the patch and look a bit more tomorrow, it's hard for me to think thoroughly after a long day :D | 16:41 |
| sshnaidm | nhicher, seems like transient yum failure | 16:41 |
| gfidente | jistr I think you spent enough time on it already | 16:41 |
| gfidente | I'll comment and vote anyway | 16:41 |
| sshnaidm | nhicher, it's here: http://logs.openstack.org/32/596432/44/check/tripleo-ci-centos-7-undercloud-containers/c08168f/logs/undercloud/home/zuul/install-undercloud.log.txt.gz | 16:41 |
| nhicher | sshnaidm: ok, it's an upstream job | 16:42 |
| weshay | nhicher, looking for which container booofed | 16:43 |
| sshnaidm | mwhahaha, EmilienM, weshay do we want to track install-undercloud.log in elasticsearch? it's logs from ansible-modify-containers role | 16:43 |
| weshay | sshnaidm, it's logged to the journal still isn't it :( ? | 16:44 |
| sshnaidm | weshay, no, it's in install-undercloud.log | 16:44 |
| sshnaidm | weshay, http://logs.openstack.org/32/596432/44/check/tripleo-ci-centos-7-undercloud-containers/c08168f/logs/undercloud/home/zuul/install-undercloud.log.txt.gz | 16:44 |
| *** agopi has joined #tripleo | 16:45 | |
| sshnaidm | weshay, just weird name for this log.. | 16:45 |
| weshay | nhicher, sshnaidm http://logs.openstack.org/32/596432/44/check/tripleo-ci-centos-7-undercloud-containers/c08168f/logs/undercloud/var/log/journal.txt.gz#_Oct_02_12_40_31 | 16:45 |
| sshnaidm | weshay, idk what's that | 16:46 |
| weshay | sshnaidm, ansible-modify-containers or tripleo-container-image-prepare? | 16:46 |
| weshay | the latter is what nhicher's job booofed on | 16:46 |
| sshnaidm | weshay, image-prepare | 16:46 |
| weshay | ya.. that goes to the journal | 16:46 |
| weshay | sshnaidm, for example http://logs.openstack.org/32/596432/44/check/tripleo-ci-centos-7-undercloud-containers/c08168f/logs/undercloud/var/log/journal.txt.gz#_Oct_02_12_38_08 | 16:47 |
| sshnaidm | weshay, please, look at link I posted | 16:47 |
| weshay | sshnaidm, ah | 16:48 |
| openstackgerrit | Merged openstack/puppet-tripleo master: adding deployment_type fact in support https://review.openstack.org/605478 | 16:48 |
| weshay | ic | 16:48 |
| weshay | MERGE PARTY | 16:48 |
| weshay | sshnaidm, the name of the log sucks but yes we should add it to logstash | 16:50 |
| gfidente | we're getting old for partying | 16:50 |
| gfidente | if it wasn't for weshay | 16:50 |
| weshay | meh | 16:50 |
| weshay | undercloud_install.log and install-undercloud.log is u l g y | 16:51 |
| *** dtantsur is now known as dtantsur|afk | 16:52 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Add log of ansible modify container role to logstash https://review.openstack.org/607316 | 16:53 |
| sshnaidm | weshay, yeah ^^ | 16:53 |
| weshay | ssbarnea, are you going to add the f-28 release files? | 16:54 |
| ssbarnea|bkp2 | weshay: yes but I not sure what to put in them for start. | 16:55 |
| weshay | ssbarnea|bkp2, how about in a review :) ? | 16:56 |
| * weshay runs | 16:56 | |
| weshay | ssbarnea|bkp2, let's chat | 16:56 |
| ssbarnea|bkp2 | weshay https://bluejeans.com/2655417928 | 16:56 |
| openstackgerrit | Jill Rouleau proposed openstack/tripleo-ansible master: Fix README filename https://review.openstack.org/605878 | 16:57 |
| openstackgerrit | Carlos Goncalves proposed openstack/tripleo-heat-templates stable/rocky: Pass parameters for TLS proxy in front of Octavia-API https://review.openstack.org/607318 | 16:59 |
| openstackgerrit | Carlos Goncalves proposed openstack/puppet-tripleo stable/rocky: Enable TLS in the internal network for Octavia API https://review.openstack.org/607319 | 16:59 |
| *** boazel has joined #tripleo | 17:01 | |
| openstackgerrit | Carlos Goncalves proposed openstack/puppet-tripleo stable/queens: Enable TLS in the internal network for Octavia API https://review.openstack.org/607322 | 17:01 |
| *** derekh has quit IRC | 17:03 | |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-quickstart-extras master: Add option to install chrome when running tempest https://review.openstack.org/607323 | 17:04 |
| *** thrash|bbl is now known as thrash | 17:08 | |
| weshay | ssbarnea|bkp2, https://review.openstack.org/#/c/607294/ | 17:08 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: Set mysql connect timeout in the undercloud https://review.openstack.org/606774 | 17:09 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: Cleanup ControllerStorageNfs role https://review.openstack.org/607015 | 17:09 |
| weshay | ssbarnea|bkp2, https://review.openstack.org/#/c/595374/ | 17:10 |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795009 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 17:10 |
| openstack | Launchpad bug 1795009 in tripleo "the scenario001/004 multinode-oooq-container job is failing for Queens and Pike for overcloud deployment error "overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:" " [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 17:10 |
| weshay | ssbarnea|bkp2, https://review.openstack.org/#/c/605843/6..7/toci_quickstart.sh | 17:11 |
| *** rf0lc0 has joined #tripleo | 17:11 | |
| mwhahaha | sshnaidm: i think so | 17:12 |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-quickstart master: Add new featureset for running tripleo-ui tests https://review.openstack.org/607325 | 17:14 |
| *** jpena is now known as jpena|off | 17:14 | |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-quickstart master: Add new featureset for running tripleo-ui tests https://review.openstack.org/607325 | 17:14 |
| *** dciabrin has quit IRC | 17:15 | |
| *** dciabrin has joined #tripleo | 17:16 | |
| *** salmankhan has quit IRC | 17:16 | |
| weshay | ssbarnea|bkp2, https://review.openstack.org/#/c/602387/13/config/release/tripleo-ci/Fedora/master.yml | 17:17 |
| *** lblanchard has joined #tripleo | 17:17 | |
| openstackgerrit | Honza Pokorny proposed openstack-infra/tripleo-ci master: Add selenium job https://review.openstack.org/607327 | 17:19 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: Move nova-metadata api to httpd wsgi https://review.openstack.org/606791 | 17:21 |
| weshay | ssbarnea|bkp2, http://codesearch.openstack.org/?q=tripleo-ci-centos-7-standalone&i=nope&files=&repos= | 17:23 |
| weshay | ssbarnea|bkp2, http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/zuul.d/multinode-jobs.yaml#n25 | 17:23 |
| *** amoralej is now known as amoralej|off | 17:24 | |
| *** trown is now known as trown|lunch | 17:24 | |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: fedora28 standalone job definition https://review.openstack.org/595374 | 17:25 |
| *** shardy has quit IRC | 17:26 | |
| *** iranzo has joined #tripleo | 17:26 | |
| *** homeski has joined #tripleo | 17:26 | |
| *** hamzy has joined #tripleo | 17:27 | |
| *** gkadam has quit IRC | 17:28 | |
| homeski | I'm trying to run thru the OSP10 -> 13 FFU. When running "openstack overcloud ffwd-upgrade run" it is failing on fast_forward_upgrade_bootstrap_role_tasks.yaml | 17:29 |
| homeski | error while evaluating conditional .... role_name == 'Controller' and ansible_hostname == Controller[0] ... 'Controller' is undefined | 17:29 |
| *** rdopiera has quit IRC | 17:29 | |
| homeski | any ideas why this is failing .. it seems to be pretty straight forward | 17:29 |
| openstackgerrit | Carlos Goncalves proposed openstack/tripleo-heat-templates stable/queens: Pass parameters for TLS proxy in front of Octavia-API https://review.openstack.org/607329 | 17:29 |
| mwhahaha | homeski: do you not have a controller role defined for 10? | 17:29 |
| * mwhahaha is guessing | 17:29 | |
| mwhahaha | dhellmann: ci is in a happier spot if you want to propose your cleanup stuff for the python3 bits | 17:29 |
| homeski | Well I checked the generated /home/stack/tripleo-ansible-inventory.yaml file and it definitely has a line "role_name: Controller" for the Controller group | 17:31 |
| homeski | I'm guessing that's where it should be pulling from | 17:31 |
| openstackgerrit | James Slagle proposed openstack/python-tripleoclient stable/rocky: Honor timeout with config-download https://review.openstack.org/607330 | 17:31 |
| homeski | and not sure if it's relevant to his playbook, but I am passing in a custom role_data.yaml (when running the prepare command) which as a role with name: Controller defined | 17:32 |
| *** lblanchard has quit IRC | 17:37 | |
| *** jabadia has quit IRC | 17:38 | |
| openstackgerrit | James Slagle proposed openstack/python-tripleoclient master: Pass stack name to enable_ssh_admin workflow https://review.openstack.org/606964 | 17:38 |
| openstackgerrit | James Slagle proposed openstack/python-tripleoclient master: Update deployment status when deploying or failed https://review.openstack.org/607182 | 17:38 |
| homeski | I'm guessing I'm missing something simple :| | 17:39 |
| *** boazel has quit IRC | 17:39 | |
| *** AJaeger has joined #tripleo | 17:39 | |
| openstackgerrit | Andreas Jaeger proposed openstack/tripleo-ansible master: Remove obsolete template release-openstack-server https://review.openstack.org/607332 | 17:41 |
| AJaeger | tripleo cores, jaosorior , quick question on ^ | 17:41 |
| AJaeger | we switched everything to publish-to-pypi, should I do that change for tripleo-ansible as well? | 17:42 |
| AJaeger | sdoran: ^ | 17:42 |
| mwhahaha | AJaeger: wasn't the difference that release-openstack-server didn't actually publish to pypi? | 17:43 |
| * mwhahaha doesn't remember what the explicit differences were | 17:43 | |
| AJaeger | mwhahaha: yes - but we're not doing that anymore, see dhellmann's recent email. | 17:43 |
| dhellmann | mwhahaha : ack, I'll propose the ones for tripleo that I've had on hold | 17:43 |
| dhellmann | I'll still wait until an off-peak time, since there are ~50 of them | 17:44 |
| mwhahaha | AJaeger: cause we want the tarbal but not the pypi stuff for ansible | 17:44 |
| mwhahaha | dhellmann: good idea :D | 17:44 |
| AJaeger | mwhahaha: http://lists.openstack.org/pipermail/openstack-dev/2018-September/135240.html | 17:44 |
| dhellmann | I didn't look at anything past the services and libraries for that job change | 17:44 |
| dhellmann | I don't really know the expectations for ansible role releases | 17:45 |
| mwhahaha | i mean it doesn't really matter, it just doesn't make sense | 17:45 |
| dhellmann | do folks install those via pip? | 17:45 |
| mwhahaha | no | 17:45 |
| *** boazel has joined #tripleo | 17:45 | |
| dhellmann | or ansible-galaxy | 17:45 |
| mwhahaha | we rpm them, but it could be a galaxy thing too | 17:45 |
| dhellmann | yeah, so we probably need a new variation on the tarball-only job that uses python 3 instead of 2 | 17:45 |
| mwhahaha | so for us the tarbal/tag is all we really want | 17:45 |
| dhellmann | or whatever it is that the job on those repos is doing now | 17:45 |
| mwhahaha | that would also be true for all the puppet stuff as well | 17:45 |
| dhellmann | does that puppet job use python somehow? | 17:46 |
| * dhellmann doesn't know how that part really works | 17:46 | |
| AJaeger | dhellmann: you converted all usage of release-openstack-server - just https://review.openstack.org/#/c/600847/ sneaked in 3 hours ago ;( | 17:46 |
| mwhahaha | we've got the basics for the reno stuff but it doesn't make sense to publish | 17:46 |
| AJaeger | mwhahaha: we have puppet-release-job | 17:46 |
| dhellmann | reno is a separate job, and that will be updated. I'm just concerned with the actual package building job | 17:47 |
| mwhahaha | yea puppet-release-job is fine then if thats not changing | 17:47 |
| mwhahaha | we need a similar one for the other non-python related things | 17:47 |
| dhellmann | AJaeger : I'm not sure what I'm supposed to be looking at in that patch | 17:47 |
| mwhahaha | which is what we were using release-openstack-server for | 17:47 |
| AJaeger | dhellmann: then let's use a different name so that it's clear it's not a generic thing... | 17:47 |
| dhellmann | yeah | 17:47 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Allow pinning of ara in undercloud-setup https://review.openstack.org/606179 | 17:48 |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Pin older versions of ara for pike/queens https://review.openstack.org/606181 | 17:48 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: Persist ceph-ansible fetch_directory using config-download https://review.openstack.org/604772 | 17:48 |
| openstackgerrit | Merged openstack/tripleo-validations stable/rocky: Fix diskspace validation when size is the limit https://review.openstack.org/605815 | 17:48 |
| openstackgerrit | Merged openstack/tripleo-validations master: Fix a minor typo in the ceilometerdb-size validation https://review.openstack.org/604353 | 17:48 |
| openstackgerrit | Merged openstack/tripleo-validations master: Fix several config file location https://review.openstack.org/604351 | 17:48 |
| openstackgerrit | Merged openstack/tripleo-validations master: Validation for no PMD cores on a NUMA node https://review.openstack.org/559168 | 17:48 |
| openstackgerrit | Merged openstack/tripleo-validations master: Removed older version of python added 3.5 https://review.openstack.org/606428 | 17:48 |
| dhellmann | release-openstack-server used to be the "tarballs but no pypi" thing | 17:48 |
| mwhahaha | yea we would like to keep that around | 17:48 |
| dhellmann | AJaeger : isn't there a job with tarball in the name? | 17:48 |
| AJaeger | dhellmann: https://review.openstack.org/#/c/600847/5/zuul.d/layout.yaml merged three hours before - and I just wanted to merge a change that removes release-openstack-server that passed since you converted all users - but now failed to merge | 17:48 |
| dhellmann | mwhahaha : yeah, we'll update it and have it work similarly, just using python 3 | 17:48 |
| dhellmann | AJaeger : ah | 17:48 |
| AJaeger | dhellmann: So, we're back to drawing board ;) | 17:49 |
| mwhahaha | AJaeger: i'll approve your removal, but it would be good to still have the release-openstack-server or something to that effect that we can use | 17:49 |
| mwhahaha | release-tarball-only or something :D | 17:49 |
| dhellmann | mwhahaha : I don't suppose anyone on the team has a little time to do that, with mentoring from me and AJaeger ? | 17:49 |
| mwhahaha | pffft time | 17:49 |
| dhellmann | otherwise I can put it on my list, but more hands would be good | 17:49 |
| dhellmann | heh | 17:49 |
| dhellmann | tell me about it | 17:50 |
| mwhahaha | if you got some docs i can take a shot | 17:50 |
| dhellmann | docs? what do you take me for? a professional? | 17:50 |
| mwhahaha | :D ok i'll take an etherpad and some guessing | 17:50 |
| AJaeger | we should definitely name it differently than release-openstack-server ;) We can take the current definition and just rename it... | 17:50 |
| *** openstackgerrit has quit IRC | 17:51 | |
| dhellmann | how does release-openstack-server build the tarball in the first place? | 17:51 |
| mwhahaha | magic i'm sure | 17:51 |
| AJaeger | dhellmann:, mwhahaha , basically revert https://review.openstack.org/#/c/531829/ and https://review.openstack.org/#/c/531830/ with different names | 17:51 |
| dhellmann | do we want something that just does "git ls-files | tar" or whatever? | 17:51 |
| *** vinaykns has joined #tripleo | 17:52 | |
| dhellmann | that gets us out of the python version issue entirely | 17:52 |
| mwhahaha | well i think we have the basic structure in python | 17:52 |
| dhellmann | which means the puppet modules don't need a setup.py any more | 17:52 |
| mwhahaha | it just doesn't make sense to publish to pypi | 17:52 |
| mwhahaha | (not refering to the puppet stuff) | 17:52 |
| AJaeger | dhellmann: http://git.openstack.org/cgit/openstack-infra/project-config/tree/playbooks/python-tarball/run.yaml | 17:52 |
| mwhahaha | the puppet stuff runs puppet to do a puppet build | 17:52 |
| AJaeger | that shows what we do today | 17:52 |
| dhellmann | mwhahaha : oh, I thought for some reason we'd stuck setup.py files in there so we could use pbr | 17:53 |
| dhellmann | maybe that was some other non-python thing | 17:53 |
| mwhahaha | dhellmann: that was for reno | 17:53 |
| dhellmann | AJaeger : ah, that's easy enough then, we can just rename the job and they can control the version of python using tox | 17:53 |
| mwhahaha | our ansible roles do have a setup.py that will result in a role | 17:53 |
| dhellmann | mwhahaha : ah, yeah, because it requires tox | 17:53 |
| mwhahaha | i'll take a stab at resurecting those patches | 17:53 |
| dhellmann | or the job does anyway | 17:53 |
| mwhahaha | yea | 17:53 |
| dhellmann | so many layers | 17:54 |
| AJaeger | mwhahaha: thanks, if you need help, please ask... | 17:54 |
| * mwhahaha does his best guessing | 17:54 | |
| *** holser__ has quit IRC | 17:54 | |
| *** hamzy has quit IRC | 17:55 | |
| dhellmann | I think if we're going to say the job is a generic tarball building thing, I would almost rather not have it rely on setup.py at all | 17:56 |
| *** iranzo has quit IRC | 17:56 | |
| mwhahaha | well we already have all the setup.py bits in place | 17:56 |
| mwhahaha | but i can take a look at what we're doing | 17:57 |
| dhellmann | yeah, I'm thinking forward to other non-python things | 17:57 |
| mwhahaha | yea | 17:57 |
| dhellmann | that also makes it clearer when choosing a job: if you are doing a "python release" you use the job that publishes to pypi; if you are doing a "tarball release" you choose that one and it works for any language | 17:57 |
| weshay | hrm.. is the retry working here? http://git.openstack.org/cgit/openstack/tripleo-common/tree/tripleo_common/image/image_uploader.py#n301 | 17:58 |
| weshay | http://logs.openstack.org/73/607173/1/gate/tripleo-ci-centos-7-containerized-undercloud-upgrades/703ce71/logs/undercloud/home/zuul/install-undercloud.log.txt.gz#_2018-10-02_16_53_02_480 | 17:58 |
| dhellmann | mwhahaha : I added a few notes to https://etherpad.openstack.org/p/python3-first around line 130 | 18:00 |
| *** ansmith has quit IRC | 18:00 | |
| *** hamzy has joined #tripleo | 18:02 | |
| *** pcaruana has quit IRC | 18:02 | |
| weshay | mwhahaha, so if this fails w/ tenacity in place http://logs.openstack.org/73/607173/1/gate/tripleo-ci-centos-7-containerized-undercloud-upgrades/703ce71/logs/undercloud/home/zuul/install-undercloud.log.txt.gz#_2018-10-02_16_53_02_480 | 18:04 |
| mwhahaha | weshay: yea i think it's because the exception that was thrown, i'll look in a bit | 18:04 |
| weshay | mwhahaha, should we consider that an infra issue and try to pursue adjustments w/ infra | 18:04 |
| weshay | k | 18:04 |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 18:10 |
| *** zaneb has quit IRC | 18:13 | |
| *** zaneb has joined #tripleo | 18:13 | |
| *** ansmith has joined #tripleo | 18:15 | |
| *** openstackgerrit has joined #tripleo | 18:17 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: Don't merge /etc/collectd.d https://review.openstack.org/606961 | 18:17 |
| rascasoft | hey folks, I'm hitting ""Error: /Stage[main]/Xinetd/Service[xinetd]/ensure: change from stopped to running failed: Systemd start for xinetd failed!" in the RDOPhase2 rocky deployment, is this a known issue? | 18:21 |
| * weshay looks around | 18:24 | |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates master: Handle LP openvswitch meta-package on upgrade https://review.openstack.org/605200 | 18:26 |
| weshay | rascasoft, http://cistatus.tripleo.org/promotion/ | 18:27 |
| *** zaneb has quit IRC | 18:27 | |
| weshay | comparing w/ fs001/35 might help | 18:27 |
| weshay | just check the hashes | 18:27 |
| rascasoft | weshay, ack, thanks | 18:27 |
| weshay | rascasoft, rdop2 does not look sane atm | 18:27 |
| *** zaneb has joined #tripleo | 18:28 | |
| weshay | rascasoft, tripleo's jobs are failing there as well | 18:28 |
| weshay | rascasoft, you are way past https://thirdparty.logs.rdoproject.org/jenkins-tripleo-quickstart-rocky-rdo_trunk-baremetal-hp_dl360_envD-single_nic_vlans-9/undercloud/home/stack/overcloud_deploy.log.txt.gz#_2018-10-02_17_16_06 | 18:29 |
| weshay | right? | 18:29 |
| weshay | rascasoft, are your logs posting to thirdparty.logs? | 18:30 |
| *** openstackgerrit has quit IRC | 18:35 | |
| *** trown|lunch is now known as trown | 18:45 | |
| *** boazel has quit IRC | 18:49 | |
| *** jaganathan has quit IRC | 19:01 | |
| *** ansmith has quit IRC | 19:01 | |
| *** ansmith has joined #tripleo | 19:01 | |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 19:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 19:10 |
| *** holser_ has joined #tripleo | 19:11 | |
| tzumainn | mwhahaha, hi - I'm looking at the standalone tripleo docs - https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/standalone.html - and was wondering if you might have some ideas on how to have isolated tenant and provisioning networks? | 19:14 |
| *** med_ has joined #tripleo | 19:18 | |
| mwhahaha | tzumainn: what do you mean | 19:20 |
| mwhahaha | tzumainn: in the documented standalone there's really no provisioning network (cause no undercloud) | 19:21 |
| tzumainn | mwhahaha, I'm configuring for overcloud ironic; my understanding is that the recommendation there would be to have distinct provisioning and tenant networks | 19:21 |
| tzumainn | I'm still kinda new to network configuration, but from the documentation it sounds like the recommendation is for overcloud ironic to use the undercloud provisioning network | 19:22 |
| tzumainn | but I'm wondering if there's some way to simulate that with standalone tripleo | 19:22 |
| mwhahaha | tzumainn: i mean you can take the standalone and run it it with ironic and use share the external network (i think), but i'm not really an expert in that so it'd probably be a good idea to ask the ironic folks for more details | 19:24 |
| mwhahaha | tzumainn: i am working on a standalone all-in-one that can be deployed via the undercloud (a normal overcloud) which you could use to actually test that network setup | 19:24 |
| *** hamzy has quit IRC | 19:25 | |
| tzumainn | mwhahaha, okay, thanks - yeah, I've tested it with the single external network and it works great, but I thought I'd try and see if there was a way to get a network architecture closer to the ones in the docs | 19:25 |
| tzumainn | mwhahaha, thanks a lot! | 19:25 |
| *** hamzy has joined #tripleo | 19:26 | |
| mwhahaha | tzumainn: https://review.openstack.org/#/c/605156/ just landed but the standalone-overcloud.yaml is an environment file that can be used to deploy a single controller that also has nova (and you can enable ironic) | 19:26 |
| mwhahaha | tzumainn: so you end up with a 1 uc + 1oc node to provide the ironic bits | 19:26 |
| tzumainn | mwhahaha, oh yeah, I've figured out how to do that | 19:26 |
| tzumainn | oh, you mean with the normal overcloud | 19:27 |
| tzumainn | I'l ltake a look, thanks! | 19:27 |
| mwhahaha | yea | 19:27 |
| mwhahaha | tzumainn: you might also chat with dtantsur|afk, bfournie or dsneddon about better network configuration options for ironic | 19:27 |
| weshay | stevebaker, mr baker.. this logs to /var/lib/mistral? http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/puppet/services/container-image-prepare.j2.yaml#n108 | 19:32 |
| mwhahaha | weshay: yea when it's run in the deployment, it's an external task so it ends up in /var/lib/mistral | 19:33 |
| weshay | hrm.. wonder why we're not picking up /var/lib/mistral | 19:34 |
| weshay | http://logs.openstack.org/32/596432/44/check/tripleo-ci-centos-7-undercloud-containers/c08168f/logs/undercloud/home/zuul/install-undercloud.log.txt.gz#_2018-10-02_12_39_02_647 | 19:34 |
| weshay | http://logs.openstack.org/32/596432/44/check/tripleo-ci-centos-7-undercloud-containers/c08168f/logs/undercloud/var/lib/ | 19:34 |
| mwhahaha | weshay: that one is not in /var/lib/mistral | 19:35 |
| mwhahaha | that one is in install-undercloud.log | 19:35 |
| mwhahaha | weshay: the undercloud version logs to the install-undercloud.log | 19:35 |
| weshay | ya. so that is as much detail as we get from that script? | 19:36 |
| * weshay going to play w/ it locally | 19:36 | |
| weshay | f.. ansible | 19:36 |
| mwhahaha | weshay: i have a change to enable debug | 19:36 |
| * weshay looks | 19:36 | |
| * mwhahaha weshay https://review.openstack.org/#/c/604886/ | 19:36 | |
| mwhahaha | currently it's stuck on info | 19:36 |
| mwhahaha | that adds --debug | 19:37 |
| mwhahaha | so we could improve the logging | 19:37 |
| * mwhahaha has a lot of outstanding patches that need landing | 19:38 | |
| weshay | mwhahaha, we would need one more to add --debug to http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/puppet/services/container-image-prepare.j2.yaml#n108 | 19:40 |
| weshay | ya? | 19:40 |
| weshay | shall I give that a go? | 19:40 |
| mwhahaha | yes | 19:40 |
| mwhahaha | you can if you desire | 19:40 |
| weshay | ya.. it's better than patching ci all day | 19:41 |
| mwhahaha | we have a Debug param you can use | 19:41 |
| mwhahaha | weshay: i've never seen CI so green today | 19:41 |
| * mwhahaha jinxes it | 19:41 | |
| weshay | YOU Bastar... /0\ | 19:41 |
| weshay | lolz | 19:41 |
| weshay | recheck | 19:41 |
| weshay | recheck | 19:41 |
| weshay | recheck | 19:41 |
| mwhahaha | amazing what happens when you stop randomly sending signals to containers :D | 19:42 |
| weshay | and you stop running meaningless validations | 19:42 |
| mwhahaha | pffft they have meaning, we just don't bother checking it :D | 19:42 |
| weshay | mwhahaha, your increase in container threads merged ya? | 19:43 |
| mwhahaha | yea | 19:43 |
| weshay | mwhahaha, pffft.. | 19:43 |
| weshay | k | 19:43 |
| mwhahaha | that merged over the weekend | 19:43 |
| mwhahaha | in master/rocky | 19:43 |
| weshay | no meaning upstream | 19:43 |
| weshay | k | 19:43 |
| weshay | like my life | 19:43 |
| weshay | pffft | 19:43 |
| mwhahaha | https://review.openstack.org/#/c/600512/15 failed in the gate with a TIMED_OUT | 19:46 |
| mwhahaha | can't win them all i guess | 19:46 |
| stevebaker | mwhahaha, weshay: I have changes to log both undercloud and overcloud prepare to /var/log/tripleo-container-image-prepare.log https://review.openstack.org/#/c/599273/ https://review.openstack.org/#/c/603944/ | 19:46 |
| weshay | stevebaker, would it make sense to create a /var/log/tripleo dir? | 19:47 |
| mwhahaha | no because you can't necessarily log to it with all the users | 19:47 |
| mwhahaha | mistral/stack/whatever | 19:47 |
| mwhahaha | it just simplifies our need to 2>&1 > /foo | 19:47 |
| weshay | job security++ | 19:48 |
| *** dprince has quit IRC | 19:51 | |
| weshay | mwhahaha, /me looking for your suggestion re: debug.. [{get_param: Debug}, '']} | 19:52 |
| weshay | is that it? | 19:52 |
| mwhahaha | weshay: you'd use a conditional in heat (there are examples) and then modify the command line with a --debug if the conditional is true | 19:53 |
| weshay | k | 19:53 |
| mwhahaha | or if it's in ansible somewhere, maybe use an ansible thing, yea | 19:54 |
| mwhahaha | with a {get_param: Debug} | 19:54 |
| weshay | aye.. just like ContainerImagePrepareLogFile | 19:54 |
| weshay | k | 19:54 |
| * weshay gives it a go | 19:54 | |
| mwhahaha | Debug is a /true/false i think | 19:55 |
| weshay | k | 19:55 |
| *** abishop_ has quit IRC | 19:56 | |
| mwhahaha | weshay: who configures the dns in the upstream instances? specifically the unbound config | 19:58 |
| mwhahaha | that's infra right? | 19:58 |
| weshay | a | 19:58 |
| weshay | ya | 19:58 |
| weshay | I've only read briefly about it | 19:58 |
| weshay | rf0lc0, may be able to assist w/ a question | 19:59 |
| EmilienM | looks like the bot is dead | 19:59 |
| mwhahaha | was looking at the https://bugs.launchpad.net/tripleo/+bug/1795718 trying to see if i could track down the error in unbound | 19:59 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 19:59 |
| mwhahaha | EmilienM: what bot? gerritbot? | 19:59 |
| EmilienM | yes | 20:00 |
| *** jabadia has joined #tripleo | 20:00 | |
| rf0lc0 | weshay, mwhahaha yes unbound sets dns. Default is opendns and google. | 20:00 |
| mwhahaha | rf0lc0: right but that comes from nodepool preset right? we're not changing it to google/opendns? | 20:01 |
| rf0lc0 | infra sets it to opendns/google | 20:01 |
| mwhahaha | k | 20:01 |
| rf0lc0 | foward.conf... let me get the right file | 20:02 |
| weshay | ugh.. /me kicks thyself.. so if debug was default in the non-containerized undercloud.. is it ok to make it a parameter_default in the current containerized in /usr/share/openstack-tripleo-heat-templates/environments/undercloud.yaml | 20:05 |
| mwhahaha | weshay: no | 20:05 |
| mwhahaha | that gets configured elsewhere | 20:05 |
| mwhahaha | weshay: debug in undercloud.conf is processed via the undercloud configuration bits in tripleoclient and gets automattically added into the undercloud_parameters.yaml that we generate | 20:08 |
| mwhahaha | weshay: so if it's false in the default config, go fix in tripleoclient but i'm pretty sure it's true | 20:08 |
| weshay | ya.. it's true by default | 20:09 |
| weshay | k. | 20:09 |
| weshay | thanks for the bread crumbs | 20:09 |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 20:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 20:10 |
| weshay | parameter_defaults: | 20:11 |
| weshay | CertmongerCA: local | 20:11 |
| weshay | CloudName: 192.168.24.2 | 20:11 |
| weshay | ContainerCli: docker | 20:11 |
| weshay | ControlPlaneStaticRoutes: [] | 20:11 |
| weshay | Debug: true | 20:11 |
| rf0lc0 | mwhahaha, not sure this is relevant... here is where dns is set during dib stage: https://github.com/openstack-infra/project-config/blob/master/nodepool/elements/nodepool-base/finalise.d/89-unbound#L46 | 20:12 |
| mwhahaha | rf0lc0: k. i'm just looking at the unbound logs and it doesn't really seem like we end up caching much | 20:13 |
| mwhahaha | rf0lc0: if you ctrl+f "returning answer from cache" from http://logs.openstack.org/73/607173/1/gate/tripleo-ci-centos-7-containerized-undercloud-upgrades/703ce71/logs/undercloud/var/lib/unbound/unbound.log.txt.gz we only return 13 things from cache | 20:14 |
| mwhahaha | we aren't actually caching registry-1.docker.io either | 20:14 |
| mwhahaha | which is what frequently fails but i'm trying to figureout if i'm reading these logs right | 20:14 |
| vinaykns | hello channel...I have a question...do we have a loadbalancer service on the overcloud..?? and does it come with the default setup..? | 20:14 |
| *** Director100b has joined #tripleo | 20:14 | |
| mwhahaha | vinaykns: we support octavia in newer versions and it does not ship enabled by default but is available | 20:15 |
| Director100b | Hi there. Is there are redhat people here? Director 13 seems to be complaining of a issing python-jwt package. | 20:15 |
| Director100b | *missing | 20:15 |
| Director100b | It's nowhere it seems. | 20:15 |
| mwhahaha | Director100b: that comes from rhel-ha channel, do you have that enabled? | 20:16 |
| vinaykns | mwhahaha: cool..thx..! | 20:16 |
| rf0lc0 | mwhahaha, I don't know how cache works there, here is the config that is read: http://logs.openstack.org/25/602425/5/check/tripleo-ci-centos-7-containers-multinode/e2bb2ec/logs/undercloud/etc/unbound/forwarding.conf.txt.gz | 20:16 |
| Director100b | mwhahaha: Yeah. I do. /me goes to double-check | 20:17 |
| mwhahaha | rf0lc0: it's cause http://logs.openstack.org/25/602425/5/check/tripleo-ci-centos-7-containers-multinode/e2bb2ec/logs/undercloud/etc/unbound/conf.d/ttl.conf.txt.gz | 20:17 |
| mwhahaha | the ttl for docker is 900 | 20:18 |
| mwhahaha | so it doesn't cache | 20:18 |
| * mwhahaha sighs | 20:18 | |
| Director100b | rhel-ha-for-rhel-7-server-rpms is enabled. | 20:18 |
| mwhahaha | Director100b: https://access.redhat.com/downloads/content/python-jwt/1.5.3-1.el7/noarch/fd431d51/package claims it's there so open a ticket with support | 20:18 |
| Director100b | yeah. One is already open (from before when I tried to update director. I installed from scratch due to something I broke, and not it won't install at all) | 20:19 |
| Director100b | I thought someone here may be aware as the support ticket is coming back a bit bleak. | 20:20 |
| Director100b | And maybe tell e a magic trick. | 20:20 |
| mwhahaha | not really other than making sure the channel is working | 20:21 |
| mwhahaha | you could just download it from the website and manually install it as a temporary workaround | 20:21 |
| mwhahaha | but it's seems odd | 20:21 |
| Director100b | aye. I'll see what they say; failing that i'll just package it up and do it the old fashioned way. | 20:25 |
| *** akrivoka has quit IRC | 20:25 | |
| *** elmaciej has joined #tripleo | 20:30 | |
| Director100b | mwhahaha: Installed it manually. Rerunning install. Let's see what happens. | 20:32 |
| mwhahaha | rf0lc0, weshay: so we can actually configure our own unbound servers because they are zuul vars btw | 20:34 |
| weshay | increase the ttl? | 20:34 |
| mwhahaha | we need to decrease it actually | 20:34 |
| mwhahaha | we're preventing docker.io from being cached | 20:34 |
| weshay | ok.. k | 20:35 |
| mwhahaha | http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/roles/configure-unbound/defaults/main.yaml#n17 | 20:35 |
| mwhahaha | cause we're setting it to 900 | 20:35 |
| mwhahaha | rather than leaving it at 0 | 20:35 |
| mwhahaha | i think | 20:35 |
| mwhahaha | i'm trying to figure out what these parms mean | 20:35 |
| *** med_ has quit IRC | 20:36 | |
| weshay | http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/zuul.d/base.yaml#n15 | 20:37 |
| mwhahaha | yea that | 20:38 |
| mwhahaha | i'm not sure if that means cache it for at least 900 | 20:38 |
| weshay | mwhahaha, of course we're the only project that has to adjust this stuff | 20:38 |
| mwhahaha | or don't cache it unless it's 900 | 20:38 |
| mwhahaha | looking at the c code for unbind, i can't actually see a reference to it being used :D | 20:39 |
| weshay | not sure why it's even set | 20:39 |
| weshay | https://review.openstack.org/#/c/578432/8/zuul.d/base.yaml | 20:39 |
| mwhahaha | i remember dmsimard was messing with it when we were having docker.io issues | 20:40 |
| mwhahaha | so it's likely a carry over from that | 20:40 |
| *** ansmith has quit IRC | 20:40 | |
| mwhahaha | though i'm not sure it ever actually helped | 20:40 |
| mwhahaha | weshay: oh there's my answer in the logs, so the reason why it failed is because we never got an actual set of servers back trying to resolve registry-1.docker.io | 20:44 |
| *** hamzy has quit IRC | 20:44 | |
| mwhahaha | it's not cached cause there's nothing there | 20:44 |
| mwhahaha | oh wait no that was the AAAA reply | 20:45 |
| *** trown is now known as trown|outtypewww | 20:55 | |
| *** raildo has quit IRC | 20:58 | |
| *** Vorrtex has quit IRC | 21:04 | |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 21:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 21:10 |
| *** gfidente has quit IRC | 21:12 | |
| *** ade_lee has quit IRC | 21:15 | |
| *** elmaciej has quit IRC | 21:18 | |
| vinaykns | I have a question...regarding listening to the qdr service running on the controller node's on the overcloud.. | 21:22 |
| vinaykns | currently I am hardcoding with a controller name to send the data to, but this won't work if any of that node is down. What I want is haproxy dynamically matching for that address | 21:25 |
| vinaykns | https://review.openstack.org/#/c/600843/2/environments/enable-ceilometer-amqp1-publisher.yaml ...but I'm not sure how to do that...any workaround for this..? | 21:26 |
| mwhahaha | vinaykns: not like that, you'd have to generate it dynamically | 21:29 |
| *** med_ has joined #tripleo | 21:29 | |
| mwhahaha | vinaykns: you'd want to move that into the service and then do some vip lookup. something like that should not live an environment file but rather a service | 21:30 |
| vinaykns | mwhahaha: does this be okay if i provide a fixed ip..? | 21:32 |
| mwhahaha | no | 21:32 |
| vinaykns | mwhahaha: could you elaborate on move into the service means..? | 21:33 |
| *** aedc has quit IRC | 21:34 | |
| mwhahaha | vinaykns: move the logic over to, https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/ceilometer-base.yaml#L153 | 21:34 |
| mwhahaha | vinaykns: where you use a new parameter to indicate you want to enable this feature to publish to the amqp endpoints and combine the logic in that file to handle it | 21:35 |
| *** moshele has joined #tripleo | 21:36 | |
| vinaykns | mwhahaha: make sense... I could update https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/ceilometer-base.yaml#L46 but still I don't know how to send the data without specifying overcloud-controller-0 or something like that.. | 21:39 |
| mwhahaha | let me see if i can dig up where that logic lives | 21:42 |
| *** holser_ has quit IRC | 21:44 | |
| jbadiapa | mwhahaha, vinaykns, could be something like https://github.com/openstack/tripleo-heat-templates/blob/85da81624f12f35eb905fecc1db5ccb4d9dbadbe/puppet/services/designate-central.yaml#L78? but for the amqp1 url | 21:44 |
| mwhahaha | yes | 21:44 |
| mwhahaha | you'd query the endpointmap for the appropriate vip | 21:44 |
| *** med_ has quit IRC | 21:45 | |
| jbadiapa | There is no endpointmap for qdr or I couldn't find ir | 21:49 |
| jbadiapa | vinaykns, https://github.com/openstack/tripleo-heat-templates/blob/85da81624f12f35eb905fecc1db5ccb4d9dbadbe/network/endpoints/endpoint_map.yaml | 21:50 |
| vinaykns | jbadiapa: oh okay...make sense...perhaps is isn't included there | 21:52 |
| vinaykns | jbadiapa: I guess I have to add those to the list. | 21:53 |
| jbadiapa | vinaykns, to here https://github.com/openstack/tripleo-heat-templates/blob/85da81624f12f35eb905fecc1db5ccb4d9dbadbe/network/endpoints/endpoint_data.yaml | 21:54 |
| *** bfournie has quit IRC | 21:54 | |
| vinaykns | jbadiapa: cool..thanks for pointing out.! | 21:55 |
| jbadiapa | mwhahaha, do you know if the service can use other different port than the default one? | 21:56 |
| mwhahaha | jbadiapa: there's a port configuration as well | 21:56 |
| mwhahaha | I think it's in NetworkMap already | 21:57 |
| mwhahaha | so it depends | 21:57 |
| vinaykns | mwhahaha, jbadiapa: thank you..:) | 22:00 |
| *** vinaykns has quit IRC | 22:05 | |
| *** rh-jelabarre has quit IRC | 22:08 | |
| *** homeski has quit IRC | 22:09 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 22:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 22:10 |
| *** boazel has joined #tripleo | 22:12 | |
| *** moshele has quit IRC | 22:24 | |
| *** jabadia has quit IRC | 22:32 | |
| *** med_ has joined #tripleo | 22:32 | |
| *** matbu has quit IRC | 22:35 | |
| *** tzumainn has quit IRC | 22:35 | |
| *** jcoufal has quit IRC | 22:36 | |
| *** tosky has quit IRC | 22:41 | |
| *** med_ has quit IRC | 22:51 | |
| *** matbu has joined #tripleo | 22:57 | |
| *** dsneddon has quit IRC | 23:01 | |
| *** dsneddon has joined #tripleo | 23:02 | |
| *** ansmith has joined #tripleo | 23:03 | |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 23:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 23:10 |
| *** agopi is now known as agopi|off | 23:41 | |
| *** agopi|off has quit IRC | 23:45 | |
| *** morazi has quit IRC | 23:52 | |
| *** rcernin has joined #tripleo | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!