*** yolanda has quit IRC | 00:08 | |
*** ooolpbot has joined #tripleo | 00:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
---|---|---|
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 00:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 00:10 |
*** ooolpbot has quit IRC | 00:10 | |
openstack | Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 00:10 |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 00:10 |
*** moshele has quit IRC | 00:25 | |
*** lblanchard has joined #tripleo | 00:34 | |
openstackgerrit | Rafael Folco proposed openstack-infra/tripleo-ci master: DNM - Add another level of parent jobs for zuul v3 https://review.openstack.org/593063 | 00:36 |
openstackgerrit | Rafael Folco proposed openstack-infra/tripleo-ci master: DNM: [PoC] Move extra playbooks to job definition https://review.openstack.org/594583 | 00:36 |
*** tzumainn has quit IRC | 00:40 | |
*** gbarros has quit IRC | 01:01 | |
*** dmellado has quit IRC | 01:22 | |
*** rcernin_ has joined #tripleo | 01:23 | |
*** gouthamr has quit IRC | 01:23 | |
*** mschuppert has quit IRC | 01:24 | |
*** stevebaker has quit IRC | 01:24 | |
*** rcernin has quit IRC | 01:25 | |
*** gbarros has joined #tripleo | 01:44 | |
*** rcernin has joined #tripleo | 02:00 | |
*** lblanchard has quit IRC | 02:01 | |
*** rcernin_ has quit IRC | 02:03 | |
*** ooolpbot has joined #tripleo | 02:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 02:10 |
openstack | Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 02:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 02:10 |
*** ooolpbot has quit IRC | 02:10 | |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 02:10 |
openstackgerrit | Merged openstack/tripleo-quickstart master: Use sudo when working with root home as non root https://review.openstack.org/593104 | 02:15 |
openstackgerrit | Merged openstack/tripleo-heat-templates master: Fix enabled tripleo-validations check https://review.openstack.org/590237 | 02:15 |
openstackgerrit | Merged openstack/python-tripleoclient master: Update reno for stable/rocky https://review.openstack.org/594263 | 02:15 |
openstackgerrit | Merged openstack/tripleo-heat-templates master: Enable collectd to connect to metrics QDR https://review.openstack.org/576057 | 02:18 |
openstackgerrit | Merged openstack/tripleo-quickstart master: we need to allow tripleo-roles to be updated in ci https://review.openstack.org/594314 | 02:18 |
*** khyr0n has joined #tripleo | 02:19 | |
*** rlandy has quit IRC | 02:25 | |
*** apetrich has quit IRC | 02:26 | |
*** dhill_ has quit IRC | 02:34 | |
*** gouthamr has joined #tripleo | 02:36 | |
*** dmellado has joined #tripleo | 02:38 | |
*** rpioso is now known as rpioso|afk | 02:43 | |
*** edmondsw has quit IRC | 02:45 | |
*** mmethot has quit IRC | 02:46 | |
*** stevebaker has joined #tripleo | 02:48 | |
*** mmethot has joined #tripleo | 02:49 | |
*** gbarros has quit IRC | 02:49 | |
*** psachin has joined #tripleo | 02:51 | |
openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: Fix HASH for periodic jobs(use tripleo-ci-testing) https://review.openstack.org/593383 | 03:01 |
*** ooolpbot has joined #tripleo | 03:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 03:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 03:10 |
*** ooolpbot has quit IRC | 03:10 | |
openstack | Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 03:10 |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 03:10 |
*** sanjayu_ has joined #tripleo | 03:20 | |
*** khyr0n has quit IRC | 03:31 | |
openstackgerrit | Merged openstack/tripleo-quickstart master: rocky is not yet promoted, adjust release file https://review.openstack.org/594357 | 03:42 |
*** Petersingh has joined #tripleo | 03:44 | |
*** Petersingh is now known as Petersingh|afk | 03:56 | |
*** morazi has quit IRC | 04:05 | |
*** khyr0n has joined #tripleo | 04:09 | |
*** ooolpbot has joined #tripleo | 04:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 04:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 04:10 |
*** ooolpbot has quit IRC | 04:10 | |
openstack | Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 04:10 |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 04:10 |
*** khyr0n has quit IRC | 04:11 | |
*** khyr0n has joined #tripleo | 04:12 | |
*** shyamb has joined #tripleo | 04:15 | |
*** shyamb has quit IRC | 04:32 | |
*** shyamb has joined #tripleo | 04:32 | |
shyamb | Hi | 04:33 |
shyamb | Is it good idea to call hiera from puppet code? | 04:33 |
shyamb | from puppet template? | 04:34 |
*** Petersingh|afk is now known as Petersingh | 04:35 | |
*** aufi has joined #tripleo | 04:47 | |
openstackgerrit | melissaml proposed openstack/tripleo-specs master: Remove the redundant word https://review.openstack.org/594799 | 04:47 |
openstackgerrit | Goutham Pacha Ravi proposed openstack/tripleo-heat-templates master: Fix bind-mount to manila's bootstrap container https://review.openstack.org/594801 | 04:52 |
openstackgerrit | Goutham Pacha Ravi proposed openstack/tripleo-heat-templates master: Fix bind-mount to manila's bootstrap container https://review.openstack.org/594801 | 04:52 |
*** moshele has joined #tripleo | 05:00 | |
*** moshele has quit IRC | 05:02 | |
gouthamr | hi bug czars, need to mark this triaged please: https://review.openstack.org/#/c/594801 | 05:08 |
gouthamr | sorry, meant the LP: https://bugs.launchpad.net/tripleo/+bug/1788337 :) | 05:09 |
openstack | Launchpad bug 1788337 in tripleo "container manila_api_db_sync fails with TLS everywhere" [Undecided,In progress] - Assigned to Goutham Pacha Ravi (gouthamr) | 05:09 |
*** ooolpbot has joined #tripleo | 05:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 05:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 05:10 |
openstack | Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 05:10 |
*** ooolpbot has quit IRC | 05:10 | |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 05:10 |
*** sdake has quit IRC | 05:13 | |
*** sdake has joined #tripleo | 05:13 | |
*** aufi has quit IRC | 05:25 | |
*** mschuppert has joined #tripleo | 05:25 | |
*** shyamb has quit IRC | 05:30 | |
*** palkar has joined #tripleo | 05:30 | |
*** sanjayu_ has quit IRC | 05:34 | |
*** shyamb has joined #tripleo | 05:36 | |
*** sdake has quit IRC | 05:50 | |
openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-specs master: Validation Framework specifications https://review.openstack.org/589169 | 05:50 |
*** sdake has joined #tripleo | 05:51 | |
*** hjensas has quit IRC | 05:52 | |
*** ykarel has joined #tripleo | 05:54 | |
*** prometheanfire has left #tripleo | 05:57 | |
*** ksambor has joined #tripleo | 05:59 | |
*** sdake has quit IRC | 06:02 | |
*** sdake has joined #tripleo | 06:04 | |
openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-heat-templates stable/ocata: DNM: test ocata https://review.openstack.org/594810 | 06:04 |
*** moshele has joined #tripleo | 06:06 | |
openstackgerrit | Kamil Sambor proposed openstack/python-tripleoclient master: Add fixtures instead mock https://review.openstack.org/579538 | 06:07 |
*** shyamb has quit IRC | 06:08 | |
*** shyamb has joined #tripleo | 06:09 | |
*** ooolpbot has joined #tripleo | 06:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 06:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 06:10 |
*** ooolpbot has quit IRC | 06:10 | |
openstack | Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 06:10 |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 06:10 |
*** jfrancoa has joined #tripleo | 06:18 | |
*** ccamacho has joined #tripleo | 06:20 | |
*** ccamacho has joined #tripleo | 06:20 | |
*** yprokule has joined #tripleo | 06:20 | |
openstackgerrit | Juan Badia Payno proposed openstack/tripleo-heat-templates master: [WIP] logging test DO NOT MERGED https://review.openstack.org/594836 | 06:22 |
openstackgerrit | Juan Badia Payno proposed openstack/tripleo-heat-templates stable/queens: [WIP] logging test DO NOT MERGED https://review.openstack.org/594838 | 06:30 |
*** gkadam has joined #tripleo | 06:31 | |
*** saneax has joined #tripleo | 06:31 | |
openstackgerrit | Juan Badia Payno proposed openstack/tripleo-heat-templates stable/pike: [WIP] logging test DO NOT MERGED https://review.openstack.org/594840 | 06:39 |
*** marios|rover has joined #tripleo | 06:40 | |
*** pcaruana has joined #tripleo | 06:41 | |
*** shyamb has quit IRC | 06:51 | |
*** shyamb has joined #tripleo | 06:52 | |
openstackgerrit | Michele Baldessari proposed openstack/puppet-pacemaker master: Abstract container backend https://review.openstack.org/584661 | 06:52 |
*** Petersingh is now known as Petersingh|afk | 06:54 | |
*** hjensas has joined #tripleo | 06:56 | |
*** agurenko has joined #tripleo | 06:57 | |
*** amoralej|off is now known as amoralej | 07:01 | |
*** dmellado has quit IRC | 07:04 | |
*** Petersingh|afk is now known as Petersingh | 07:05 | |
*** dmellado has joined #tripleo | 07:06 | |
*** rcernin has quit IRC | 07:07 | |
*** mschuppert has quit IRC | 07:07 | |
*** noama has joined #tripleo | 07:08 | |
*** bogdando has joined #tripleo | 07:10 | |
*** ooolpbot has joined #tripleo | 07:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 07:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 07:10 |
*** ooolpbot has quit IRC | 07:10 | |
openstack | Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 07:10 |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 07:10 |
*** dmellado has quit IRC | 07:12 | |
*** tosky has joined #tripleo | 07:13 | |
*** Tengu has joined #tripleo | 07:15 | |
*** sshnaidm|afk is now known as sshnaidm | 07:17 | |
*** dmellado has joined #tripleo | 07:19 | |
*** jpich has joined #tripleo | 07:20 | |
*** dtantsur|afk is now known as dtantsur | 07:22 | |
*** f1 has joined #tripleo | 07:23 | |
*** f1 is now known as florianf | 07:23 | |
openstackgerrit | Harald Jensås proposed openstack/tripleo-heat-templates master: Composable networks - L3 routed networks (1/3) https://review.openstack.org/582180 | 07:26 |
openstackgerrit | Harald Jensås proposed openstack/tripleo-heat-templates master: Composable networks - L3 routed networks (2/3) https://review.openstack.org/582181 | 07:26 |
openstackgerrit | Harald Jensås proposed openstack/tripleo-heat-templates master: Composable networks - L3 routed networks (3/3) https://review.openstack.org/582301 | 07:26 |
*** rdopiera has joined #tripleo | 07:26 | |
*** shyamb has quit IRC | 07:27 | |
*** moguimar has joined #tripleo | 07:30 | |
*** ccamacho has quit IRC | 07:33 | |
*** Petersingh is now known as Petersingh|lunch | 07:34 | |
*** akrivoka has joined #tripleo | 07:34 | |
*** moguimar has quit IRC | 07:36 | |
*** shardy has joined #tripleo | 07:38 | |
*** assassin has joined #tripleo | 07:40 | |
*** jtomasek has joined #tripleo | 07:41 | |
*** aufi has joined #tripleo | 07:43 | |
*** tosky has quit IRC | 07:43 | |
*** tosky has joined #tripleo | 07:43 | |
openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Collect overcloud statistics with ARA https://review.openstack.org/578462 | 07:44 |
*** dxiri has joined #tripleo | 07:46 | |
*** mschuppert has joined #tripleo | 07:47 | |
*** jpena|off is now known as jpena | 07:47 | |
*** jaganathan has quit IRC | 07:52 | |
*** ooolpbot has joined #tripleo | 08:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 08:10 |
openstack | Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 08:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 08:10 |
*** ooolpbot has quit IRC | 08:10 | |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 08:10 |
openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: Increase timeout from 180 mins to 200 mins https://review.openstack.org/594905 | 08:16 |
*** amoralej is now known as amoralej|brb | 08:16 | |
*** ccamacho has joined #tripleo | 08:16 | |
*** Petersingh|lunch is now known as Petersingh | 08:18 | |
*** akrivoka has quit IRC | 08:20 | |
*** ccamacho has quit IRC | 08:25 | |
*** ccamacho has joined #tripleo | 08:26 | |
*** holser_ has joined #tripleo | 08:26 | |
*** pcaruana has quit IRC | 08:28 | |
*** pcaruana has joined #tripleo | 08:30 | |
*** shyamb has joined #tripleo | 08:41 | |
openstackgerrit | Harald Jensås proposed openstack/tripleo-heat-templates master: Remove uneececary conditionals in network-environment jinja https://review.openstack.org/594906 | 08:43 |
*** dxiri has quit IRC | 08:46 | |
*** amoralej|brb is now known as amoralej | 08:47 | |
openstackgerrit | Steven Hardy proposed openstack/tripleo-quickstart-extras master: WIP tet changing default for PREPARE_ARGS https://review.openstack.org/594907 | 08:48 |
shardy | marios: Hey, I pushed ^^ to see if it works, will comment again on the review | 08:48 |
*** sri_ has joined #tripleo | 08:49 | |
sshnaidm | does anybody know which use runs ansible-playbook when installing overcloud? is it root or tripleo-admin? | 08:50 |
shardy | sshnaidm: it's heat-admin by default IIRC, with some become: true blocks | 08:51 |
*** akrivoka has joined #tripleo | 08:52 | |
shardy | you can see it via grep ansible_ssh_user in tripleo-common FYI | 08:52 |
shardy | and in workbooks/deployment.yaml (ssh_user) | 08:52 |
sri_ | shardy, quick question is there way i can skip ntp for simple overcloud deployment in stable/queens ? | 08:53 |
*** paramite_ has joined #tripleo | 08:54 | |
shardy | sri_: you could try either setting the NtpServer parameter to an empty list: | 08:55 |
shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/time/ntp.yaml#L36 | 08:55 |
shardy | or | 08:55 |
shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/overcloud-resource-registry-puppet.j2.yaml#L207 | 08:55 |
shardy | you could set the OS::TripleO::Services::Ntp service to OS::Heat::None in your resource registry | 08:56 |
shardy | or you could pass a custom ControllerServices parameter with only the services you want | 08:56 |
sri_ | shardy, got it, thank you :) | 08:57 |
sri_ | shardy, the reason I wanted to do this is ntp port 123/udp is disabled in my network, it will take a 3 to 4 days to complete that firewall request | 08:58 |
sshnaidm | shardy, hmm.. I see tripleo-admin here in inventory class: https://github.com/openstack/tripleo-common/blob/dfb9c24d86cbdeb95f8ea82d04eb32bb48d54bbf/tripleo_common/actions/ansible.py#L595 | 08:59 |
*** tosky has quit IRC | 08:59 | |
*** tosky has joined #tripleo | 09:00 | |
marios | ack thanks shardy will check | 09:00 |
sshnaidm | shardy, I look at files that are created in mistral folder and they're of user "42430".. is it being deleted..? | 09:02 |
sshnaidm | bogdando, Tengu maybe you know? ^^ | 09:02 |
shardy | sshnaidm: which mistral folder is that? | 09:03 |
shardy | the inventory output etc is probably generated as the mistral user | 09:03 |
jaosorior_ | bandini: could you check this out https://review.openstack.org/#/c/593491/ ? | 09:04 |
shardy | dtantsur: Hey if you have any notes re https://review.openstack.org/#/c/576856/ I plan to try it today, overall looks good to me | 09:05 |
*** moshele has quit IRC | 09:05 | |
sshnaidm | shardy, /var/lib/mistral/overcloud/ | 09:05 |
shardy | did you rebuild all the kolla containers or just the mistral one to test this? | 09:05 |
*** moshele has joined #tripleo | 09:05 | |
sshnaidm | d0ugal, do you know if ansible-playbook in overcloud deployment runs with mistral user tripleo-admin or heat-admin..? | 09:06 |
shardy | sshnaidm: interesting, I see the same, but that's created by mistral not ansible IIRC | 09:06 |
shardy | sshnaidm: I wonder if it's a uid mapping issue because mistral is now running in a container | 09:07 |
sshnaidm | shardy, yeah, container user will have this | 09:07 |
shardy | the ansible mistral action will now run inside the mistral-executor container, so I guess that explains it | 09:07 |
openstackgerrit | Kamil Sambor proposed openstack/python-tripleoclient master: Add fixtures instead mocks https://review.openstack.org/581342 | 09:08 |
shardy | IIRC we updated the kolla uid mappings in a few places before to avoid it | 09:08 |
sshnaidm | shardy, I see, so ansible-playbook is actually running from within mistral container.. | 09:08 |
shardy | sshnaidm: yes, which is why we recently had to mount some dirs to ensure roles etc from the host are available | 09:08 |
shardy | it'd be good in due course to move away from that though | 09:09 |
sshnaidm | shardy, do you know when I can see volume mappings of this container? | 09:09 |
shardy | sshnaidm: sec | 09:09 |
*** paramite_ has quit IRC | 09:09 | |
sshnaidm | shardy, now I understand why creating files didn't work :) they were created inside the container.. | 09:09 |
shardy | sshnaidm: docker inspect mistral_executor shows them under HostConfig/Binds | 09:10 |
*** ooolpbot has joined #tripleo | 09:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 09:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 09:10 |
*** ooolpbot has quit IRC | 09:10 | |
openstack | Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 09:10 |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 09:10 |
*** aufi has quit IRC | 09:10 | |
sshnaidm | shardy, thanks, found it here: http://logs.openstack.org/62/578462/22/check/tripleo-ci-centos-7-scenario002-multinode-oooq-container/3d98420/logs/undercloud/var/log/extra/docker/containers/mistral_executor/docker_info.log.txt.gz | 09:12 |
shardy | sshnaidm: ah nice, I wasn't sure if we captured that in CI | 09:13 |
sshnaidm | shardy, yeah, we do docker inspect on all containers | 09:13 |
*** akrivoka has quit IRC | 09:15 | |
*** chem has joined #tripleo | 09:19 | |
*** akrivoka has joined #tripleo | 09:19 | |
openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Collect overcloud statistics with ARA https://review.openstack.org/578462 | 09:20 |
sshnaidm | d0ugal, do you know maybe if user (tripleo-admin) that runs ansible-playbook from mistral executor container have permissions to create files in /var/lib/mistral ? | 09:21 |
*** shyam89 has joined #tripleo | 09:22 | |
d0ugal | sshnaidm: Sorry, was in a meeting. ansible-playbook should run as the mistral user. | 09:22 |
d0ugal | sshnaidm: I believe they should have permissions - isn't that the reason that folder exists? but I'm not sure. | 09:23 |
*** amoralej is now known as amoralej|brb | 09:24 | |
*** akrivoka has quit IRC | 09:24 | |
sshnaidm | d0ugal, yeah, seems like so, thanks | 09:25 |
*** leanderthal has joined #tripleo | 09:25 | |
sshnaidm | d0ugal, but if I want to use some special plugin for ansible, seems like I need to install it in container, right? I don't see mapping of python module there.. | 09:26 |
*** shyamb has quit IRC | 09:27 | |
chandankumar | d0ugal: Hello | 09:27 |
chandankumar | d0ugal: Can we do a tag release for mistral-tempest-plugin? | 09:27 |
*** ykarel_ has joined #tripleo | 09:28 | |
*** akrivoka has joined #tripleo | 09:29 | |
dtantsur | shardy: will post something today; got stuck yesterday with trying to update code in a container.. ended up screwing my undercloud twice | 09:30 |
*** ykarel has quit IRC | 09:31 | |
*** akrivoka has quit IRC | 09:33 | |
shardy | dtantsur: ack yeah that was the part I was unsure about, happy to help figure it out if needed though | 09:33 |
*** moshele has quit IRC | 09:34 | |
d0ugal | sshnaidm: correct, I am not sure how you would install it in the container | 09:34 |
d0ugal | chandankumar: Sure | 09:35 |
chandankumar | d0ugal: I am prepating the patch then for release | 09:35 |
d0ugal | chandankumar: great, let me know and I'll +1 it | 09:35 |
dtantsur | shardy: I ended up with this script (run from tripleo-common checkout): http://paste.openstack.org/show/728578/ | 09:35 |
d0ugal | chandankumar: I'm going out for a bit, back in ~1hr | 09:35 |
dtantsur | I'll post the remainign instructions later | 09:35 |
d0ugal | dtantsur: neat. | 09:36 |
d0ugal | well, messy but also neat ;) | 09:36 |
dtantsur | no, it's NOT neat :D it's horrible, but it works | 09:36 |
d0ugal | lol | 09:36 |
dtantsur | I especially like retries around 'docker cp', because it fails without clear reason A LOT | 09:36 |
d0ugal | ugh | 09:36 |
* d0ugal will bbiab | 09:37 | |
shardy | wow, yeah, err, well whatever works I guess :) | 09:37 |
shardy | I was thinking we'd build a new tripleo-common rpm with dlrn then rebuild the container referencing the local repo | 09:38 |
shardy | but not got any step-by-step notes on that | 09:38 |
*** aevoo has joined #tripleo | 09:38 | |
shardy | having it automated somewhere would be helpful, since we do that in CI and all developers must need to do the same sometimes | 09:38 |
dtantsur | shardy: this is going to be much longer, I suspect.. but I'd be glad to see a script | 09:38 |
shardy | dtantsur: yeah, I'll take a look at what CI does when I've had more coffee ;) | 09:39 |
shardy | I know the rpm build part, it's the container rebuild with repo override that I'm not sure on | 09:39 |
shardy | dtantsur: the other option is to just test it with mistral on baremetal I guess ;) | 09:40 |
shardy | I'll take a look at the container rebuild thing though | 09:41 |
*** Petersingh_ has joined #tripleo | 09:41 | |
*** akrivoka has joined #tripleo | 09:42 | |
*** Mantorok has quit IRC | 09:42 | |
sshnaidm | d0ugal, if ara has rpm, I suppose it should be done somewhere here? https://github.com/openstack/tripleo-common/blob/master/container-images/tripleo_kolla_template_overrides.j2#L48 | 09:42 |
*** Mantorok has joined #tripleo | 09:43 | |
*** moshele has joined #tripleo | 09:45 | |
*** Petersingh has quit IRC | 09:45 | |
openstackgerrit | Javier Peña proposed openstack-infra/tripleo-ci master: Replace call to map-project-name script with rdopkg https://review.openstack.org/552612 | 09:48 |
openstackgerrit | Javier Peña proposed openstack-infra/tripleo-ci master: Replace call to map-project-name script with rdopkg https://review.openstack.org/552612 | 09:49 |
*** akrivoka has quit IRC | 09:53 | |
*** shyam89 has quit IRC | 09:54 | |
*** ccamacho has quit IRC | 09:54 | |
*** ccamacho has joined #tripleo | 09:55 | |
shardy | dtantsur: FYI https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/tips_tricks.html#testing-a-code-fix-in-a-container shows how to rebuild the container (kudos bandini!) | 09:55 |
*** amoralej|brb is now known as amoralej | 09:56 | |
*** ykarel_ is now known as ykarel | 09:58 | |
dtantsur | shardy: sure, but involved rebuilding the undercloud.. and from my experience, you also need to manually kill the existing container, since the installer may not replace it. | 09:58 |
*** akrivoka has joined #tripleo | 09:59 | |
*** akrivoka has quit IRC | 10:01 | |
*** shyam89 has joined #tripleo | 10:02 | |
dtantsur | shardy: re "testing mistral on baremetal", building non-containerized underclouds with quickstart is no longer possible.. | 10:03 |
*** shyamb has joined #tripleo | 10:03 | |
*** jaosorior_ has quit IRC | 10:05 | |
*** akrivoka has joined #tripleo | 10:06 | |
*** akrivoka has quit IRC | 10:06 | |
*** akrivoka has joined #tripleo | 10:06 | |
*** shyam89 has quit IRC | 10:07 | |
shardy | dtantsur: I'm pretty sure it would be possible, but just not using instack-undercloud, I'll give it a try later | 10:09 |
shardy | e.g you could kill the container then re-run the deploy with the heat config pointing at the puppet/services/ files for mistral | 10:09 |
shardy | I know that's become a poorly tested path in CI lately though | 10:10 |
*** ooolpbot has joined #tripleo | 10:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 10:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 10:10 |
*** ooolpbot has quit IRC | 10:10 | |
openstack | Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 10:10 |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 10:10 |
dtantsur | shardy: wonderful, now hitting a missing dependency of ironicclient | 10:11 |
shardy | dtantsur: Hmm, don't we already use that for the various baremetal actions? | 10:12 |
dtantsur | shardy: I don't think we build configdrives | 10:13 |
shardy | dtantsur: Hmm, well all the nova deployed nodes have config drives, but I'm not sure how/where that happens | 10:13 |
dtantsur | shardy: it may be a dependency of the nova package | 10:14 |
shardy | yeah I guess it must be as it contains the nova user/meta-data | 10:14 |
shardy | but I was thinking of all the provide workflow etc | 10:14 |
openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-heat-templates stable/pike: DNM: test pike 004 https://review.openstack.org/594925 | 10:19 |
dtantsur | shardy: this stuff does not use configdrive building feature of ironicclient, but metalsmith does | 10:20 |
*** Petersingh_ is now known as Petersingh | 10:20 | |
*** aufi has joined #tripleo | 10:21 | |
shardy | dtantsur: ack | 10:22 |
openstackgerrit | Javier Peña proposed openstack-infra/tripleo-ci master: Replace call to map-project-name script with rdopkg https://review.openstack.org/552612 | 10:22 |
dtantsur | shardy: posted a comment with rough testing instructions | 10:23 |
shardy | dtantsur: thanks! | 10:23 |
Tengu | hmm. that selinux thing in container is a pain. | 10:28 |
Tengu | there are some weird things. | 10:28 |
*** ykarel is now known as ykarel|lunch | 10:29 | |
*** ykarel|lunch is now known as ykarel|away | 10:34 | |
d0ugal | sshnaidm: Interesting, I hadn't see that file before! That does look like the correct place. | 10:37 |
d0ugal | chandankumar: Did you create the release patch? | 10:37 |
*** sanjayu_ has joined #tripleo | 10:53 | |
*** saneax has quit IRC | 10:53 | |
*** sanjayu__ has joined #tripleo | 10:55 | |
*** apetrich has joined #tripleo | 10:58 | |
*** sanjayu_ has quit IRC | 10:58 | |
*** boazel has joined #tripleo | 10:59 | |
*** electrichead has quit IRC | 10:59 | |
*** slagle has quit IRC | 11:01 | |
*** Petersingh is now known as Petersingh|afk | 11:04 | |
*** ooolpbot has joined #tripleo | 11:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 11:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 11:10 |
*** ooolpbot has quit IRC | 11:10 | |
openstack | Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 11:10 |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 11:10 |
*** shyamb has quit IRC | 11:11 | |
*** dxiri has joined #tripleo | 11:12 | |
*** akrivoka_ has joined #tripleo | 11:14 | |
*** akrivoka_ has quit IRC | 11:14 | |
*** jaosorior has joined #tripleo | 11:15 | |
*** shyamb has joined #tripleo | 11:17 | |
chandankumar | d0ugal: https://review.openstack.org/#/c/594930/ | 11:17 |
*** ccamacho has quit IRC | 11:18 | |
*** ykarel|away has quit IRC | 11:19 | |
*** jpena is now known as jpena|lunch | 11:20 | |
openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart master: allow tripleo-roles to be updated in ci https://review.openstack.org/594933 | 11:24 |
*** akrivoka_ has joined #tripleo | 11:24 | |
*** ccamacho has joined #tripleo | 11:25 | |
ssbarnea|ruck | marios: ^^ i hope that is what was missing. | 11:25 |
tbarron | jaosorior: bogdando: bandini: pls review https://review.openstack.org/#/c/594801 which | 11:27 |
tbarron | jaosorior: bogdando: bandini is like https://review.openstack.org/#/c/539498/ but for manila rather than cinder | 11:28 |
jaosorior | tbarron: sure, have you had a chance to test it out? | 11:28 |
tbarron | jaosorior: gouthamr deployed with it and the bind mount is there | 11:28 |
*** abishop has quit IRC | 11:28 | |
tbarron | jaosorior: his deployment has some issue with the api connection so I'm redeploying fresh this morning | 11:29 |
tbarron | jaosorior: overcloud deploy completed, manila api service is running fine with the bind mount for the db | 11:29 |
tbarron | jaosorior: but he may not have the free ipa setup and connection just right, seems independent of this fix though | 11:30 |
*** akrivoka_ has quit IRC | 11:30 | |
tbarron | jaosorior: also he had to redeploy with telemetry disabled since it appears to have the same issue :( | 11:32 |
marios|rover | ssbarnea|ruck: ack lets see what ci makes of it (the master job was also failing but for different reason so it might not make them green yet ;) ) | 11:32 |
marios|rover | ssbarnea|ruck: but yeah that's what i meant in the comment | 11:33 |
*** rh-jelabarre has joined #tripleo | 11:33 | |
*** dhill_ has joined #tripleo | 11:33 | |
*** edmondsw has joined #tripleo | 11:33 | |
tbarron | jaosorior: bogdando: ty | 11:34 |
*** Petersingh|afk is now known as Petersingh | 11:37 | |
openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart master: DNM: test fs008 tempest https://review.openstack.org/594934 | 11:38 |
sshnaidm | tosky, just interesting if it will work ^^ | 11:38 |
openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-heat-templates stable/ocata: DNM: test ocata https://review.openstack.org/594810 | 11:39 |
*** saneax has joined #tripleo | 11:39 | |
*** sanjayu__ has quit IRC | 11:42 | |
*** saneax has quit IRC | 11:42 | |
openstackgerrit | Adriano Petrich proposed openstack/tripleo-quickstart-extras master: WIP validate that the mistral error is still happening. https://review.openstack.org/594935 | 11:43 |
*** morazi has joined #tripleo | 11:45 | |
bandini | tbarron: ops was too slow ;) | 11:51 |
sshnaidm | mwhahaha, hi, in which step do we update containers in CI now with that new ansible role? | 11:52 |
sshnaidm | EmilienM, stevebaker ^^ | 11:53 |
shardy | sshnaidm: which new ansible role? For overcloud deployments the containers are always started/restarted via paunch in whatever step the templates define them to start | 11:53 |
sshnaidm | shardy, I mean https://github.com/openstack/ansible-role-tripleo-modify-image | 11:54 |
*** leanderthal has quit IRC | 11:55 | |
sshnaidm | seems like I'm not familiar with that new workflow.. | 11:55 |
sshnaidm | weshay, ^^ | 11:55 |
*** raildo has joined #tripleo | 11:56 | |
weshay | sshnaidm, that is executed in the undercloud install | 11:56 |
weshay | http://git.openstack.org/cgit/openstack/tripleo-docs/tree/doc/source/install/advanced_deployment/container_image_prepare.rst#n83 | 11:58 |
*** boazel has quit IRC | 11:58 | |
sshnaidm | weshay, trying to find logs of that.. | 11:58 |
tbarron | bandini: ty anyways! | 11:59 |
jaosorior | #startmeeting TripleO Security Squad | 12:00 |
openstack | Meeting started Wed Aug 22 12:00:22 2018 UTC and is due to finish in 60 minutes. The chair is jaosorior. Information about MeetBot at http://wiki.debian.org/MeetBot. | 12:00 |
openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 12:00 |
*** openstack changes topic to " (Meeting topic: TripleO Security Squad)" | 12:00 | |
openstack | The meeting name has been set to 'tripleo_security_squad' | 12:00 |
*** lhinds has joined #tripleo | 12:00 | |
jaosorior | hey lhinds ! | 12:01 |
weshay | sshnaidm, it's in the stinkin journal | 12:01 |
weshay | sshnaidm, https://review.openstack.org/#/c/593716/ | 12:01 |
jaosorior | raildo, redrobot, are you around? | 12:01 |
lhinds | jaosorior: hey | 12:01 |
raildo | o/ | 12:01 |
jaosorior | I'll wait a bit to see if more folks log in | 12:02 |
jaosorior | Tengu: ^^ | 12:02 |
Tengu | jaosorior: :) | 12:02 |
*** abishop has joined #tripleo | 12:03 | |
*** lblanchard has joined #tripleo | 12:03 | |
*** moguimar has joined #tripleo | 12:04 | |
moguimar | o/ | 12:04 |
jaosorior | o/ | 12:04 |
jaosorior | Alright, lets start | 12:04 |
jaosorior | #topic SELinux for containers | 12:04 |
*** openstack changes topic to "SELinux for containers (Meeting topic: TripleO Security Squad)" | 12:04 | |
Tengu | EmilienM: -^ might be interesting for you as well :) | 12:05 |
jaosorior | So, Tengu has been involved with work related to moving from docker to podman, and on the way, he started checking out the SELinux integration for containers, a topic which we had in our view | 12:05 |
jaosorior | Tengu: IIRC, you started looking at this by running the containerized undercloud with podman, and SELinux enabled, right? | 12:05 |
Tengu | indeed. Following EmilienM first steps in order to get an undercloud running on podman instead of docker. | 12:06 |
jaosorior | So, Tengu run into some SELinux issues, which he tracked down in this trello card https://trello.com/c/hNkI15a7/1-selinux-issues | 12:06 |
*** amoralej is now known as amoralej|lunch | 12:06 | |
jaosorior | and they're not as much issues as I expected :D | 12:06 |
chandankumar | Ruck/Rover https://review.rdoproject.org/r/#/c/15732/ we are updating tempestconf to 2.0.0 in queens if you see any failures related to temepstconfiguraiton let us know | 12:06 |
*** electrichead has joined #tripleo | 12:06 | |
openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Fix network lines rendering https://review.openstack.org/594938 | 12:06 |
Tengu | indeed, but they are nasty :). | 12:06 |
jaosorior | they are | 12:07 |
jaosorior | So, whoever is interested in working on this, please contact me and Tengu about it, so we can put you up to speed. It's quite interesting work :D | 12:07 |
Tengu | some of the issues are "normal" and require a specific selinux policy. | 12:07 |
Tengu | but most of them exists only because of bad practices :] | 12:08 |
jaosorior | right, we need to divide which of these issues should we rectify ASAP, and which should have an "exception" in the selinux policy (and hopefully get fixed later) | 12:08 |
jaosorior | the first of the issues is the way we use docker/podman with puppet in order to generate the configurations; basically we bind-mount the /etc/puppet directory (which has a selinux label of etc_t) into the container to a temporary directory, and then attempt to copy that into the /etc/puppet directory in the container | 12:09 |
jaosorior | this is not allowed, as containers only have access to reading and executing stuff with etc_t | 12:10 |
*** ooolpbot has joined #tripleo | 12:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 12:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 12:10 |
*** ooolpbot has quit IRC | 12:10 | |
openstack | Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 12:10 |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 12:10 |
jaosorior | creating stuff from it, and writing to it, isn't allowed | 12:10 |
jaosorior | with the hopes of following an approach where we'll have an immutable host, we're going with the approach of having dedicated hieradata for the containers | 12:10 |
Tengu | also, the first thing docker-puppet.sh does is an rm -rf /etc/puppet/ssl directory in the container. | 12:10 |
Tengu | second step is to add a file in the /etc/puppet/hieradata directory | 12:11 |
Tengu | both actions are forbidden. | 12:11 |
Tengu | first one can be avoided by copying only wanted files. second one is trickier. | 12:11 |
*** ssbarnea|ruck has quit IRC | 12:13 | |
jaosorior | right, though having dedicated hieradata for the containers would solve these issues (I think) since we would then have those files with the needed selinux labels, and copy them as needed, trying to keep docker-puppet.py's functionality | 12:13 |
jaosorior | So, either we copy the hieradata to a temp location on the host, and relabel that, or we straight generate the hieradata on a container volume | 12:13 |
*** leanderthal has joined #tripleo | 12:14 | |
jaosorior | Anyway, this is the stuff that we've been discussing lately, and we'll keep this trello card updated: https://trello.com/c/hNkI15a7/1-selinux-issues | 12:15 |
jaosorior | Any questions/feedback/interest in this topic? | 12:15 |
weshay | sshnaidm, that patch is failing on file not found | 12:16 |
weshay | not sure why | 12:16 |
sshnaidm | weshay, because you use "shell: |" and need to use "shell: >" | 12:17 |
weshay | bah.. | 12:17 |
jaosorior | #topic Secret Management update | 12:18 |
openstackgerrit | Sagi Shnaidman proposed openstack/ansible-role-tripleo-modify-image master: log modify image to a log file for humans https://review.openstack.org/593716 | 12:18 |
*** openstack changes topic to "Secret Management update (Meeting topic: TripleO Security Squad)" | 12:18 | |
jaosorior | redrobot, moguimar, raildo: anything you wanna bring up on this topic? | 12:18 |
sshnaidm | weshay, ^^ | 12:18 |
moguimar | o/ | 12:18 |
moguimar | started working on the castellan drive | 12:18 |
moguimar | found out today that the castellan-vault tests are not working | 12:18 |
moguimar | already diagnosed the cause | 12:18 |
moguimar | from vault 0.10.0 forward there is a change in the API | 12:19 |
raildo | jaosorior, not from my side | 12:19 |
openstackgerrit | Sagi Shnaidman proposed openstack/ansible-role-tripleo-modify-image master: log modify image to a log file for humans https://review.openstack.org/593716 | 12:19 |
moguimar | so castellan fails to talk to a vault server >= 0.10.0 | 12:19 |
jaosorior | moguimar: so, is the castellan driver broken as well? | 12:19 |
moguimar | yep, I filed a bug on launchpad | 12:20 |
jaosorior | crap | 12:20 |
jaosorior | thanks for filing it | 12:20 |
moguimar | #link https://bugs.launchpad.net/castellan/+bug/1788375 | 12:20 |
openstack | Launchpad bug 1788375 in castellan "API changes in vault 0.10.0 causes test to fail." [Undecided,New] | 12:20 |
*** jpena|lunch is now known as jpena | 12:20 | |
openstackgerrit | Sagi Shnaidman proposed openstack/ansible-role-tripleo-modify-image master: log modify image to a log file for humans https://review.openstack.org/593716 | 12:20 |
moguimar | but the fix is quite simple | 12:20 |
*** dprince has joined #tripleo | 12:20 | |
moguimar | I already have a fix in progress | 12:20 |
moguimar | tests passing and all, just need to make it backward compatible with vault < 0.10.0 | 12:21 |
jaosorior | ade_lee: are we supposed to have access to castellan's launchpad? | 12:21 |
jaosorior | moguimar: when you have a fix let me know. thanks for working on this. | 12:21 |
*** trown|outtypewww is now known as trown | 12:21 | |
moguimar | I've analyzed the wireshark logs and the vault client itself does some http request to fetch API version | 12:21 |
jaosorior | moguimar: right, it does discovery. Maybe we can do that upon first interaction and cache the result. | 12:22 |
moguimar | so I'll add the same behaviour in the castellan driver | 12:22 |
moguimar | yep | 12:22 |
jaosorior | awesome | 12:22 |
jaosorior | moguimar: thanks for this work | 12:22 |
moguimar | o/ | 12:22 |
jaosorior | #topic Any other business | 12:24 |
*** openstack changes topic to "Any other business (Meeting topic: TripleO Security Squad)" | 12:24 | |
jaosorior | Anything else folks want to bring up to the meeting? | 12:24 |
openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart-extras master: WIP: Update default for THT resource registry https://review.openstack.org/594944 | 12:27 |
jaosorior | Alright folks! thanks for joining! | 12:28 |
moguimar | o/ | 12:28 |
electrichead | thanks jaosorior | 12:28 |
jaosorior | Just a reminder, the security squad meeting is now every two weeks | 12:28 |
jaosorior | so, talk to you here in two weeks! | 12:28 |
jaosorior | #endmeeting | 12:28 |
*** openstack changes topic to "Welcome to Rocky | CI Status: GREEN | https://docs.openstack.org/tripleo-docs/latest/" | 12:28 | |
moguimar | yup | 12:28 |
openstack | Meeting ended Wed Aug 22 12:28:48 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 12:28 |
openstack | Minutes: http://eavesdrop.openstack.org/meetings/tripleo_security_squad/2018/tripleo_security_squad.2018-08-22-12.00.html | 12:28 |
weshay | anyone have the link to the fedora tripleo etherpad? | 12:28 |
openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/tripleo_security_squad/2018/tripleo_security_squad.2018-08-22-12.00.txt | 12:28 |
openstack | Log: http://eavesdrop.openstack.org/meetings/tripleo_security_squad/2018/tripleo_security_squad.2018-08-22-12.00.log.html | 12:28 |
*** agopi has quit IRC | 12:28 | |
*** dxiri has quit IRC | 12:29 | |
*** thrash|g0ne is now known as thrash | 12:30 | |
*** akrivoka_ has joined #tripleo | 12:30 | |
*** akrivoka has quit IRC | 12:31 | |
*** rlandy has joined #tripleo | 12:32 | |
*** psachin has quit IRC | 12:32 | |
*** akrivoka has joined #tripleo | 12:32 | |
*** akrivoka has quit IRC | 12:33 | |
*** shyamb has quit IRC | 12:35 | |
*** toure has joined #tripleo | 12:38 | |
*** rnoriega_ has joined #tripleo | 12:38 | |
*** shyamb has joined #tripleo | 12:42 | |
*** akrivoka has joined #tripleo | 12:44 | |
openstackgerrit | Kamil Sambor proposed openstack/python-tripleoclient master: Add fixtures instead mock https://review.openstack.org/579538 | 12:44 |
Tengu | jaosorior: I was right from the very start in fact. | 12:47 |
*** tzumainn has joined #tripleo | 12:47 | |
jaosorior | Tengu: ? | 12:47 |
Tengu | jaosorior: cp: failed to restore the default file creation context: Permission denied | 12:47 |
Tengu | + cp -a /tmp/puppet-etc/auth.conf /tmp/puppet-etc/hiera.yaml /tmp/puppet-etc/hieradata /tmp/puppet-etc/modules /tmp/puppet-etc/puppet.conf /etc/puppet/ | 12:47 |
Tengu | so it's not even the fact we actually push the "docker.json" with the step. | 12:48 |
Tengu | it's the fact we can't restore properly the labels. | 12:48 |
Tengu | i.e. the cp itself | 12:48 |
Tengu | and doing temporary directory will probably NOT do any good | 12:48 |
jaosorior | what context do they have? | 12:48 |
Tengu | wait, modifing. | 12:48 |
*** agopi has joined #tripleo | 12:56 | |
*** tbonds has joined #tripleo | 12:56 | |
Tengu | jaosorior: http://paste.openstack.org/show/728590/ | 12:58 |
openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Collect overcloud statistics with ARA https://review.openstack.org/578462 | 12:58 |
Tengu | jaosorior: this is from the container directly. | 12:58 |
*** Petersingh has quit IRC | 12:58 | |
Tengu | MAYBE it's an issue with the container and libpod. | 12:58 |
dtantsur | folks, I have some bad news. It seems that the very basic and default deployment of a rocky overcloud fails with: sudo: no tty present and no askpass program specified | 13:00 |
dtantsur | this is on running: Command: /usr/bin/sudo /bin/nova-manage cell_v2 discover_hosts --verbose | 13:00 |
*** jcoufal has joined #tripleo | 13:01 | |
openstackgerrit | Brent Eagles proposed openstack/puppet-tripleo master: Use logrotate copytruncate with neutron services https://review.openstack.org/582661 | 13:01 |
*** aevoo has quit IRC | 13:02 | |
*** ssbarnea has joined #tripleo | 13:03 | |
*** boazel has joined #tripleo | 13:03 | |
Tengu | jaosorior: replacing the "cp -a" with a "cp -dR" seems to make it go far, far further. | 13:03 |
*** amoralej|lunch is now known as amoralej | 13:05 | |
jaosorior | uhm | 13:06 |
jaosorior | interesting | 13:06 |
jaosorior | maybe this is just the way it's supposed to be | 13:07 |
jaosorior | and we just didn't take into account the container's SELinux context | 13:07 |
*** ssbarnea is now known as ssbarnea_ | 13:07 | |
Tengu | ^^ | 13:07 |
jaosorior | Tengu: notice c266,c558 for /etc/puppet | 13:07 |
dtantsur | shardy: are you aware of any problems with rocky deployment? (see above) | 13:07 |
Tengu | jaosorior: yep, because it's create from within the container | 13:07 |
jaosorior | while /tmp/puppet-etc has an c0,c0 (which doesn't show up cause it's the default... which comes from the host) | 13:07 |
Tengu | yep | 13:08 |
Tengu | jaosorior: and the error is: can't restore. | 13:08 |
jaosorior | I completely disregarded that | 13:08 |
jaosorior | yeah, that's the expected behavior then | 13:08 |
Tengu | yep | 13:08 |
Tengu | sooo | 13:08 |
Tengu | drop -a | 13:08 |
Tengu | add -dR | 13:08 |
Tengu | and we're good. | 13:08 |
jaosorior | sounds good | 13:08 |
Tengu | in fact -a == -dR --preserve=ALL | 13:08 |
Tengu | we can --preserve some other things, but not 100% sure it's needed. | 13:09 |
Tengu | running a new deploy with the modified code. | 13:09 |
Tengu | will check what's the next error. | 13:09 |
*** ooolpbot has joined #tripleo | 13:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 13:10 |
openstack | Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea) | 13:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 13:10 |
*** ooolpbot has quit IRC | 13:10 | |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 13:10 |
shardy | dtantsur: not aware, CI looks OK and I deployed yesterday and things seemed to be working - when do you see that error? | 13:10 |
jaosorior | shardy, bandini could I get a review for this https://review.openstack.org/#/c/593491/ ? | 13:12 |
*** boazel has quit IRC | 13:12 | |
*** rh-jelabarre has quit IRC | 13:13 | |
*** ssbarnea_ has quit IRC | 13:13 | |
openstackgerrit | Luigi Toscano proposed openstack/tripleo-heat-templates master: Bind mount the he database client settings in sahara_db_sync https://review.openstack.org/594980 | 13:13 |
*** ssbarnea_ has joined #tripleo | 13:13 | |
*** mjturek has joined #tripleo | 13:14 | |
Tengu | jaosorior: crossing fingers, I'm at the "step 1". Hopefully it will do what I want :). | 13:15 |
Tengu | there's also an "rsync" command, will check what's going on with that one. | 13:15 |
*** ssbarnea_ has quit IRC | 13:16 | |
*** ssbarnea|ruck has joined #tripleo | 13:16 | |
dtantsur | shardy: on trying to deploying anything on a clean undercloud.. literally just openstack overcloud deploy --templates | 13:17 |
openstackgerrit | Luigi Toscano proposed openstack/tripleo-heat-templates master: Bind mount the database client settings in sahara_db_sync https://review.openstack.org/594980 | 13:18 |
*** ssbarnea|ruck has quit IRC | 13:18 | |
*** ssbarnea|ruck has joined #tripleo | 13:18 | |
Tengu | jaosorior: rsync: opendir "/etc/pki/ca-trust/extracted" failed: Permission denied (13) next error :). | 13:19 |
openstackgerrit | Daniel Alvarez proposed openstack/tripleo-quickstart master: DNM: OVN tempest test https://review.openstack.org/587830 | 13:20 |
jaosorior | Tengu: funky, I wonder what happened there. | 13:20 |
jaosorior | oh yeah | 13:20 |
jaosorior | cert_t | 13:20 |
jaosorior | so that I do think we should add to the policy | 13:20 |
*** etingof has joined #tripleo | 13:20 | |
Tengu | probably yes. | 13:20 |
Tengu | will audit2allow in order to check. | 13:21 |
*** shyamb has quit IRC | 13:21 | |
Tengu | allow container_t cert_t:dir read; allow container_t cert_t:file read; | 13:21 |
Tengu | seems fine. | 13:21 |
*** etingof has quit IRC | 13:21 | |
Tengu | jaosorior: hmmm. allow container_t security_t:file write; ?? | 13:21 |
jaosorior | no | 13:21 |
Tengu | ah, wait. | 13:21 |
jaosorior | security_t is for selinux | 13:21 |
Tengu | yes | 13:21 |
jaosorior | cert_t is for certs | 13:21 |
Tengu | will truncate the audit. | 13:22 |
openstackgerrit | Martin Schuppert proposed openstack/tripleo-heat-templates master: Move nova-metadata api to httpd wsgi https://review.openstack.org/582623 | 13:26 |
*** akrivoka_ has quit IRC | 13:28 | |
*** hjensas has quit IRC | 13:28 | |
openstackgerrit | Martin Schuppert proposed openstack/puppet-tripleo master: Do not create metadata ssl proxy if we have metadata api via httpd wsgi https://review.openstack.org/595001 | 13:30 |
openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Avoid SELinux issues with the "cp -a" command https://review.openstack.org/595003 | 13:31 |
Tengu | jaosorior: EmilienM -^^ | 13:31 |
*** shardy is now known as shardy_mtg | 13:31 | |
sri_ | dsneddon, beagles, i am trying to deploy overcloud with dvr enabled in stable/qeens , I've created my roles with /roles/ComputeDVR.yaml and I've generated network templates using ./process-templates.py , but in compute node os-net-config not generating based on my computedvr.yaml config it's generating generic compute templates(default). here is my config http://paste.openstack.org/show/728594/ | 13:33 |
*** jcoufal has quit IRC | 13:33 | |
Tengu | jaosorior: so basically, now, the policy looks like: http://paste.openstack.org/show/728595/ | 13:35 |
Tengu | I'm not happy with the security_t and setfscreate. | 13:36 |
*** sri_ has quit IRC | 13:37 | |
*** sri_ has joined #tripleo | 13:37 | |
jaosorior | Tengu: neither am I | 13:38 |
*** boazel has joined #tripleo | 13:38 | |
Tengu | ;) | 13:41 |
Tengu | jaosorior: in mtg - will check that after | 13:41 |
openstackgerrit | Florian Fuchs proposed openstack/tripleo-validations stable/queens: Fix node health validation https://review.openstack.org/595006 | 13:42 |
dtantsur | shardy_mtg: rebased and now my script stopped working >_< I guess I'll put this work on hold until somebody comes with a working procedure of developing on the undercloud. | 13:43 |
openstackgerrit | Merged openstack-infra/tripleo-ci master: Remove tripleo.sh --bootstrap-subnodes add ceph loop device https://review.openstack.org/583195 | 13:43 |
openstackgerrit | Merged openstack/tripleo-heat-templates master: Group fast_forward_upgrade_tasks tasks into blocks https://review.openstack.org/587235 | 13:43 |
*** dtrainor has quit IRC | 13:44 | |
Tengu | jaosorior: will create a temporary policy with the cert_t thingy + ovs socket, and see what's crashing. | 13:46 |
jaosorior | Tengu: sounds good to me | 13:47 |
*** jcoufal has joined #tripleo | 13:47 | |
Tengu | jaosorior: hm, do you have a good cheat-sheet for building custom selinux modules? | 13:50 |
jaosorior | I don't : | 13:50 |
jaosorior | :/ | 13:50 |
Tengu | hmm. ok. I just known the audit2allow -M + semodule -i | 13:51 |
dtantsur | shardy_mtg, mwhahaha, posted https://bugs.launchpad.net/tripleo/+bug/1788414 so that we track the problems with development we have now. Feel free to contribute ideas, suggestions, etc. | 13:51 |
openstack | Launchpad bug 1788414 in tripleo "No easy way for local debugging on the containerized undercloud" [Critical,Confirmed] | 13:51 |
*** akrivoka has quit IRC | 13:51 | |
*** akrivoka has joined #tripleo | 13:52 | |
openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Fix standalone home dir variable https://review.openstack.org/595010 | 13:52 |
jaosorior | bandini: thanks for the review | 13:53 |
mwhahaha | dtantsur: so i guess it assumes openstack projects have their own way of developing their features. So being able to test things in a non-deployment fashion would help reduce the criticality of that. I do agree there needs to be a better way to test tripleo-common and such in containers but that also points to our inability to validate changes outside of a live environment | 13:54 |
openstackgerrit | Merged openstack/tripleo-heat-templates master: undercloud: revert to using the iscsi deploy interface by default https://review.openstack.org/585370 | 13:57 |
openstackgerrit | Merged openstack/tripleo-quickstart-extras master: add delorean-current to repolist for updates https://review.openstack.org/592577 | 13:59 |
*** gouthamr has quit IRC | 14:00 | |
*** dmellado has quit IRC | 14:01 | |
*** stevebaker has quit IRC | 14:01 | |
*** ksambor has quit IRC | 14:01 | |
*** shardy_mtg is now known as shardy | 14:01 | |
shardy | dtantsur: there should be no need to rebuild the undercloud, only restart the container, as you mention re-running paunch via the undercloud install should be enough | 14:01 |
shardy | dtantsur: thanks for the bug, can you please share your quickstart config, and I'll reproduce and add some ideas on how we can streamline things | 14:02 |
shardy | or even just restarting the container with the new image I guess | 14:02 |
dtantsur | shardy: well, any quickstart config, I don't see why it would matter. | 14:03 |
dtantsur | even without quickstart | 14:03 |
shardy | dtantsur: well you said there was a nova error - my local quickstart env built yesterday is working fine | 14:04 |
dtantsur | shardy: I realized that it may be because I updated tripleo-common from an older checkout | 14:04 |
shardy | dtantsur: Ok cool, let me take a pass at the container restart steps then | 14:04 |
dtantsur | after rebasing my ugly script stopped working :( | 14:04 |
shardy | Ok lets see if we can come up with a slightly less ugly and working one :) | 14:05 |
dtantsur | shardy: thanks! we need something (semi)automatic, otherwise the life of occasional contributors like me becomes too miserable | 14:05 |
shardy | dtantsur: yeah I agree | 14:05 |
openstackgerrit | Merged openstack/tripleo-heat-templates master: Fix bind-mount to manila's bootstrap container https://review.openstack.org/594801 | 14:06 |
openstackgerrit | Merged openstack/tripleo-heat-templates master: Clarify ironic classic driver removal reno https://review.openstack.org/593041 | 14:06 |
*** rh-jelabarre has joined #tripleo | 14:06 | |
openstackgerrit | Tom Barron proposed openstack/tripleo-heat-templates stable/queens: Fix bind-mount to manila's bootstrap container https://review.openstack.org/595014 | 14:07 |
*** dtrainor has joined #tripleo | 14:08 | |
*** hjensas has joined #tripleo | 14:09 | |
*** haleyb has joined #tripleo | 14:10 | |
*** ooolpbot has joined #tripleo | 14:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 14:10 |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 14:10 |
*** ooolpbot has quit IRC | 14:10 | |
*** moshele has quit IRC | 14:10 | |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Bind mount tripleo.cnf for tls everywhere work for gnocchi https://review.openstack.org/595018 | 14:12 |
*** mwhahaha changes topic to "Welcome to Rocky | CI Status: ORANGE - RDO Cloud unavailable| https://docs.openstack.org/tripleo-docs/latest/" | 14:13 | |
weshay | marios|rover, ssbarnea|ruck join #rdo please | 14:15 |
weshay | marios|rover, ssbarnea|ruck rdo cloud just went down | 14:15 |
sri_ | dsneddon, beagles, nerver mind i pass the network-environment.yaml as last that's fixed the issue, but little bit tricky to get the order right | 14:16 |
openstackgerrit | Merged openstack/tripleo-heat-templates master: Assert container's ssl cert exists before upgrade. https://review.openstack.org/589082 | 14:18 |
Tengu | jaosorior: puppet wants to apply selinux labels. | 14:20 |
Tengu | jaosorior: but the two policies I've created drastically reduce the audit.log content :). | 14:21 |
Tengu | so I'm really happy now. Will update the trello. | 14:22 |
*** ssbarnea|ruck has quit IRC | 14:23 | |
*** toure is now known as toure|biab | 14:24 | |
jaosorior | Tengu: can we track which labels it tries to apply? maybe we can move that out of puppet. | 14:25 |
Tengu | jaosorior: http://paste.openstack.org/show/728602/ | 14:28 |
Tengu | jaosorior: fact is, puppet MODULES might want to set the labels on the fly for either new or existing files. | 14:29 |
Tengu | jaosorior: file { "...": ensure => (file|directory), selabel => ...} for instance. | 14:29 |
Tengu | (not sure about the option name, but you get it) | 14:29 |
weshay | rascasoft, rasca? | 14:31 |
jaosorior | Tengu: isn't that handled by a puppet provider? could we set that provider to be noop? | 14:40 |
*** raildo has quit IRC | 14:40 | |
*** raildo has joined #tripleo | 14:42 | |
bogdando | PTAL https://review.openstack.org/#/c/587431, jaosorior ? | 14:44 |
bogdando | it was 2 +2, just rebased | 14:45 |
*** jrist has joined #tripleo | 14:45 | |
*** gouthamr has joined #tripleo | 14:50 | |
*** bugzy has joined #tripleo | 14:51 | |
*** raildo_ has joined #tripleo | 14:51 | |
*** morazi has quit IRC | 14:51 | |
openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart master: Include overcloud-deploy to the default tags https://review.openstack.org/594133 | 14:52 |
openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Collect overcloud statistics with ARA https://review.openstack.org/578462 | 14:53 |
*** bugzy_ has quit IRC | 14:53 | |
Tengu | jaosorior: hm. nope. | 14:53 |
d0ugal | Workflow squad meeting in 5 mins. https://etherpad.openstack.org/p/tripleo-workflows-squad-status /cc rbrady,apetrich,thrash,toure,jtomasek | 14:54 |
Tengu | jaosorior: the "file" type has the selinux support directly embedded, and check the puppet facts for that. | 14:54 |
Tengu | oh. | 14:54 |
Tengu | wait. | 14:54 |
Tengu | MAYBE.... we can override that fact locally in the container? | 14:54 |
*** raildo has quit IRC | 14:54 | |
Tengu | jaosorior: selinux => true if we set it to "false", the file resources should not try to enforce the labels/context. | 14:55 |
shardy | dtantsur: FYI I did a first pass on some notes here https://etherpad.openstack.org/p/tripleo-rpm-container-image-notes | 14:55 |
*** mmedvede is now known as mmedvede_ | 14:55 | |
Tengu | but it may create some other issues. | 14:55 |
Tengu | let's try. | 14:55 |
shardy | probably we need a script and some optimization of those steps, but hopefully it helps understand the overview | 14:55 |
*** mmedvede_ is now known as mmedvede | 14:55 | |
dtantsur | shardy: will read after the meetings, thansk | 14:55 |
*** tzumainn has quit IRC | 14:56 | |
openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Make sure we properly set container related facts during upgrade. https://review.openstack.org/595143 | 14:57 |
chem | jfrancoa: ^ | 14:58 |
openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart-extras master: WIP: Update default for THT resource registry https://review.openstack.org/594944 | 14:58 |
*** moguimar has quit IRC | 14:58 | |
*** apetrich has quit IRC | 14:58 | |
dtantsur | shardy: oh yeah, we do need a script for that. I can barely understand it, let alone remember :) | 14:58 |
jfrancoa | chem: thank you, I'll check it | 14:58 |
chem | jfrancoa: untested, unrun, but sure it works! | 14:59 |
shardy | dtantsur: heh yeah a script would be good, but hopefully that helps in the meantime - we can probably cut/paste a script to get you building tripleo-common and restarting all the mistral containers | 14:59 |
dtantsur | shardy: btw, how long does it long take? | 15:00 |
shardy | dtantsur: not sure as I was cut/pasting notes between steps - the longest is the dlrn-setup (which should be 1-time) and the dlrn build | 15:00 |
shardy | the docker part is very fast | 15:00 |
dtantsur | k, I wonder if we can do it within 2 minutes | 15:01 |
shardy | let me try to create a script and run it end to end | 15:01 |
*** ssbarnea|ruck has joined #tripleo | 15:02 | |
mwhahaha | shardy: i've been reworking the oooq build test package ansible role to be a standalone thing which may help wrap that build thing | 15:02 |
mwhahaha | https://github.com/mwhahaha/tripleo-f28-testbed/tree/master/roles/bootstrap-packages | 15:03 |
*** dxiri has joined #tripleo | 15:03 | |
mwhahaha | accepts a yaml with gerrit changes and gives you a repo to use | 15:03 |
shardy | mwhahaha: ack nice, yeah that may be easier, I just wanted to capture the steps initially | 15:03 |
*** dxiri has quit IRC | 15:03 | |
*** dxiri has joined #tripleo | 15:04 | |
*** tonyb has quit IRC | 15:04 | |
*** aufi has quit IRC | 15:05 | |
*** morazi has joined #tripleo | 15:05 | |
*** aufi has joined #tripleo | 15:05 | |
Tengu | jaosorior: faking the fact using the FACTER_selinux=false doesn't seem to do it - at least not completly.... getting closer though. | 15:06 |
shardy | dtantsur: the dlrn build takes just over 2mins for me, but that can possibly be improved as atm it's e.g cloning rdoinfo every time | 15:07 |
*** rnoriega_ is now known as rnoriega | 15:07 | |
dtantsur | shardy: ideally yes. 2+ minutes is a bit too much.. | 15:07 |
mwhahaha | ... | 15:08 |
shardy | yeah, better than redeploying the undercloud though ;) | 15:08 |
dtantsur | absolutely | 15:08 |
dtantsur | but when working on my patch I had 10+ iteration a day. it would be 20 minutes just waiting for DLRN :) | 15:08 |
shardy | another option might be to just setup.py install in the Dockerfile I guess | 15:09 |
mwhahaha | compared to everything else we do on a regular basis 20 mins spread over the course of the day is not that much | 15:09 |
mwhahaha | i think i spent 20 mins waiting for images to download | 15:09 |
dtantsur | well, it adds to everything, not replaced it | 15:09 |
dtantsur | 20 mins here, 20 mins there, y'know.. | 15:09 |
shardy | sec let me try the hack-install-from source via docker build | 15:10 |
*** pcaruana has quit IRC | 15:10 | |
*** ooolpbot has joined #tripleo | 15:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 15:10 |
*** ooolpbot has quit IRC | 15:10 | |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 15:10 |
*** rh-jelabarre has quit IRC | 15:11 | |
dtantsur | isn't ^^^ already fixed in nova? | 15:11 |
mwhahaha | it's probably not availabel yet in promotion | 15:12 |
dtantsur | I thought it's blocking promotion. Maybe I'm confusing something. | 15:12 |
mwhahaha | it has to make it into packaging, etc | 15:13 |
mwhahaha | it should be fixed but with rdo cloud dead, who knows | 15:13 |
dtantsur | CI of Schroedinger | 15:19 |
shardy | dtantsur: I added an alternative that does setup.py install intead | 15:20 |
shardy | should be pretty fast, seems to build but I've not tested the resulting image | 15:20 |
dtantsur | I guess it's better if it works | 15:21 |
shardy | seems to start so worth trying I think | 15:23 |
EmilienM | weshay: anyone looking at stable/rocky failures in tripleoclient, | 15:27 |
EmilienM | ? | 15:27 |
EmilienM | http://logs.openstack.org/61/594261/1/check/tripleo-ci-centos-7-undercloud-containers/8e99210/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2018-08-22_00_29_50 | 15:27 |
EmilienM | amoralej: ^ | 15:27 |
dsneddon | sri_, you might have run into this bug? https://review.openstack.org/#/c/592652 | 15:28 |
weshay | EmilienM, the lord is watching everything at all times | 15:29 |
EmilienM | yup | 15:29 |
* weshay gets coffee | 15:29 | |
EmilienM | we are waiting for this branch to work in CI | 15:29 |
EmilienM | so we can release | 15:29 |
weshay | EmilienM, rdo is down man | 15:30 |
EmilienM | I know | 15:30 |
weshay | can we release rocky w/o a promotion? | 15:30 |
EmilienM | no | 15:30 |
weshay | EmilienM, I think NOT | 15:30 |
*** openstackgerrit has quit IRC | 15:31 | |
amoralej | EmilienM, that job is trying to pull from /docker.io/tripleorocky/centos-binary-barbican-api:current-tripleo | 15:32 |
amoralej | have we uploaded containers there? | 15:32 |
amoralej | i can't see them there | 15:32 |
EmilienM | since we didn't have a first promotion, I suspect no | 15:32 |
amoralej | yes, that's also my guess | 15:33 |
EmilienM | but we could workaround by pushing the containers from latest promotion in master no? | 15:33 |
*** openstackgerrit has joined #tripleo | 15:33 | |
openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: Avoid ignoring failure to collect missing folders https://review.openstack.org/594077 | 15:33 |
amoralej | EmilienM, latest before branching | 15:33 |
weshay | EmilienM, /me gets patch | 15:33 |
weshay | EmilienM, rocky jobs should be using master containers | 15:34 |
amoralej | there is a https://trunk.rdoproject.org/centos7-rocky/current-tripleo/ | 15:34 |
Tengu | jaosorior: ok. so now I have the var_lib_t thingy: Error: Could not set 'present' on ensure: Permission denied - /var/lib/ironic/httpboot/inspector.ipxe20180822-11-11oty8d.lock at /etc/puppet/modules/ironic/manifests/inspector.pp:336 | 15:34 |
Tengu | jaosorior: guess we can't avoid adding the var_lib_t write access... at least for now. | 15:34 |
weshay | EmilienM, amoralej https://review.openstack.org/#/c/594357/ | 15:34 |
amoralej | ack | 15:35 |
EmilienM | weshay: ok let me 'recheck' | 15:35 |
weshay | EmilienM, ya.. your patch ran on old code | 15:37 |
weshay | EmilienM, come on bro | 15:37 |
weshay | hive mind | 15:37 |
EmilienM | I put high expectations in you weshay | 15:37 |
*** aufi has quit IRC | 15:44 | |
*** dmellado has joined #tripleo | 15:45 | |
*** sri_ has quit IRC | 15:46 | |
openstackgerrit | melissaml proposed openstack/tripleo-heat-templates master: Remove the duplicated word https://review.openstack.org/595234 | 15:49 |
*** sri_ has joined #tripleo | 15:49 | |
*** pcaruana has joined #tripleo | 15:49 | |
jaosorior | Tengu: the option there is to remove the security labeling option from that specific container | 15:51 |
Tengu | jaosorior: for the security_t thingy? | 15:51 |
*** tzumainn has joined #tripleo | 15:51 | |
jaosorior | Tengu: --security-opt label:disable | 15:51 |
jaosorior | Tengu: no, for the ironic container | 15:52 |
Tengu | ah | 15:52 |
Tengu | jaosorior: so that's for the var_lib_t then? or is it unrelated? | 15:52 |
jaosorior | Tengu: correct, for the var_lib_t thing | 15:52 |
jaosorior | Tengu: I had a blog post on that at some point... | 15:52 |
jaosorior | let me fetch it | 15:52 |
*** gkadam is now known as gkadam-afk | 15:52 | |
jaosorior | Tengu: http://jaormx.github.io/2018/selinux-and-docker-notes/ | 15:53 |
Tengu | jaosorior: hm, what's the best to do in fact? disabling this flag, or...? | 15:53 |
jaosorior | Tengu: write, if a container REALLY needs to do something extra (like the ironic container) then we disable selinux labeling for that specific container | 15:53 |
jaosorior | this way, the rest of the containers stay secure, while this special one... well... remains doing it's special thing :) | 15:53 |
*** leanderthal has quit IRC | 15:54 | |
Tengu | ok. makes sense. | 15:54 |
jaosorior | Tengu: at some point we can come back to it and figure out if we can change the behavior of the container | 15:54 |
Tengu | jaosorior: still, while checking the logs of the containers, I get error with rsync and the PKI sync. will dig into this a bit more. | 15:54 |
*** paramite_ has joined #tripleo | 15:58 | |
Tengu | damn. that's the archive done at the end of the puppet initial run.... | 15:59 |
openstackgerrit | RedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/560445 | 16:00 |
*** rpioso|afk is now known as rpioso | 16:02 | |
Tengu | fact is, we don't need to archive the PKI - it's a mounted volume. | 16:04 |
*** yprokule has quit IRC | 16:04 | |
*** rdopiera has quit IRC | 16:04 | |
*** gouthamr has quit IRC | 16:05 | |
Tengu | jaosorior: guess your proposal won't be OK, since all containers actually write in /var/lib: https://github.com/openstack/tripleo-heat-templates/blob/master/docker/docker-puppet.py#L264-L266 | 16:06 |
Tengu | and also L257 | 16:06 |
jaosorior | Tengu: yeah; we talked about that with dprince, and shardy; ideally we should write to docker volumes instead of /var/lib/config-data/... | 16:06 |
Tengu | ;) | 16:07 |
jaosorior | Tengu: but I guess dealing with that is out of scope for now | 16:07 |
*** morazi has quit IRC | 16:07 | |
Tengu | so we might want to enable the var_lib_t write directly. | 16:07 |
jaosorior | so, for now, the only thing we can do is add the var_lib_t write permission | 16:07 |
jaosorior | yeah | 16:07 |
Tengu | "cool". | 16:07 |
jaosorior | :( | 16:07 |
Tengu | I have three policies in order to get a clean way to disable them later. | 16:07 |
Tengu | oh wow. already that late? | 16:08 |
*** jpena is now known as jpena|off | 16:09 | |
weshay | mwhahaha, EmilienM do I need to make sure ntpdate is just available on the undercloud or do I need to ensure it's installed.. I think installed | 16:09 |
mwhahaha | weshay: so both | 16:10 |
*** ooolpbot has joined #tripleo | 16:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 16:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1788426 | 16:10 |
*** ooolpbot has quit IRC | 16:10 | |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 16:10 |
openstack | Launchpad bug 1788426 in tripleo "Tracking bug: RDO Cloud is down" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 16:10 |
mwhahaha | weshay: i was going to propose a patch to get it into the fedora stable repo for rdo but then rdo cloud ate it | 16:10 |
openstackgerrit | David Peacock proposed openstack/tripleo-common master: add Python 3 to requirements https://review.openstack.org/595242 | 16:10 |
chandankumar | mwhahaha: is there a way to create reproducable env using standalone mode in offline mode? | 16:11 |
mwhahaha | weshay: so if you configure ValidateNtp to false, then you don't need it installed and just need it available for install | 16:11 |
*** paramite_ has quit IRC | 16:11 | |
mwhahaha | chandankumar: if you pull down all the containers a head of time to a registry, maybe? | 16:11 |
chandankumar | mwhahaha: i need to try that | 16:12 |
weshay | chandankumar, what do you mean offline | 16:13 |
mwhahaha | chandankumar: it uses a similar method to the overcloud so as along as you can use the container prepare to point to a local registry, it should work | 16:13 |
mwhahaha | chandankumar: it would also assume a local copy of the tripleo rpm repo(s) as well | 16:13 |
chandankumar | weshay: in devstack, we have a offline mode, like we installed devstack do ubstack.sh then shut down the vm then restsrat the vm and re-run stack.sh everything is back to normal | 16:14 |
chandankumar | weshay: I am looking for something that like scenario | 16:14 |
weshay | wouldn't that be super | 16:15 |
Tengu | jaosorior: in fact, we should use named volumes, no bind-mount, when we want persistent data to be written... Right? | 16:15 |
chandankumar | it would be useful for developer developing stuff on a laptop | 16:15 |
*** raildo has joined #tripleo | 16:16 | |
*** raildo_ has quit IRC | 16:19 | |
*** morazi has joined #tripleo | 16:19 | |
Tengu | jaosorior: ok. and apparently, we don't have a selinux relabelling issued by puppet... the file /var/lib/ironic/httpboot/inspector.ipxe20180822-11-17p9bsk.lock comes from puppet, as it's wanting to create /var/lib/ironic/httpboot/inspector.ipxe from a template. The audit.log shows we can't change the file group to "ironic-inspector" apparently. | 16:21 |
Tengu | and apparently, that part is managed by a "write" with security_t OR setfscreate. that's the last issues I get for now. | 16:22 |
*** rh-jelabarre has joined #tripleo | 16:28 | |
*** moshele has joined #tripleo | 16:28 | |
jaosorior | Tengu: lets talk about this tomorrow; gotta head off | 16:34 |
jaosorior | Tengu: and the assertion about named volumes is correct :) that's the ideal thing. | 16:34 |
Tengu | jaosorior: ok :) | 16:37 |
Tengu | jaosorior: see you tomorrow then! | 16:37 |
Tengu | have a good one | 16:37 |
Tengu | will also sign off. | 16:37 |
Tengu | pretty long day. | 16:37 |
*** toure|biab is now known as toure | 16:38 | |
*** jpich has quit IRC | 16:40 | |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Bind mount tripleo.cnf for tls everywhere work for telemetry https://review.openstack.org/595018 | 16:44 |
*** gouthamr has joined #tripleo | 16:46 | |
*** gkadam-afk is now known as gkadam | 16:49 | |
*** ykarel has joined #tripleo | 16:51 | |
*** trown is now known as trown|lunch | 16:52 | |
*** agurenko has quit IRC | 16:53 | |
*** gkadam has quit IRC | 16:54 | |
*** trozet has quit IRC | 17:05 | |
*** gbarros has joined #tripleo | 17:10 | |
*** ooolpbot has joined #tripleo | 17:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 17:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1788426 | 17:10 |
*** ooolpbot has quit IRC | 17:10 | |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 17:10 |
openstack | Launchpad bug 1788426 in tripleo "Tracking bug: RDO Cloud is down" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 17:10 |
*** rh-jelabarre has quit IRC | 17:15 | |
*** ykarel has quit IRC | 17:20 | |
sri_ | dsneddon, ack | 17:21 |
*** ykarel has joined #tripleo | 17:21 | |
openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart master: Use become when doing recursive chown https://review.openstack.org/595265 | 17:21 |
weshay | mwhahaha, EmilienM do you want a lp tracking the progress of f28? I don't care.. just let me know how you guys want to iterate | 17:21 |
weshay | http://paste.openstack.org/show/728619/ | 17:21 |
*** vinaykns has joined #tripleo | 17:21 | |
weshay | guess the step is PODMAN | 17:21 |
weshay | < . . . . | 17:22 |
*** vinaykns has left #tripleo | 17:22 | |
EmilienM | bandini: why are we overriding FACTER_hostname in docker-puppet.py | 17:22 |
EmilienM | ? | 17:22 |
EmilienM | facter should be able to get $::hostname by itself, no? | 17:22 |
*** morazi has quit IRC | 17:23 | |
*** ykarel_ has joined #tripleo | 17:24 | |
mwhahaha | weshay: that's already covered in my etherpad | 17:24 |
mwhahaha | weshay: w/ a bug and patches | 17:24 |
*** ykarel_ has quit IRC | 17:24 | |
* weshay looks again | 17:24 | |
*** ykarel_ has joined #tripleo | 17:24 | |
*** morazi has joined #tripleo | 17:25 | |
mwhahaha | weshay: https://review.openstack.org/593963 and https://review.openstack.org/#/c/594499 | 17:25 |
*** rnoriega has quit IRC | 17:25 | |
*** bcafarel has quit IRC | 17:25 | |
*** mmedvede has quit IRC | 17:25 | |
*** rf0lc0 has quit IRC | 17:25 | |
*** redrobot has quit IRC | 17:25 | |
*** mburned has quit IRC | 17:25 | |
*** jpena|off has quit IRC | 17:25 | |
*** panda|off has quit IRC | 17:25 | |
*** marios has quit IRC | 17:25 | |
*** EmilienM has quit IRC | 17:25 | |
*** jschlueter has quit IRC | 17:25 | |
*** rakhmerov has quit IRC | 17:25 | |
*** weshay has quit IRC | 17:25 | |
*** ericyoung has quit IRC | 17:25 | |
*** Damjanek has quit IRC | 17:25 | |
*** corvus has quit IRC | 17:25 | |
bandini | EmilienM: sec lemme find it | 17:25 |
*** ykarel has quit IRC | 17:26 | |
*** ykarel_ has quit IRC | 17:27 | |
*** gouthamr has quit IRC | 17:29 | |
*** holser_ has quit IRC | 17:30 | |
*** EmilienM has joined #tripleo | 17:30 | |
openstackgerrit | Luigi Toscano proposed openstack/tripleo-heat-templates stable/queens: Add FFU upgrade steps to Sahara services https://review.openstack.org/595270 | 17:32 |
*** ChanServ sets mode: +v EmilienM | 17:33 | |
*** jpena has joined #tripleo | 17:36 | |
*** bogdando has quit IRC | 17:41 | |
*** rodrigods has joined #tripleo | 17:45 | |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: DNM - docker-puppet: don't force FACTER_hostname https://review.openstack.org/595274 | 17:45 |
*** rfolco has joined #tripleo | 17:50 | |
*** weshay has joined #tripleo | 17:51 | |
shardy | mwhahaha: Hi, do you happen to know if there are any other examples of standalone where you only have one nic? | 17:55 |
shardy | I'm debugging some issues where the only physical nic gets added to br-ctlplane but then the networking into and out of the (baremetal) box is broken from that point | 17:55 |
shardy | wanted to check if there are known issues before I dig into it | 17:56 |
mwhahaha | shardy: yes it's in the docs | 17:56 |
*** rfolco has quit IRC | 17:56 | |
mwhahaha | shardy: oh wait no i just have the one | 17:56 |
mwhahaha | shardy: you have to configure the single nic with the equivalent of what was on the box previously | 17:56 |
mwhahaha | shardy: so you have to add the default route into the standalone params you pass in and stuff | 17:57 |
shardy | mwhahaha: ack I'm working on a config based from the docs, must be getting something wrong | 17:57 |
shardy | I couldn't see where $IP was being set for the public interface, only NeutronPublicInterface | 17:57 |
mwhahaha | shardy: we basically move the network config from the existing nic, add a br-ctrplane on that nic and setup the ip/routes via that. | 17:58 |
weshay | shardy, I think the upstream standalone job uses br-ex | 17:58 |
mwhahaha | shardy: it's in tripleoclient, let me dig it up | 17:58 |
shardy | Ok so we take down e.g em1, add it to the bridge then bring the same ip back up | 17:58 |
mwhahaha | shardy: yea | 17:58 |
shardy | weshay: ack thanks, I suspect I'm using the wrong bridge, just about to reproduce | 17:59 |
*** mmedvede has joined #tripleo | 18:00 | |
mwhahaha | shardy: right so we create the NeutronPhysicalBridge on the NeutronPublicInterface | 18:01 |
weshay | man the upstream images are 8.3G | 18:01 |
mwhahaha | shardy: the deployed-server portmap we create is done here https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/tripleo_deploy.py#L390 | 18:02 |
mwhahaha | and of course our network config is https://github.com/openstack/tripleo-heat-templates/blob/master/net-config-standalone.j2.yaml#L84 | 18:03 |
mwhahaha | i think we basically are hardcoding br-ctlplane | 18:03 |
*** rfolco has joined #tripleo | 18:05 | |
shardy | mwhahaha: ah thanks for the links - I've rebuilt the box and trying again with the exact config from the docs | 18:07 |
mwhahaha | shardy: yea i think out of the box you have to use br-ctlplane, probably should document that better | 18:08 |
mwhahaha | you could create a new br to use instead of like em1 | 18:08 |
mwhahaha | as the NeutronPublicInterface | 18:08 |
mwhahaha | but the NeutronPublicBridge has to be br-ctlplane | 18:08 |
shardy | Ok thanks for the help all, seems better this time \o/ | 18:14 |
*** shardy is now known as shardy_afk | 18:15 | |
*** sri_ has quit IRC | 18:15 | |
*** ChanServ has quit IRC | 18:16 | |
*** khyr0n has quit IRC | 18:18 | |
*** stevebaker has joined #tripleo | 18:18 | |
*** trown|lunch is now known as trown | 18:21 | |
*** ChanServ has joined #tripleo | 18:22 | |
*** barjavel.freenode.net sets mode: +o ChanServ | 18:22 | |
*** dmellado has quit IRC | 18:22 | |
*** stevebaker has quit IRC | 18:23 | |
*** dxiri has quit IRC | 18:24 | |
*** bcafarel has joined #tripleo | 18:27 | |
*** tbonds has quit IRC | 18:30 | |
EmilienM | weshay: stable/rocky jobs look green so far | 18:31 |
EmilienM | see 594261 in zuul | 18:32 |
EmilienM | so far it's green except update/upgrades | 18:32 |
EmilienM | but we can figure these ones later | 18:32 |
EmilienM | so if that pass, the only blocker to release is to get a promotion | 18:32 |
EmilienM | (for that we need RDO cloud to be back I guess) | 18:32 |
weshay | sec | 18:33 |
*** electrichead has quit IRC | 18:34 | |
openstackgerrit | Rafael Folco proposed openstack-infra/tripleo-ci master: DNM: Test dryrun https://review.openstack.org/595308 | 18:40 |
*** assassin has quit IRC | 18:42 | |
*** dtantsur is now known as dtantsur|afk | 18:43 | |
*** moshele has quit IRC | 18:46 | |
*** tosky has quit IRC | 18:47 | |
*** mburned has joined #tripleo | 18:47 | |
weshay | EmilienM, /me looks | 18:49 |
weshay | EmilienM, ah.. this is looking good | 18:50 |
*** amoralej is now known as amoralej|off | 18:51 | |
*** redrobot has joined #tripleo | 18:52 | |
EmilienM | weshay: yes. | 18:54 |
EmilienM | now we need this @#! promotion | 18:54 |
*** jfrancoa has quit IRC | 19:08 | |
*** mjturek has quit IRC | 19:11 | |
weshay | ade_lee, I added an alert bug here | 19:34 |
weshay | but I don't see the bot running | 19:34 |
weshay | ade_lee, check out rh-openstack-dev and tripleo bugs w/ alert tags | 19:35 |
ade_lee | weshay, ack thanks | 19:35 |
*** assassin has joined #tripleo | 19:43 | |
openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: Move toci BASH variables to ansible https://review.openstack.org/589448 | 19:43 |
*** pcaruana has quit IRC | 19:47 | |
*** panda has joined #tripleo | 19:55 | |
openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: Remove tripleo.sh --bootstrap-subnodes from toci_gate_test.sh https://review.openstack.org/587012 | 19:56 |
*** gbarros has quit IRC | 20:01 | |
mwhahaha | weshay, EmilienM: i deployed an centos7 overcloud on f28 \o/ | 20:08 |
weshay | mwhahaha, nice | 20:08 |
weshay | mwhahaha, so we just have to wait on a few of these patches to land | 20:09 |
mwhahaha | yea | 20:09 |
mwhahaha | most are approved | 20:09 |
mwhahaha | so soonish | 20:09 |
weshay | k. cool.. I'm trying locally w/ the upstream fedora28 image.. fk. uuug 8.2gb | 20:09 |
mwhahaha | the cloud image? | 20:10 |
weshay | ya | 20:11 |
weshay | https://nb01.openstack.org/images/ | 20:11 |
weshay | 8.6 | 20:11 |
mwhahaha | weshay: wow they are all really big, wonder why | 20:13 |
*** Damjanek has joined #tripleo | 20:14 | |
*** dprince has quit IRC | 20:16 | |
mwhahaha | so i guess the next step would be to try and build a fedora overcloud image | 20:20 |
*** morazi has quit IRC | 20:21 | |
weshay | mwhahaha, ya.. so let me see how far I can get w/ this image.. the stock image will NOT work | 20:21 |
mwhahaha | weshay: what are you using that image for? the overcloud? | 20:21 |
weshay | and the partitions are a little different and some other small changes between centos and fed | 20:21 |
mwhahaha | ah | 20:22 |
weshay | mwhahaha, atm.. I'm using it for the undercloud | 20:22 |
mwhahaha | the standard fedora28 cloud image worked fine for the undercloud | 20:22 |
mwhahaha | my tests were with my python3 packages tho | 20:22 |
mwhahaha | which have yet to land in rdo | 20:22 |
weshay | mwhahaha, ya.. I have to customize and expand the partitions etc | 20:23 |
weshay | in a way that is slightly different than centos | 20:24 |
weshay | so I'm just checking to see if the upstream image is any different | 20:24 |
mwhahaha | weshay: that's weird i didn't run into that | 20:26 |
mwhahaha | cloud-init issues maybe? | 20:26 |
mwhahaha | weshay: oh wait you're using libvirt right? does cloud-init run? | 20:26 |
*** tonyb has joined #tripleo | 20:26 | |
weshay | no sir | 20:26 |
*** itlinux has joined #tripleo | 20:27 | |
mwhahaha | no to which part | 20:28 |
mwhahaha | you running against ovb then? | 20:28 |
weshay | mwhahaha, actually I take that back.. but that is not what is used for libvirt | 20:28 |
weshay | mwhahaha, you were running on the cloud though | 20:28 |
itlinux | hello guys, I found this command in the docs.openstack.org openstack overcloud update --nodes controller-0 will this command work on a bm in pike? no containers used. Talking to some of the great guys in this channel told me that they did not test it against not container pike. Thanks | 20:29 |
mwhahaha | weshay: right so you'd have to inject fake cloud-init user config | 20:29 |
weshay | ya | 20:29 |
mwhahaha | weshay: so if you use libvirt you can just tell cloud-init to expand the disk | 20:29 |
weshay | mwhahaha, also trying to rip out a lot of this stuff | 20:29 |
weshay | so I'm not very interested in changing much of it | 20:29 |
weshay | mwhahaha, ya | 20:29 |
mwhahaha | well i'd assume that's the same problem with using the centos cloud images on libvirt? :D | 20:30 |
* mwhahaha shrugs | 20:30 | |
mwhahaha | it's a minor image tweak in comparison to the rest of our hacks | 20:30 |
weshay | mwhahaha, we do. .but virt-customize is used | 20:30 |
weshay | I need to tear most of this out.. | 20:30 |
mwhahaha | yea | 20:30 |
weshay | stand up a couple nodes.. run toci | 20:30 |
weshay | that's all | 20:30 |
mwhahaha | wouldn't be a problem if our provisioning system wasn't so tightly coupled with the rest | 20:31 |
* mwhahaha hides | 20:31 | |
weshay | run infra playbooks, run tripleo playbooks | 20:31 |
*** stevebaker has joined #tripleo | 20:31 | |
weshay | mwhahaha, hey.. at least we have one | 20:31 |
*** khyr0n has joined #tripleo | 20:31 | |
weshay | mwhahaha, if upstream nodepool had an answer to libvirt multinode.. I'd take it | 20:31 |
weshay | so :P | 20:32 |
mwhahaha | weshay: https://i.imgur.com/t96UURT.jpg | 20:32 |
*** ericyoung has joined #tripleo | 20:33 | |
weshay | heh.. true | 20:33 |
*** gbarros has joined #tripleo | 20:33 | |
mwhahaha | stevebaker: btw i figured out my problem from yesterday. old containers that don't properly support push_destination: true | 20:33 |
mwhahaha | stevebaker: so we just have to wait on a promotion | 20:33 |
mwhahaha | speaking of promotions, weshay what are we blocked on for master promotions (other than rdo cloud being dead) | 20:33 |
* weshay shows | 20:34 | |
weshay | sec | 20:34 |
openstackgerrit | Rafael Folco proposed openstack-infra/tripleo-ci master: DNM: [PoC] Enable featureset override https://review.openstack.org/594511 | 20:34 |
*** tbonds has joined #tripleo | 20:34 | |
weshay | mwhahaha, 593383 | 20:34 |
mwhahaha | k | 20:35 |
weshay | the upstream image appeears to be working | 20:35 |
mwhahaha | weshay: looks like the ntp getting added to the fedora stable repo soonish (patch is merging) | 20:45 |
mwhahaha | https://review.rdoproject.org/r/15823 | 20:45 |
*** raildo has quit IRC | 20:45 | |
openstackgerrit | Aakarsh proposed openstack/tripleo-quickstart master: DNM: Testing browbeat's sysmetric collection https://review.openstack.org/595351 | 20:45 |
*** jcoufal has quit IRC | 20:49 | |
openstackgerrit | Merged openstack/tripleo-common master: nova_metadata healthcheck script https://review.openstack.org/594320 | 20:51 |
openstackgerrit | Merged openstack/tripleo-ui master: Imported Translations from Zanata https://review.openstack.org/594049 | 20:51 |
openstackgerrit | Alan Bishop proposed openstack/tripleo-common stable/queens: Add missing service for DockerCinderVolumeImage https://review.openstack.org/595357 | 20:53 |
*** trown is now known as trown|outtypewww | 20:54 | |
*** florianf has quit IRC | 20:57 | |
*** jpena is now known as jpena|off | 20:57 | |
*** jtomasek has quit IRC | 20:57 | |
*** toure is now known as toure|gone | 20:59 | |
*** abishop has quit IRC | 21:04 | |
*** morazi has joined #tripleo | 21:08 | |
*** ooolpbot has joined #tripleo | 21:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 21:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1788426 | 21:10 |
*** ooolpbot has quit IRC | 21:10 | |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 21:10 |
openstack | Launchpad bug 1788426 in tripleo "Tracking bug: RDO Cloud is down" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 21:10 |
*** gouthamr has joined #tripleo | 21:11 | |
openstackgerrit | Merged openstack/python-tripleoclient master: Allow custom UC/standalone passwords https://review.openstack.org/587431 | 21:12 |
openstackgerrit | Merged openstack/python-tripleoclient master: Use binary io for object saving https://review.openstack.org/594563 | 21:12 |
openstackgerrit | Merged openstack/tripleo-common master: Open tarball in binary mode https://review.openstack.org/594560 | 21:15 |
*** itlinux has quit IRC | 21:15 | |
*** holser_ has joined #tripleo | 21:18 | |
*** dprince has joined #tripleo | 21:21 | |
*** ubijtsa has joined #tripleo | 21:29 | |
*** sri_ has joined #tripleo | 21:30 | |
*** assassin has quit IRC | 21:31 | |
*** boazel has quit IRC | 21:33 | |
*** dansmith is now known as htimsnad | 21:34 | |
openstackgerrit | Merged openstack-infra/tripleo-ci master: Fix HASH for periodic jobs(use tripleo-ci-testing) https://review.openstack.org/593383 | 21:43 |
openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Updated skip list for Tempest https://review.openstack.org/591982 | 21:43 |
*** agopi has quit IRC | 21:52 | |
EmilienM | bandini: FWIW, https://review.openstack.org/#/c/595274/ seems to pass | 21:56 |
mwhahaha | a likely story | 22:01 |
* mwhahaha tries a deploy of a fedora28 overcloud image | 22:05 | |
* mwhahaha thinks this will not go well | 22:05 | |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt https://review.openstack.org/591540 | 22:08 |
weshay | mwhahaha, waste of time | 22:08 |
weshay | mwhahaha, did you build the images? | 22:08 |
mwhahaha | SHHHHHH DON'T RUIN MY FUN | 22:08 |
mwhahaha | weshay: yes | 22:08 |
mwhahaha | only had to extract the kernel by hand and do some hacking of tripleo-common | 22:09 |
*** ooolpbot has joined #tripleo | 22:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 22:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1788426 | 22:10 |
*** ooolpbot has quit IRC | 22:10 | |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 22:10 |
openstack | Launchpad bug 1788426 in tripleo "Tracking bug: RDO Cloud is down" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 22:10 |
EmilienM | fun times | 22:10 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt https://review.openstack.org/591540 | 22:11 |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: WIP - Podman support for docker-puppet https://review.openstack.org/588655 | 22:12 |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: DNM - test podman https://review.openstack.org/590587 | 22:12 |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Avoid SELinux issues with the "cp -a" command https://review.openstack.org/595003 | 22:13 |
EmilienM | ideally we would wait to get a rocky promotion before we release rc1 | 22:14 |
EmilienM | but what if it doesn't happen next week? | 22:14 |
EmilienM | I think it's tolerable to release rc1 in our current state | 22:14 |
EmilienM | unless we have a critical bug that really prevents the deployment of Rocky version | 22:15 |
mwhahaha | let's see what happens this week | 22:16 |
mwhahaha | btw my fedora image ironic'd | 22:16 |
mwhahaha | so the provisioning worked | 22:16 |
* mwhahaha is shocked | 22:16 | |
* mwhahaha waits for ansible to fail horribly | 22:17 | |
EmilienM | you're so positive | 22:17 |
mwhahaha | i think if this works i'll just take tomorrow off and buy a lottery ticket | 22:17 |
*** tzumainn has quit IRC | 22:19 | |
mwhahaha | {"msg": "No package openstack-heat-agents available.", "failed": true ... | 22:20 |
mwhahaha | well that's solvable | 22:20 |
mwhahaha | ok so my image build was just bad, but good to know it provisions fine and ansible ran | 22:21 |
EmilienM | :-O | 22:21 |
EmilienM | test with podman if you need failures | 22:21 |
mwhahaha | NO | 22:21 |
mwhahaha | YOU KEEP YOUR DIRTY PODMAN TO YOURSELF | 22:22 |
EmilienM | ourselves* | 22:22 |
mwhahaha | naw uh | 22:22 |
mwhahaha | it's all yours | 22:22 |
mwhahaha | oh fiddle sticks my image build failed because i'm using a file:/// repo so disk image builder doesn't like that | 22:23 |
*** holser_ has quit IRC | 22:26 | |
*** rcernin has joined #tripleo | 22:34 | |
openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: WIP,DNM fedora28 standalone https://review.openstack.org/595374 | 22:34 |
*** holser_ has joined #tripleo | 22:34 | |
weshay | mwhahaha, wouldn't we be better off.. just focusing on the standalone? | 22:35 |
mwhahaha | weshay: in the short term yea, but at some point we're going to need a python3 overcloud node | 22:35 |
mwhahaha | and if it works... | 22:35 |
weshay | mwhahaha, will we NEEED it | 22:36 |
weshay | and for how long | 22:36 |
mwhahaha | right now i'm doing a basic pass to see where our gaps are | 22:36 |
weshay | anyway.. ya it's good idea | 22:36 |
mwhahaha | given what i've found i'm not that concerned on the standalone | 22:36 |
weshay | mwhahaha, ideally I'd like to get a fed-28 standalone running | 22:36 |
mwhahaha | since i managed to deploy an overcloud succesfully | 22:36 |
weshay | ok | 22:37 |
*** sri_ has quit IRC | 22:37 | |
mwhahaha | standalone is basically an undercloud which we got working but that's not where most of our hidden python bits are | 22:37 |
weshay | k | 22:37 |
mwhahaha | i'm more concerned about our mistral workflow bits | 22:38 |
mwhahaha | and hidden tripleo-common stuff | 22:38 |
*** sri_ has joined #tripleo | 22:38 | |
mwhahaha | weshay: do we have stable/queens scenario timeouts tracked somewhere? | 22:40 |
mwhahaha | weshay: https://review.openstack.org/#/c/595014/ | 22:40 |
mwhahaha | or is that related to the one alert | 22:40 |
weshay | sorry.. your message timed out, please call back later | 22:40 |
mwhahaha | weshay: https://www.youtube.com/watch?v=dIiZ3vvZ78s | 22:41 |
* weshay looks for the bug | 22:41 | |
weshay | kr | 22:41 |
weshay | fkr | 22:41 |
weshay | even | 22:41 |
weshay | mwhahaha, ah good.. | 22:42 |
weshay | ur patch is on queens | 22:42 |
weshay | soooooooooooooooooooooo | 22:42 |
weshay | mwhahaha, https://review.openstack.org/#/c/594933/ | 22:42 |
weshay | mwhahaha, so I don't think we should use wild cards in the include_pkgs | 22:43 |
weshay | because as soon as you stand up a new repo.. boom | 22:43 |
weshay | it's in c | 22:43 |
weshay | ci | 22:43 |
weshay | so your yum update patch is not in mix yet | 22:43 |
mwhahaha | weshay: hmmm | 22:44 |
mwhahaha | but we shouldn't have that many packages | 22:44 |
weshay | so you are still hitting https://bugs.launchpad.net/tripleo/+bug/1786764 | 22:44 |
openstack | Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Fix released] - Assigned to Sorin Sbarnea (ssbarnea) | 22:44 |
mwhahaha | weshay: why did this suddenly become a problem | 22:45 |
weshay | mwhahaha, meaning.. why did containers all of a sudden become a problem? | 22:46 |
mwhahaha | yea | 22:46 |
weshay | the update issue has been there | 22:46 |
weshay | mwhahaha, don't make me paste that irc blob from infra about dns servers and such | 22:47 |
mwhahaha | right so i'll be coming back to my own little world | 22:47 |
mwhahaha | when you solve it, let me know | 22:47 |
* mwhahaha runs | 22:47 | |
weshay | :) | 22:47 |
*** ChanServ has quit IRC | 22:49 | |
mwhahaha | turns out i just built a blank fedora cloud image | 23:01 |
mwhahaha | so i guess it would have worked if i enabled package installs | 23:01 |
* mwhahaha goes and builds an overcloud-full | 23:01 | |
*** ChanServ has joined #tripleo | 23:03 | |
*** barjavel.freenode.net sets mode: +o ChanServ | 23:03 | |
*** dprince has quit IRC | 23:09 | |
*** holser_ has quit IRC | 23:10 | |
*** ooolpbot has joined #tripleo | 23:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1786764 | 23:10 |
openstack | Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea) | 23:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1787910 | 23:10 |
*** ooolpbot has quit IRC | 23:10 | |
openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 23:10 |
*** slagle has joined #tripleo | 23:17 | |
*** rpioso is now known as rpioso|afk | 23:18 | |
*** sri_ has quit IRC | 23:19 | |
*** sri_ has joined #tripleo | 23:20 | |
*** rlandy has quit IRC | 23:39 | |
*** gbarros has quit IRC | 23:46 | |
*** gouthamr has quit IRC | 23:48 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!