Friday, 2018-11-09

« Thursday, 2018-11-08 Index Saturday, 2018-11-10 »
*** aedc has quit IRC00:00
*** vinaykns has quit IRC00:02
*** bfournie has quit IRC00:06
*** slaweq has joined #tripleo00:11
*** saneax has quit IRC00:15
*** slaweq has quit IRC00:16
*** rlandy is now known as rlandy|bbl00:18
*** pcaruana has quit IRC00:25
*** phuongnh has joined #tripleo00:25
*** slaweq has joined #tripleo01:11
*** slaweq has quit IRC01:16
openstackgerritSteve Baker proposed openstack/tripleo-common master: WIP implement python based uploader  https://review.openstack.org/61601801:33
openstackgerritSteve Baker proposed openstack/tripleo-common master: WIP make python uploader the default  https://review.openstack.org/61601901:33
*** dhill_ has quit IRC01:48
*** tzumainn has quit IRC01:50
*** iurygregory has quit IRC02:09
*** hamzy has joined #tripleo02:15
*** lblanchard has quit IRC02:19
*** jrist has quit IRC02:30
*** jrist has joined #tripleo02:43
*** mschuppert has quit IRC02:46
*** mrsoul has quit IRC02:46
*** bshephar has joined #tripleo02:59
openstackgerrit98k proposed openstack/tripleo-validations master: Add python 3.6 unit test job  https://review.openstack.org/61674002:59
*** slaweq has joined #tripleo03:11
*** slaweq has quit IRC03:15
openstackgerrit98k proposed openstack/tripleo-ui master: Update min tox version to 2.0  https://review.openstack.org/61675203:23
openstackgerritMerged openstack/tripleo-heat-templates master: Check if source directory does exist before linking  https://review.openstack.org/61391203:31
*** radeks__ has joined #tripleo03:46
*** rlandy|bbl is now known as rlandy04:07
openstackgerritSteve Baker proposed openstack/tripleo-heat-templates master: Crane: Add deployment/crane-puppet.yaml  https://review.openstack.org/60950804:20
openstackgerrit98k proposed openstack/tripleo-heat-templates master: Add python 3.6 unit test job  https://review.openstack.org/61676704:30
openstackgerritSteve Baker proposed openstack/tripleo-common master: WIP implement python based uploader  https://review.openstack.org/61601804:31
openstackgerritSteve Baker proposed openstack/tripleo-common master: WIP make python uploader the default  https://review.openstack.org/61601904:31
openstackgerrit98k proposed openstack/tripleo-heat-templates master: Update min tox version to 2.0  https://review.openstack.org/61676804:32
*** janki has joined #tripleo04:36
*** ykarel has joined #tripleo04:56
*** openstackstatus has quit IRC04:59
*** slaweq has joined #tripleo05:11
*** slaweq has quit IRC05:16
*** saneax has joined #tripleo05:23
*** openstack has joined #tripleo07:08
*** ChanServ sets mode: +o openstack07:08
*** etingof has joined #tripleo07:08
*** mschuppert has joined #tripleo07:10
*** pcaruana has joined #tripleo07:21
*** skramaja has joined #tripleo07:34
*** jtomasek has joined #tripleo07:35
*** brault has joined #tripleo07:39
*** slaweq has joined #tripleo07:45
*** aedc has joined #tripleo07:48
openstackgerritChristophe Fontaine proposed openstack/tripleo-heat-templates master: Update of vpp-agent  https://review.openstack.org/61684207:50
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60429808:00
*** iurygregory has joined #tripleo08:00
*** akrivoka has joined #tripleo08:04
*** cylopez has joined #tripleo08:07
*** shardy has joined #tripleo08:07
dciabrinchem, morning o/ looks like we're good with https://review.openstack.org/#/c/616249/2 ?08:15
openstackgerritCédric Jeanneret proposed openstack/tripleo-specs master: New Spec for Stein: in-flight validations.  https://review.openstack.org/60200708:16
*** paramite has joined #tripleo08:16
*** florianf|afk is now known as florianf08:20
*** cylopez has left #tripleo08:21
Tenguflorianf: hey :). You're here just in time - care to check that thing? https://review.openstack.org/#/c/602007/  I have some new ideas, allowing to get a bigger view/thing.08:27
ksamborHey all, quick question. Where in tripleo code we take care of installing additional packages during split-stack deployment?08:28
*** phuongnh has quit IRC08:33
*** phuongnh has joined #tripleo08:34
openstackgerritQuique Llorente proposed openstack/python-tripleoclient master: WIP: Use corrent ansible-playbook cmd for py3  https://review.openstack.org/61657908:34
florianfTengu: Oh, sure!08:44
openstackgerritAna Krivokapic proposed openstack/tripleo-validations master: Fix disk space validation  https://review.openstack.org/61595908:44
*** ykarel is now known as ykarel|lunch08:49
therveksambor: OS::TripleO::DeployedServer::Bootstrap08:52
openstackgerritKamil Sambor proposed openstack/paunch master: Revert "Change naming of key arguments in runner"  https://review.openstack.org/61685508:54
openstackgerritAna Krivokapic proposed openstack/tripleo-common master: Fix execution of custom vs default validations  https://review.openstack.org/59958108:57
Tenguakrivoka: need any help on the disk space validation? as I'm the one that changed it to something more complicated than its first iteration... :)08:58
therveksambor: Also I don't think https://review.openstack.org/616855 is required because it was already changed previously, but I thought I mentioned it...08:58
ksamborou, kk08:58
*** jpich has joined #tripleo08:58
akrivokaTengu: if you could review my patch, that would be great :)08:59
akrivokaTengu: https://review.openstack.org/#/c/615959/08:59
Tenguakrivoka: on my way then :)08:59
Tenguakrivoka: hmm. wouldn't "mtab" be a nice source for the disk mounts?08:59
ksamborthanks therve for point this out09:00
akrivokaTengu: don't have it on my undercloud09:01
akrivokaTengu: I went with what comes with the default install09:01
Tenguakrivoka: it's the state file for the mounts in fact. but it has too much useless things in fact.09:01
therveksambor: No problem09:02
*** skramaja_ has joined #tripleo09:03
*** skramaja has quit IRC09:03
*** sanjayu_ has joined #tripleo09:04
Tenguakrivoka: hmm. "item.item.min_size"? does it really work? I'd think there would be a reference to "volumes" var in there - but I'm not 100% sure with ansible loops.09:04
akrivokaTengu: I tested it and it worked... let me know if you find otherwise09:05
Tenguakrivoka: ok. there are some black magic in ansible anyway :/09:05
*** saneax has quit IRC09:06
*** sanjayu__ has joined #tripleo09:08
*** sanjayu_ has quit IRC09:10
*** iurygregory has quit IRC09:12
*** rfolco|off is now known as rfolco09:14
*** iurygregory has joined #tripleo09:14
*** bogdando has joined #tripleo09:14
openstackgerritMarios Andreou proposed openstack/tripleo-quickstart-extras master: Add custom environment files for standalone  https://review.openstack.org/61482009:16
openstackgerritMarios Andreou proposed openstack/tripleo-quickstart-extras master: Add custom environment files for standalone  https://review.openstack.org/61482009:16
openstackgerritMarios Andreou proposed openstack/tripleo-quickstart-extras master: Add custom environment files for standalone  https://review.openstack.org/61482009:20
*** panda|rover|off is now known as panda|rover09:24
openstackgerritMarios Andreou proposed openstack/tripleo-quickstart master: Enable low memory for standalone  https://review.openstack.org/61482409:24
*** derekh has joined #tripleo09:42
openstackgerritMerged openstack/tripleo-heat-templates master: Mount /usr/share/openstack-octavia-amphora-images into mistral-executor  https://review.openstack.org/61460909:46
*** ykarel|lunch is now known as ykarel09:47
*** ykarel has quit IRC09:49
openstackgerritCarlos Goncalves proposed openstack/tripleo-heat-templates stable/rocky: Mount /usr/share/openstack-octavia-amphora-images into mistral-executor  https://review.openstack.org/61686209:49
openstackgerritMartin Mágr proposed openstack/tripleo-common stable/rocky: Add health check script for Sahara API  https://review.openstack.org/61230309:50
openstackgerritMartin Mágr proposed openstack/tripleo-common stable/rocky: Add health checks for Swift services  https://review.openstack.org/61389309:50
*** ykarel has joined #tripleo09:50
openstackgerritThomas Herve proposed openstack/paunch master: Remove log message when container doesn't exist  https://review.openstack.org/61621609:52
*** salmankhan has joined #tripleo09:54
openstackgerritMartin André proposed openstack/tripleo-quickstart-extras master: Remove unused ansible.cfg  https://review.openstack.org/60717709:55
*** ykarel_ has joined #tripleo09:56
*** ykarel has quit IRC09:58
*** iurygregory has quit IRC09:58
*** ykarel__ has joined #tripleo10:00
*** ykarel__ is now known as ykarel10:01
*** ykarel_ has quit IRC10:02
openstackgerritMerged openstack/tripleo-common master: Stop installing octavia-amphora-image package  https://review.openstack.org/61463110:06
*** phuongnh has quit IRC10:07
openstackgerritCarlos Goncalves proposed openstack/tripleo-common stable/rocky: Stop installing octavia-amphora-image package  https://review.openstack.org/61686710:07
*** iurygregory has joined #tripleo10:08
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180240910:10
*** ooolpbot has quit IRC10:10
openstackLaunchpad bug 1802409 in tripleo "cannot create directory "/var/lib/tempestdata" Permission denied" [Critical,In progress] - Assigned to Derek Higgins (derekh)10:10
*** yprokule has joined #tripleo10:13
openstackgerritMartin Mágr proposed openstack/tripleo-common stable/rocky: Add health check script for Sahara API  https://review.openstack.org/61230310:14
openstackgerritMartin Mágr proposed openstack/tripleo-common stable/rocky: Add health checks for Swift services  https://review.openstack.org/61389310:14
*** yprokule has quit IRC10:15
openstackgerritMerged openstack/tripleo-heat-templates master: Do not purge the Ironic Inspector dhcp-hostsdir  https://review.openstack.org/61336110:15
openstackgerritMerged openstack/tripleo-heat-templates master: Move UndercloudPostDeployment to python  https://review.openstack.org/61454010:15
openstackgerritMerged openstack/tripleo-quickstart master: Switch upgrade job to master to master  https://review.openstack.org/60694810:15
openstackgerritMerged openstack/tripleo-common stable/queens: Revert "Fix nova-placement healthcheck"  https://review.openstack.org/61654810:15
*** skramaja_ is now known as skramaja10:20
*** yprokule has joined #tripleo10:27
openstackgerritMarios Andreou proposed openstack/tripleo-quickstart master: Add new featureset for standalone-scenario001 job  https://review.openstack.org/61687110:32
openstackgerritMarios Andreou proposed openstack-infra/tripleo-ci master: Add new tripleo-ci-centos-7-standalone-scenario001 job  https://review.openstack.org/61687210:33
mariosmwhahaha: ^10:33
*** sambetts_ is now known as sambetts|afk10:33
openstackgerritMerged openstack/os-net-config master: Apply IP/netmask/route/MTU changes without bouncing interfaces.  https://review.openstack.org/60639610:33
mariosmwhahaha: it worked :) http://logs.openstack.org/24/614824/9/check/tripleo-ci-fedora-28-standalone/03c9b71/logs/undercloud/home/zuul/standalone.sh.txt.gz10:33
marios  -e /usr/share/openstack-tripleo-heat-templates/environments/low-memory-usage.yaml \10:33
*** suuuper has joined #tripleo10:37
*** iurygregory is now known as iury|lunch10:56
openstackgerritAna Krivokapic proposed openstack/tripleo-validations master: Fix disk space validation  https://review.openstack.org/61595910:59
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180240911:10
openstackLaunchpad bug 1802409 in tripleo "cannot create directory "/var/lib/tempestdata" Permission denied" [Critical,In progress] - Assigned to Derek Higgins (derekh)11:10
*** ooolpbot has quit IRC11:10
*** aedc has quit IRC11:11
*** suuuper has quit IRC11:12
*** rodolof has joined #tripleo11:12
*** gfidente has joined #tripleo11:16
*** radeks__ has quit IRC11:20
*** raildo has joined #tripleo11:22
*** suuuper has joined #tripleo11:25
openstackgerritSorin Sbarnea proposed openstack-infra/tripleo-ci master: Collect dnf logs and config  https://review.openstack.org/61580011:30
openstackgerritSorin Sbarnea proposed openstack-infra/tripleo-ci master: Collect dnf logs and config  https://review.openstack.org/61580011:30
*** rfolco is now known as rfolco|doctor11:32
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates stable/rocky: Do not purge the Ironic Inspector dhcp-hostsdir  https://review.openstack.org/61688211:34
*** rook has joined #tripleo11:35
rookbogdando RE:IPC between containers -- how can i easily chnage the containers to have --ipc=host?11:36
*** iury|lunch is now known as iurygregory11:36
bogdandorook: via paunch composer and tht11:36
bogdandorook: http://git.openstack.org/cgit/openstack/paunch/tree/paunch/builder/compose1.py#n7511:37
*** thrash|g0ne is now known as thrash11:37
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Fix the undercloud-process-count validation  https://review.openstack.org/61688611:37
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Create clouds.yaml for the undercloud  https://review.openstack.org/61585311:37
bogdandosomething very similar to 'net: host', to support 'ipc: host'11:37
bogdandorook: oh, it already has it http://git.openstack.org/cgit/openstack/paunch/tree/paunch/builder/compose1.py#n3611:38
bogdandoso just modify some t-h-t docker templates to use ipc: host alongside net: host11:38
rookbogdando: i see - I wonder if this is worth the effort? I have identified multiple areas that were using more memory, and each group has been taken care of it.11:38
bogdandorook: as I replied to the subject mail thread, I think we can only try it and compare11:38
rookbogdando: this problem would have been in 13 too...11:39
rookthat is my only point.11:39
bogdandoI don't know how to check otherwise, if shared /dev/shm can help python things in containers11:39
rooki see11:39
bogdandoI think it won't unless python code leverages /dev/shm11:39
bogdandocan't say for python/cpython internals11:39
hjensasAnyone got time to review https://review.openstack.org/613683? bug: #180191611:47
openstackbug 1801916 in tripleo " multiple-nic sample template only add routes to External and Tenant interface" [High,In progress] https://launchpad.net/bugs/1801916 - Assigned to Harald Jensås (harald-jensas)11:47
*** dprince has joined #tripleo11:49
*** aedc has joined #tripleo11:56
*** dtantsur is now known as dtantsur|brb11:59
*** vpickard_ is now known as vpickard12:00
*** ykarel_ has joined #tripleo12:09
rookbogdando if you want to work with me to build out a POC let me know.12:10
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180240912:10
*** ooolpbot has quit IRC12:10
openstackLaunchpad bug 1802409 in tripleo "cannot create directory "/var/lib/tempestdata" Permission denied" [Critical,In progress] - Assigned to Derek Higgins (derekh)12:10
*** ykarel__ has joined #tripleo12:10
*** ykarel has quit IRC12:11
*** janki has quit IRC12:11
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Fix the undercloud-process-count validation  https://review.openstack.org/61688612:14
*** ykarel_ has quit IRC12:14
*** ykarel__ is now known as ykarel12:18
ykarelhjensas, hi12:18
hjensasykarel: hello12:18
dciabrinbogdando, ping12:19
ykarelhjensas, are u aware of issue in which jobs timing out at https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build/52f38a1/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2018-11-08_15_37_5312:19
ykarelthis is seen in promotion job12:19
ykarelstuck around, Run deployment UndercloudCtlplaneNetworkDeployment12:19
hjensasykarel: I have not seen that before.12:26
*** rh-jelabarre has joined #tripleo12:29
hjensasykarel: there are no changes to that Deployment since quite some time.12:30
*** panda|rover is now known as panda|rover|lch12:31
hjensasykarel: we have one change fixing an issue in the gate: https://review.openstack.org/614739 - But it seems strange that the install is stuck there.12:31
ykarelhjensas, seeing the review ^^ i asked u to check if u know about the jobs getting stuck12:32
openstackgerritCédric Jeanneret proposed openstack/tripleo-specs master: New Spec for Stein: in-flight validations.  https://review.openstack.org/60200712:33
hjensasykarel: That deployment runs a python script, we can see it accessing neutron here:12:36
hjensashttps://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build/52f38a1/logs/undercloud/var/log/containers/neutron/server.log.txt.gz#_2018-11-08_15_37_56_46112:36
ykarelso may be some signal issue12:37
hjensasykarel: afaict it does the get request, checking if the ctlplane network already exist.12:38
hjensasykarel: Then I see req-a5cee308-0847-41b4-a972-fdd429e71416 - which Request body: {u'network': {u'provider:network_type': u'flat', u'name': u'ctlplane', u'provider:physical_network': u'ctlplane', u'mtu': 1350}}  <- Creating the network.12:39
hjensasykarel: after that it should create segments and subnets, but I don't see those requests coming in.12:39
*** EmilienM is now known as EvilienM12:40
ykarelhjensas, okk checking a success job12:41
ykarelhjensas, in SUCCESS job i can see referece to ctlplane-subnet: https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build/a5e3452/logs/undercloud/var/log/containers/neutron/server.log.txt.gz12:43
openstackgerritYurii Prokulevych proposed openstack/tripleo-upgrade master: Cleanup irrelevant scripts from previous releases.  https://review.openstack.org/61537012:44
therveCan someone review https://review.openstack.org/#/c/616708/ ?12:45
EvilienMjaosorior: ^12:47
jaosoriorthere are a lot of scenario failures there12:47
*** radeks__ has joined #tripleo12:47
jaosoriorlets see if it's related12:47
jaosoriorEvilienM: those failures seem to be related with the container tagging role12:48
EvilienMjaosorior: oops12:49
jaosoriorEvilienM: have you checked out what's the issue there?12:49
EvilienMjaosorior: I'm looking now12:51
jaosoriorthanks12:51
hjensasykarel: are these normal? https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build/52f38a1/logs/undercloud/var/log/containers/mistral/mistral-db-manage.log.txt.gz#_2018-11-08_15_27_40_37412:52
jaosoriorhjensas: that sure doesn't look normal12:52
jaosoriord0ugal: ^^12:52
jaosoriorhjensas: or... maybe it is12:53
ykarelhjensas, yes i have seen these before so seems unrelated12:53
jaosoriorhjensas: I think that's from mistral's populate command12:53
jaosoriorthe populate command grabs all the object's that mistral supports and tries to fill in the relevant actions. If it fails, it just skips them.12:53
*** ansmith has joined #tripleo12:53
jaosoriorso if that's the case, it's normal and it's not an issue12:53
hjensasykarel: jaosorior: ok, thanks. I will keep digging.12:54
jaosoriorhjensas: what are you debugging?12:54
hjensasjaosorior: promotion jobs are timing out.12:55
jaosoriorhjensas: got logs?12:55
hjensasjaosorior: It seems its doing the get, then the network create here: https://github.com/openstack/tripleo-heat-templates/blob/master/extraconfig/post_deploy/undercloud_ctlplane_network.py#L47-L7112:55
*** rodolof has quit IRC12:55
hjensasjaosorior: then nothing more in neutron-server.log related to the creation of subnets etc.12:56
jaosoriorfunky12:56
hjensasjaosorior: very. :)12:56
hjensasjaosorior: logs https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build/52f38a1/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2018-11-08_15_37_5312:57
EvilienMjaosorior: it's weird12:58
EvilienM"Error response from daemon: no such id: 192.168.24.1:8787/tripleomaster/centos-binary-cinder-backup:current-tripleo-updated-20181109093052"12:59
jaosoriorEvilienM: life is weird12:59
EvilienMbut the container is there: http://logs.openstack.org/20/614820/13/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/edfb2a7/logs/undercloud/var/log/extra/docker/docker_allinfo.log.txt.gz12:59
EvilienMhowever it wasn't pull on overcloud http://logs.openstack.org/20/614820/13/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/edfb2a7/logs/subnode-2/var/log/extra/docker/docker_allinfo.log.txt.gz12:59
EvilienMso the issue is before the tagging12:59
EvilienMlet me see history if we changed something on that13:00
ykarelhjensas, jaosorior looks like i get to something13:00
jaosoriorykarel: ?13:00
ykarelhjensas, jaosorior hieradata-overrides-classic-undercloud.yaml vs hieradata-overrides-t-h-t-undercloud.yaml in containerized_undercloud vs not containerized_undercloud13:01
ykareland this should be happeing after containerized_undercloud smart default patch13:01
EvilienM2018-11-09 11:16:15 |         "2018-11-09 11:16:07,083 ERROR: 21240 -- ERROR configuring designate",13:01
EvilienMscenario003 is broken because of designate13:01
EvilienMhttp://logs.openstack.org/20/614820/13/check/tripleo-ci-centos-7-scenario003-multinode-oooq-container/2dfb121/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-11-09_11_16_1513:01
jaosoriordammit designate!13:01
EvilienMbnemec: can you please file a bug and take a look? ^13:02
EvilienMscenario001 and 002 are broken because cinder containers aren't all pulled on the overcloud, I'm looking why.13:02
*** rlandy has joined #tripleo13:04
hjensasykarel: I'm not following with regards to the hiera overrides?13:05
ykarelhjensas, am still digging but that change(smart undercloud defaults) looks suspicious, in failing job i see following two changes to undercloud.conf13:08
ykarelcontainer_images_file = /home/zuul/containers-prepare-parameter.yaml13:08
ykarelhieradata_override = ./hieradata-overrides-t-h-t-undercloud.yaml13:08
*** dtantsur|brb is now known as dtantsur13:09
*** abishop has joined #tripleo13:10
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180240913:10
*** ooolpbot has quit IRC13:10
openstackLaunchpad bug 1802409 in tripleo "cannot create directory "/var/lib/tempestdata" Permission denied" [Critical,In progress] - Assigned to Derek Higgins (derekh)13:10
ykarelhjensas, jaosorior can missing "ironic::drivers::deploy::http_port": "3816" cause such issue?13:11
mandreEvilienM, jaosorior, chkumar|pto: can you confirm we need https://review.openstack.org/#/c/605980/ in stable/rocky?13:12
mandrejobs are failing with "mkdir: cannot create directory ‘/var/lib/tempestdata’: Permission denied"13:12
mandrehttp://logs.openstack.org/75/615775/1/check/tripleo-ci-centos-7-undercloud-oooq/5505a48/logs/undercloud/home/zuul/tempest.log.txt.gz13:12
jaosoriormandre: I think this addresses that https://review.openstack.org/#/c/616708/113:13
ykareli see, The default httpd ipxe port is 8080, but this conflicts with the telnet13:13
ykarel# server which streams the console log in CI. So we allow override here.13:13
mandrejaosorior: oh cool, thanks13:14
therveEvilienM: What's the designate issue?13:14
EvilienMjaosorior: mhh something weirds happen13:14
EvilienMtherve: I didn't investigate but I linked the logs13:14
EvilienMjaosorior: so it's not the docker tag patch that we landed this week13:15
*** trown|outtypewww is now known as trown13:15
EvilienMjaosorior: it's really missing containers on the overcloud13:15
therveEvilienM: Right the *whole* logs :D13:15
EvilienMjaosorior: https://www.diffchecker.com/pcyCS0yT13:15
therveClass[Designate::Db] is already declared; ?13:15
openstackgerrit98k proposed openstack/puppet-crane master: Add bugs link in README  https://review.openstack.org/61691113:18
EvilienMjaosorior: do we have a bug report yet?13:18
*** jcoufal has joined #tripleo13:19
jaosoriorEvilienM: not yet. I just found out when you guys asked me to check the tempest patch.13:20
EvilienMI know mwhahaha was looking at it last night13:20
EvilienMjaosorior: i'm comparing rpms13:21
*** ykarel has quit IRC13:22
hjensasykarel: I'm not sure.13:23
*** ykarel has joined #tripleo13:23
EvilienMjaosorior: using logstash to see when it started exactly13:24
*** aufi has quit IRC13:28
*** rfolco|doctor is now known as rfolco13:29
chemdciabrin: so the fix seem to work, I've added more people on the review list to have this merge soonish13:29
therveEvilienM: https://review.openstack.org/#/c/612455/ broke puppet-tripleo13:29
dciabrinchem, w00t!13:30
EvilienMtherve: oops13:31
openstackgerritThomas Herve proposed openstack/puppet-tripleo master: Fix designate db instantation  https://review.openstack.org/61692013:31
therveEvilienM: ^^ I think should fix it13:32
EvilienMok13:32
EvilienMjaosorior: I have a query in logstash13:33
EvilienMhttp://logstash.openstack.org/#dashboard/file/logstash.json?query=build_status%3A%20FAILURE%20AND%20message%3A%20%5C%22Error%20response%20from%20daemon%3A%20no%20such%20id%3A%20192.168.24.1%3A8787%2Ftripleomaster%2Fcentos-binary-cinder-backup%3Acurrent-tripleo-updated-*%5C%2213:33
openstackgerritMerged openstack/tripleo-quickstart-extras master: Run atop for monitoring deployment  https://review.openstack.org/61535713:35
EvilienMjaosorior: it started on Nov 513:35
EvilienMlogstash says it's https://review.openstack.org/#/c/614827/ :/13:36
d0ugalhjensas, jaosorior: yeah, it is normal :)13:37
EvilienMjaosorior: I'm filling a bug13:37
bogdandodciabrin: pong13:37
d0ugalhjensas, jaosorior: in that case it means that python-vitrageclient isn't installed - but that is fine since we don't need it. The log could be better tho'13:38
bogdandorook: yes, we can try that13:38
jaosoriorthansk EvilienM13:40
EvilienMhttps://bugs.launchpad.net/tripleo/+bug/180251113:40
openstackLaunchpad bug 1802511 in tripleo "Error when tagging cinder-backup container for Pacemaker" [Critical,Triaged] - Assigned to Emilien Macchi (emilienm)13:40
jaosoriord0ugal: thanks!13:40
EvilienMI'm workingo n a patch otherwise I'll revert13:40
EvilienMthe scenarios are non voting anyway13:40
jaosoriorEvilienM: we should still try to keep them green :)13:41
EvilienMI know13:41
EvilienMbut let me try fixing it13:41
dciabrinbogdando, quick question on logrotation https://github.com/openstack/puppet-tripleo/blob/master/templates/logrotate/containers_logrotate.conf.erb#L12. is a empty file /var/lib/containers/*/*.log removed due to notifempty or missing ok? or is it kept untouch?13:41
* Tengu will paint his screen green13:41
*** sanjayu__ has quit IRC13:42
*** skramaja has quit IRC13:43
bogdandodciabrin: it could be older than 14 days13:43
bogdandoor cuz of minsize 113:43
dciabrinbogdando, i'm afraid if a service doesn't log often, an empty file could be deleted by the logrotate container13:44
bogdandothose two options are there to enforce GDPR13:44
bogdandoand that's normal13:44
bogdandoIIUC13:44
bogdandolike personal data related info must be force expired and all of that, even if logrotate refuses to manage such files13:45
Tengudciabrin: what would happen if a file is deleted? usually the app logging stuff will create it anyway... ?13:45
*** jpich has quit IRC13:45
bogdandodciabrin: ^^13:45
dciabrinTengu, not if the service keeps a filedescriptor open to that logfile13:45
dciabrinTengu, if would keep logging into something that is not accessible anymore13:45
dciabrinbogdando, see my point?13:46
Tenguhmm. yeah, but I have some doubt about a service that doesn't log anything in 14 days... ?13:46
bogdandodciabrin: what can we do to that? if a file is not managed by logrotate, we can do nothing13:46
bogdandobut force delete it13:46
openstackgerritMerged openstack/tripleo-quickstart master: Use container images mirrored on dockerhub for CI  https://review.openstack.org/61544513:46
openstackgerritMerged openstack/python-tripleoclient master: Undercloud: Disable persisting VIPs in /etc/hosts  https://review.openstack.org/61309413:46
bogdando... to enforce those egpr things13:46
bogdando*gdpr13:46
Tengubogdando: maybe replace the delete by a truncate, and, after 30 days, really delete it if empty?13:47
bogdandothe rest is up to the process holding it13:47
Tengutruncate -s 0 will empty the whole file.13:47
Tengustill ugly, but..13:47
dciabrinTengu, bogdando well this is certainly a way of interpreting it. another way is " i want to make sure the functional logs for my service are still accessible even when my service don't log for a long time"13:47
bogdandoTengu: well, probably worth trying13:47
dciabrinTengu, truncate is ok, I'm worried about the deletion that may happen13:47
Tengudciabrin: in the case you describe, the operator SHOULD copy the logs of unused service elsewhere for archive purpose.13:48
bogdandodciabrin: yeah, but that contradicts gdpr :)13:48
*** lblanchard has joined #tripleo13:48
dciabrinTengu, bogdando iiuc, is a file is to be truncated, it will be copied before anyway13:48
bogdandoso one has to chose which one to break13:48
Tenguand if the operator copies the log, this is out of openstack/tripleo control. so GDPR OK13:48
Tengudciabrin: not with "truncate" command.13:48
dciabrinTengu, bogdando I'm not sure we're saying the same thing.13:48
dciabrinTengu, bogdando it's perfectly fine for mysql to not log something for 20 days. what would happend with current logrotation rules?13:49
bogdandodciabrin: I got it, yes, hopefully truncate could improve that13:49
Tengudciabrin: mysql, no log for 20 days while it's accessed every second or so?13:49
dciabrinTengu, mysql doesn't log SQL access.13:49
dciabrinTengu, haproxy can, my mysql doesn't log read nor write13:50
bogdandoso if sparse logging resumed after 15 days, it goes to a new file and not to the removed and unlined one13:50
bogdandomakes sense13:50
bogdandoTengu, dciabrin: ^^ ++13:50
bogdandounlinked*13:50
Tengubogdando: ok. so yeah, truncate after 14d, and remove empty files after a longer period.13:50
Tengutruncate -s0 is the right thing to do imho.13:50
*** morazi has joined #tripleo13:50
bogdandoTengu: care to push a patch and backports?13:51
bogdandothanks! :)13:51
Tengubogdando: gimme a link? :)13:51
Tengudon't remember in what project you pushed the stuff :D13:51
Tenguah, replacing https://github.com/openstack/puppet-tripleo/blob/master/templates/logrotate/containers_logrotate.conf.erb#L2313:51
dciabrinbogdando, Tengu i think the log rotation already works like that with https://bugzilla.redhat.com/show_bug.cgi?id=158551613:51
openstackdciabrin: Error: Error getting bugzilla.redhat.com bug #1585516: NotPermitted13:51
dciabrincopy/truncate takes care of that13:52
bogdandodciabrin: that is also a case for unmanaged files13:52
bogdandothat logrotate does not care of13:52
dciabrinmy question is what happens in the edge case where a service (mysql) holds a fd to a file that has not been updated for more than 14 days.13:52
bogdandobut still has to be enforced as expired off /var/log/containers13:52
dciabrinwill that file get deleted? in which case, mysql logging won't recover13:52
bogdandothink of that postrotate find -delete as a cron job13:52
*** jpich has joined #tripleo13:53
bogdandojust causally related to logrotate working on logs13:53
openstackgerritMerged openstack/tripleo-heat-templates master: Rework neutron/own agent wrapper tools for podman  https://review.openstack.org/60697513:53
openstackgerritMerged openstack/tripleo-heat-templates stable/queens: switch documentation job to new PTI  https://review.openstack.org/61591113:53
bogdandoit could be as well moved to a cron job13:53
openstackgerritMerged openstack/tripleo-heat-templates stable/queens: Change log directory for OpenDaylight  https://review.openstack.org/61245913:53
Tengubogdando: should I add a new "find" command in order to delete files older than "2x the purge_after_days"?13:53
bogdandoif you wish, but I can think of collisions to logrotate than13:53
bogdandopainful ones...13:53
bogdandoTengu: that would break the very meaning of that param I'm affraid13:54
Tengubogdando: not really, since the files will be TRUNCATED to 0 after said purge_after_days period13:54
Tenguso empty files13:54
bogdandomay be instead it should be truncated by that time and purged by dat +113:54
bogdandoday + 113:54
Tengunope13:55
Tenguthat won't correct dciabrin issue13:55
bogdandohm13:55
bogdandook then13:55
bogdandowfm13:55
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: tripleo-container-tag: pull image before tagging  https://review.openstack.org/61693613:55
Tenguand we want to clean the inodes13:55
EvilienMjaosorior: ^13:55
Tengucool :)13:55
EvilienMthat should do it13:55
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Make build-test-packages more resilient  https://review.openstack.org/61693713:55
EvilienMjaosorior: I'm following http://zuul.openstack.org/stream/d4b40935946c4316ad78858e15966719?logfile=console.log which is scenario001 with my patch. Let's see how it works. If it fails we revert my patch13:58
EvilienMis that ok?13:58
*** mcornea has joined #tripleo13:58
dciabrinTengu, bogdando tbh I don't buy that GDPR mandates that the main log file of a service must be deleted if it's empty just because it's been created more than 14 days ago. the find at the end of the logrotate looks too broad to me13:59
*** cylopez has joined #tripleo13:59
*** cylopez has left #tripleo13:59
Tengubogdando: is the file name pattern known? i.e. can we add "-and -not -name *.log" ?14:00
Tenguthat would prevent some issues.14:00
Tengubut that means we must ensure all log files have a .log suffix.14:00
Tenguand I'm pretty sure this isn't the case.14:00
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: Enable testing using ansible molecule  https://review.openstack.org/61367214:01
Tengubogdando: just for my understanding - the find you added was for the case where there were "old logs not properly managed with the new logrotate config" right?14:02
*** aufi has joined #tripleo14:02
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: Implement tripleo-docker-rm role  https://review.openstack.org/61520014:05
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: Implement tripleo-docker-rm role  https://review.openstack.org/61520014:05
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: WIP - upgrade: remove Docker containers now managed by Podman  https://review.openstack.org/61520914:05
ksamborHi all,  where in tripleo code we take care of installing additional packages during split-stack deployment?14:06
EvilienMnhicher, weshay_ruck: is anyone looking at vexxhost errors? I have one here with OVB: https://logs.rdoproject.org/23/614023/5/openstack-experimental/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master-vexxhost/c8acacf/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-11-08_23_29_4514:06
EvilienMnhicher, weshay_ruck : not sure it's related to my patch (I enabled podman on the undercloud)14:07
*** akrivoka has quit IRC14:07
*** akrivoka has joined #tripleo14:08
EvilienMnhicher, weshay_ruck : https://logs.rdoproject.org/23/614023/5/openstack-experimental/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master-vexxhost/c8acacf/logs/undercloud/var/log/containers/nova/nova-compute.log.txt.gz#_2018-11-08_23_12_53_37314:08
EvilienMnhicher, weshay_ruck : sorry for spam - the actual issue is here: https://logs.rdoproject.org/23/614023/5/openstack-experimental/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master-vexxhost/c8acacf/logs/undercloud/var/log/containers/nova/nova-compute.log.txt.gz#_2018-11-08_23_12_51_68314:08
EvilienM"Failure prepping block device: InternalServerError: Unable to attach VIF cacf147c-eba0-49cc-a30f-1da4bbe040d9 because Ironic can not update Neutron port cacf147c-eba0-49cc-a30f-1da4bbe040d9 MAC address to match physical MAC address fa:16:3e:b6:c3:78 (HTTP 500)"14:09
EvilienMI don't know if it's a known bug on master with OVB14:09
openstackgerritCédric Jeanneret proposed openstack/puppet-tripleo master: Ensure we do not remove logs with very, very few traffic  https://review.openstack.org/61694514:09
Tengudciabrin: bogdando -^^14:09
*** tzumainn has joined #tripleo14:10
Tengustill. not 100% sure it's the best way to handle the issue.14:10
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180240914:10
openstackLaunchpad bug 1802409 in tripleo "cannot create directory "/var/lib/tempestdata" Permission denied" [Critical,In progress] - Assigned to Derek Higgins (derekh)14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180251114:10
*** ooolpbot has quit IRC14:10
openstackLaunchpad bug 1802511 in tripleo "Error when tagging cinder-backup container for Pacemaker" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)14:10
EvilienMhas anyone tried to deploy master on BM lately?14:10
TenguEvilienM: "on BM" - so my undercloud deploy on a VM doesn't count? :)14:11
EvilienMI mean, OVB for example14:11
EvilienMhjensas: looks similar to https://bugs.launchpad.net/tripleo/+bug/174967114:12
openstackLaunchpad bug 1749671 in tripleo "Overcloud installation fails with "Failure prepping block device." error" [Critical,Fix released] - Assigned to Harald Jensås (harald-jensas)14:12
EvilienMi'm filling a new bug14:12
hjensasEvilienM: sounds like a server build failed but the server's port was not deleted. Then rescheduling picks the same ironic node for some other server, and when ironic tries to set the mac address of the neutron port connecting to the new nova server there is a duplicate mac address error raised by neutron`14:15
EvilienMI flled a new bug: https://bugs.launchpad.net/tripleo/+bug/180251914:15
openstackLaunchpad bug 1802519 in tripleo "Overcloud can't get deployed: Ironic fails to update MAC address on Neutron port" [Critical,Triaged]14:15
hjensasEvilienM: ack, coincidentely I'm working on a patch that should fix this in ironic today.14:17
hjensasEvilienM: https://review.openstack.org/53998114:17
*** moguimar has joined #tripleo14:18
weshay_ruckEvilienM, was looking at the bug you just opened on scen00314:19
* weshay_ruck switches to vexx14:19
EvilienMweshay_ruck: therve has a fix14:19
EvilienMfor designate14:19
*** moguimar has quit IRC14:19
bogdandoTengu, dciabrin: that's how I understood the requirement. Files should not be restricted to *.log or anyhow14:21
bogdandofell free to alter that folks14:21
bogdandofeel14:21
bogdandoTengu: >old logs not properly managed with the new logrotate config14:22
bogdandoanything in /var/logs/containers14:22
bogdandonot just logs14:22
bogdandoassuming that using such a path implies storing some sensitive info14:23
Tenguok14:23
*** agopi|brb is now known as agopi14:23
weshay_ruckEvilienM, that's the only job I've seen fail on vexx w/ that at least today/yesterday14:23
Tenguwell... I don't see better way to handle that then... dciabrin ?14:23
bogdandofalling into GDPR requirements. But that is still my understanding14:23
weshay_ruckEvilienM, you cool w/ a rdo check to retest?14:23
weshay_ruckand I'll keep digging14:23
EvilienMweshay_ruck: yes in fact it's what I was about to do on a nother patch without podman14:23
weshay_ruckEvilienM, we have data from other jobs14:24
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: CI test - never merge  https://review.openstack.org/61695414:24
weshay_ruckdon't think you need to do that14:24
EvilienMok14:24
bogdandoand thinking of /var/log for host logs, it each time reminds me of Linux readiness for GDPR :)14:24
EvilienMweshay_ruck: so all is green?14:24
EvilienMweshay_ruck: why would we hit this issue then?14:24
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Add custom env and container cli option to standalone upgrade.  https://review.openstack.org/61695714:25
weshay_ruckEvilienM, no there was something that failing in tempest and the undercloud. I need to get to the direct logs though14:25
bogdandojaosorior, Tengu, dciabrin: we still have yet that part adressed btw :)14:25
weshay_ruckmin14:25
jaosoriorbogdando: ??14:25
bogdandoI mean /var/log on host14:25
dciabrinbogdando, Tengu well there's two things here. 1) I want to have a mysql.log when I get called out to fix a problem. 2) I would challenge the fact that technical every log file stored on computers in europe would have to be truncate to not hold more than 14days worth of history.14:25
bogdandoand its contents force expiration14:25
*** iurygregory has quit IRC14:25
*** yolanda has joined #tripleo14:25
bogdandoeven if logrotate thinks it's fine to keep it there14:26
weshay_ruckEvilienM, https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master-vexxhost14:26
weshay_ruckfyi14:26
*** iurygregory has joined #tripleo14:26
weshay_ruckEvilienM, so w/o podman most recent https://logs.rdoproject.org/33/614633/3/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/0daaf82/14:27
weshay_ruckfailed in tempest14:27
Tengubogdando, dciabrin: we might want to ask a lawyer on that topic. Just to be on the safe side.14:27
Tenguanyway, I have to run away now...14:27
weshay_ruckUnhandled error: MaxRetryError: HTTPConnectionPool(host=u'172.17.0.16', port=9696): Max retries exceeded with url: /v2.0/extensions.json14:28
bogdandodciabrin: > would challenge the fact that technical every log file stored on computers in europe would have to be truncate to not hold more than 14days worth of history.14:28
bogdandowould you bet a 20$ mil on that statement?14:28
TenguI'll come back Monday, maybe I'll be able to reach 1 or 2 lawyers of my friends :).14:28
bogdandoanyway, that 14 days period is just a default btw14:28
*** ykarel is now known as ykarel|away14:29
bogdandojaosorior: I meant that https://bugzilla.redhat.com/show_bug.cgi?id=1578731#c514:31
openstackbugzilla.redhat.com bug 1578731 in puppet-tripleo "Allow logrotate configuration to control log retention based on age" [High,On_qa] - Assigned to rhos-maint14:31
bogdandosorry for being out of context14:31
weshay_ruckEvilienM, ugh.. typical friday, there is some weird results from the fs001/35 jobs on vex atm as well in the very latest :(14:32
EvilienMweshay_ruck: ok so it's not just me right?14:32
EvilienMweshay_ruck: can I let the alert on this LP?14:32
bogdandodciabrin, Tengu: my take is it is more safe to have nothing left in logs than removing files containing 100 of positions falling into the regulation and missing a one small file with a 101 th of it14:33
*** ykarel|away has quit IRC14:34
bogdandoand getting sued for that haha14:34
weshay_ruckEvilienM, this lp = https://bugs.launchpad.net/tripleo/+bug/1802519 ya?  So if I could recreate I would say yes, but I'd like to see it twice before an alert is set unless you are sure of the cause14:34
openstackLaunchpad bug 1802519 in tripleo "Overcloud can't get deployed: Ironic fails to update MAC address on Neutron port" [Critical,Triaged]14:34
chemweshay_ruck: I have my bottle of champagne at the ready, and the review has a lot of +2, what is missing to/prevent from merging https://review.openstack.org/#/c/604706/14:35
weshay_ruckchem, is that just wine in france :)14:36
weshay_ruckchem, +1 to merge from me :)14:36
weshay_ruckfriggin awesomeness14:37
nhicherEvilienM: no I didn't check these jobs since sshnaidm|off manage them14:39
*** moguimar has joined #tripleo14:39
weshay_rucknhicher, ya.. something's up atm. trying to determine what14:39
weshay_rucksagi is off today14:39
chemweshay_ruck: well Champagne is a bunch of french grappe producers that made it clear that the sparkling white wine they produce has to be bough at a premium.14:40
weshay_ruckwe should try that..14:41
weshay_ruckOSPange14:41
EvilienMweshay_ruck: ok14:42
openstackgerrit98k proposed openstack/puppet-tripleo master: Add bugs link in README  https://review.openstack.org/61696614:43
chemmwhahaha: weshay_ruck  (and everybody else) thanks!  Will try to keep an eye on that jobs during the we given I don't consume to much overpriced sparkling white wine.14:44
weshay_rucknhicher, this is odd.. the latest vexhost jobs I see "ansible_user_dir": "/root", these jobs fail, and the last successful jobs have             "ansible_user_dir": "/home/zuul",14:45
weshay_ruckchem++14:45
weshay_ruckchem++14:45
weshay_ruckreally nice work on the two singlenode update/upgrade jobs14:45
chempiece of cake :)14:46
nhicherweshay_ruck: I will check logs14:46
EvilienM"piece of cake" lol14:47
weshay_rucknhicher, k.. /me checking for changes to qs14:47
weshay_ruckEvilienM, chem is a cool mo fo14:47
EvilienM:-O14:47
EvilienMin american it means a nice guy?14:47
mwhahahamaybe, maybe not14:48
mwhahaha:o14:48
openstackgerrit98k proposed openstack/puppet-tripleo master: Add bugs link in README  https://review.openstack.org/61696614:49
chemhehe, like it14:51
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Add fedora28 support for quickstart  https://review.openstack.org/59165214:51
honzajtomasek: looks like the keystone bug might be related to a recent refactor (they ported the server to flask)14:59
honzajtomasek: have you seen that bug?  i'm pretty sure we're the only project in tripleo that uses the options request or cors in general15:00
*** moguimar has quit IRC15:00
jaosoriorbogdando: oh, taht15:01
jaosoriorbogdando: what's missing from there? Just the journal?15:01
bogdandojaosorior: yes, journal and probably everything in /var/log of hardware nodes15:02
bogdandonot sure we can address that for tripleo though, it's rather for distros15:03
jaosoriorbogdando: yeah, we don't actually manage the journal in tripleo15:03
jaosoriorwe might need to just provide an ansible playbook so folks can do that15:03
*** moguimar has joined #tripleo15:06
rookbogdando: do you want to make the changes on my undercloud?15:07
bogdandorook: I could push a DNM patch for tht gerrit15:08
bogdandoso you can apply it locally to tht dir15:08
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180240915:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180251115:10
*** ooolpbot has quit IRC15:10
openstackLaunchpad bug 1802409 in tripleo "cannot create directory "/var/lib/tempestdata" Permission denied" [Critical,In progress] - Assigned to Derek Higgins (derekh)15:10
openstackLaunchpad bug 1802511 in tripleo "Error when tagging cinder-backup container for Pacemaker" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)15:10
*** Vorrtex has joined #tripleo15:10
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: DNM testing with ipc=host for all containers  https://review.openstack.org/61697415:12
bogdandorook: https://review.openstack.org/61697415:12
*** Vorrtex has quit IRC15:13
bogdandoor if you wish, you can just cd into tht dir at undercloud and run https://pastebin.com/RJ2d65r415:14
bogdando*grep -rP15:14
jtomasekhonza: ah, yeah, that sounds reasonable15:15
*** kopecmartin is now known as kopecmartin|off15:15
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: multinode-containers: include Podman service  https://review.openstack.org/61697615:16
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: WIP / fs010: enable Podman on the overcloud  https://review.openstack.org/61252615:16
EvilienMTengu: ^ this is why I think podman wasn't well configured15:17
*** mjturek has joined #tripleo15:20
*** artom has quit IRC15:21
*** artom has joined #tripleo15:21
*** Vorrtex has joined #tripleo15:23
*** morazi has quit IRC15:24
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Enable Scale up/redeploy of a deployment  https://review.openstack.org/60793115:26
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Introduce zuul.projects and executed pre-run playbooks for reproducer  https://review.openstack.org/61357215:27
weshay_ruckEvilienM, I like using fs10 to evaluate because it runs all the time15:27
weshay_ruck+115:27
*** morazi has joined #tripleo15:28
openstackgerritMerged openstack/puppet-pacemaker master: Skip some tests for Ubuntu 18.04 and puppet 4.8.  https://review.openstack.org/61624915:28
EvilienMweshay_ruck: cool bro15:31
weshay_ruckthe neutron WHAT?15:33
openstackgerritKamil Sambor proposed openstack/tripleo-heat-templates stable/queens: Load openvswitch module from the host  https://review.openstack.org/61698315:34
*** hkominos_ has joined #tripleo15:34
*** rascasoft has quit IRC15:36
*** akrzos has quit IRC15:36
*** jcoufal has quit IRC15:36
*** rnoriega has quit IRC15:36
*** jcoufal has joined #tripleo15:36
*** akrzos has joined #tripleo15:36
*** rnoriega has joined #tripleo15:36
*** rascasoft has joined #tripleo15:38
weshay_ruckEvilienM, mwhahaha so this should get f28 where we need it re: hash https://review.openstack.org/#/c/615140/  I'm seeing a new error "missing python module paunch"15:41
weshay_ruckI don't see a bug on it yet15:41
mwhahahaweshay_ruck: we have a patch15:42
weshay_ruckI suppose moving to paunch would fix that15:42
weshay_rucker..15:42
weshay_ruckpodman15:42
mwhahahai think it's fixed by https://review.openstack.org/#/c/616203/15:42
openstackgerritwes hayutin proposed openstack/tripleo-heat-templates master: Use correct python version for puppet-docker.py  https://review.openstack.org/61620315:44
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart-extras master: WIP: enable fedora-28 for the reproducer  https://review.openstack.org/60249215:51
openstackgerritwes hayutin proposed openstack/tripleo-heat-templates master: Use correct python version for puppet-docker.py  https://review.openstack.org/61620315:51
*** ksambor has quit IRC15:51
*** bnemec is now known as beekneemech15:53
beekneemechEvilienM: Do we run tripleo-ci on puppet modules? Wondering if we need to add a job to puppet-designate to catch stuff like this in the future.15:53
*** ayoung has joined #tripleo15:53
mwhahahabeekneemech: we do15:54
*** dsneddon has quit IRC15:54
mwhahahabeekneemech: not scenarios15:54
mwhahahabeekneemech: as we convert the sceanrios to standalone we can add it to puppet-designate15:55
beekneemechmwhahaha: Okay, sounds good.15:55
mwhahahabeekneemech: alternatively we can construct a designate only standalone that we could run on designate/puppet-designate15:55
weshay_ruckmwhahaha, are you holding back on https://review.openstack.org/#/c/614820/ re: chem's comments15:56
mwhahahaweshay_ruck: no i am not, though it sounds like teh standalone-upgrade can be replaced with this feature15:56
mwhahahaweshay_ruck: not sure why they duped the standalone to standalone-upgrade if it was just to ad an environment file but i'll take a look later15:57
weshay_ruckya.. imho the thought was just to let chem run with it, to prove it could be done and add value upstream15:57
mwhahahayea so once we have my change we can probably swap it out and remove it15:58
weshay_ruckaye15:58
*** iurygregory is now known as iurygregory|mtg15:58
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO  https://review.openstack.org/56722416:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044516:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60429816:00
*** jistr is now known as jistr|call16:00
chemmwhahaha: weshay_ruck is right, I did it the copy/paste way just to keep stuff clean and at the beginning I wasn't sure how much this would have to change from the original16:01
*** moguimar has quit IRC16:02
chemmwhahaha: when the test is merged, I'll revisit and check how to refactor this thing into the standalone role.16:02
*** moguimar has joined #tripleo16:03
rookbogdando: so, once that is complete, just reinstall the undercloud?16:05
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Make build-test-packages more resilient  https://review.openstack.org/61693716:07
openstackgerritSorin Sbarnea proposed openstack/tripleo-heat-templates master: DNM: test patch on vexxhost  https://review.openstack.org/61463316:08
openstackgerritSorin Sbarnea proposed openstack/tripleo-heat-templates master: DNM: test patch on vexxhost  https://review.openstack.org/61463316:08
*** janki has joined #tripleo16:09
*** ooolpbot has joined #tripleo16:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180240916:10
openstackLaunchpad bug 1802409 in tripleo "cannot create directory "/var/lib/tempestdata" Permission denied" [Critical,In progress] - Assigned to Derek Higgins (derekh)16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180251116:10
*** ooolpbot has quit IRC16:10
openstackLaunchpad bug 1802511 in tripleo "Error when tagging cinder-backup container for Pacemaker" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)16:10
dciabrinchem ok to unblock https://review.openstack.org/#/c/609998/ now that https://review.openstack.org/#/c/616249/ merged?16:11
chemdciabrin: oh, boy stuff merge so quickly in there :)16:12
dciabrinchem, haha clearly faster than t-h-t16:13
chemdciabrin: I'm having a quick second look at the code, but mostly it's faith in bandini readme16:13
dciabrinchem, yeah he tested it fairly extensively and had the previous revision checked by mcornea iirc16:14
chemdciabrin: should we trigger a recheck to get the second +2 ?16:16
dciabrinchem yep works for me16:16
chemdciabrin: well, it's so cheap that I did it actually16:16
dciabrinchem exactly :D16:16
*** jistr|call is now known as jistr16:17
*** moguimar has quit IRC16:18
*** moguimar has joined #tripleo16:22
*** aufi has quit IRC16:23
*** rook is now known as rook-pto16:23
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Make build-test-packages more resilient  https://review.openstack.org/61693716:26
rajinirHi are the gates reliable? I don't see them passing on the stable branches... https://review.openstack.org/#/c/616685/16:26
*** moguimar has quit IRC16:27
rajinirI have been rechecking many patches for a couple of weeks now no improvement16:28
openstackgerritMerged openstack/tripleo-upgrade master: Use Jinja2 macro to remove redundancy in workarounds.sh.j2.  https://review.openstack.org/61298716:30
*** yprokule has quit IRC16:31
*** moguimar has joined #tripleo16:31
*** moguimar has quit IRC16:32
*** dprince has quit IRC16:35
*** d0ugal has quit IRC16:36
openstackgerritHarald Jensås proposed openstack/os-net-config stable/rocky: Apply IP/netmask/route/MTU changes without bouncing interfaces.  https://review.openstack.org/61699216:36
*** d0ugal has joined #tripleo16:36
mwhahahachem: 604706 is going to merge conflict with another change int he gate, you may want to rebase if you want to save a bunch of time16:37
mwhahahachem: 613250 is the change you'll want to rebase on top of16:37
mwhahahaalso gate reset :(16:37
*** bogdando has quit IRC16:38
chemmwhahaha: oki, thanks for the heads up, I'll do that then.16:38
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Make build-test-packages more resilient  https://review.openstack.org/61693716:38
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: Enable testing using ansible molecule  https://review.openstack.org/61367216:39
openstackgerritMarios Andreou proposed openstack/tripleo-quickstart-extras master: Create reproducer_extra_vars.yml to replace env_vars_to_src.sh  https://review.openstack.org/61699316:39
openstackgerritMarios Andreou proposed openstack-infra/tripleo-ci master: DNM: testing shortwiring the run-v3 to not run toci-gate-test testing the reproducer. The real patch we are depending on is the parent  https://review.openstack.org/61699416:39
*** d0ugal has quit IRC16:41
*** d0ugal has joined #tripleo16:42
*** aedc has quit IRC16:42
*** d0ugal has quit IRC16:44
*** d0ugal has joined #tripleo16:44
openstackgerritAthlan-Guyot sofer proposed openstack-infra/tripleo-ci master: New workflow for standalone upgrade.  https://review.openstack.org/60470616:45
abishopanyone know about DLMs and tripleo?16:46
abishoptripleo has an "Etcd" service, but who distributes the python-etcd3 libraries? (I can't find them)16:46
*** iurygregory|mtg has quit IRC16:46
*** d0ugal has quit IRC16:47
*** d0ugal has joined #tripleo16:47
chemmwhahaha: oki, done, and thanks again.16:47
*** d0ugal has quit IRC16:48
weshay_ruckjaosorior, mwhahaha ya.. so you guys may be familiar w/ the bm env that we have and rascasoft have... so there is some discussion and plans to propose to bring those real env.. to the tripleo promotion level16:48
*** quiquell|off has quit IRC16:48
*** d0ugal has joined #tripleo16:48
weshay_ruckjaosorior, mwhahaha I can fill in some details, but it's something for you guys to think about for a bit16:48
weshay_ruckwhat iyho would be required for that to be successful w/o being a burden16:49
rascasoftweshay_ruck, what can I say :) better late than ever :)16:49
weshay_rucklolz16:49
mwhahahaweshay_ruck: my issue with baremetal in the process is that i don't have a baremetal env to reproduce any reported issues16:49
mwhahahaweshay_ruck: so it's dangerous to block on that if we don't have resources to dev/troubleshoot16:49
rascasoftmwhahaha, that's something we can take care about16:49
*** dsneddon has joined #tripleo16:50
rascasoftmwhahaha, because you know BM == REAL == End users16:50
mwhahaharascasoft: yea i'm aware, i just don't have an env to work with :D16:50
weshay_ruckrascasoft, alway's keeping it real16:50
rascasoftmwhahaha, that's what I'm saying16:50
mwhahaharascasoft: i have raised this elsewhere to some folks16:50
rascasoftmwhahaha, we can organize things so that when there's a problem, the env is available16:51
*** shardy has quit IRC16:51
rascasoftmwhahaha, that's the story of my life (re: raised this elsewhere)16:51
mwhahahapretty much16:52
*** morazi has quit IRC16:53
mwhahahaabishop: etcd was for kubernetes, so i'm not sure we support it for any openstack stuff16:54
jaosoriormwhahaha: I think some neutron plugin was using it16:54
mwhahahaif cores are around, please take a look at https://review.openstack.org/#/c/616936/ to fix the failing scenarios on master16:54
abishopmwhahaha: hmm, ok. cinder is in need of a dlm, so I'm on the hunt16:55
mwhahahajaosorior: we don't support it's deployment with openstack stuff, so if it is we'll probably need to look at that16:55
mwhahahaabishop: might just check with rdo on how to get that stuff added16:55
abishopmwhahaha: ack, might come to that16:55
abishopjaosorior: I think midonet uses zookeeper, but I don't think we support that as a tripleo service16:56
openstackgerritHarald Jensås proposed openstack/os-net-config stable/queens: Apply IP/netmask/route/MTU changes without bouncing interfaces.  https://review.openstack.org/61699616:58
jaosoriorAlright folks, I'm off for the weekend now! Maybe see you in Berlin! :D16:59
jaosoriorhave a good weekend!16:59
*** dtantsur is now known as dtantsur|afk17:00
*** agopi is now known as agopi|brb17:07
mwhahahaabishop: yea in the past midonet has installed zookeeper theirselves via tripleo but it's all their own code17:09
*** ooolpbot has joined #tripleo17:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180240917:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180251117:10
*** ooolpbot has quit IRC17:10
openstackLaunchpad bug 1802409 in tripleo "cannot create directory "/var/lib/tempestdata" Permission denied" [Critical,In progress] - Assigned to Derek Higgins (derekh)17:10
*** jpich has quit IRC17:10
openstackLaunchpad bug 1802511 in tripleo "Error when tagging cinder-backup container for Pacemaker" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)17:10
abishopmwhahaha: I like that ooo can deploy etcd3, and I'm pursuing the python libs in #rdo17:10
*** morazi has joined #tripleo17:11
*** agopi|brb has quit IRC17:12
*** chem has quit IRC17:20
*** derekh has quit IRC17:37
*** gfidente is now known as gfidente|afk17:37
*** rlandy is now known as rlandy|brb17:39
*** trown is now known as trown|lunch17:40
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: Do not ignore NtpServer by default  https://review.openstack.org/61700117:46
*** rlandy|brb is now known as rlandy17:49
*** florianf is now known as florianf|afk17:54
mwhahahaugh sshnaidm|off, weshay_ruck: atop has broken f28 http://logs.openstack.org/05/614005/5/check/tripleo-ci-fedora-28-standalone/5371f2e/job-output.txt.gz#_2018-11-09_16_16_06_42314317:57
weshay_ruckugh.. looking17:57
mwhahahaneed to disable that ig uess17:58
mwhahahacause no epel17:58
weshay_ruckmwhahaha, it's not ignored17:58
weshay_ruck?17:58
weshay_rucksorry in 1-1 as well17:59
mwhahahaoh yea it is17:59
openstackgerritAlan Bishop proposed openstack/tripleo-docs master: Update ceph OSD setup for standalone  https://review.openstack.org/61700317:59
mwhahahanm, though still a nasty bit of error in the log17:59
weshay_ruckhttp://logs.openstack.org/05/614005/5/check/tripleo-ci-fedora-28-standalone/5371f2e/job-output.txt.gz#_2018-11-09_16_22_44_89190317:59
mwhahahahttp://logs.openstack.org/05/614005/5/check/tripleo-ci-fedora-28-standalone/5371f2e/logs/delorean_logs/54/80/54803f9cf702764725a046a861ffefb4a5fb8d12_dev/rpmbuild.log.txt.gz18:00
mwhahahadlrn build is bad18:00
* mwhahaha sighs18:00
*** agopi_ has joined #tripleo18:00
* mwhahaha goes fixing build-test-packages18:00
*** agopi_ is now known as agopi18:02
openstackgerritBen Nemec proposed openstack/tripleo-docs master: Add documentation on deploying Designate  https://review.openstack.org/58583318:05
*** akrivoka has quit IRC18:07
*** akrivoka has joined #tripleo18:07
*** mjturek has quit IRC18:08
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180240918:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180251118:10
*** ooolpbot has quit IRC18:10
openstackLaunchpad bug 1802409 in tripleo "cannot create directory "/var/lib/tempestdata" Permission denied" [Critical,In progress] - Assigned to Derek Higgins (derekh)18:10
openstackLaunchpad bug 1802511 in tripleo "Error when tagging cinder-backup container for Pacemaker" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)18:10
*** suuuper has quit IRC18:11
openstackgerritAlex Schultz proposed openstack/ansible-role-chrony master: Move default values to defaults  https://review.openstack.org/61700718:21
*** dprince has joined #tripleo18:24
openstackgerritAlex Schultz proposed openstack/ansible-role-chrony master: Add tripleo-standalone for coverage  https://review.openstack.org/61700918:25
openstackgerritAlex Schultz proposed openstack/ansible-role-chrony master: Add tripleo-standalone for coverage  https://review.openstack.org/61700918:26
openstackgerritAlex Schultz proposed openstack/ansible-role-chrony master: Move default values to defaults  https://review.openstack.org/61700718:30
openstackgerritMerged openstack/ansible-role-chrony master: Update min tox version to 2.0  https://review.openstack.org/61174418:36
openstackgerritMerged openstack/ansible-role-chrony master: Remove setup.py check from pep8 job  https://review.openstack.org/61424918:36
*** salmankhan has quit IRC18:38
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Make build-test-packages more resilient  https://review.openstack.org/61693718:38
*** trown|lunch is now known as trown18:45
*** morazi has quit IRC19:08
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180240919:10
openstackLaunchpad bug 1802409 in tripleo "cannot create directory "/var/lib/tempestdata" Permission denied" [Critical,In progress] - Assigned to Derek Higgins (derekh)19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180251119:10
*** ooolpbot has quit IRC19:10
openstackLaunchpad bug 1802511 in tripleo "Error when tagging cinder-backup container for Pacemaker" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)19:10
openstackgerritMerged openstack/tripleo-heat-templates master: Enable _member_ role for undercloud install.  https://review.openstack.org/61167719:19
*** janki has quit IRC19:22
*** agopi_ has joined #tripleo19:30
*** agopi has quit IRC19:32
*** hkominos_ has left #tripleo19:45
openstackgerritRussell Bryant proposed openstack/tripleo-heat-templates master: Add OVN environment that works with standalone mode.  https://review.openstack.org/61702219:48
*** Vorrtex has quit IRC19:50
*** Vorrtex has joined #tripleo19:52
*** agopi_ is now known as agopi19:55
*** sambetts|afk has quit IRC20:00
*** sambetts_ has joined #tripleo20:02
*** morazi has joined #tripleo20:03
*** ooolpbot has joined #tripleo20:12
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:12
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180240920:12
openstackLaunchpad bug 1802409 in tripleo "cannot create directory "/var/lib/tempestdata" Permission denied" [Critical,In progress] - Assigned to Derek Higgins (derekh)20:12
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180251120:12
*** ooolpbot has quit IRC20:12
openstackLaunchpad bug 1802511 in tripleo "Error when tagging cinder-backup container for Pacemaker" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)20:12
*** salmankhan has joined #tripleo20:32
*** salmankhan has quit IRC20:36
*** gfidente|afk has quit IRC20:37
*** Vorrtex has quit IRC20:40
*** dprince has quit IRC20:57
openstackgerritThomas Herve proposed openstack/paunch master: Remove log message when container doesn't exist  https://review.openstack.org/61621621:09
*** ooolpbot has joined #tripleo21:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180240921:10
openstackLaunchpad bug 1802409 in tripleo "cannot create directory "/var/lib/tempestdata" Permission denied" [Critical,In progress] - Assigned to Derek Higgins (derekh)21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180251121:10
*** ooolpbot has quit IRC21:10
openstackLaunchpad bug 1802511 in tripleo "Error when tagging cinder-backup container for Pacemaker" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)21:10
*** paramite has quit IRC21:11
mwhahahafiled a bug for the failure in the gate, https://bugs.launchpad.net/tripleo/+bug/180259221:12
openstackLaunchpad bug 1802592 in tripleo "overcloud deploy fails on a KeyError: 'plans'" [High,Triaged]21:12
*** abishop has quit IRC21:15
*** jcoufal has quit IRC21:15
openstackgerritAlex Schultz proposed openstack/python-tripleoclient master: Only return plans if available from the message  https://review.openstack.org/61703521:19
mwhahahathrash, d0ugal: can you check my assumptions on https://review.openstack.org/#/c/617035/21:21
thrashmwhahaha: quick glance says that's correct21:22
mwhahahacool thanks21:22
*** rlandy has quit IRC21:24
*** agopi has quit IRC21:34
*** lblanchard has quit IRC21:47
*** lblanchard has joined #tripleo21:47
*** toure is now known as toure|gone21:49
openstackgerritAlex Schultz proposed openstack/tripleo-common master: Pass execution_id to tripleo.ansible-playbook.  https://review.openstack.org/60606421:50
*** vpickard is now known as vpickard_21:52
*** raildo has quit IRC22:00
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180240922:10
openstackLaunchpad bug 1802409 in tripleo "cannot create directory "/var/lib/tempestdata" Permission denied" [Critical,In progress] - Assigned to Derek Higgins (derekh)22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180251122:10
*** ooolpbot has quit IRC22:10
openstackLaunchpad bug 1802511 in tripleo "Error when tagging cinder-backup container for Pacemaker" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)22:10
*** boazel has joined #tripleo22:13
*** boazel_ has quit IRC22:16
*** trown is now known as trown|outtypewww22:26
*** jtomasek has quit IRC22:31
openstackgerritRadoslaw Smigielski proposed openstack/python-tripleoclient master: [FFU] Fix sensitive hostname in FFU upgrade run command  https://review.openstack.org/61654222:38
*** rh-jelabarre has quit IRC22:41
*** betherly has joined #tripleo23:02
*** betherly has quit IRC23:06
*** radeks__ has quit IRC23:08
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180240923:10
openstackLaunchpad bug 1802409 in tripleo "cannot create directory "/var/lib/tempestdata" Permission denied" [Critical,In progress] - Assigned to Derek Higgins (derekh)23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/180251123:10
*** ooolpbot has quit IRC23:10
openstackLaunchpad bug 1802511 in tripleo "Error when tagging cinder-backup container for Pacemaker" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)23:10
*** lblanchard has quit IRC23:19
*** mcornea has quit IRC23:48
*** slaweq has quit IRC23:57
« Thursday, 2018-11-08 Index Saturday, 2018-11-10 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!