Monday, 2019-08-19

*** BjoernT has joined #openstack-kolla00:43
*** BjoernT has quit IRC00:49
*** spsurya has joined #openstack-kolla01:09
*** redrobot has quit IRC02:23
*** Guest90568 has joined #openstack-kolla02:29
*** Guest90568 is now known as redrobot02:32
*** BjoernT has joined #openstack-kolla02:45
*** dave-mccowan has quit IRC03:46
*** BjoernT has quit IRC04:43
*** dpawlik has joined #openstack-kolla05:01
*** skramaja has joined #openstack-kolla05:09
*** janki has joined #openstack-kolla05:24
*** mrunge has quit IRC05:52
*** mrunge_ has joined #openstack-kolla05:52
*** mrunge_ is now known as mrunge05:56
*** jbadiapa has joined #openstack-kolla06:05
*** skramaja_ has joined #openstack-kolla06:07
*** skramaja has quit IRC06:07
*** threestrands has quit IRC06:34
*** threestrands has joined #openstack-kolla06:35
*** shyamb has joined #openstack-kolla06:55
shyambHi06:55
shyambI am mounting a nfs share on nova_libvirt container06:56
shyambbut container does not have nfs mount helper package06:56
*** shyamb has quit IRC07:00
*** shyam89 has joined #openstack-kolla07:00
*** shyam89 has quit IRC07:06
*** shyamb has joined #openstack-kolla07:06
openstackgerritMerged openstack/kolla-ansible master: Remove nova [DEFAULT]firewall_driver option  https://review.opendev.org/66813907:07
mnasiadkashyamb: use template override to install it - build your own image07:15
shyambmnasiadka: yes, thanks07:16
*** threestrands has quit IRC07:19
*** lemko has joined #openstack-kolla07:21
openstackgerritMerged openstack/kolla-ansible master: Use default BGP speaker driver in bgp_dragent.ini  https://review.opendev.org/67511107:31
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Use fluentd image labels  https://review.opendev.org/67641307:32
*** dpawlik has quit IRC07:34
*** rpittau|afk is now known as rpittau07:35
hrwmorning07:41
yoctozeptomorning07:43
*** ivve has joined #openstack-kolla07:45
hrwyoctozepto: commented your comment07:46
hrwfluentd--07:46
yoctozeptohrw: can see nothing new :/07:47
hrwyoctozepto: ah, not noticed your comment to my comment to your comment ;D07:48
yoctozeptohttp://www.quickmeme.com/img/19/19df7503a5dd5885a309b94c48c561de832f45726a77dac00b2d518317694b78.jpg07:49
hrw;D07:49
hrwhttps://review.opendev.org/#/c/677063 argh07:52
hrwyoctozepto: can you suggest patch which removes that k8s module? so we can get fluentd building again in my patch and then remove k8s stuff in other07:53
hrwin case someone needs it07:54
hrwI am one step from patch to make tripleo CI non-voting07:59
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: CI: make TripleO non voting  https://review.opendev.org/67713608:02
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Add a spec for ceph-ansible  https://review.opendev.org/54498008:02
*** dougsz has joined #openstack-kolla08:02
hrwdone08:03
hrwdougsz: elo08:03
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Add a spec for ceph-ansible  https://review.opendev.org/54498008:04
mnasiadkahrw: only one step away? :)08:04
mnasiadkamorning guys08:05
hrwmnasiadka: ;D08:05
hrwmnasiadka: tripleo sorts out their issue for several days now. and it blocks whole work in our project08:06
mnasiadkahrw: yeah, I know :)08:06
mnasiadkahrw: it's funny that a RedHat employee proposes that :)08:06
hrwmnasiadka: Red Hat - we have space in name ;d08:06
stingrayzamorning k-a folks :) happy monday08:10
*** skramaja has joined #openstack-kolla08:12
*** skramaja_ has quit IRC08:12
hrwstingrayza: elo08:13
hrwyoctozepto: and that fluentd fix needs to go to stein too08:15
hrwand probably to other branches too08:15
*** shyamb has quit IRC08:24
hrwfuck08:27
yoctozeptohrw: all right, fluentd here I come08:27
hrwin short: we are fucked when it comes to CI08:27
hrwor we join fluentd fix with disabling voting for tripleo or we wait for tripleo to fix their issue08:28
hrw#$Tu324789t7389065720!08:28
mnasiadkahrw: wonder what the name will be in 3-4 years :D08:30
hrwmnasiadka: will see08:30
mnasiadkahrw: btw, I think tripleo doesn't use fluentd :)08:31
hrwmnasiadka: fluentd broke ubuntu CI jobs.08:31
mnasiadkahrw: you mean activesomething plugin for fluent? :)08:32
hrwyes08:32
hrws/plugin/gem08:32
hrwas it is gem used by one of plugins08:32
openstackgerritRadosław Piliszek proposed openstack/kolla master: Unbreak CI: fluentd and TripleO dependency  https://review.opendev.org/67714408:32
mnasiadkahrw: as yoctozepto probably proposed, let's dump k8s plugins in fluentd, we retired kolla-k8s long time ago08:32
yoctozeptomnasiadka: yeah, let's :-)08:32
yoctozeptosame second08:32
yoctozeptotripleo has some issue with td-agent atm08:33
mnasiadkayoctozepto: the same comes with all those kubernetes related images, we probably should deprecate them as soon as possible :)08:33
yoctozeptoso they ain't using it but it breaks them08:33
hrwyoctozepto: +2+208:33
yoctozeptoff logic :D08:33
yoctozeptomnasiadka: yeah, kubes should be gone08:33
yoctozeptoI think there is some note about them being obsoleted08:33
yoctozeptowill have to look08:33
yoctozeptonah, there is none08:35
yoctozeptobut they are not used in k-a at all08:35
yoctozeptoseems the deployment implementation never really started off08:36
yoctozeptoor maybe it's in this wicked kolla-kubernetes repo08:36
yoctozeptotoo afraid to check08:36
yoctozeptomnasiadka: https://review.opendev.org/67714408:36
yoctozeptohttps://review.opendev.org/#/q/project:openstack/kolla-kubernetes08:37
yoctozeptoscary08:37
yoctozeptohttps://review.opendev.org/565221 <- this what I saw :D08:38
yoctozeptohrw: hopefully only that unbound kubernetes plugins is causing ruby issues08:39
hrwyoctozepto: yes. so far only this one08:39
hrwchecked08:39
yoctozeptogr808:39
yoctozeptodougsz gave some some fluentd-plugins-to-remove examples08:42
yoctozeptohttps://review.opendev.org/#/c/675614/24/docker/fluentd/Dockerfile.j2@2808:42
yoctozeptosome more*08:42
yoctozeptohrw, mnasiadka: ^08:42
yoctozeptogrok is definitely used atm08:43
yoctozeptosecure forward is not08:43
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible stable/stein: Use default BGP speaker driver in bgp_dragent.ini  https://review.opendev.org/67714608:43
*** Blinkiz has joined #openstack-kolla08:56
BlinkizHello. To automate installation of OpenStack from baremetal, is bifrost recommended? Is that project going strong? Or should I put my time somewhere else?08:58
BlinkizIt seems like bifrost deploys ubuntu by default. It says so in documentation. I have gotten the feeling that kolla prefers centos. Right?08:59
yoctozeptoBlinkiz: kayobe deliverable guys, including our PTL, mgoddard, consider bifrost the right path09:00
yoctozeptokayobe is kolla-on-bifrost09:00
Blinkizyoctozepto, thank you for the guidance :)09:01
yoctozeptoBlinkiz: kolla supports ubuntu and centos equally09:01
yoctozeptosince some weeks ago kayobe is part of kolla project09:02
yoctozeptoso this channel is proper place to discuss it too09:02
Blinkizalright. thanks09:02
yoctozeptojust the relevant folks need to show up09:02
openstackgerritMark Goddard proposed x/kayobe-config stable/stein: Update .gitreview for Stein  https://review.opendev.org/67715109:03
openstackgerritMark Goddard proposed x/kayobe-config stable/stein: Update upper constraints for Stein  https://review.opendev.org/67715209:03
yoctozeptopersonally I did not use bifrost due to time constraints09:03
*** dpawlik has joined #openstack-kolla09:03
yoctozeptothe issue was with ipa not picking up my glorious fujitsu systems properly09:03
yoctozeptoadd infiniband driver support to that and you get: let's use what works already09:04
yoctozepto:D09:04
openstackgerritMichal Nasiadka proposed openstack/kolla master: Add fluentd_binary and fluentd_version labels  https://review.opendev.org/67641109:05
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Use fluentd image labels  https://review.opendev.org/67641309:06
Blinkizyeah. I got you. Mellanox has started to support Ubuntu 18.04 since a few weeks back for ASAP2. Accelerated Switchning and packet processing. So as long as openvswitch also support Ubuntu, then ubuntu is good to go for all my dependencies. My colleagues here at work prefer ubuntu so I may use that instead of centos.09:07
mnasiadkayoctozepto: so seems like fluentd needs a bit more love :)09:08
yoctozeptoBlinkiz: in here ceph runs over infiniband as this is our fastest network, the stock driver in centos had issues so we replace it with upstream's09:11
yoctozeptowhich works pretty well09:12
BlinkizI see..09:12
yoctozeptoand the process is already prepared in foreman09:12
yoctozeptoso bifrost would be no added value atm09:12
yoctozeptobut added baggage ;p09:12
yoctozeptomnasiadka: yeah, like all kolla09:13
yoctozeptoubuntu unbroken09:13
BlinkizI have previous wanted to use Foreman but after seeing a youtube video about Foreman I turned the other way. It seems over complicated, almost bloated.09:13
stingrayzaI think if you have hundreds of hosts running a few set templates, foreman is great. but it takes some time to get set up and is probably overkill in smaller deployments.09:16
stingrayzaI also don't like the fact that it is mostly puppet based, when I'm trying to ansible the world09:17
stingrayza(I know you can extend it to do some ansible stuff, but the core is still puppet)09:17
BlinkizI have a question about Bifrost. On there product page it says that Ubuntu is supported up to 16.04. Is this correct? Where is 18.04 support?09:17
yoctozeptostingrayza: the project "kill foreman in UCO" is officially approved by me09:17
yoctozepto(UCO is my unit at university)09:17
*** janki has quit IRC09:17
yoctozeptoso don't worry09:18
yoctozeptoit's there because it works09:18
stingrayzaXD XD09:18
yoctozeptoand the first to dump otherwise09:18
BlinkizOn Kayobe page it says Centos 7.6 and Ubuntu 16.04 is supported. So why not Ubuntu 18.04?09:19
BlinkizI mean, has development halted?09:19
yoctozeptoBlinkiz: I read their sources on, like, Saturday? and they have Ubuntu 18.04 in there09:19
*** tonythomas has joined #openstack-kolla09:19
yoctozeptojust no idea how well supported09:19
hrwBlinkiz: Ubuntu development? maybe09:19
yoctozeptoBlink: just try, let them know09:20
yoctozeptolet us (kayobe) know09:20
yoctozeptoour power comes from community :-)09:20
Blinkiz:)09:20
openstackgerritMark Goddard proposed x/kayobe-config-dev stable/stein: Update upper constraints for stein  https://review.opendev.org/67698709:23
openstackgerritMark Goddard proposed x/kayobe-config stable/stein: Update upper constraints for Stein  https://review.opendev.org/67715209:23
openstackgerritScott Solkhon proposed openstack/kolla master: Add effective and permitted capability to blackbox exporter  https://review.opendev.org/67715609:27
openstackgerritMerged openstack/kolla-ansible master: CI - remove unused required-projects  https://review.opendev.org/66663410:03
*** shyamb has joined #openstack-kolla10:11
*** hamdyk has joined #openstack-kolla10:13
*** dougsz has quit IRC10:17
openstackgerritMark Goddard proposed openstack/kolla-ansible master: CI: Test accessing dashboard  https://review.opendev.org/67641210:20
*** hjensas has joined #openstack-kolla10:24
yoctozeptoI thought Zull would finish checking that unbreak by now but it seems it went crazy and started jobs it shouldn't10:28
yoctozeptohttp://zuul.openstack.org/status10:28
yoctozeptoit queued all jobs we have10:28
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Set default timeout to 60 seconds for docker stop  https://review.opendev.org/66989710:29
*** Wasaac has quit IRC10:30
*** Wasaac has joined #openstack-kolla10:31
*** dougsz has joined #openstack-kolla10:35
*** shyamb has quit IRC10:36
*** skramaja has quit IRC10:39
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Set default timeout to 60 seconds for docker stop  https://review.opendev.org/66989710:50
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Set default timeout to 60 seconds for docker stop  https://review.opendev.org/66989710:54
*** shyamb has joined #openstack-kolla10:58
*** dave-mccowan has joined #openstack-kolla11:06
*** rohit02 has joined #openstack-kolla11:08
*** hjensas has quit IRC11:08
rohit02i am facinf error while configuring multinode kolla-ansible ""msg": "The conditional check 'kolla_external_vip_interface not in ansible_interfaces' failed."11:09
*** gfidente has joined #openstack-kolla11:10
rohit02has anyone face this issue earlier11:11
*** hjensas has joined #openstack-kolla11:16
*** dougsz has quit IRC11:17
*** kplant has joined #openstack-kolla11:20
yoctozeptorohit02: make sure the interface you specified is present on controller nodes you added to inventory11:20
yoctozeptospecifically kolla_external_vip_interface11:20
yoctozeptoit's a variable you should have set in globals.yml11:20
rohit02i have alrady set in global.yml file11:31
rohit02complete error is "Unable to look up a name or access an attribute in template string ({% if kolla_external_vip_interface not in ansible_interfaces %} True {% else %} False {% endif %}).\nMake sure your variable name does not contain invalid characters like '-': argument of type 'StrictUndefined' is not iterable\n\nThe error appears to have been in '/usr/local/share/kolla-ansible/ansible/roles/haproxy/tasks/precheck.yml': line 85, column 3, but11:32
rohit02may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Checking the kolla_external_vip_interface is present\n  ^ here\n"11:32
yoctozeptoah, that makes it a different thingy11:36
yoctozeptodoes your interface have '-' in its name?11:37
openstackgerritMark Goddard proposed openstack/kolla-ansible master: CI: Test accessing dashboard  https://review.opendev.org/67641211:42
mnasiadkayoctozepto: can we just have the tripleo non-voting & removing from gating change as a separate one, so it's easy to revert it when they start behaving?11:48
hrwmnasiadka: nope11:49
rohit02no '-' in interface name.....interface name is ens19211:49
hrwmnasiadka: we have two independent failures11:49
mnasiadkadamn11:49
*** janki has joined #openstack-kolla11:50
hrwmnasiadka: welcome in 'fetch random binary and ship' world11:50
mnasiadkahrw: well, blame ruby guys that they don't restrict their gems to check what ruby version you have installed...11:52
mgoddardBlinkiz: re kayobe, we support centos on the control plane hosts. Ubuntu is supported only for the host on which you run kayobe. We should update as I'm sure it would work on more recent ubuntu than 16.0411:52
mgoddardBlinkiz: if you're interested in trying it out, let me know11:53
mgoddardBlinkiz: this is an easy way to get started in a test environment: https://github.com/stackhpc/a-universe-from-nothing11:53
hrwmnasiadka: ? pip does not check either11:53
hrwmnasiadka: the only to blame imho are td-agent guys for shipping old ruby11:54
mnasiadkahrw: well true11:54
yoctozeptohrw, mnasiadka: then blame both11:54
hrwyoctozepto: can I blame whoever used td-agent instead of fluentd?11:55
*** dougsz has joined #openstack-kolla11:55
Blinkizmgoddard, thank you. :)11:55
yoctozeptohrw: or fluentd instead of td-agent :-)11:56
openstackgerritMichal Nasiadka proposed openstack/kolla master: Add fluentd_binary and fluentd_version labels  https://review.opendev.org/67641111:56
mnasiadkahrw: as long as OS distro maintainers don't give a shite about Ruby - we need to use td-agent :D11:57
mnasiadkaat least in CentOS case11:57
hrwmnasiadka: Debian and Ubuntu ship Ruby 2.5+11:57
mnasiadkahrw: so why use td-agent there instead of fluentd? :)11:57
yoctozeptore: many unnecessary jobs11:57
yoctozepto<frickler> yoctozepto: 2019-08-19 10:04:33,230 DEBUG zuul.layout: [e: d383ebbbfc1248a3b59af867de2eaba2] The configuration of job <Job kolla-ansible-ubuntu-source-ironic branches: {MatchAny:{BranchMatcher:master}} source: opendev/base-jobs/zuul.d/jobs.yaml@master#25> is changed by <Change 0x7f185d716908 openstack/kolla 677144,1>; ignoring file matcher11:57
yoctozepto<frickler> so it seems that indeed the reconfiguraton triggered by 666634 causes new jobs to be added because the file matcher is ignored in this special situation11:58
hrwmnasiadka: Debian uses fluentd only11:58
hrwmnasiadka: https://review.opendev.org/676909 waits for working CI11:58
yoctozeptoguys, are not we overengineering it - just build modern ruby toolkits in those containers11:58
yoctozeptothat's what they are for anyways11:59
mnasiadkahrw: debian x86_64 uses td-agent :)11:59
hrwmnasiadka: no, it just was not built so no one noticed ;D11:59
yoctozeptoscrap "ruby" meant any11:59
mnasiadkayoctozepto: well, that's against some common logic in this project :)11:59
mnasiadkayoctozepto: building something that distro should deliver :)12:00
yoctozeptowe should rethink the principles then ;-)12:00
*** weshay_pto is now known as weshay12:00
*** lemko has quit IRC12:00
hrwheh... distro deliver...12:00
hrwwhen hell froze and pigs will fly12:01
mnasiadkaI'm not saying we should not - just saying what was the approach until now :)12:01
*** henriqueof has joined #openstack-kolla12:01
hrwmnasiadka: when I joined Kolla it was even stronger12:02
mnasiadkahrw: sometimes all those if/elif/endif statements 700 hundred times in Dockerfile.j2 give me headaches :)12:02
mnasiadkahrw: especially in fluentd Dockerfile :)12:02
hrwmnasiadka: I think that we should be able now to remove all those "sorry, no binaries for this distro/arch/buildtype" things as we have UNBUILDABLE_IMAGES now12:03
mnasiadkayeah12:03
mnasiadkathis is an infinite journey...12:03
mnasiadkayoctozepto: the number of jobs ignited by the CI unblocker is crazy ;)12:06
yoctozeptomnasiadka: they appeared later due to merge in k-a repo12:06
yoctozeptoZuul quirk12:06
openstackgerritChandan Kumar (raukadah) proposed openstack/kolla master: Include iputils package to tempest container  https://review.opendev.org/67719112:07
yoctozeptomnasiadka, hrw: I don't know if we shouldn't satisfy AJaeger in a follow-up12:11
yoctozeptobecause it takes time to check all jobs12:11
yoctozeptoand the tripleo job finishes quickly with that failure12:12
hrwyoctozepto: add a comment12:12
mnasiadkayoctozepto: just do it, it's like over 3 hours for a CI run :)12:13
*** vmixor has joined #openstack-kolla12:13
yoctozeptohrw, mnasiadka: guys, you want me to make it run again or not because your signals were unclear :D12:13
hrwyoctozepto: add a comment for AJaeger and merge12:14
yoctozeptohrw: you mean as in in-gerrit reply?12:15
hrwyes12:15
hrw 16 files changed, 28 insertions(+), 160 deletions(-)12:16
hrwsmall docker/ cleanup12:16
hrwin progress12:16
hrw 26 files changed, 71 insertions(+), 285 deletions(-)12:20
hrw 45 files changed, 116 insertions(+), 487 deletions(-)12:28
yoctozeptogeeez, it grows :O12:29
yoctozeptohrw, mnasiadka: https://review.opendev.org/67714412:30
yoctozeptoZuul is finishing12:30
yoctozeptoplease slap w+1 when done12:30
yoctozeptomy comment is there12:30
yoctozeptoi will be commuting for some time now12:30
yoctozeptotill later12:30
hrwok12:31
hrw+W given12:36
goldyfruito/12:36
*** spsurya has quit IRC12:43
hrw 82 files changed, 167 insertions(+), 713 deletions(-)12:43
hrwthat will be fun to review12:47
*** shyamb has quit IRC12:48
hrwlet push12:49
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Cleanup: drop checks handled by UNBUILDABLE_IMAGES  https://review.opendev.org/67719412:50
hrwin reality it is only 9 added lines - rest are indents12:55
*** dougsz has quit IRC12:59
*** dougsz has joined #openstack-kolla13:03
openstackgerritMerged openstack/kolla master: Unbreak CI: fluentd and TripleO dependency  https://review.opendev.org/67714413:18
hrwYES!13:19
hrw6 rechecks sent13:19
*** janki has quit IRC13:20
openstackgerritMarcin Juszkiewicz proposed openstack/kolla stable/stein: Unbreak CI: fluentd and TripleO dependency  https://review.opendev.org/67720213:20
openstackgerritMarcin Juszkiewicz proposed openstack/kolla stable/stein: Unbreak CI: fluentd and TripleO dependency  https://review.opendev.org/67720213:23
hrwyoctozepto: tripleo in gate dropped13:24
*** rohit02 has quit IRC13:27
openstackgerritChandan Kumar (raukadah) proposed openstack/kolla master: Include iputils package to tempest container  https://review.opendev.org/67719113:36
openstackgerritChandan Kumar (raukadah) proposed openstack/kolla master: Include iputils package to tempest container  https://review.opendev.org/67719113:37
*** BjoernT has joined #openstack-kolla13:52
mgoddardsweet13:54
hrwmgoddard: take a look at "Cleanup: drop checks handled by UNBUILDABLE_IMAGES"  https://review.opendev.org/677194 one13:55
hrwwithout even commenting in review - just would like to see first opinion13:56
mgoddardhrw: I think it makes sense. Only downside is that it's no longer obvious from the Dockerfile that e.g binary support is lacking14:03
*** dougsz has quit IRC14:05
mnasiadkamgoddard: then it's the famous support matrix :)14:05
mgoddardtrue14:05
mnasiadkaI'll start it tomorrow14:05
BlinkizAnyone here know about a script/project/container that can automatically get TLS certificate from lets Encrypt every ~third month? Or is it something I need to code together myself in ansible?14:11
hrwBlinkiz: certbot you mean? in cron?14:11
BlinkizAnd of course work with kolla-Ansible. Meaning giving haproxy the right cert(s) on all control nodes14:12
Blinkizhrw, certbot/cron is on a single machine, kolla-ansible control nodes is many machines. So.. yeah.. asking for guidance if exist any existing tool.14:14
openstackgerritIsaac Prior proposed openstack/kolla-ansible master: Removes monasca_grafana persistent volume  https://review.opendev.org/67721114:14
openstackgerritChandan Kumar (raukadah) proposed openstack/kolla stable/stein: Use openstack-tempest-all for rpm tempest container  https://review.opendev.org/67721214:16
*** cah_link has joined #openstack-kolla14:16
*** dpawlik has quit IRC14:20
openstackgerritChandan Kumar (raukadah) proposed openstack/kolla stable/stein: Use openstack-tempest-all for rpm tempest container  https://review.opendev.org/67721214:20
openstackgerritIsaac Prior proposed openstack/kolla-ansible master: Removes monasca_grafana persistent volume  https://review.opendev.org/67721114:21
*** KeithMnemonic has joined #openstack-kolla14:25
*** dougsz has joined #openstack-kolla14:26
*** cah_link has quit IRC14:27
hrwBlinkiz: and thanks for reminding - cleaned up my letsencrypt certs14:29
kplantthat'd be a cool integration, kolla and letsencrypt via certbot14:30
yoctozeptohrw: thx14:34
*** hamdyk has quit IRC14:34
*** BjoernT_ has joined #openstack-kolla14:45
yoctozeptomgoddard: replied re that proposed bifrost change14:47
*** BjoernT has quit IRC14:47
BlinkizDoes OpenStack/Kolla have a way to distribute files between control nodes? Am planning my implementation of Let's Encrypt certificate. :)14:49
BlinkizI mean.. not by running Kolla deploy. I mean automatically.14:50
openstackgerritRadosław Piliszek proposed openstack/kolla master: CI: follow-up on TripleO demotion - no gating  https://review.opendev.org/67722514:51
yoctozeptoBlinkiz: you can have custom mounted docker volumes pointing to host directory14:54
yoctozeptouse ansible to deploy your certs14:54
yoctozeptoremember to signal to haproxy they got changed14:54
Blinkizyoctozepto, yeah, but which control node is master in keepalived? Only that one can get a new cert with ACME http -01challenge response.14:55
kplantyoctozepto: could probably `docker restart haproxy` no?14:55
yoctozeptokplant: downtime?14:55
kplantis an haproxy restart that noticeable?14:56
kplantwell.. a successful haproxy restart :p14:56
yoctozeptoBlinkiz: ah, you do it this way, I see, hmm14:56
kplantBlinkiz: whichever node has the VIP is the current 'master'14:56
yoctozeptokplant: if you have requests coming at all times...14:56
yoctozeptoso yes14:56
kplantyou could also snipe VRRP and learn it that way14:56
yoctozeptoeasier to check the VIP14:57
yoctozeptoansible playbook run against all the machines14:57
yoctozeptogetting facts14:57
yoctozeptoset which has the VIP address14:57
Blinkizyoctozepto, I can for my own domain use DNS verification but customers I do not have control over. Am planning to just ask them to point there subdomain to a A record. With this I can get certificates for customers domains if I use http-01 challenge response14:57
yoctozeptodelegate certbot to it14:57
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Enable TripleO CI as voting one again  https://review.opendev.org/67722814:58
yoctozeptoBlinkiz: me and kplant prepared a solution for you :-)14:58
hrwso we will not forget to enable it back ;D14:58
hrwyoctozepto: it is on top of your one14:58
Blinkizyoctozepto, yeah, it is possible if I run certbot (logic) on a management node outside the cluster. Am thinking about a way to let the cluster solve it14:59
yoctozeptohrw: good thinking15:00
BlinkizBut the more I think about it the more I see that it needs to be an outside machine solving the renewal of tls certificates.15:00
Blinkizprobably my deploy machine where kolla-ansible is running from15:00
yoctozeptoBlinkiz: nah, it can run from inside the cluster15:00
hrwhttps://review.opendev.org/#/c/674240/ (Debian CI) got green from Zuul. +2 +W please15:01
yoctozeptoyou could orchestrate all crons to check if they are master15:01
yoctozeptoand only then proceed15:01
hrwit's deps are in zuul15:01
Blinkizyoctozepto, okay. What is your suggestion how I get the new certificate to the control nodes that is not currently master?15:01
yoctozeptocloudnull: how bad is that tripleo ci issue - I see you, folks, are rechecking and rechecking but it fails on same jobs15:01
yoctozeptofyi, we disable tripleo in our gate to unblock ourselves but we are staying vigilant15:02
cloudnullits a frustrating one. not related to that change.15:02
yoctozeptoBlinkiz: rsync15:02
cloudnullit may be worth making that job non-voting15:02
yoctozeptocloudnull: yeah, I see it's more general15:02
cloudnullonce we get stable again we can re-enable it15:03
yoctozeptocloudnull: ping us please if we miss it15:03
cloudnull++15:03
yoctozeptohrw prepared the tripleo glory restoration patch15:03
cloudnullI'll make a note to make that job voting again sometime in the after-now15:03
yoctozeptocloudnull: https://review.opendev.org/67722815:03
yoctozeptoyou can recheck that once it is fixed15:04
yoctozeptowe care for your comfort15:04
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Use fluentd image labels  https://review.opendev.org/67641315:04
cloudnull+115:04
yoctozeptoyour friendly kolla team15:04
cloudnullthank you yoctozepto sorry for the issues15:04
yoctozeptono problem, we all make mistakes from time to time15:04
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Use fluentd image labels  https://review.opendev.org/67641315:07
yoctozeptohrw: did you try building bifrost with that mariadb change15:10
yoctozepto?15:10
hrwyoctozepto: not15:18
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: Copy Nova role as a basis for the Nova cell role  https://review.opendev.org/67565815:18
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: [WIP] Factor out nova-cell role  https://review.opendev.org/67565915:18
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: Copy Nova role as a basis for the Nova cell role  https://review.opendev.org/67565815:19
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: [WIP] Factor out nova-cell role  https://review.opendev.org/67565915:19
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: Minimal support for deploying multiple instances of MariaDB  https://review.opendev.org/61975615:22
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: Copy Nova role as a basis for the Nova cell role  https://review.opendev.org/67565815:22
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: [WIP] Factor out nova-cell role  https://review.opendev.org/67565915:22
*** factor has quit IRC15:32
*** factor has joined #openstack-kolla15:32
*** ivve has quit IRC15:33
*** factor has quit IRC15:40
*** factor has joined #openstack-kolla15:41
openstackgerritChandan Kumar (raukadah) proposed openstack/kolla master: Include iputils package to tempest container  https://review.opendev.org/67719115:43
*** shyamb has joined #openstack-kolla15:47
openstackgerritMerged openstack/kolla master: Fixes wrong local_setting.py file issue for Ubuntu binary deployment.  https://review.opendev.org/66840915:50
hrwmnasiadka: can you rebase https://review.opendev.org/#/c/675614 patch? will be easier to probably rebase https://review.opendev.org/#/c/675614/ on top16:00
hrwhttps://review.opendev.org/676909 I meant (at the end)16:00
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: debian: add some images to unbuildable list  https://review.opendev.org/67690516:01
hrwyoctozepto: your comment accepted ^^16:01
mnasiadkahrw: yeah, will rebase first thing tomorrow morning, I'm rather focused now on https://review.opendev.org/#/c/676413/9 and the dependent change :)16:03
hrwsure16:03
*** gfidente has quit IRC16:03
hrwmnasiadka: I think that after merging all those fluentd changes it will look like new... so much rewriting16:04
mnasiadkahrw: might be :)16:05
*** rpittau is now known as rpittau|afk16:07
hrwif rpm then td-agent unless !x86 elif deb then fluentd unless (ubuntu x86);fi16:07
*** icarusfactor has joined #openstack-kolla16:07
yoctozeptohrw: like poop sprinkled with perfumes you mean? ;-)16:07
hrwyoctozepto: s/sprinkled with/drawn in/16:08
yoctozeptoagreed16:08
hrwmnasiadka: will steal your fluentd_user stuff to https://review.opendev.org/#/c/676909/16:08
vmixorHi, I have created some changes to kolla and kolla-ansible to support deploy of Stein on Debian Buster. I used code from master branches and packages from Stein backports (http://buster-stein.debian.net/debian). Now I have deployment in quite operational state, I think. I have been instructed to try push my changes to upstream.16:08
vmixorTherefore I would be glad for some advice and/or guidance about next steps I should do.16:08
*** factor has quit IRC16:09
yoctozeptohrw, vmixor: hereby I link you with the debian bound16:09
*** henriqueof has quit IRC16:09
hrwvmixor: the problem is that those are Stein packages. so code should go to stable/stein. but that's also new feature while we do not backport features16:09
hrwyoctozepto: I asked vmixor to bring it to public channel16:10
yoctozeptoah, understood16:10
yoctozeptoI don't mind backporting to the latest stable as long as we can be *sure* we are not affecting stability16:11
yoctozeptoI feel deployment projects should not really fully freeze their deployment solution for the latest stable16:11
hrwvmixor: send patches to gerrit16:12
yoctozeptoas master deploying master is probably fencing off many people from use16:12
mgoddardI don't think we should get into the habit of backporting features16:12
yoctozeptovmixor, hrw: yeah, master will accept them probably16:12
mgoddardif you need a new feature on an old branch, maintain a fork16:13
hrwmgoddard: agreed16:13
yoctozeptook then, I'm outvoted :D16:13
yoctozeptothen debian is train+ only16:13
mgoddardstable branches should be stable16:13
hrwvmixor: I more look at those patches as 'let us see how good Debian/binary target works'16:13
yoctozeptohrw: you did not mention binary16:14
yoctozeptotill now16:14
hrwyoctozepto: vmixor has Debian/binary with Stein packages.16:14
yoctozeptoI see16:14
yoctozeptohmm16:14
vmixoryoctozepto, hrw: yes, I have used binary packages from Stein backports for Buster16:15
hrwyoctozepto: most of work can be merged and kept in Train so will be useful when Train packages arrive16:15
yoctozeptovmixor: is debian planning to support train soon? are you building some packages off master branches?16:16
hrwyoctozepto: most of packages are around RC or after release16:16
yoctozeptowe have such packages for centos iirc16:16
yoctozeptonot sure about ubuntu though16:16
yoctozeptoI see16:16
hrwyoctozepto: ubuntu and centos do builds from git. Debian OpenStack team iirc not16:16
yoctozeptobut could they? that would make it easier to introduce k-a debian-binary16:17
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: fluentd: there is no 'td-agent' for Debian/x86-64  https://review.opendev.org/67690916:17
yoctozeptoso far there don't seem to be any breaking changes between stein and train noticed so it might be only a hypothetical issue16:18
hrwyoctozepto: ask on #debian-openstack @oftc?16:18
yoctozeptohrw: too lazy ;-)16:18
*** igordc has joined #openstack-kolla16:18
mnasiadkahrw: sure16:20
hrwvmixor: once you send a patch I will review, comment16:21
vmixorhrw: ok, I will create patchset for current master16:23
openstackgerritMerged openstack/kolla stable/stein: Unbreak CI: fluentd and TripleO dependency  https://review.opendev.org/67720216:30
yoctozeptohrw, mnasiadka, mgoddard: https://review.opendev.org/677225 and be done with this stuff for now (master & stein)16:32
yoctozepto(stein just merged)16:32
*** shyamb has quit IRC16:43
hrwvmixor: thanks16:46
vmixorhrw: thank you too.16:49
vmixorhrw: one more question: Is threre some blueprint for Stein support for Debian?16:50
*** jonaspaulo has joined #openstack-kolla16:54
openstackgerritMerged openstack/kolla master: Add pciutils package to cyborg-agent and zun-compute  https://review.opendev.org/67693217:01
*** henriqueof has joined #openstack-kolla17:17
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: [WIP] Factor out nova-cell role  https://review.opendev.org/67565917:19
*** raukadah has joined #openstack-kolla17:24
raukadahHello Kolla dev!17:24
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Implement TLS encryption for internal endpoints  https://review.opendev.org/66355517:26
openstackgerritChandan Kumar (raukadah) proposed openstack/kolla stable/stein: Use openstack-tempest-all for rpm tempest container  https://review.opendev.org/67721217:26
raukadahplease have a look at these reviews https://review.opendev.org/#/c/677191/ and https://review.opendev.org/#/c/677212/17:26
raukadahthanks@17:26
openstackgerritDoug Szumski proposed openstack/kolla master: WIP: Support DB bootstrap from Nova Conductor container  https://review.opendev.org/67726317:36
openstackgerritDoug Szumski proposed openstack/kolla master: WIP: Support DB bootstrap from Nova Conductor container  https://review.opendev.org/67726317:43
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: [WIP] Factor out nova-cell role  https://review.opendev.org/67565917:44
openstackgerritMerged openstack/kolla-ansible master: Removes monasca_grafana persistent volume  https://review.opendev.org/67721117:46
*** icarusfactor has quit IRC17:53
hrwvmixor: no18:25
* hrw out18:25
hrwsee you in a week - will be back on 27th (yoctozepto, mnasiadka, mgoddard)18:26
mnasiadkahrw: have fun ;)18:26
*** strigazi has quit IRC18:37
*** strigazi has joined #openstack-kolla18:38
*** dougsz has quit IRC19:03
yoctozeptohrw: take care ;-)19:06
*** ivve has joined #openstack-kolla19:09
*** dougsz has joined #openstack-kolla19:21
*** factor has joined #openstack-kolla19:22
*** icarusfactor has joined #openstack-kolla19:44
*** kplant has quit IRC19:44
*** factor has quit IRC19:46
*** dougsz has quit IRC20:29
*** pomac has joined #openstack-kolla20:30
*** vmixor has quit IRC20:38
*** BjoernT_ has quit IRC21:36
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Cleanup: drop checks handled by UNBUILDABLE_IMAGES  https://review.opendev.org/67719421:49
hrwyoctozepto: if you have time then please rebase https://review.opendev.org/672700 (Docker: daemon.json). your zun changes broke it ;D21:49
* hrw -> out21:49
*** jonaspaulo has quit IRC22:19
*** tonythomas has quit IRC22:21
*** dave-mccowan has quit IRC22:22
*** dave-mccowan has joined #openstack-kolla22:25
*** dave-mccowan has quit IRC22:32
*** ivve has quit IRC22:34
*** KeithMnemonic has quit IRC22:49
*** EmilienM|pto has quit IRC22:54
*** EmilienM has joined #openstack-kolla22:56
*** KeithMnemonic has joined #openstack-kolla23:14

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!