Wednesday, 2017-12-06

*** dardelean has joined #openstack-kolla00:01
*** Pavo has quit IRC00:01
*** Pavo has joined #openstack-kolla00:03
*** affie_ has quit IRC00:06
*** lrensing has quit IRC00:09
*** zhubingbing has joined #openstack-kolla00:09
*** signed8bit_Zzz is now known as signed8b_00:10
*** Pavo has quit IRC00:13
*** zhubingbing has quit IRC00:14
*** threestrands has quit IRC00:25
*** Guest1563 has quit IRC00:26
*** signed8b_ is now known as signed8bit_Zzz00:36
*** dardelean has quit IRC00:45
*** hongbin has quit IRC00:49
*** tovin07_ has joined #openstack-kolla00:49
*** hieulq has joined #openstack-kolla00:50
*** kelv_ has quit IRC00:54
*** zhubingbing has joined #openstack-kolla00:56
*** zhubingbing has quit IRC01:00
*** itlinux has quit IRC01:00
*** dardelean has joined #openstack-kolla01:01
*** zshi has joined #openstack-kolla01:05
*** vhosakot has quit IRC01:05
*** dardelean has quit IRC01:06
*** yangyapeng has joined #openstack-kolla01:14
*** lukepatrick has joined #openstack-kolla01:15
*** goldyfruit has joined #openstack-kolla01:23
*** kevzha01 has joined #openstack-kolla01:28
*** kevzha01 is now known as kevinz01:28
*** goldyfruit has quit IRC01:32
*** goldyfruit has joined #openstack-kolla01:33
*** goldyfruit has quit IRC01:34
*** kiennt26 has joined #openstack-kolla01:36
*** tovin07_ has quit IRC01:36
*** caowei has joined #openstack-kolla01:39
*** keper7 has quit IRC01:40
*** genek has joined #openstack-kolla01:40
*** tovin07_ has joined #openstack-kolla01:46
*** signed8bit_Zzz is now known as signed8b_01:46
*** jbrooks has joined #openstack-kolla01:49
*** zhurong has joined #openstack-kolla01:49
openstackgerritJeffrey Zhang proposed openstack/kolla master: Add squid cache proxy to speed up build  https://review.openstack.org/52145001:57
*** oanson has quit IRC02:12
*** signed8b_ is now known as signed8bit_Zzz02:12
*** oanson has joined #openstack-kolla02:12
*** Pavo has joined #openstack-kolla02:13
openstackgerritJeffrey Zhang proposed openstack/kolla master: Add squid cache proxy to speed up build  https://review.openstack.org/52145002:16
*** pbourke_ has quit IRC02:25
*** dave-mccowan has joined #openstack-kolla02:26
*** zhangfei has joined #openstack-kolla02:26
*** pbourke_ has joined #openstack-kolla02:27
*** dave-mccowan has quit IRC02:30
*** hieulq has quit IRC02:32
*** unicell has quit IRC02:33
*** Pavo has quit IRC02:33
*** hieulq has joined #openstack-kolla02:34
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Add note about overwriting policy.json file  https://review.openstack.org/51118602:37
*** Pavo has joined #openstack-kolla02:44
daidvberendt, Overwriting policy.json file note patch is ready to go :D #link https://review.openstack.org/51118602:44
*** itlinux has joined #openstack-kolla02:45
*** dave-mccowan has joined #openstack-kolla02:45
*** itlinux has quit IRC02:53
*** signed8bit_Zzz has quit IRC03:00
*** signed8bit has joined #openstack-kolla03:01
*** zhurong has quit IRC03:02
*** ArminderSingh has quit IRC03:03
*** codenamelxl has joined #openstack-kolla03:05
*** ArminderSingh has joined #openstack-kolla03:07
*** zhurong has joined #openstack-kolla03:10
*** caowei has quit IRC03:14
*** caowei has joined #openstack-kolla03:14
*** goldyfruit has joined #openstack-kolla03:17
*** zshi has quit IRC03:27
openstackgerritChason Chan proposed openstack/kolla-ansible master: Add vitrage ansible role  https://review.openstack.org/43260803:33
*** kristaps_ has joined #openstack-kolla03:57
*** lvdombrkr has quit IRC03:58
*** andyjones2001 has quit IRC03:59
*** ntpttr_ has quit IRC03:59
*** ntpttr_ has joined #openstack-kolla03:59
*** mgkwill has quit IRC04:07
*** mgkwill_ has joined #openstack-kolla04:07
*** caowei has quit IRC04:12
*** caowei has joined #openstack-kolla04:12
openstackgerritJeffrey Zhang proposed openstack/kolla master: [DNM]Test squash docker images layers for current built image  https://review.openstack.org/52587104:20
*** zhurong has quit IRC04:21
*** dave-mccowan has quit IRC04:23
openstackgerritJeffrey Zhang proposed openstack/kolla master: [DNM]Test squash docker images layers for current built image  https://review.openstack.org/52587104:24
*** Pavo has quit IRC04:27
*** lukepatrick has quit IRC04:29
*** david-lyle has joined #openstack-kolla04:32
*** janki has joined #openstack-kolla04:33
*** jbrooks has quit IRC04:40
*** tovin07_ has quit IRC04:41
*** daidv has quit IRC04:41
*** duonghq has quit IRC04:41
*** daidv has joined #openstack-kolla04:42
*** tovin07_ has joined #openstack-kolla04:42
*** duonghq has joined #openstack-kolla04:42
*** david-lyle has quit IRC04:42
*** goldyfruit has quit IRC04:48
*** janki has quit IRC04:54
*** janki has joined #openstack-kolla05:01
inc0Jeffrey4l: squash looks interesting05:04
Jeffrey4linc0, yes. tested locally,which works.05:04
inc0how long does it take to squash layers?05:05
inc0we could add cleanups if squash works05:05
Jeffrey4linc0, less than 1 min for certain image.05:06
Jeffrey4linc0, yes.05:06
*** coolsvap has joined #openstack-kolla05:07
inc0tomorrow I'll fix ceph thing and hopefully this will help CI05:08
Jeffrey4lcool.05:08
Jeffrey4lbtw, i am trying to bump ceph to Luminous in Q.05:08
inc0yeah that's the plan05:08
inc0but we still need to have fix pike05:08
*** zhurong has joined #openstack-kolla05:08
Jeffrey4lyes. ubuntu ;(05:09
Jeffrey4lhttps://review.openstack.org/#/q/I7e9fd2760ae3202f3a84833bb65edfdcffa67fe405:09
Jeffrey4lcould you review these two patches? inc005:09
*** threestrands has joined #openstack-kolla05:09
*** threestrands has quit IRC05:09
inc0done05:10
Jeffrey4lthanks.05:10
inc0for some reason master periodic job didn't run for last couple of days05:10
inc0I'll check with infra tomorrow05:11
Jeffrey4lok. maybe i broke something ;(05:11
Jeffrey4land this week we need release b205:12
Jeffrey4lhttp://logs.openstack.org/periodic/git.openstack.org/openstack/kolla/master/kolla-build-centos-source/05:13
Jeffrey4linc0, period job run.05:14
inc0well this ran05:14
*** skramaja has joined #openstack-kolla05:14
inc0well build works but not publish ones05:15
inc028/11 was last time publish job ran for master05:15
Jeffrey4lok. guess one of my patch break it.05:16
inc0look at it plz, I'll check it tomorrow myself05:16
inc0pike seems to be ok05:17
inc0ocata*05:17
inc0pike is also 8 days old05:17
Jeffrey4lhttp://logs.openstack.org/periodic/git.openstack.org/openstack/kolla/master/kolla-build-centos-source/cf1edb2/job-output.txt.gz#_2017-12-05_07_02_19_46801505:18
Jeffrey4lhere is failed.05:19
Jeffrey4lfound the root cause: http://logs.openstack.org/periodic/git.openstack.org/openstack/kolla/master/kolla-build-centos-source/cf1edb2/ara/result/025313a9-653d-42d1-9dd8-af49bb1058db/05:23
Jeffrey4lbut no idea why and how to fix.05:23
inc0I'll check with infra tomorrow05:26
Jeffrey4linc0, i am checking this now ;)05:30
Jeffrey4ldmsimard, you may like this https://review.openstack.org/52587105:30
Jeffrey4land here is the result: http://logs.openstack.org/71/525871/2/check/kolla-build-ubuntu-binary/48309e8/logs/docker-info.txt.gz05:31
*** k_mouza has joined #openstack-kolla05:32
SamYapleJeffrey4l: https://github.com/openstack/kolla/blob/master/.zuul.yaml#L4505:34
SamYaplei beleive you need to define the secret on the job itself05:34
SamYaplesecrets are very tricky about inheriting and stuff05:34
Jeffrey4lyes. it should be root issue.05:35
Jeffrey4li will revert the patch.05:36
SamYapleah i see yo ujust made that change05:36
Jeffrey4lyes..05:37
*** k_mouza has quit IRC05:37
openstackgerritJeffrey Zhang proposed openstack/kolla master: [DNM]Test squash docker images layers for current built image  https://review.openstack.org/52587105:41
*** mdnadeem has joined #openstack-kolla05:43
*** janki has quit IRC05:44
openstackgerritJeffrey Zhang proposed openstack/kolla master: [DNM] Test without squash  https://review.openstack.org/52588205:46
*** zhangfei has quit IRC05:53
*** janki has joined #openstack-kolla05:56
*** andyjones2001 has joined #openstack-kolla05:57
openstackgerritJeffrey Zhang proposed openstack/kolla master: Use seperate publish jobs for period jobs  https://review.openstack.org/52588406:00
openstackgerritJeffrey Zhang proposed openstack/kolla master: Use separate publish jobs in periodic pipeline  https://review.openstack.org/52588406:03
SamYapleJeffrey4l: thats a good patch. ive been recommending that layout from day 1 :D06:04
SamYapleits what we do in loci06:05
Jeffrey4lSamYaple, ah, i'd like the solution ;D06:05
Jeffrey4lSamYaple, how do you squash layers in loci?06:05
Jeffrey4lthe same tools? or?06:05
SamYapleJeffrey4l: we dont!06:06
SamYapleno need to squash layers when you dont build with layered images06:06
*** zhangfei has joined #openstack-kolla06:06
Jeffrey4li see.06:06
SamYaplei meant the publish jobs patch06:06
SamYapleill give your squash layers patch a look though06:07
Jeffrey4lhrm. sorry for mis-understand.06:07
*** zshi has joined #openstack-kolla06:07
Jeffrey4lSamYaple, re jobs. yes. i thought merge the jobs may works. ;p06:07
SamYapleah yea docker-squash. i played with that a long time ago06:07
SamYapleyou can certainly do that, but youre largest layer ends up being openstack-base, because of all the build tools06:08
Jeffrey4llatest patch is testing the layers and total size. and check whether it will generated less size of registry.06:08
SamYapleyou would have to squash *through* the parent layers to see real improvements06:08
*** janonymous has joined #openstack-kolla06:09
Jeffrey4lSamYaple, in that way, the total size will be increased.06:09
Jeffrey4lone image one layer. and those images shared nothing.06:09
SamYapleJeffrey4l: i know. thats why we do it a completely different way in LOCI06:10
SamYaplethe individual layered images in kolla are fairly well optmoized at this point06:10
SamYapleits just that the totalsize gets really large06:10
openstackgerritMerged openstack/kolla stable/ocata: Change more repo url to infra mirror  https://review.openstack.org/52549706:11
Jeffrey4lSamYaple, this patch can support one image on layer. just remove the "--form <layer>" parameter from docker-squash.06:11
Jeffrey4li will implement this, and may someone like it.06:11
SamYapleyes. just be careful with thebuild times, its doubles the IO needed and makes builds take alot longer06:12
Jeffrey4lyes. btw, docker-squash support use /dev/shm for the in-middle file. so that will be helpful for this .06:13
SamYapleJeffrey4l: cool!06:14
openstackgerritXinliang Liu proposed openstack/kolla-ansible master: Make qemu use nova user on debian  https://review.openstack.org/52589106:14
openstackgerritJeffrey Zhang proposed openstack/kolla master: [DNM]Test squash all the layers  https://review.openstack.org/52589206:15
*** rwsu has quit IRC06:17
openstackgerritJeffrey Zhang proposed openstack/kolla master: [DNM]Test squash docker images layers for current built image  https://review.openstack.org/52587106:21
openstackgerritJeffrey Zhang proposed openstack/kolla master: [DNM]Test squash all the layers  https://review.openstack.org/52589206:21
masbergood afternoon, is there any roadmap to support iluminus?06:21
openstackgerritJeffrey Zhang proposed openstack/kolla master: [DNM] Test without squash  https://review.openstack.org/52588206:21
masberand aprox date?06:22
Jeffrey4lmasber, we will support luminous in Q06:22
masberJeffrey4l, sorry what does "Q" means?06:22
masberah queens06:22
Jeffrey4lmasber, yes ;D06:22
masbersorry :)06:22
masberso in six months?06:23
Jeffrey4lno need to sorry. i should say Q cycle06:23
Jeffrey4lmasber, no more six months. Q will be release in March next year.06:23
masberoh06:24
masberok ok06:24
*** xinliang has quit IRC06:27
*** rwsu has joined #openstack-kolla06:30
openstackgerritXinliang Liu proposed openstack/kolla master: Revert "nova-libvirt: fix kvm permission issue"  https://review.openstack.org/52590006:35
*** janki has quit IRC06:36
*** janki has joined #openstack-kolla06:38
*** xinliang has joined #openstack-kolla06:39
openstackgerritcaowei proposed openstack/kolla master: Remove designate-pool-manager container  https://review.openstack.org/52590606:45
*** gkadam has joined #openstack-kolla06:54
*** cah_link has joined #openstack-kolla07:14
*** jascott1 has joined #openstack-kolla07:33
*** jascott1 has quit IRC07:37
*** athomas has joined #openstack-kolla07:51
*** wojdec has joined #openstack-kolla08:08
*** shardy has joined #openstack-kolla08:08
*** dardelean has joined #openstack-kolla08:14
*** wojdec1 has joined #openstack-kolla08:26
*** wojdec has quit IRC08:28
hrwmorning08:29
hrwsomeone know tool which can do "docker rmi" on regexps?08:29
hrwlike: docker rmi kolla/*08:29
*** affie has joined #openstack-kolla08:30
*** zhurong has quit IRC08:33
SamYaplehrw: `docker images | awk '/kolla/ {print $1}' | xargs docker rmi`08:33
hrwthanks08:34
*** affie has quit IRC08:35
*** cah_link has quit IRC08:37
*** codenamelxl has quit IRC08:37
*** genek has quit IRC08:39
*** mgoddard has joined #openstack-kolla08:42
*** wlmbasson has quit IRC08:45
*** wlmbasson has joined #openstack-kolla08:48
*** bubuwifi_ has quit IRC08:48
*** mewald has joined #openstack-kolla08:49
*** b_bezak has joined #openstack-kolla08:49
mewaldmorning, boostrap-servers fails with "The error was: 'dict object' has no attribute u'ansible_ens3'" Is this a know issue or just me using it wrong?08:49
duonghqmewald, do you have NIC ens3?08:51
*** bubuwifi_ has joined #openstack-kolla08:52
*** zhurong has joined #openstack-kolla08:53
mewaldyeah, if I run "ansible -i /etc/kolla/inventory control-0 -m setup" I can see the variable08:55
mewaldthis is the full error: https://gist.github.com/anonymous/bc46c11f9443980a56c12d9340bc993e08:55
mewaldit happens in the jinja template for /etc/hosts08:56
mewaldwell looks like the code has changed :D its "blockinfile" now08:56
*** gfidente has joined #openstack-kolla08:57
mewaldthe fact is there: https://gist.github.com/anonymous/22feccd5918cc1af1e0fba3aec0e211a09:00
*** hieulq_ has joined #openstack-kolla09:01
*** zhangfei has quit IRC09:02
*** hieulq_ has quit IRC09:04
*** hieulq has quit IRC09:04
*** dardelean_ has joined #openstack-kolla09:04
*** hieulq has joined #openstack-kolla09:06
*** hieulq_ has joined #openstack-kolla09:07
*** hieulq has quit IRC09:08
*** hieulq_ has quit IRC09:08
*** jmccarthy has joined #openstack-kolla09:08
*** ktibi has joined #openstack-kolla09:09
*** gfidente has quit IRC09:09
*** k_mouza has joined #openstack-kolla09:09
*** hieulq has joined #openstack-kolla09:10
*** gfidente has joined #openstack-kolla09:10
*** gfidente has quit IRC09:10
*** gfidente has joined #openstack-kolla09:10
*** bubuwifi__ has joined #openstack-kolla09:12
openstackgerritJianghua Wang proposed openstack/kolla-ansible master: WIP: XenAPI: Support neutron configuration  https://review.openstack.org/52599909:13
*** gfidente has quit IRC09:15
*** bubuwifi_ has quit IRC09:15
*** gfidente has joined #openstack-kolla09:17
*** tvignaud has quit IRC09:18
*** mrunge_ has joined #openstack-kolla09:19
kolla-slack<aslancimbom100> What the hell are kata containers09:20
*** mrunge has quit IRC09:21
*** dardelean has quit IRC09:22
*** dardelean has joined #openstack-kolla09:23
dardelean_morning09:26
*** dardelean has quit IRC09:27
*** affie has joined #openstack-kolla09:29
*** mrunge_ is now known as mrunge09:30
*** affie_ has joined #openstack-kolla09:30
*** gfidente has quit IRC09:31
*** gfidente has joined #openstack-kolla09:33
*** gfidente has quit IRC09:33
*** gfidente has joined #openstack-kolla09:33
*** affie has quit IRC09:33
*** tvignaud has joined #openstack-kolla09:35
*** serlex has joined #openstack-kolla09:35
*** serlex has quit IRC09:44
*** affie_ has quit IRC09:50
openstackgerritMerged openstack/kolla master: base: use install_packages macro instead of calling APT  https://review.openstack.org/51336209:52
*** gfidente has quit IRC09:57
*** rmart04 has joined #openstack-kolla09:57
hrwyay; :D09:57
*** zhangfei has joined #openstack-kolla09:58
*** affie_ has joined #openstack-kolla10:00
*** gfidente has joined #openstack-kolla10:01
*** gfidente has quit IRC10:01
*** gfidente has joined #openstack-kolla10:01
*** gfidente has quit IRC10:03
*** gfidente has joined #openstack-kolla10:13
*** gfidente has quit IRC10:13
*** gfidente has joined #openstack-kolla10:13
*** kiennt26 has quit IRC10:15
*** k_mouza has quit IRC10:22
*** k_mouza has joined #openstack-kolla10:24
*** tovin07_ has quit IRC10:27
*** sambetts|afk is now known as sambetts10:32
*** skramaja_ has joined #openstack-kolla10:33
*** skramaja has quit IRC10:33
*** zhurong has quit IRC10:37
*** skramaja has joined #openstack-kolla10:37
*** skramaja_ has quit IRC10:38
*** genek has joined #openstack-kolla10:47
mewaldWhat do I need to do to this this: get_docker_client\\n    return docker.APIClient\\nAttributeError: \\'module\\' object has no attribute \\'APIClient\\'\\n'10:49
mewaldI remember it was something with the docker-py version but I can't recall the solution10:49
openstackgerritcaowei proposed openstack/kolla-ansible master: Implement designate-producer service  https://review.openstack.org/52602810:52
*** caowei has quit IRC10:56
*** dardelean has joined #openstack-kolla10:57
openstackgerritMerged openstack/kolla stable/pike: Change more repo url to infra mirror  https://review.openstack.org/52549611:01
*** zhangfei has quit IRC11:08
*** genek_ has joined #openstack-kolla11:20
*** lpetrut has joined #openstack-kolla11:22
*** genek has quit IRC11:23
kolla-slack<aslancimbom100> mewald, is docker running on the host where the error occurs?11:35
*** yangyapeng has quit IRC11:36
*** yangyapeng has joined #openstack-kolla11:36
*** k_mouza has quit IRC11:40
lxdong2017-12-06 19:43:38.696 7 ERROR nova.compute.manager [instance: 923f243a-c4f3-4320-a168-f8c2d7f0e3e6] 2017-12-06T11:43:08.531494Z qemu-kvm: -drive file=rbd:volumes/volume-c552c6bd-9f58-45a0-af7e-9ff3e88f8b17:id=cinder:auth_supported=cephx\;none:mon_host=103.234.143.151\:6789\;103.234.143.152\:6789,file.password-secret=virtio-disk0-secret0,format=raw,if=none,id=drive-virtio-disk0,serial=c552c6bd-9f58-45a0-af7e-9ff3e88f8b17,cache11:46
lxdongOperation not permitted11:46
*** janki has quit IRC11:52
*** kiennt26 has joined #openstack-kolla11:54
*** vincent_vdk has left #openstack-kolla11:56
*** openstackgerrit has quit IRC12:03
*** k_mouza has joined #openstack-kolla12:03
*** lxdong has quit IRC12:10
*** kiennt26 has quit IRC12:11
*** dave-mccowan has joined #openstack-kolla12:13
*** lxdong has joined #openstack-kolla12:16
*** yangyapeng has quit IRC12:18
*** yangyapeng has joined #openstack-kolla12:18
*** ansmith has quit IRC12:21
*** yangyapeng has quit IRC12:25
*** keper7 has joined #openstack-kolla12:30
*** openstackgerrit has joined #openstack-kolla12:30
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Move Cinder backend passwords to passwords.yml  https://review.openstack.org/52181712:30
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Enable nfs driver volume snapshots by default  https://review.openstack.org/52148712:39
*** janki has joined #openstack-kolla12:46
*** b_bezak has quit IRC12:48
*** dardelea_ has joined #openstack-kolla12:48
*** yangyapeng has joined #openstack-kolla12:50
*** dardelea_ has quit IRC12:53
*** skramaja has quit IRC13:01
mewaldwhat the name of the slack team for kolla? where is it documented?13:05
*** janonymous has quit IRC13:06
*** goldyfruit has joined #openstack-kolla13:06
*** goldyfruit has quit IRC13:09
*** k_mouza has quit IRC13:13
*** k_mouza has joined #openstack-kolla13:14
*** rhallisey has joined #openstack-kolla13:20
mewaldI would like to join the slack channel but have no info how to do that13:20
*** kbaegis has joined #openstack-kolla13:26
*** dardelea_ has joined #openstack-kolla13:30
*** mewald has quit IRC13:31
*** dardelea_ has quit IRC13:34
*** dardelea_ has joined #openstack-kolla13:49
*** ansmith has joined #openstack-kolla13:51
*** jmccarthy has left #openstack-kolla13:51
*** ansmith has quit IRC13:56
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Add config necessary to use Barbican with Murano  https://review.openstack.org/52563113:58
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Add the missing hosts group for designate project  https://review.openstack.org/52607513:58
*** goldyfruit has joined #openstack-kolla14:00
*** b_bezak has joined #openstack-kolla14:01
*** ansmith has joined #openstack-kolla14:03
*** coolsvap has quit IRC14:04
*** mdnadeem has quit IRC14:09
keper7hey all, both the new ocata and pike images are failing at the "haproxy : Waiting for virtual IP to appear" step for me. Also fluentd is constantly restarting. This is on a fresh physical 3-node cluster. This is my globals.yml: https://pastebin.com/u3nCapVh   - Has anyone seen this behaviour?14:10
*** pbourke_ is now known as pbourke14:12
lxdongHas anyone met this error?emergency14:13
*** david-lyle has joined #openstack-kolla14:13
*** mchlumsky has joined #openstack-kolla14:16
*** dave-mccowan has quit IRC14:17
*** gkadam has quit IRC14:18
*** zhubingbing has joined #openstack-kolla14:20
*** signed8bit has quit IRC14:24
*** signed8bit has joined #openstack-kolla14:25
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Add config necessary to use Barbican with Murano  https://review.openstack.org/52563114:27
*** keper7 has quit IRC14:31
*** david-lyle has quit IRC14:31
dardelea_keper7 I have the same issues with the fluentd cotainer on stable/pike14:34
dardelea_for the haproxy bit, I had the same issues, let me see what I i14:34
*** salv-orlando has joined #openstack-kolla14:35
*** signed8bit has quit IRC14:37
*** jtriley has joined #openstack-kolla14:38
dardelea_keper7 do you have that IP on the interface if you do a "ip a"?14:38
dardelea_keper7 please see this https://ask.openstack.org/en/question/102634/kolla-waiting-for-virtual-ip-to-appear/14:39
*** duonghq has quit IRC14:45
*** duonghq has joined #openstack-kolla14:45
*** duonghq_ has joined #openstack-kolla14:46
*** jbrooks has joined #openstack-kolla14:59
*** jbrooks has quit IRC15:03
*** salv-orlando has quit IRC15:03
*** owalsh_ has joined #openstack-kolla15:03
*** owalsh has quit IRC15:05
*** salv-orlando has joined #openstack-kolla15:10
*** emccormick has quit IRC15:12
*** salv-orlando has quit IRC15:13
*** k_mouza_ has joined #openstack-kolla15:16
*** janki has quit IRC15:17
*** k_mouza has quit IRC15:19
*** jbrooks has joined #openstack-kolla15:21
*** k_mouza_ has quit IRC15:24
*** k_mouza has joined #openstack-kolla15:25
*** k_mouza has quit IRC15:25
*** k_mouza has joined #openstack-kolla15:26
*** jbrooks has quit IRC15:26
*** david-lyle has joined #openstack-kolla15:26
*** rmart04 has quit IRC15:26
*** rmart04 has joined #openstack-kolla15:27
lxdonghttps://bugs.launchpad.net/kolla-ansible/+bug/1670676 shows it was fixed which I ran into these days in 5.0.1 dev 24 http://paste.openstack.org/show/628135/15:28
openstackLaunchpad bug 1670676 in kolla-ansible ocata "boot from volume failed" [Critical,Fix committed] - Assigned to Jeffrey Zhang (jeffrey4l)15:28
lxdongjeffrey41 around?15:29
lxdongjeffrey4l around?15:29
*** kbaegis1 has joined #openstack-kolla15:30
*** kbaegis1 has quit IRC15:30
*** owalsh_ is now known as owalsh15:32
*** kbaegis has quit IRC15:32
*** david-lyle has quit IRC15:34
*** salv-orlando has joined #openstack-kolla15:37
*** ktibi has quit IRC15:38
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Add config necessary to use Barbican with Murano  https://review.openstack.org/52563115:42
*** lpetrut has quit IRC15:43
*** lpetrut has joined #openstack-kolla15:44
*** salv-orlando has quit IRC15:45
inc0good morning15:46
*** egonzalez has joined #openstack-kolla15:47
*** dave-mccowan has joined #openstack-kolla15:47
duonghqmorning inc015:52
*** jamesbenson has joined #openstack-kolla15:52
*** bubuwifi has quit IRC15:55
*** emccormick has joined #openstack-kolla15:57
inc0meeting time folks:)16:00
*** lpetrut has quit IRC16:02
*** zhubingbing has quit IRC16:03
*** emccormick has quit IRC16:03
*** keper7 has joined #openstack-kolla16:04
portdirectAnyone else from kolla through in kubecon?16:12
*** rmart04 has quit IRC16:14
*** vhosakot has joined #openstack-kolla16:18
*** lukepatrick has joined #openstack-kolla16:22
inc0I'd bet sdake is there somewher16:24
inc0e16:24
jamesbensoninc0: I'm following the meeting been disconnected for a while.  So it looks like dockerhub now has all of the images!16:25
inc0all the versions, subset of images;)16:25
inc0but large subset16:26
inc0and only source builds16:26
jamesbensonawesome, and the masters are updated nightly?16:26
*** signed8bit has joined #openstack-kolla16:26
inc0all the branches16:26
inc0although it didn't fire up last couple of days16:26
jamesbensonthis is going to be a huge life saver!  great job!16:26
inc0I need to check what's wrong16:26
jamesbensonI saw 10 hrs ago on the ones I looked at...16:27
inc0but yeah, it's supposed to take top of the branch nightly16:27
jamesbensonok, 10hrs for ocata... 6/8 days for pike/queens16:27
inc0ocata works, pike and queens doesn't some patch messed it I guess16:27
jamesbensonyeah16:28
inc0I'll look at it today16:28
jamesbensonstellar!  So now we just reference the branch and it gets the latest...  Is there a way to snapshot the exact day/version of the branch in hub?16:28
inc0well we don't do it, but you're free to do so16:29
inc0just create your own dockerhub account16:29
inc0pull ours16:29
inc0change tag, push16:29
jamesbensongotcha, awesome, really awesome!16:29
inc0we'll add more testing to it soon too16:30
inc0so it will build->deploy->push16:30
inc0not just build16:30
*** lxdong has quit IRC16:30
dasTorinc0, if i can help/assist you with the quickstart guide i am happy to do so16:30
inc0dasTor: let me get something started, then please review16:31
dasTorit's pretty important in my opinion16:31
inc0we can work on review:)16:31
inc0yes, I agree, arguably most important part of the project;)16:31
dasTorokay16:31
inc0thanks16:32
egonzalezchason just took a look into vitrage code and the have dbsyncs  and stuff like that, we should rework the role to make it data stored properly in a db16:33
*** TxGirlGeek has joined #openstack-kolla16:34
egonzalezchason missed that during rework, basically is the same as other services, create db, and bootstrap. *remember need to add dbsync command into vitrage-api image extend-start.sh16:35
chasonegonzalez OK, I will update it. :D16:37
rwelluminc0: Can you look at https://review.openstack.org/#/c/523490/ please? Really waiting on Serguei but he's MIA and I'd like to get it merged. Especially with Pike images available.16:44
openstackgerritMerged openstack/kolla master: Upgrade the rst convention of the Contributor Guide  https://review.openstack.org/51257616:44
inc0yes I've seen it16:44
inc0it looks ok to me and already have 2 +2s16:45
rwellumok cool, kfox1111 you think it's Workflow ready then?16:45
kfox1111yeah.16:47
*** egonzalez has quit IRC16:47
rwellumty16:47
rwelluminc0: have you seen this (pretty sure you have): https://www.openstack.org/news/view/365/kata-containers-project-launches-to-build-secure-container-infrastructure16:48
inc0I know the people who did it;)16:48
inc0in fact, they sit in just behind cubical-wall from me16:48
kfox1111done.16:49
rwellumty kfox111116:49
rwelluminc0: :)16:49
inc0it's pretty cool tech16:49
*** lpetrut has joined #openstack-kolla16:50
rwellumhyper came to my company and talked their way through it.16:50
kfox1111clear containers and hyper together?16:50
inc0yeah16:50
rwelluminc0: if you look at the CEO comments - Kolla comes up16:50
inc0unfrtunately kolla won't run on this16:51
kfox1111and out of the openstack unbrella and not cncf?16:51
inc0no net=host support (obviously)16:51
inc0yup kfox111116:51
kfox1111inc0: but control planes could be stamped out on it.16:51
*** lukepatrick has quit IRC16:51
inc0sure16:51
*** lukepatrick has joined #openstack-kolla16:52
inc0it's really just nova-libvirt (I don't want my vms to run in my vms) and neutron agents16:52
*** mcnanci has quit IRC16:52
inc0also vms having access to network is useful16:52
kfox1111just please tell me it will be archetected like a cncf project and not like an openstack one. :/16:52
inc0it's written in golang fwiw16:52
kfox1111right. network nodes and compute nodes wouldn't work. but those are easy. :)16:52
inc0I mean it's a thing now, it's not "planned, we'll do it"16:53
kfox1111oh good.16:53
inc0you can deploy it16:53
inc0now16:53
kfox1111interesting.16:53
*** jbrooks has joined #openstack-kolla16:53
*** hongbin has joined #openstack-kolla16:54
kfox1111still reading the anouncement...16:54
kfox1111so is it just a clear container/hyper replacement (merging of the two projects?)16:54
inc0merging if anything, not sure how different kata is from either16:55
kfox1111https://katacontainers.io/16:55
kfox1111yeah... diagram looks like the same from cc/hyper.16:56
kfox1111still alittle puzzled by the openstack association.16:56
inc0well, OpenStack wants to look #cloudnative(tm)16:57
kfox1111any discussion on it being used by stackkube?16:57
inc0no clue16:58
kfox1111yeah, but will that cause skism with the cncf which is suposed to be a partner?16:58
kfox1111that puts openstack in a more competitive role.16:58
inc0don't get too deep into politics there16:58
inc0it's all money16:58
kfox1111heh.16:58
inc0and I think that was the plan16:58
*** salv-orlando has joined #openstack-kolla16:58
kfox1111politics and money are just different forms of power.16:59
kfox1111its all about power.16:59
kfox1111so... it sounds like cri-o will play nice with it too?17:00
kfox1111interesting...17:00
inc0it's oci-compliant17:00
inc0so yeah17:00
kfox1111https://github.com/kata-containers/community17:00
kfox1111either thats old, or openstack is willing to do somethign different.17:01
inc0yeah17:01
kfox1111though it was released 2 days ago...17:01
*** duonghq has quit IRC17:01
inc0that's actually interesting17:01
inc0it's OpenStack foundation but not part of OpenStack17:02
* kfox1111 nods.17:02
inc0and it was pull request not gerrit review17:02
kfox1111so the openstack foundation is acting just like the cncf in that case.17:02
rwellumIt seems like the perfect thing for Openstack to me. VM's that act like containers. Maintains the basic strength of Openstack (multi-tenancy etc) while adding features of containers.17:02
inc0there is more to multitenancy than that17:03
kfox1111it reuses 0 of opensatcck though.17:03
kfox1111openstack17:03
inc0it's an alternative to openstack if anything17:03
kfox1111thats like if dnsmasq came under the openstack umbrella.17:03
rwellumYeah maybe that was a reach :)17:03
kfox1111I would welcome an alternative to nova for sure but this is a different animal entirely.17:04
*** athomas has quit IRC17:04
kfox1111inc0: their getting started link takes you to that community url.17:04
kfox1111so they have nothing posted.17:04
inc0but I know that clear already works17:04
inc0clear containers that is17:05
kfox1111so does hyper.17:05
inc0so probably codedump will happen17:05
kfox1111but that means it might be a long time before they work together.17:05
inc0or they did under the hood, without a name, I honestly have no idea17:05
inc0either way17:06
inc0clear containers work17:06
inc0and is oci-compliant17:06
kfox1111yeah. I've been keeping a close eye on both.17:06
kfox1111I had a slight preference to hyper as you could run docker and hyper on the same node, and select pods to land on either via annotation.17:06
kfox1111which would be super useful.17:07
kfox1111ideally you could then use docker to stand up hyper that way.17:07
*** itlinux has joined #openstack-kolla17:08
*** salv-orlando has quit IRC17:09
*** salv-orlando has joined #openstack-kolla17:10
*** david-lyle has joined #openstack-kolla17:10
inc0bbiaf, going to get tired physically17:13
*** salv-orlando has quit IRC17:14
kfox1111not gonna ask. :)17:21
openstackgerritMerged openstack/kolla-kubernetes master: Stop kolla-kubernetes pike nova-libvirt containers crashing.  https://review.openstack.org/52349017:23
*** salv-orlando has joined #openstack-kolla17:26
*** salv-orlando has quit IRC17:26
*** salv-orlando has joined #openstack-kolla17:27
kfox1111inc0: there are other repos in that org. so there is some code. no pointers for how to deploy it though, so seems very early on.17:28
*** sambetts is now known as sambetts|afk17:31
*** salv-orlando has quit IRC17:32
*** david-lyle has quit IRC17:32
*** david-lyle has joined #openstack-kolla17:32
*** wojdec1 has quit IRC17:34
*** affie_ has quit IRC17:36
*** vhosakot has quit IRC17:38
*** vhosakot has joined #openstack-kolla17:39
*** bmace has joined #openstack-kolla17:39
*** lrensing has joined #openstack-kolla17:40
kfox1111https://kccncna17.sched.com/event/CU81/kata-containers-hypervisor-based-container-runtime-xu-wang-hyperhq-samuel-ortiz-intel17:45
kfox1111yeah, they call it a merge of projects17:45
*** david-lyle has quit IRC17:45
*** rmart04 has joined #openstack-kolla17:46
kfox1111yeah, they call it a merge of projects17:47
kfox1111the claim is they will do a multitenant demo.17:47
kfox1111I wonder if that is the openstack angle.17:47
kfox1111keystone/neutron integration.17:47
*** jbrooks has quit IRC17:47
*** rmart04 has left #openstack-kolla17:47
*** shardy has quit IRC17:54
*** signed8bit is now known as signed8bit_Zzz17:56
*** jascott1 has joined #openstack-kolla17:56
*** lrensing has quit IRC17:56
*** k_mouza_ has joined #openstack-kolla17:56
*** jascott1 has quit IRC17:59
*** jascott1 has joined #openstack-kolla17:59
*** k_mouza has quit IRC18:00
*** k_mouza_ has quit IRC18:01
*** rhallisey has quit IRC18:01
*** rhallisey has joined #openstack-kolla18:02
yankcrimeregarding mariadb backups, has anyone looked at implementing this blueprint yet: https://blueprints.launchpad.net/kolla/+spec/database-backup-recovery18:10
yankcrime(the blueprint doesn't offer any real opinions on approach, just some suggestions)18:10
*** salv-orlando has joined #openstack-kolla18:10
yankcrimecurious to see if anyone's started to tackle it yet or if there's been further debate on what the approach should be18:10
*** david-lyle has joined #openstack-kolla18:13
*** david-lyle has quit IRC18:18
*** david-lyle has joined #openstack-kolla18:18
*** unicell has joined #openstack-kolla18:19
*** unicell has quit IRC18:20
*** dave-mccowan has quit IRC18:20
*** salv-orlando has quit IRC18:21
*** salv-orlando has joined #openstack-kolla18:21
*** dardelea_ has quit IRC18:22
*** salv-orlando has quit IRC18:26
*** vhosakot has quit IRC18:31
*** gkadam has joined #openstack-kolla18:32
*** david-lyle has quit IRC18:33
harlowjainc0 qq, i forget if its been discussed, but has there been any specs on bigger sets of testing of the images being published to docker hub18:45
harlowjalike deploying them via kolla-ansible into a mini-cloud18:46
harlowjaklindgren_ ^18:48
harlowjaand thennnnnn validating via rally | tempest annnd then putting like a tag on the image saying 'verified'18:48
harlowjaor something else like that18:48
*** signed8bit_Zzz is now known as signed8bit18:50
*** unicell has joined #openstack-kolla18:54
*** rmart04 has joined #openstack-kolla18:56
*** TxGirlGeek has quit IRC19:00
kfox1111harlowja: that would be great. :)19:01
kfox1111I've been pushign for that for a while.19:01
kfox1111I think inc0 got trunk builds pushing now?19:02
kfox1111so those would have had to have made it through some gate testing.19:02
kfox1111I really want stable versions though.19:02
*** affie_ has joined #openstack-kolla19:04
inc0harlowja: we discussed this as 2 different things but yes19:05
harlowjaya, it was talked about at our standup19:05
harlowjaand i knew u guys had talked about it and ...19:05
inc0problem with too much of testing right now is that zuul doesn't support passing artifacts between jobs19:05
harlowjamy manager assumed it already existed, and i'm like ummmm, don't assume that :-/19:06
inc0so while we can do smoke test build->deploy->push in same job19:06
harlowjawhy not just go outside of zuul?19:06
inc0but I don't know how to do build job->pass images to multiple test scenarios->push if all completes19:06
kfox1111can we make abuild job that just pushes the containers to tarblls,19:06
harlowjapush to some staging docker hub repo?19:07
harlowjathen have something, not zuul kick off?19:07
*** mgoddard has quit IRC19:07
kfox1111then test jobs that pull from there and add an annotation of some kind?19:07
harlowja(if zuul is a problem, just don't use it, lol)19:07
* harlowja just saying19:07
inc0well issue is that will be bandwidth killer19:07
harlowjayup19:07
inc0if somehow we could ensure set of jobs would land in same nodepool19:07
kfox1111we pull from the hub for everything else?19:07
*** TxGirlGeek has joined #openstack-kolla19:07
inc0we can have one long standing build job with local registry19:07
inc0and other pull19:07
kfox1111so... with the repo hook I showed you, we could build all the ocntainers right in one dockerhub job I think.19:08
harlowjahmmmmm19:08
kfox1111we could tag that latest. so there is only ever one.19:08
inc0then we have no testing whatsoever19:09
kfox1111then the infra job could pull that, test them,19:09
kfox1111and when done, push the final tested set somewhere?19:09
inc0it's ugly and hacky in any form :/19:09
kfox1111yeah. but f ar less worse then the easy to use deployment procedure starting with "step 1, build the universe from source"19:09
harlowjado u guys want to start an etherpad (maybe one already exists)19:09
inc0I'll start etherpad19:10
harlowjacool beans :)19:10
harlowjakfox1111 ya, it got into the question at standup about <yes i know kolla folks want this> but everyone doesn't have enough time/resources....19:10
inc0also quick win will be when I'm done with multinode deploy job (I'm very close) I'll just turn publish job to multinode build and deploy19:10
inc0that will give us *some* testing right away19:10
harlowjawhich imho is a shitty answer, lol, but reality i guess :-/19:10
kfox1111harlowja: thats part of it. and part of it was waiting for infra to stop blocking us on waiting for zuulv3.19:10
inc0https://etherpad.openstack.org/p/kolla-publisher-testing19:10
harlowjathx19:13
* harlowja sending that around internally19:13
harlowjahopefully, just maybe we can do help here, lol19:13
inc0that'd be great19:13
inc0I'll ping infra to join convo too19:13
harlowjacool19:13
*** TxGirlGeek has quit IRC19:14
inc0one of *very very hacky* ideas is to have like 10 node job19:15
inc01 node build 3 node kolla-ansible deployment 3 node kolla-k8s deployment 3 node another scenario deployment19:15
inc0but that'd be nightmare to write and orchestrate19:16
*** jascott1 has quit IRC19:16
*** jascott1 has joined #openstack-kolla19:16
*** jascott1 has quit IRC19:17
*** jascott1 has joined #openstack-kolla19:17
yankcrimeso the comment at the bottom of https://blueprints.launchpad.net/kolla/+spec/database-backup-recovery - was there an etherpad for that discussion (denver ptg)?19:17
*** TxGirlGeek has joined #openstack-kolla19:17
harlowjainc0 maybe start simpler, klindgren_ has some ansible scripts that maybe, just maybe we can combine with nodepool to do some basic stuff19:18
harlowjai'd rather start somewhere than full on all the things :-P19:19
*** edolnx has joined #openstack-kolla19:20
inc0well as I said one thing we can definetly do is to find *one* scenario to test19:20
inc0it can be complex, worthwhile scenario19:20
inc0but singular19:20
inc0then our publish job will be just that one19:20
*** signed8bit has quit IRC19:21
edolnxGreetings all - I'm getting a error when restarting fluentd container: `{{ check_common_containers.results|selectattr('item.key', 'equalto', service_name)|first }}: no test named 'equalto'` any ideas?19:21
*** wwriverrat has joined #openstack-kolla19:22
inc0edolnx: check your ansible/jinja2 versions19:23
inc0afair that had something to do with jinja2 version being old19:23
inc0ansible 2.3 should be fine and faily modern jinja219:23
edolnxI thought that too, my venv has ansible (2.4.2.0) and Jinja2 (2.10)19:24
edolnxwhcih I believe are safe19:24
*** rhallisey has quit IRC19:24
*** rhallisey has joined #openstack-kolla19:25
*** dave-mccowan has joined #openstack-kolla19:25
inc0hmm...can you install jinja2>2.8 systemwide?19:26
harlowjaSpamapS said he's poke at some of the zuulv3 questions on there, so hopefully we can work through this inc0 kfox1111  :)19:27
edolnxDoes that need to be done on the node running ansible-kolla or all nodes @inc0?19:27
inc0edolnx: I think just ansible node19:28
inc0aka deploy node19:28
inc0would be awesome harlowja SpamapS, I just ran out of ideas myself19:28
kfox1111+119:28
inc0and with all chaos regarding even migrating our gates to zuulv3 I put that out for later19:28
inc0if you guys could help, I'd really appreciate that19:29
rwellumedolnx: I have: Ansible version:    2.2.0.0, Jinja2 version:     2.8.1 - especially the Jinja version I've never been able to use anything past this.19:29
kfox1111I've been meaning to try out my quine-registry idea with a compute-kit set of images.19:30
edolnxJust upgraded jinja - let's see if that helps...19:30
*** clarkb has joined #openstack-kolla19:30
inc0hey clarkb :)19:30
kfox1111that might be a good way of passing around all the containers.19:30
inc0clarkb: we have some notes here https://etherpad.openstack.org/p/kolla-publisher-testing19:30
edolnxyep, that did it - now on to new and exciting errors. Yay.19:31
rwellumStupid question - why don't we publish images post gate tests?19:31
inc0basically our issue boils down to this: we would like to have tight testing for our dockerhub publisher19:31
rwellumedolnx: what version worked?19:31
inc0but we can't figure out how to pass artifacts (built images) between jobs19:31
kfox1111rwellum: thats what we're talking about.19:31
edolnxI just did a `pip install --upgrade jinja2` which upgraded to 2.1019:31
harlowjainc0 i do believe SpamapS knows all the things :)19:31
kfox1111rwellum: the issue is, there is a fanout of tests and then should fan back in for final publish.19:32
rwellumedolnx: you had 2.10 before?19:32
SpamapSo/19:32
edolnxrwellum: In my venv but not globally19:32
*** affie_ has quit IRC19:32
kfox1111ie, we don't test out of a single job, and then push. we test several jobs in aparallel and need them all to pass first.19:32
*** harlowja has quit IRC19:32
rwellumedolnx: oh ok.19:32
SpamapSSo, it's not actually hard to do artifacts between parent/child jobs19:32
clarkbinc0: the trivial method would be to have a "root" job that makes the image and uploads it. Then you run child jobs that download that image and uses it19:32
SpamapSIt's just that nobody has copleted the work.19:33
inc0SpamapS the allknowing would be a wonderful help here, but if clarkb the galant could give us any feedback, I would be grateful19:33
clarkbthen if everything passes you can have a single leaf job that marks the image good and if fails you'd delete it19:33
kfox1111then its not trivail? :)19:33
SpamapSclarkb: actually you might want to kepe it if it fails.. for debugging.19:33
clarkbwell the non trivialness comes from the size of kolla's images19:33
clarkbSpamapS: good point19:33
SpamapSkeep19:33
SpamapSSo to me the only real challenge is where to put it, and how to clean artifacts up.19:34
inc0clarkb: well, we don't have any intermediate place to publish...unless you want us to use dockerhub, that will be really heavy on traffic19:34
clarkbbut you should be able to set something like that up and see how bad the image size ends up being in practice19:34
SpamapSI presume we don't want to fill up infra's log storage swift.19:34
kfox1111inc0: I think it shouldn't be too bad due ot layering?19:34
SpamapSSo need maybe another swift account somewhere, and a periodic job that deletes old stuff out of it.19:34
*** goldyfruit has quit IRC19:34
clarkbinc0: aiui the plan has alawys been publish to dockerhub then we'll cache dockerhub in our cloud regions19:34
inc0well registry was ~2.5gb tarballed best case scenario19:34
clarkbinc0: I think it would be nice to stick to that plan until we show that it does not work19:35
inc0clarkb: that's what happens now...for images that are already published19:35
SpamapSclarkb: is that publish to dockerhub in post?19:35
inc0it's about pre-publishing and validating before publish19:35
clarkbSpamapS: yes I think so19:35
SpamapSI thought the challenge was check/gate artifacts.19:35
clarkbSpamapS: you can use the same system there19:35
clarkbproblem is your caching won't be as effective19:36
kfox1111yeah. we want to test before we push. (or repush with tested tags)19:36
clarkb(since its new artifacts each time)19:36
kfox1111currently we push rather untested thigns. :(19:36
SpamapSRight19:36
*** harlowja has joined #openstack-kolla19:36
kfox1111and each parallel test should test the exact same artefect, rather then building a new one.19:36
inc0yeah that's hard requirement19:37
clarkbkfox1111: yup you'd have a job that runs first to make one artifact, then once that succeeds your other jobs run and pull that artifact19:37
clarkbits a graph of jobs19:37
kfox1111so you have one job that runs the length of the other jobs?19:37
SpamapSNo19:38
SpamapSYou'd have a job that runs fully, building an artifact, uploading it somewhere, and returning the URL where that artifact can be found.19:38
SpamapSThen many jobs that depend on that one.19:38
kfox1111so where does the 2+gig artefect land?19:38
clarkbno you have on job that runs first, then multiple jobs, then possibly cleanup type job(s) to follow19:38
clarkbkfox1111: dockerhub19:39
kfox1111I'm with you there. then the question is just where.19:39
kfox1111ok.19:39
inc0I guess if we'd just push with tag like pike-staging19:39
kfox1111I'm good with that.19:39
kfox1111yeah.19:39
kfox1111pike-untested.19:39
SpamapSwait19:39
SpamapSpush from where?19:39
kfox1111or a seperate repo foo-testign19:39
inc0SpamapS: periodic job19:39
inc0we do that already19:39
kfox1111periodic build job.19:39
inc0I'd use regular kolla with -staging suffix19:39
inc0retagging will nto really upload anything then19:40
inc0I think dockerhub is intelligent like that19:40
kfox1111inc0: true.19:40
kfox1111well, sort of.19:40
kfox1111it won't take up any extra space. not sure it doesn't upload still though.19:40
inc0also some people, for some reason might want to use -staging too19:40
inc0I don't think it does19:40
inc0well, ok, we can try that19:40
kfox1111true.if they want to bleed, not my problem. :)19:40
kfox1111ok. sounds like we have a plan. :)19:41
SpamapSI must have missed something. So this is testing already-landed things?19:41
SpamapS(Thought we were talking about gate/check)19:41
kfox1111this is the, refresh our released containers.19:41
inc0SpamapS: we push images today, just don't have proper sanity testing for them19:41
*** lrensing has joined #openstack-kolla19:41
*** wwriverrat has quit IRC19:41
*** wwriverrat has joined #openstack-kolla19:41
inc0no, no gate/check - periodic19:41
SpamapSGot it.19:42
SpamapSWhy tag it at all? You get an ID right?19:42
kfox1111once a day, check to see if any containers have out of date thingies in them. if so, trigger a rebuild, tests, repush.19:42
SpamapSzuul_return that ID19:42
rwellumSo what breaks images typically? It seems to me that if nova breaks in a kolla container, nova should know about it? So do we have to test stable/pike much for example?19:42
inc0you can't do docker pull dasdaw2adq2daz19:42
kfox1111libvirt version update. ;)19:42
kfox1111inc, technically you can :sha256:xxxx  Ithink.19:42
inc0btw having -staging would allow us to better reproduce failures too19:43
kfox1111+119:43
inc0ok, sounds like a plan19:43
SpamapSYou can also have versions right? LIke there's FROM: ubuntu and FROM: ubuntu/xenial .. can you have  a tag like, kolla/20171206194348 ?19:44
kfox1111yeah. I think we can move forward.19:44
kfox1111you can, yeah..19:44
SpamapSPoint being, if you make it a standing tag, you have a concurrency problem to deal with and a race. If you make it time/sha/uuid based then you don't have to race to finish testing before the next periodic kicks off.19:45
*** affie_ has joined #openstack-kolla19:46
kfox1111hmm...19:47
kfox1111the build takes less then an hour. is it likely that more then one would ever pile up?19:47
kfox1111can you make a job so that it only ever runs one, or can you get multiple stil?19:48
SpamapSHow long does it take for you to debug a fail?19:49
SpamapSYou can have an "only one running" yes, that's what semaphore is for.19:49
kfox1111debug can be longer then the window if you relaize you ned to debug and pull a copy.19:50
SpamapSBut the point is, say it fails.. the next one starts in 8 hours. You need to docker pull it before that happens or it is lost forever and you can't reproduce the fail.19:50
kfox1111so you have 22ish hours to save it off?19:50
kfox1111yeah.19:50
*** TxGirlGeek has quit IRC19:50
*** jascott1 has quit IRC19:50
kfox1111I'm kind of ok with that window.19:50
SpamapSHopefully it doesn't fail on Saturdays ;)19:50
kfox1111its less risky then to flood the hub accidently with too many images, and they cut us off.19:51
*** jascott1 has joined #openstack-kolla19:51
*** rwsu has quit IRC19:51
SpamapSOr maybe make your periodic only run on days when you are around to debug. :)19:51
kfox1111if its transient, then its not too much of a problem. if it happens more then one time in a row, you at least have the latest.19:51
kfox1111hysenbuild. :)19:51
*** affie_ has quit IRC19:52
*** goldyfruit has joined #openstack-kolla19:52
kfox1111gotta head out for a while. thanks for the help everyone. :)19:52
inc0if it fails and succeeds after 24 hrs, I probably won't do excersize of figuring out failure19:54
inc0unless it's persistent, then I'll pay more attention to it19:54
inc0and download images failed19:54
*** jascott1 has quit IRC19:55
*** k_mouza has joined #openstack-kolla19:59
SpamapSDoes dockerhub automatically clean up images which have no tags I wonder?19:59
SpamapSLike if you retag kolla-stage away from an image, do they GC the old one?19:59
harlowjaif its like artifactory nope :-P but not sure, ha19:59
*** b_bezak has quit IRC20:00
SpamapSYeah docker doesn't do it locally either.20:00
SpamapShence.. rmi cron jobs :-P20:00
harlowjaya...20:00
*** b_bezak has joined #openstack-kolla20:00
harlowjaall i keep on thinking about is nova used to be like that, lol20:01
harlowjawith regard to glance images downloaded20:01
SpamapSYeah, clean things up people.20:03
*** b_bezak has quit IRC20:05
*** affie_ has joined #openstack-kolla20:06
*** k_mouza has quit IRC20:07
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: Fix publisher jobs  https://review.openstack.org/52617220:08
SamYapleinc0: https://review.openstack.org/#/c/525884/20:09
inc0damn I missed it20:10
inc0nvm20:10
SamYapletoo slow!20:10
inc0thanks SamYaple20:11
inc0well, better fix it twice than none at all20:11
SamYapletwice the work for the same result. perfect business model20:11
SamYapleno wonder opensource is dying20:11
SamYaple(im kidding)20:11
inc0too late, I already lost all faith in myself20:12
SamYaplehaha20:12
SamYapleyou going to dublin?20:12
inc0not sure20:12
inc0I hope20:12
SamYaplecome to dublin! pbourke doesnt drink whiskey. we need to show him how20:13
inc0ultimately it's not my call, but problem is I like scotch whiskey so I can get lynched in irish pubs:(20:13
SamYapleim counting on it20:14
*** gkadam has quit IRC20:14
inc0yeah, you're just waiting for opportunity to have this good ol' days pitchfork and torch play?20:14
inc0;)20:14
SamYaplei just figure if you get blitzed youll buy me whiskey20:15
inc0don't underestimate alcohol tolerance of Polish blood20:16
*** jascott1 has joined #openstack-kolla20:18
*** TxGirlGeek has joined #openstack-kolla20:24
kfox1111k8s cleans up docker images I think.20:27
kfox1111not a native docker feature though I think.20:27
*** affie_ has quit IRC20:31
*** duonghq_ has quit IRC20:31
*** wlmbasson has quit IRC20:31
*** cinerama` has quit IRC20:31
*** kklimonda has quit IRC20:31
*** kong has quit IRC20:31
*** berendt has quit IRC20:31
*** zhenguo has quit IRC20:31
*** dgonzalez has quit IRC20:31
*** spsurya has quit IRC20:31
*** sgordon has quit IRC20:31
*** dave-mccowan has quit IRC20:31
*** unicell has quit IRC20:31
*** hongbin has quit IRC20:31
*** xinliang has quit IRC20:31
*** andyjones2001 has quit IRC20:31
*** hrw has quit IRC20:31
*** mcnanci has joined #openstack-kolla20:32
*** retr0h has quit IRC20:34
*** bachp has quit IRC20:35
*** mkrai has quit IRC20:35
*** nhlfr has quit IRC20:35
*** lrensing has quit IRC20:38
*** mikal has quit IRC20:38
*** jascott1 has quit IRC20:39
*** chason has quit IRC20:39
*** jascott1 has joined #openstack-kolla20:39
*** jamesbenson has quit IRC20:39
*** itlinux has quit IRC20:40
*** chason has joined #openstack-kolla20:40
*** mikal has joined #openstack-kolla20:43
*** lpetrut has quit IRC20:45
*** affie_ has joined #openstack-kolla20:47
*** duonghq_ has joined #openstack-kolla20:47
*** wlmbasson has joined #openstack-kolla20:47
*** cinerama` has joined #openstack-kolla20:47
*** kklimonda has joined #openstack-kolla20:47
*** berendt has joined #openstack-kolla20:47
*** kong has joined #openstack-kolla20:47
*** zhenguo has joined #openstack-kolla20:47
*** dgonzalez has joined #openstack-kolla20:47
*** spsurya has joined #openstack-kolla20:47
*** sgordon has joined #openstack-kolla20:47
*** lrensing has joined #openstack-kolla20:47
*** dave-mccowan has joined #openstack-kolla20:47
*** unicell has joined #openstack-kolla20:47
*** hongbin has joined #openstack-kolla20:47
*** xinliang has joined #openstack-kolla20:47
*** andyjones2001 has joined #openstack-kolla20:47
*** hrw has joined #openstack-kolla20:47
*** livelace-link has quit IRC20:49
*** calbers has quit IRC20:51
*** livelace-link has joined #openstack-kolla20:51
*** itlinux has joined #openstack-kolla20:53
*** livelace-link has quit IRC20:53
*** livelace-link has joined #openstack-kolla20:54
*** calbers has joined #openstack-kolla20:55
*** affie_ has quit IRC20:55
*** affie_ has joined #openstack-kolla20:55
*** livelace-link has quit IRC20:56
*** livelace-link has joined #openstack-kolla20:57
*** livelace-link has quit IRC20:59
*** harlowja has quit IRC21:00
*** Pavo has joined #openstack-kolla21:03
*** harlowja has joined #openstack-kolla21:03
*** salv-orlando has joined #openstack-kolla21:08
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Quickstart rewrite  https://review.openstack.org/52620021:12
inc0uff21:12
inc0that felt good21:12
*** itlinux has quit IRC21:13
*** rhallisey has quit IRC21:15
*** salv-orlando has quit IRC21:16
*** salv-orlando has joined #openstack-kolla21:17
*** lrensing has quit IRC21:19
*** salv-orlando has quit IRC21:21
*** rmart04 has quit IRC21:22
*** itlinux has joined #openstack-kolla21:26
*** affie_ has quit IRC21:28
*** affie_ has joined #openstack-kolla21:29
*** harlowja has quit IRC21:32
*** ansmith has quit IRC21:35
*** lukepatrick has quit IRC21:38
*** keper7 has quit IRC21:38
*** openstackgerrit has quit IRC21:38
*** genek_ has quit IRC21:38
*** hawi has quit IRC21:38
*** dasTor has quit IRC21:38
*** SamYaple has quit IRC21:38
*** pomac has quit IRC21:38
*** pc_m has quit IRC21:38
*** mandre has quit IRC21:38
*** rich1iu has quit IRC21:38
*** david-lyle has joined #openstack-kolla21:39
*** jamesbenson has joined #openstack-kolla21:45
*** david-lyle has quit IRC21:47
*** SamYaple has joined #openstack-kolla21:49
*** Pavo has quit IRC21:53
*** gfidente has quit IRC21:57
*** dardelea_ has joined #openstack-kolla21:58
*** itlinux has quit IRC21:59
*** lukepatrick has joined #openstack-kolla22:01
*** keper7 has joined #openstack-kolla22:01
*** openstackgerrit has joined #openstack-kolla22:01
*** genek_ has joined #openstack-kolla22:01
*** hawi has joined #openstack-kolla22:01
*** dasTor has joined #openstack-kolla22:01
*** pomac has joined #openstack-kolla22:01
*** pc_m has joined #openstack-kolla22:01
*** mandre has joined #openstack-kolla22:01
*** rich1iu has joined #openstack-kolla22:01
*** dardelea_ has quit IRC22:02
*** jtriley has quit IRC22:10
rwelluminc0: cathartic?22:11
rwellumkfox1111: still around?22:12
kfox1111kind of. whats up?22:15
inc0rwellum: yes, that's the word22:17
rwelluminc0: +1 - reading it now.22:17
rwellumkfox1111: looking at fluentd - did you get it up and running - i.e logs to an ELK somewhere? I applied your patch and I see the logs streaming to stderr/stdout - but my fluentd setup is wrong I think.22:18
kfox1111I used fluent-bit and I got the logs out. I never completed the pipeline to ek.22:19
kfox1111but should work.22:19
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: Fix deployment of ceph  https://review.openstack.org/52547022:19
rwellumYeah using the same. I think I just need to figure out the fluent-bit setting to point to my elk22:21
rwellumAlong the lines of https://kubernetes.io/docs/concepts/services-networking/service/#services-without-selectors22:22
*** salv-orlando has joined #openstack-kolla22:27
kfox1111where is your elasticsearch?22:28
*** zshi has quit IRC22:31
*** ansmith has joined #openstack-kolla22:31
*** salv-orlando has quit IRC22:32
*** jascott1 has quit IRC22:33
*** jascott1 has joined #openstack-kolla22:34
rwellumIt's on a VM in the same openstack cloud - I can reach it via a ping from the VM I am deploying kolla-k8s on.22:34
kfox1111can you curl it?22:35
kfox1111you may have to update the security group on it.22:36
*** harlowja has joined #openstack-kolla22:36
rwellumApparently not22:36
rwellumhttps://www.irccloud.com/pastebin/283A6eIZ/22:38
rwellumhttps://www.irccloud.com/pastebin/EBXnBbvE/22:39
*** jascott1 has quit IRC22:40
*** jascott1 has joined #openstack-kolla22:41
*** jascott1 has quit IRC22:42
*** jascott1 has joined #openstack-kolla22:46
kfox1111you need to change the type to es.22:46
kfox1111or setup a fluentd where the fluentd config is pointing.22:46
*** jascott1 has quit IRC22:46
rwellumInstead of type: forward - type: es ?22:47
rwellumBTW I was wrong about curl22:47
rwellumhttps://www.irccloud.com/pastebin/STiYANT2/22:48
kfox1111yeah.22:50
kfox1111otherwise, it will try to talk to a fluentd using the fluentd config info.22:51
rwellumWhich is where? All defaults I guess.22:53
kfox1111see https://github.com/kubernetes/charts/blob/master/stable/fluent-bit/templates/config.yaml for details.22:54
rwellumOh wow that's useful :)22:55
*** jamesbenson has quit IRC22:56
*** salv-orlando has joined #openstack-kolla22:57
kfox1111:)22:57
rwellumI had it book marked - but I'm still figuring out how everything goes together.22:58
*** jamesbenson has joined #openstack-kolla22:59
*** jamesbenson has quit IRC23:03
*** salv-orlando has quit IRC23:05
*** salv-orlando has joined #openstack-kolla23:05
*** salv-orlando has quit IRC23:10
*** genek_ has quit IRC23:11
*** TxGirlGeek has quit IRC23:11
*** TxGirlGeek has joined #openstack-kolla23:14
*** Pavo has joined #openstack-kolla23:16
*** Pavo has quit IRC23:17
*** Pavo has joined #openstack-kolla23:19
*** david-lyle has joined #openstack-kolla23:22
*** Pavo has quit IRC23:23
*** Pavo has joined #openstack-kolla23:25
*** openstack has joined #openstack-kolla23:30
*** ChanServ sets mode: +o openstack23:30
*** lukepatrick has quit IRC23:37
*** david-lyle has quit IRC23:44
*** salv-orlando has joined #openstack-kolla23:46
*** affie_ has quit IRC23:47
*** salv-orlando has quit IRC23:50
*** salv-orlando has joined #openstack-kolla23:51
*** JinVan has joined #openstack-kolla23:54
*** salv-orlando has quit IRC23:55
*** david-lyle has joined #openstack-kolla23:57
*** JinVan has quit IRC23:57
*** salv-orlando has joined #openstack-kolla23:57
*** JinVan has joined #openstack-kolla23:57
*** genek has joined #openstack-kolla23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!