Wednesday, 2017-02-08

clarkbok I put the three zanata changes in the same gerrit topic https://review.openstack.org/#/q/topic:zanata-upgrade reviews much appreciated. Feel free to not approve if you want me to do that when I am ready to watch things00:00
*** esikachev has joined #openstack-infra00:00
*** esikachev has quit IRC00:04
*** rbrndt has quit IRC00:04
*** ijw has joined #openstack-infra00:05
*** ihrachys has joined #openstack-infra00:06
*** baoli has quit IRC00:06
*** thorst_ has joined #openstack-infra00:08
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Inherit playbooks as expected  https://review.openstack.org/43048300:09
*** annegentle has quit IRC00:09
openstackgerritKhai Do proposed openstack-infra/git-review master: Allow git review to set votes  https://review.openstack.org/28150400:10
*** xarses has quit IRC00:10
*** sdake_ is now known as sdake00:10
*** ijw has quit IRC00:11
*** salv-orlando has quit IRC00:11
armaxclarkb: random curiosity00:14
armaxdoes infra have the ability to run jobs on opensuse nodes?00:14
armaxI see only fedora, centos and ubuntu in project-config00:15
clarkbarmax: not right this moment, but if someone does the work to get suse images built then we could. Basically just need the images built00:15
armaxclarkb: gotcha00:15
armaxok, makes sens00:15
armaxe00:15
clarkbmaybe I should work on that as I type this from a tumbleweed install00:15
clarkb(I don't actully have time for that)00:15
armaxis there a published repository of images that we use?00:16
clarkbarmax: ya we have centos, fedora, debian, and ubuntu currently00:16
clarkbarmax: no we build them all ourselves using disk image builder00:16
clarkbarmax: and one of the things onj the todo list is the host those images publicly but its not done yet00:16
armaxI see,  I was wondering if there was a less convoluted way than grepping through project-config00:16
*** rbuzatu has quit IRC00:16
armaxI mean, knowing what images are available for consumption00:17
*** ihrachys has quit IRC00:17
armaxclarkb: I might get some time in a not so distant future00:17
*** annegentle has joined #openstack-infra00:17
clarkbarmax: there is a url that you can hit to get the image listing let me look00:17
armaxand I’d be happy to give this a shot00:18
clarkbarmax: http://nodepool.openstack.org/image-list is what we have uploaded to clouds00:18
* armax looks00:18
clarkbarmax: http://nodepool.openstack.org/dib-image-list is our dib built images that get uploaded to those clouds00:18
armaxI see00:18
armaxso one will have to build an image through diskbuilder, pass it on and you’d upload it on nodepool?00:19
clarkbarmax: you configure disk image builder things in nodepool.yaml and then nodepool builds it locally and uploads to the clouds00:19
openstackgerritJoshua Harlow proposed openstack-infra/devstack-gate master: DNM: Playing with pytracemalloc  https://review.openstack.org/42984700:19
clarkbarmax: so ya step one would be getting local image builds working, then add to nodepool00:19
*** arxcruz has quit IRC00:20
armaxcool00:21
armaxI’ll ponder on this some more00:21
*** esikachev has joined #openstack-infra00:21
clarkbarmax: I have a secret want of using tumbleweed to test much newer software :)00:21
clarkbarmax: and then use that for periodic jobs or post jobs or something00:21
*** annegentle has quit IRC00:21
clarkbarmax: since a common request for a long time has been really new kernel/libvirt/ovs etc00:21
*** tuanluong has joined #openstack-infra00:21
armaxwell, don’t let me stop you though :)00:22
fungiarmax: we also have debian and maybe gentoo00:22
clarkbarmax: its been super stable for me locally, I just don't have time to go through ahd figure out dib and yast and infra's puppet and how they all mash together00:22
* fungi isn't sure how far prometheanfire got with the gentoo node implementation yet00:22
*** hongbin has quit IRC00:24
*** cody-somerville_ has joined #openstack-infra00:25
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Re-enable test_abandoned_not_timer  https://review.openstack.org/42798500:25
*** esikachev has quit IRC00:26
clarkbarmax: likely the most painful part will be getting our exim and iptables type stuff working. Since thats baseline we configure on all hosts and seems to be slightly different on each distro00:30
clarkbarmax: specifically I have noticed that firewall management is "weird" compared to what I am used to on tumbleweed00:30
openstackgerritMerged openstack/diskimage-builder master: python-brickclient: accommodate python2/3 changes  https://review.openstack.org/42516200:32
*** sflanigan has joined #openstack-infra00:33
*** sflanigan has joined #openstack-infra00:33
clarkbgrr elasticsearch still unhappy. Will try restarting 07 now as thats the third host with current inablility to initialize shards00:33
clarkbbut thinking may need a full restart00:33
*** arxcruz has joined #openstack-infra00:37
*** mriedem has joined #openstack-infra00:37
*** adrian_otto has joined #openstack-infra00:38
*** david-lyle has quit IRC00:42
*** cody-somerville_ has quit IRC00:43
armaxgood datapoint, I’ll take it into account00:43
*** wolverineav has joined #openstack-infra00:44
*** thorst_ has quit IRC00:45
*** wolverineav has quit IRC00:48
*** zhurong has joined #openstack-infra00:51
*** aeng has quit IRC00:51
*** tqtran has quit IRC00:51
openstackgerritAdam Gandelman proposed openstack-infra/zuul feature/zuulv3: Re-enable test_footer_message  https://review.openstack.org/43048600:53
*** _ody has left #openstack-infra00:53
*** aeng has joined #openstack-infra00:54
*** s-shiono_ has joined #openstack-infra00:55
*** esikachev has joined #openstack-infra00:56
*** s-shiono has quit IRC00:57
*** mtanino has quit IRC01:00
*** claudiub has quit IRC01:02
*** wolverineav has joined #openstack-infra01:03
*** david-lyle has joined #openstack-infra01:04
*** esikachev has quit IRC01:04
*** dtardivel has quit IRC01:04
*** bexelbie has quit IRC01:06
*** ijw has joined #openstack-infra01:06
*** Guest81238 has quit IRC01:08
*** wolverineav has quit IRC01:10
*** yarkot has quit IRC01:10
*** tphummel has quit IRC01:11
*** zz_dimtruck is now known as dimtruck01:12
*** ijw has quit IRC01:12
*** ijw has joined #openstack-infra01:12
*** rbuzatu has joined #openstack-infra01:13
*** pahuang_ has joined #openstack-infra01:15
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Implement node equality  https://review.openstack.org/43049101:15
*** pahuang has quit IRC01:16
ianychoiclarkb, good evening wow thanks a lot01:16
clarkbianychoi: hi there, so I think we are ready to try again as soon as those get reviewd :)01:17
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Implement node equality  https://review.openstack.org/43049101:18
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Add __repr__ to jobtree  https://review.openstack.org/43049301:18
*** thorst_ has joined #openstack-infra01:19
*** thorst_ has quit IRC01:19
ianychoiclarkb, yep recently I am busy with i18n team - hard StringFreeze & docs team pdf generation but I am seeing now also the conversation in #zanata channel. +1 great appreciated01:19
*** sree has joined #openstack-infra01:22
*** hurgleburgler has quit IRC01:24
*** sdake has quit IRC01:24
*** ijw has quit IRC01:24
*** liujiong has joined #openstack-infra01:25
*** wolverineav has joined #openstack-infra01:25
*** tongli has joined #openstack-infra01:29
*** dprince has joined #openstack-infra01:29
*** Julien-zte has joined #openstack-infra01:31
*** rfolco has joined #openstack-infra01:33
*** rfolco has quit IRC01:33
*** yamamoto has quit IRC01:34
*** yanyanhu has joined #openstack-infra01:35
*** yamamoto has joined #openstack-infra01:38
*** gildub has quit IRC01:43
*** yamamoto has quit IRC01:43
*** Daisy has joined #openstack-infra01:43
*** dprince has quit IRC01:44
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Re-enable test_abandoned_not_timer  https://review.openstack.org/42798501:44
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Implement node equality  https://review.openstack.org/43049101:44
*** Apoorva has quit IRC01:44
*** wolverineav has quit IRC01:44
*** rbuzatu has quit IRC01:47
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Inherit playbooks as expected  https://review.openstack.org/43048301:47
*** john-davidge has joined #openstack-infra01:48
*** baoli has joined #openstack-infra01:48
*** wolverineav has joined #openstack-infra01:50
clarkbianychoi: my current goal is to get translate-dev up as soon as possible, then once ocata translating dies down be ready to upgrade production for pike01:51
*** john-davidge has quit IRC01:52
ianychoiclarkb, I completely agree with you. For openstack-id issues, IMO we can test after merging your patch. I am ready to wait for +2s from infra cores01:55
ianychoiclarkb, also I will go to PTG. Hope that more possible issues on production will be discussed (unfortunately Zanata dev teams will not able to come..)01:57
*** rfolco has joined #openstack-infra01:57
*** rfolco has quit IRC01:57
*** MasterOfBugs has quit IRC01:57
*** wolverineav has quit IRC01:58
*** sree has quit IRC01:59
*** sree has joined #openstack-infra02:00
*** esikachev has joined #openstack-infra02:01
*** ihrachys has joined #openstack-infra02:01
*** srobert has quit IRC02:03
*** sree has quit IRC02:04
*** adrian_otto has quit IRC02:04
*** esikachev has quit IRC02:06
*** amotoki has joined #openstack-infra02:07
*** hurgleburgler has joined #openstack-infra02:09
*** pvaneck has quit IRC02:11
*** winggundamth_ has joined #openstack-infra02:16
*** thorst_ has joined #openstack-infra02:19
*** cody-somerville has quit IRC02:19
*** wolverineav has joined #openstack-infra02:21
*** thorst_ has quit IRC02:24
*** wolverineav has quit IRC02:26
*** adrian_otto has joined #openstack-infra02:26
*** gildub has joined #openstack-infra02:30
*** winggundamth has quit IRC02:31
*** gongysh has joined #openstack-infra02:31
*** gongysh has quit IRC02:32
*** tongli has quit IRC02:38
*** krtaylor has joined #openstack-infra02:40
*** esikachev has joined #openstack-infra02:40
*** yamamoto has joined #openstack-infra02:40
*** sdake has joined #openstack-infra02:41
*** rbuzatu has joined #openstack-infra02:44
*** dhinesh has quit IRC02:44
*** winggundamth_ has quit IRC02:45
*** vsaienko has joined #openstack-infra02:45
*** esikachev has quit IRC02:47
*** winggundamth_ has joined #openstack-infra02:47
*** zhurong has quit IRC02:51
*** cody-somerville has joined #openstack-infra02:55
*** thorst_ has joined #openstack-infra02:55
*** cody-somerville has quit IRC02:55
*** cody-somerville has joined #openstack-infra02:55
*** thorst_ has quit IRC02:55
*** ihrachys has quit IRC02:57
*** ihrachys has joined #openstack-infra02:58
*** Daisy has quit IRC02:58
*** ihrachys has quit IRC02:58
*** Daisy has joined #openstack-infra02:59
*** ihrachys has joined #openstack-infra02:59
*** ihrachys has quit IRC02:59
*** Daisy has quit IRC03:00
*** Daisy has joined #openstack-infra03:00
*** amotoki has quit IRC03:00
*** ihrachys has joined #openstack-infra03:00
*** ihrachys has quit IRC03:00
*** yamamoto has quit IRC03:01
*** sdake has quit IRC03:01
*** sdake has joined #openstack-infra03:03
*** Daisy has quit IRC03:05
*** Daisy has joined #openstack-infra03:05
*** Daisy has quit IRC03:05
*** Daisy has joined #openstack-infra03:06
*** mriedem has quit IRC03:07
*** mtanino has joined #openstack-infra03:07
*** esikachev has joined #openstack-infra03:08
*** amotoki has joined #openstack-infra03:10
*** Rockyg has quit IRC03:12
*** esikachev has quit IRC03:12
*** esikachev has joined #openstack-infra03:13
*** amotoki has quit IRC03:15
*** esikachev has quit IRC03:17
*** rbuzatu has quit IRC03:18
*** amotoki has joined #openstack-infra03:19
*** vikrant has joined #openstack-infra03:20
*** knangia has quit IRC03:20
*** gildub has quit IRC03:21
*** mtanino has quit IRC03:22
*** tlian has quit IRC03:23
*** vsaienko has quit IRC03:25
*** tojuvone has quit IRC03:28
*** tojuvone has joined #openstack-infra03:32
*** yamahata has quit IRC03:33
*** zhurong has joined #openstack-infra03:33
*** armax has quit IRC03:35
*** armax has joined #openstack-infra03:36
*** esikachev has joined #openstack-infra03:36
*** armax has quit IRC03:36
*** amotoki has quit IRC03:40
*** esikachev has quit IRC03:40
*** Daisy has quit IRC03:40
*** Daisy has joined #openstack-infra03:41
*** Daisy has quit IRC03:45
*** Daisy has joined #openstack-infra03:47
*** gouthamr has quit IRC03:49
*** links has joined #openstack-infra03:50
*** hongbin has joined #openstack-infra03:51
*** Daisy has quit IRC03:51
*** sdake_ has joined #openstack-infra03:54
*** Daisy has joined #openstack-infra03:55
*** thorst_ has joined #openstack-infra03:56
*** sdake has quit IRC03:57
*** Daisy has quit IRC04:00
*** thorst_ has quit IRC04:01
openstackgerritKhai Do proposed openstack/python-jenkins master: Extend support for views in folder(s)  https://review.openstack.org/41435804:03
*** esikachev has joined #openstack-infra04:03
*** aeng has quit IRC04:04
*** zhurong has quit IRC04:04
*** rlandy has quit IRC04:05
*** pahuang_ has quit IRC04:05
*** esikachev has quit IRC04:07
*** esikachev has joined #openstack-infra04:08
*** amotoki has joined #openstack-infra04:11
*** nicolasbock has quit IRC04:12
*** esikachev has quit IRC04:13
*** john-davidge has joined #openstack-infra04:14
*** nunchuck has quit IRC04:14
*** rbuzatu has joined #openstack-infra04:15
*** john-davidge has quit IRC04:19
*** ihrachys has joined #openstack-infra04:19
*** aeng has joined #openstack-infra04:21
*** tphummel has joined #openstack-infra04:22
*** pahuang_ has joined #openstack-infra04:23
*** wolverineav has joined #openstack-infra04:23
*** askb has quit IRC04:24
*** larainema has joined #openstack-infra04:25
*** dimtruck is now known as zz_dimtruck04:26
*** hongbin has quit IRC04:27
*** ijw has joined #openstack-infra04:27
*** wolverineav has quit IRC04:28
*** esikachev has joined #openstack-infra04:28
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Re-enable test_abandoned_not_timer  https://review.openstack.org/42798504:29
*** faizy has joined #openstack-infra04:30
*** askb has joined #openstack-infra04:30
*** ijw has quit IRC04:31
*** esikachev has quit IRC04:33
ianwlifeless (?) : what's the deal with a PBR release?  i just spent far too long figuring out that the "warnerror" for sphinx isn't actually warning but it was fiddled with in master04:34
*** csomerville has joined #openstack-infra04:35
*** cody-somerville has quit IRC04:35
*** wolverineav has joined #openstack-infra04:46
*** dhinesh has joined #openstack-infra04:46
*** baoli has quit IRC04:46
*** baoli has joined #openstack-infra04:47
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Fix up doc errors  https://review.openstack.org/43056104:48
*** psachin has joined #openstack-infra04:49
*** rbuzatu has quit IRC04:49
*** wolverineav has quit IRC04:50
*** sree has joined #openstack-infra04:53
*** thorst_ has joined #openstack-infra04:57
*** yamahata has joined #openstack-infra04:59
*** baoli has quit IRC05:00
*** thorst_ has quit IRC05:02
*** pgadiya has joined #openstack-infra05:03
*** gongysh has joined #openstack-infra05:03
*** amotoki has quit IRC05:03
*** amotoki has joined #openstack-infra05:04
*** Daisy has joined #openstack-infra05:06
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Fix up doc errors  https://review.openstack.org/43056105:07
*** bhavik1 has joined #openstack-infra05:10
*** bhavik1 has quit IRC05:16
*** zz_dimtruck is now known as dimtruck05:17
*** csomerville has quit IRC05:20
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Re-enable test_abandoned_not_timer  https://review.openstack.org/42798505:26
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Re-enable test_idle  https://review.openstack.org/43056805:26
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Re-enable test_idle  https://review.openstack.org/43056805:28
*** dimtruck is now known as zz_dimtruck05:29
*** nadya has joined #openstack-infra05:32
*** flepied has joined #openstack-infra05:35
*** ihrachys has quit IRC05:35
*** gildub has joined #openstack-infra05:44
*** salv-orlando has joined #openstack-infra05:45
*** rbuzatu has joined #openstack-infra05:45
*** salv-orlando has quit IRC05:50
*** ijw has joined #openstack-infra05:54
*** wolverineav has joined #openstack-infra05:55
*** thorst_ has joined #openstack-infra05:58
*** nadya has quit IRC05:59
*** Daisy has quit IRC06:00
*** wolverineav has quit IRC06:00
AJaegerianw, jhesketh, yolanda_ : Could you put https://review.openstack.org/429261 and https://review.openstack.org/429033 on your review queue, please? These are python-db job cleanups.06:00
*** Daisy has joined #openstack-infra06:00
*** yolanda_ has quit IRC06:02
*** yolanda_ has joined #openstack-infra06:02
*** thorst_ has quit IRC06:03
*** Daisy has quit IRC06:05
*** flepied has quit IRC06:05
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs  https://review.openstack.org/35921506:05
*** aeng has quit IRC06:08
*** pahuang_ has quit IRC06:08
*** dhinesh has quit IRC06:12
*** Daisy has joined #openstack-infra06:12
*** amotoki_ has joined #openstack-infra06:13
*** adrian_otto has quit IRC06:13
*** amotoki has quit IRC06:14
*** Daisy has quit IRC06:17
*** ijw has quit IRC06:19
*** rbuzatu has quit IRC06:19
*** yujunz has joined #openstack-infra06:19
*** zz_dimtruck is now known as dimtruck06:20
*** abregman has joined #openstack-infra06:21
*** aeng has joined #openstack-infra06:21
*** pahuang_ has joined #openstack-infra06:22
*** zhurong has joined #openstack-infra06:22
*** ijw has joined #openstack-infra06:24
*** gildub has quit IRC06:25
*** anilvenkata has joined #openstack-infra06:25
*** sdake_ is now known as sdake06:27
*** dimtruck is now known as zz_dimtruck06:30
*** claudiub has joined #openstack-infra06:32
*** wolverineav has joined #openstack-infra06:33
*** Daisy has joined #openstack-infra06:36
*** wolverineav has quit IRC06:38
*** nadya has joined #openstack-infra06:39
*** wolverineav has joined #openstack-infra06:39
*** cody-somerville_ has joined #openstack-infra06:40
*** esikachev has joined #openstack-infra06:40
openstackgerritMerged openstack/diskimage-builder master: Fix up doc errors  https://review.openstack.org/43056106:42
*** ijw has quit IRC06:42
*** adrian_otto has joined #openstack-infra06:43
*** zhurong has quit IRC06:43
*** wolverineav has quit IRC06:44
*** knangia has joined #openstack-infra06:44
*** salv-orlando has joined #openstack-infra06:46
*** zhurong has joined #openstack-infra06:49
*** tqtran has joined #openstack-infra06:50
*** vsaienko has joined #openstack-infra06:50
*** salv-orlando has quit IRC06:51
*** ijw_ has joined #openstack-infra06:52
*** tqtran has quit IRC06:54
*** kzaitsev_mb has quit IRC06:55
*** gildub has joined #openstack-infra06:55
openstackgerritMerged openstack-infra/tripleo-ci master: allow centos-opstools repo to be enabled for multinode jobs  https://review.openstack.org/42780206:58
*** wolverineav has joined #openstack-infra06:58
*** rcernin has joined #openstack-infra06:59
*** hurgleburgler has quit IRC06:59
*** Daisy has quit IRC07:00
*** Daisy has joined #openstack-infra07:00
*** Daisy has quit IRC07:01
*** Daisy has joined #openstack-infra07:01
*** salv-orlando has joined #openstack-infra07:02
*** florianf has joined #openstack-infra07:03
*** salv-orl_ has joined #openstack-infra07:05
*** wolverineav has quit IRC07:05
*** zhurong has quit IRC07:05
*** salv-orlando has quit IRC07:09
*** wolverineav has joined #openstack-infra07:10
*** vsaienko has quit IRC07:11
*** wolverineav has quit IRC07:15
*** wolverineav has joined #openstack-infra07:15
*** Daisy has quit IRC07:16
*** rbuzatu has joined #openstack-infra07:16
*** Daisy has joined #openstack-infra07:16
lifelessianw: you want to do one? when I was active we generally did one a week if there were commits to release07:18
*** tphummel has quit IRC07:19
*** wolverineav has quit IRC07:20
*** zz_dimtruck is now known as dimtruck07:21
*** andreas_s has joined #openstack-infra07:22
*** sdake_ has joined #openstack-infra07:24
*** esikachev has quit IRC07:24
*** sdake has quit IRC07:26
*** gildub has quit IRC07:26
*** pcaruana has joined #openstack-infra07:29
*** adriant has quit IRC07:32
*** dimtruck is now known as zz_dimtruck07:32
*** adrian_otto has quit IRC07:33
*** tesseract has joined #openstack-infra07:35
*** rbuzatu has quit IRC07:36
*** rbuzatu has joined #openstack-infra07:36
*** david-lyle_ has joined #openstack-infra07:37
*** Daisy has quit IRC07:41
*** Daisy has joined #openstack-infra07:41
*** yujunz is now known as yujunz[away]07:41
*** abregman has quit IRC07:41
*** yujunz[away] is now known as yujunz07:42
*** abregman has joined #openstack-infra07:42
ianwlifeless: urgh, i dunno.  i guess it seems a little risky because it seems that warnerror was disabled with https://review.openstack.org/#/c/171309/ (sep 2015) but renabled https://review.openstack.org/#/c/229951/ (jun 2016) but no release since07:42
*** Daisy_ has joined #openstack-infra07:42
*** yujunz is now known as yujunz[away]07:42
AJaegerianw, lifeless: Once we released ocata, let's do a pbr release, please07:42
ianwwhich means there's probably  1.5 years of rst error bitrot unnoticed that will hit for anything using warnerror07:43
ianwAJaeger: yeah, it seems like a bad time to release it, though requirements should avoid big issues i guess07:44
*** yujunz[away] is now known as yujunz07:44
AJaegerianw: let's update to sphinx 1.5 at the same time - that will show even more problems ;)07:44
*** yujunz is now known as yujunz[away]07:44
*** yujunz[away] is now known as yujunz07:44
*** Daisy has quit IRC07:45
AJaegerianw: do you have permissions to do a release of pbr and will do it then?07:45
ianwAJaeger: i do not, i'm not really active on it07:45
ianwi just noticed this because i was overhauling dib documentation and couldn't understand why it wasn't stopping for errors07:46
AJaegerianw: looking at ACLs: the release team can do it any time for us. Let me push an update to release repo and mark it WIP for now ;)07:47
*** nadya has quit IRC07:48
*** matrohon has joined #openstack-infra07:48
*** salv-orlando has joined #openstack-infra07:50
AJaegerianw, lifeless https://review.openstack.org/43061807:52
*** salv-orl_ has quit IRC07:52
*** yamamoto has joined #openstack-infra07:56
*** yamamoto has quit IRC07:56
AJaegerAnd library freeze ends sooner than I expected - so we might do this later this week07:57
*** ijw_ has quit IRC07:59
*** thorst_ has joined #openstack-infra07:59
AJaegerianw, lifeless would be nice if you could +1 https://review.openstack.org/43061808:01
openstackgerritMerged openstack-infra/project-config master: Add a non-voting job testing ironic with new "ipmi" driver  https://review.openstack.org/42970908:02
*** thorst_ has quit IRC08:04
AJaegeryolanda_: Good morning! Could you put https://review.openstack.org/429261 and https://review.openstack.org/429033 on your review queue as well, please? These are python-db job cleanups.08:04
openstackgerritMerged openstack-infra/project-config master: Add networking-cisco tests for stable/ocata  https://review.openstack.org/43025808:05
*** sdake_ is now known as dake08:05
openstackgerritMerged openstack-infra/project-config master: Enable ngs upgrade for ironic-multitenant job  https://review.openstack.org/43023108:06
*** salv-orlando has quit IRC08:10
openstackgerritMerged openstack-infra/project-config master: Step #1 - Shutting down Nova-Docker  https://review.openstack.org/43023208:13
*** liujiong_66 has joined #openstack-infra08:13
*** liujiong has quit IRC08:13
*** sdake has joined #openstack-infra08:14
openstackgerritMerged openstack-infra/project-config master: Create ansible-role-fedmsg project  https://review.openstack.org/42983508:14
*** nmathew has joined #openstack-infra08:14
*** huanxie has quit IRC08:15
*** huanxie has joined #openstack-infra08:16
*** ggnel_t has quit IRC08:17
*** dake has quit IRC08:17
*** yamamoto has joined #openstack-infra08:20
*** markus_z has joined #openstack-infra08:21
ianwAJaeger: as a bit of docs expert I would be interested if you have any thoughts on https://review.openstack.org/#/c/430537/ .  it's creating a custom directive to generate dependency info in docs.  i can't see this done much in any of our other docs, it seems ok but wondering if there's anything i missed08:23
*** zz_dimtruck is now known as dimtruck08:23
*** rossella_s has joined #openstack-infra08:24
*** yamamoto has quit IRC08:26
openstackgerritgengchc2 proposed openstack/os-testr master: Remove support for py34  https://review.openstack.org/43063908:27
*** david-lyle_ has quit IRC08:28
AJaegerianw: dhellmann is really the expert for that. I looked over the change and didn't notice any problems08:30
openstackgerritMerged openstack-infra/project-config master: refstack: Use normal python jobs  https://review.openstack.org/42926108:30
*** chlong has quit IRC08:30
ianwhow does something sit for that long then bnemec and I find the same thing within a few hours of each other :)08:31
*** chlong has joined #openstack-infra08:31
*** winggundamth__ has joined #openstack-infra08:32
*** stevemar has quit IRC08:33
*** dimtruck is now known as zz_dimtruck08:33
*** esikachev has joined #openstack-infra08:33
*** stakeda has quit IRC08:35
*** winggundamth_ has quit IRC08:35
AJaeger;)08:35
*** stevemar has joined #openstack-infra08:35
openstackgerritMerged openstack-infra/project-config master: manila: Use normal python jobs  https://review.openstack.org/42903308:36
*** hashar has joined #openstack-infra08:37
*** tosky has joined #openstack-infra08:39
*** panda|sick is now known as panda08:43
*** yujunz has quit IRC08:43
*** yujunz has joined #openstack-infra08:44
*** dingyichen has quit IRC08:46
*** liujiong_66 is now known as liujiong08:46
*** jpich has joined #openstack-infra08:46
*** dizquierdo has joined #openstack-infra08:48
*** tqtran has joined #openstack-infra08:52
*** ralonsoh has joined #openstack-infra08:56
*** tqtran has quit IRC08:56
*** salv-orlando has joined #openstack-infra08:57
*** markus_z has quit IRC08:59
*** zzzeek has quit IRC09:00
*** abregman has quit IRC09:01
*** tosky has quit IRC09:02
*** zzzeek has joined #openstack-infra09:02
*** ralonsoh_ has joined #openstack-infra09:05
*** ralonsoh has quit IRC09:05
*** ccamacho has joined #openstack-infra09:08
*** s-shiono_ has quit IRC09:08
*** amotoki_ has quit IRC09:09
*** yujunz has quit IRC09:12
*** monester_ has joined #openstack-infra09:12
*** markus_z has joined #openstack-infra09:14
monester_hi, who can merge this patch https://review.openstack.org/#/c/323744/ ? Khai Do put +2 there.09:15
*** apetrich has quit IRC09:15
*** tosky has joined #openstack-infra09:16
*** yujunz[away] has joined #openstack-infra09:17
*** Daisy_ has quit IRC09:17
*** Daisy has joined #openstack-infra09:18
*** jordanP has joined #openstack-infra09:21
*** yujunz[away] has quit IRC09:22
*** Daisy has quit IRC09:22
AJaegermonester_: see https://review.openstack.org/#/admin/projects/openstack-infra/jenkins-job-builder,access - and if you ask such a question, best to explain which of our 70+ repos you mean09:23
*** zz_dimtruck is now known as dimtruck09:24
monester_AJaeger: thanks09:25
*** apetrich has joined #openstack-infra09:25
*** apetrich has quit IRC09:26
*** apetrich has joined #openstack-infra09:26
*** Daisy has joined #openstack-infra09:29
*** sree_ has joined #openstack-infra09:30
*** sree_ is now known as Guest9502109:31
*** Daisy has quit IRC09:33
*** sree has quit IRC09:33
*** openstackgerrit has quit IRC09:34
*** dimtruck is now known as zz_dimtruck09:34
*** lucas-afk is now known as lucasagomes09:34
*** links has quit IRC09:35
*** derekh has joined #openstack-infra09:35
*** john-davidge has joined #openstack-infra09:38
*** nadya has joined #openstack-infra09:38
*** openstackgerrit has joined #openstack-infra09:38
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Re-enable multiple gerrit connection test  https://review.openstack.org/40669909:38
*** pkoniszewski_ has joined #openstack-infra09:38
*** kukacz_ has joined #openstack-infra09:39
*** ansiwen_ has joined #openstack-infra09:39
*** abadger2000 has joined #openstack-infra09:40
*** Daisy has joined #openstack-infra09:40
*** pleia2_ has joined #openstack-infra09:40
*** clayg_ has joined #openstack-infra09:40
*** SotK_ has joined #openstack-infra09:41
*** Guest66676 has joined #openstack-infra09:41
*** hashar is now known as hasharAway09:42
lifelessAJaeger: done09:43
*** med_` has joined #openstack-infra09:43
*** kshort_away has joined #openstack-infra09:43
*** lbragstad_ has joined #openstack-infra09:43
*** anthonyper_ has joined #openstack-infra09:44
*** topol_ has joined #openstack-infra09:44
*** dolphm_ has joined #openstack-infra09:44
*** gtmanfre- has joined #openstack-infra09:44
*** rhagarty_ has joined #openstack-infra09:44
*** laurelm_ has joined #openstack-infra09:44
*** jklare_ has joined #openstack-infra09:44
*** mandre_ has joined #openstack-infra09:44
*** jasondotstar_ has joined #openstack-infra09:44
*** mnencia_ has joined #openstack-infra09:44
*** therve_ has joined #openstack-infra09:44
*** dims_ has joined #openstack-infra09:44
*** dansmith_ has joined #openstack-infra09:44
*** cschwede_ has joined #openstack-infra09:44
*** patriciadomin_ has joined #openstack-infra09:44
*** ryotagami_ has joined #openstack-infra09:44
*** timrc has joined #openstack-infra09:44
*** sfinucan_ has joined #openstack-infra09:44
*** coreycb` has joined #openstack-infra09:44
*** filler_ has joined #openstack-infra09:44
*** Daisy has quit IRC09:45
*** sbadia_ has joined #openstack-infra09:45
*** jordanP has quit IRC09:45
*** ccamacho has quit IRC09:45
*** anilvenkata has quit IRC09:45
*** pahuang_ has quit IRC09:45
*** askb has quit IRC09:45
*** vikrant has quit IRC09:45
*** ociuhandu has quit IRC09:45
*** patriciadomin has quit IRC09:45
*** akscram has quit IRC09:45
*** anthonyper has quit IRC09:45
*** kevinshort has quit IRC09:45
*** pleia2 has quit IRC09:45
*** dims has quit IRC09:45
*** jtomasek has quit IRC09:45
*** pkoniszewski has quit IRC09:45
*** filler has quit IRC09:45
*** cinerama has quit IRC09:45
*** dmsimard has quit IRC09:45
*** topol has quit IRC09:45
*** Odd_Bloke has quit IRC09:45
*** cschwede has quit IRC09:45
*** ansiwen has quit IRC09:45
*** dansmith has quit IRC09:45
*** kukacz has quit IRC09:45
*** dolphm has quit IRC09:45
*** mnencia has quit IRC09:45
*** ryotagami has quit IRC09:45
*** skraynev has quit IRC09:45
*** therve has quit IRC09:45
*** med_ has quit IRC09:45
*** jasondotstar has quit IRC09:45
*** laurelm has quit IRC09:45
*** Krenair has quit IRC09:45
*** mandre has quit IRC09:45
*** abadger1999 has quit IRC09:45
*** sfinucan has quit IRC09:45
*** zeroDivisible has quit IRC09:45
*** gtmanfred has quit IRC09:45
*** clayg has quit IRC09:45
*** timrc_ has quit IRC09:45
*** lbragstad has quit IRC09:45
*** rosmaita has quit IRC09:45
*** nhandler has quit IRC09:45
*** coreycb has quit IRC09:45
*** hrubi has quit IRC09:45
*** rhagarty has quit IRC09:45
*** SotK has quit IRC09:45
*** Guest66666 has quit IRC09:45
*** jklare has quit IRC09:45
*** sbadia has quit IRC09:45
*** kshort_away is now known as kevinshort09:45
*** sbadia_ is now known as sbadia09:45
*** mandre_ is now known as mandre09:45
*** dolphm_ is now known as dolphm09:45
*** mnencia_ is now known as mnencia09:45
*** dansmith_ is now known as dansmith09:45
*** laurelm_ is now known as laurelm09:45
*** anthonyper_ is now known as anthonyper09:45
*** filler_ is now known as filler09:45
*** gtmanfre- is now known as gtmanfred09:45
*** jklare_ is now known as jklare09:45
*** hrubi has joined #openstack-infra09:46
*** kzaitsev_mb has joined #openstack-infra09:46
*** rosmaita has joined #openstack-infra09:46
*** dansmith is now known as Guest4449209:46
*** gtmanfred is now known as Guest9740409:46
*** ryotagami_ is now known as ryotagami09:46
*** Krenair has joined #openstack-infra09:46
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Test solution for sshd bug.  https://review.openstack.org/43068809:46
*** zeroDivisible has joined #openstack-infra09:47
*** knangia has quit IRC09:50
*** vikrant has joined #openstack-infra09:51
*** dmsimard has joined #openstack-infra09:51
*** pahuang_ has joined #openstack-infra09:51
openstackgerritMatthieu Huin proposed openstack-infra/jenkins-job-builder master: Add a publisher for MQTT  https://review.openstack.org/42776509:51
*** links has joined #openstack-infra09:51
*** jordanP has joined #openstack-infra09:52
*** nadya has quit IRC09:52
*** anilvenkata has joined #openstack-infra09:52
*** jtomasek has joined #openstack-infra09:52
*** askb has joined #openstack-infra09:52
*** Odd_Bloke has joined #openstack-infra09:52
*** nhandler has joined #openstack-infra09:52
*** skraynev has joined #openstack-infra09:52
*** akscram has joined #openstack-infra09:53
*** nadya has joined #openstack-infra09:53
openstackgerritSharat Sharma proposed openstack-infra/project-config master: Publish meteos documents  https://review.openstack.org/43069209:53
*** mhickey has joined #openstack-infra09:54
*** ccamacho has joined #openstack-infra09:54
*** wolverineav has joined #openstack-infra09:55
*** florianf has quit IRC09:55
*** florianf has joined #openstack-infra09:57
*** nadya has quit IRC09:58
*** dtantsur|afk is now known as dtantsur09:59
*** wolverineav has quit IRC09:59
*** thorst_ has joined #openstack-infra10:00
*** yamahata has quit IRC10:01
*** ralonsoh__ has joined #openstack-infra10:02
*** Julien-zte has quit IRC10:03
*** ralonsoh_ has quit IRC10:04
therve_I don't know if anyone noticed or cares, but testr results are broken in nova: http://logs.openstack.org/36/395436/23/check/gate-nova-python35-db/118aa52/testr_results.html.gz10:04
*** thorst_ has quit IRC10:04
*** wolverineav has joined #openstack-infra10:05
jpichAJaeger: Good morning! Looking at http://status.openstack.org/openstack-health/#/job/tripleo-ui-propose-translation-update it looks like the job hasn't run in a couple of days (we definitely have new translations completed as of this morning). I'm trying to figure out why, though if you have any pointer as to where I might want to look I'd appreciate it too :)10:05
*** ralonsoh__ is now known as ralonsoh10:06
openstackgerritKyrylo Shatskyy proposed openstack-infra/jenkins-job-builder master: Add support for 'Groovy Label Assignment Plugin'  https://review.openstack.org/42841910:07
*** liujiong has quit IRC10:08
*** ociuhandu has joined #openstack-infra10:14
*** abregman has joined #openstack-infra10:15
*** wolverineav has quit IRC10:16
openstackgerritSteven Dake proposed openstack-infra/project-config master: Demote trunk gating job to experimental  https://review.openstack.org/43070410:16
*** yamamoto has joined #openstack-infra10:17
*** dizquierdo has quit IRC10:18
*** yanyanhu has quit IRC10:19
openstackgerritMerged openstack-infra/irc-meetings master: Update chair for neutron meetings  https://review.openstack.org/43048010:21
*** clayg_ is now known as clayg10:22
*** yamamoto has quit IRC10:22
jpichAJaeger: I think it might not be only the tripleo-ui translation jobs that haven't run since the 6th... unless the job only shows up in the list if translations were actually proposed, maybe?10:23
*** Guest95021 has quit IRC10:24
*** zz_dimtruck is now known as dimtruck10:25
*** SotK_ is now known as SotK10:26
*** bexelbie has joined #openstack-infra10:31
*** bexelbie has quit IRC10:32
*** nadya has joined #openstack-infra10:33
*** sambetts|afk is now known as sambetts10:33
*** dimtruck is now known as zz_dimtruck10:34
AJaegerjpich: something is wrong indeed ;(10:35
AJaegerinfra-root, periodic and post queues are running since 28hours. Did anything happen to the static node?10:35
AJaegerjpich: see http://status.openstack.org/zuul/10:36
*** wolverineav has joined #openstack-infra10:36
*** bexelbie has joined #openstack-infra10:36
jpichAJaeger: Oh, I see...10:37
AJaeger#status log post and periodic jobs are not running, seems proposal node is down10:37
openstackstatusAJaeger: finished logging10:37
jpichAJaeger: Is there something I can do to help?10:38
AJaegerjpich: no. We have to wait for an admin to investigate and fix. Hope that the information above will be read10:38
AJaegerthanks, jpich10:38
*** pblaho has joined #openstack-infra10:38
jpichAJaeger: Ok! Thanks for following up10:39
*** bexelbie has joined #openstack-infra10:40
*** wolverineav has quit IRC10:41
*** claudiub has quit IRC10:42
*** bexelbie has joined #openstack-infra10:42
*** jbernard has quit IRC10:42
*** vsaienko has joined #openstack-infra10:43
Zaramorning!10:44
*** hasharAway has quit IRC10:46
*** jbernard has joined #openstack-infra10:48
*** salv-orlando has quit IRC10:49
*** winggundamth__ has quit IRC10:49
*** links has quit IRC10:50
*** rfolco has joined #openstack-infra10:50
*** hashar has joined #openstack-infra10:52
*** wolverineav has joined #openstack-infra10:53
jheskethAJaeger, jpich: I'm taking a look10:55
jpichjhesketh: Thank you!10:55
*** pkoniszewski_ is now known as pkoniszewski10:56
*** wolverineav has quit IRC10:57
*** dtantsur is now known as dtantsur|relocat10:57
*** dtantsur|relocat is now known as dtantsur|bbl10:57
*** vsaienko has quit IRC11:00
*** vsaienko has joined #openstack-infra11:01
*** electrofelix has joined #openstack-infra11:03
*** nicolasbock has joined #openstack-infra11:05
*** florianf has joined #openstack-infra11:05
*** links has joined #openstack-infra11:06
*** dizquierdo has joined #openstack-infra11:07
*** sdake has quit IRC11:11
*** sdake has joined #openstack-infra11:11
*** claudiub has joined #openstack-infra11:12
*** pgadiya has quit IRC11:12
*** markus_z has quit IRC11:13
*** denisra has joined #openstack-infra11:13
*** wolverineav has joined #openstack-infra11:15
*** nadya has quit IRC11:19
*** wolverineav has quit IRC11:20
*** patriciadomin_ is now known as patriciadomin11:20
*** yamamoto has joined #openstack-infra11:20
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Test solution for sshd bug.  https://review.openstack.org/43068811:26
*** pgadiya has joined #openstack-infra11:26
*** yamamoto has quit IRC11:26
*** sfinucan_ is now known as sfinucan11:28
*** nmathew has quit IRC11:29
*** vsaienko has quit IRC11:31
*** salv-orlando has joined #openstack-infra11:33
*** wolverineav has joined #openstack-infra11:34
jheskethAJaeger, jpich: looks like the proposal node was disconnected from the static zuul-launcher.. I couldn't see why that might have happened in the logs, but restarting that launcher appears to pick up the node and is processing the translation update jobs currently11:35
*** yujunz has joined #openstack-infra11:36
jhesketh#status log The proposal node had disconnected from the static zuul-launcher. Restarting the launcher has restored connection and proposal jobs are running again11:36
openstackstatusjhesketh: finished logging11:36
*** tuanluong has quit IRC11:37
jpichjhesketh: Great, thank you for the resolution!11:37
jheskethno worries :-)11:37
AJaegerthanks, jhesketh11:37
*** sdague has joined #openstack-infra11:38
*** wolverineav has quit IRC11:39
*** ociuhandu has quit IRC11:39
*** markus_z has joined #openstack-infra11:39
*** Daisy has joined #openstack-infra11:41
*** jkilpatr has quit IRC11:41
*** Julien-zte has joined #openstack-infra11:42
*** sdake has quit IRC11:43
*** Daisy has quit IRC11:45
openstackgerritOksana Voshchana proposed openstack-infra/devstack-gate master: WIP test to ironic with wsgi  https://review.openstack.org/43085411:49
*** dtantsur|bbl is now known as dtantsur11:49
*** jimbaker has quit IRC11:50
*** cinerama has joined #openstack-infra11:52
*** Daisy has joined #openstack-infra11:52
*** gongysh has quit IRC11:53
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Test solution for sshd bug.  https://review.openstack.org/43068811:53
*** vsaienko has joined #openstack-infra11:53
openstackgerritSean Dague proposed openstack-infra/devstack-gate master: WIP: testing dsconf merge  https://review.openstack.org/43085711:53
*** wolverineav has joined #openstack-infra11:54
*** Daisy has quit IRC11:56
*** brunograz has joined #openstack-infra11:58
*** brunograz has quit IRC11:58
*** brunograz has joined #openstack-infra11:58
*** brunograz has left #openstack-infra11:58
*** wolverineav has quit IRC11:59
*** thorst_ has joined #openstack-infra12:00
*** jkilpatr has joined #openstack-infra12:02
*** jkilpatr has quit IRC12:03
*** jkilpatr has joined #openstack-infra12:04
*** thorst_ has quit IRC12:05
*** pgadiya has quit IRC12:06
*** dave-mccowan has joined #openstack-infra12:07
*** raildo has joined #openstack-infra12:12
*** hashar has quit IRC12:12
*** vsaienko has quit IRC12:14
*** wolverineav has joined #openstack-infra12:14
*** vsaienko has joined #openstack-infra12:15
*** faizy has quit IRC12:16
openstackgerritMerged openstack-infra/subunit2sql master: H402 hacking have been deprecated  https://review.openstack.org/41772712:17
*** wolverineav has quit IRC12:18
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Add ocata and pike to proposal node  https://review.openstack.org/43086812:19
*** flepied has joined #openstack-infra12:19
AJaegerinfra-root, I guess I now know what's the problem with translation jobs is. We added ocata translation jobs and forgot to add it to the proposal regex ;(. Change 430868 should handle this.12:19
*** lucasagomes is now known as lucas-hungry12:22
*** mptacekx has joined #openstack-infra12:22
jheskethAJaeger: interesting, I wonder why the proposal node was also not connected12:22
jheskethAJaeger: should we push that change through?12:22
jheskethmy only thought is that it likely requires a restart of zuul :-s12:22
*** wolverineav has joined #openstack-infra12:24
*** david-lyle has quit IRC12:24
AJaegerinfra-root, please review 430868 and then please online the proposal node several times until the periodic queue is emptied12:24
AJaegerjhesketh: no, we need to online the proposal node now 10 times ...12:25
AJaegerjhesketh: so, push the change throught if you agree with it - and then let's online the node12:25
AJaegerit will not have an affect on queued jobs, only on new ones. But if we restart zuul, we loose the 180 changs in the post queue12:25
ianychoiOh....12:26
jheskethAJaeger: yeah, I'm just seeing if zuul reloads the functions.py12:26
*** zz_dimtruck is now known as dimtruck12:26
ianychoiI did not know that this part is also related with translation..12:26
jheskethAJaeger: okay, looks like it's reimported12:27
*** ricardoas has quit IRC12:27
jheskethAJaeger: I'll put that through12:27
ianychoiThanks a lot for dealing with this AJaeger and jhesketh12:28
*** wolverineav has quit IRC12:28
*** sandanar has joined #openstack-infra12:29
*** ggnel_t has joined #openstack-infra12:30
*** ricardoas has joined #openstack-infra12:30
openstackgerritEvgeny Antyshev proposed openstack-infra/zuul master: Set GIT_SSH env when updating repository  https://review.openstack.org/43087212:32
*** jkilpatr has quit IRC12:32
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Test solution for sshd bug.  https://review.openstack.org/43068812:32
AJaegerjhesketh: in hte past we needed to online the proposal node for each job already queued again - so new periodic jobs tomorrow will be fine but those queued already are the problem12:32
jheskethyeah, I was checking how the parameter functions were loaded, but they are done on each reload so we should be fine12:33
*** hashar has joined #openstack-infra12:33
*** sree has joined #openstack-infra12:34
*** salv-orlando has quit IRC12:34
*** yujunz has quit IRC12:35
*** yamamoto has joined #openstack-infra12:35
*** Julien-zte has quit IRC12:35
*** vikrant has quit IRC12:35
*** dimtruck is now known as zz_dimtruck12:36
openstackgerritMerged openstack-infra/tripleo-ci master: Disable predictable placement tests in oooq jobs  https://review.openstack.org/43034912:37
openstackgerritMerged openstack-infra/project-config master: Add ocata and pike to proposal node  https://review.openstack.org/43086812:37
*** sree_ has joined #openstack-infra12:38
*** sree_ is now known as Guest8610212:39
*** sree has quit IRC12:39
*** thorst_ has joined #openstack-infra12:41
*** kzaitsev_mb has quit IRC12:42
openstackgerritDavanum Srinivas (dims) proposed openstack-infra/project-config master: Step #3 - Retiring nova-docker  https://review.openstack.org/43088212:43
*** ccamacho is now known as ccamacho|lunch12:44
*** jkilpatr has joined #openstack-infra12:45
*** coreycb` has quit IRC12:48
*** coreycb has joined #openstack-infra12:48
*** yamamoto has quit IRC12:48
*** vsaienko has quit IRC12:50
*** vsaienko has joined #openstack-infra12:51
openstackgerritTomasz Trębski proposed openstack-infra/project-config master: Update CI for monasca-log-api  https://review.openstack.org/42401712:52
*** kzaitsev_mb has joined #openstack-infra12:53
*** wolverineav has joined #openstack-infra12:53
*** huanxie has quit IRC12:53
*** gongysh has joined #openstack-infra12:55
*** huanxie has joined #openstack-infra12:56
*** yuval has joined #openstack-infra12:57
*** pleia2_ is now known as pleia212:57
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Add option for scenarios jobs promotion  https://review.openstack.org/42581912:57
*** wolverineav has quit IRC12:58
*** ociuhandu has joined #openstack-infra12:58
*** askb has quit IRC12:58
*** esikachev has quit IRC13:04
*** esikachev has joined #openstack-infra13:05
*** rhallisey has joined #openstack-infra13:05
*** psachin has quit IRC13:07
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Designate: Use normal python jobs  https://review.openstack.org/42885113:09
*** esikachev has quit IRC13:09
*** vsaienko has quit IRC13:11
openstackgerritZara proposed openstack-infra/storyboard master: Hacky WIP to provide postgresl support for StoryBoard  https://review.openstack.org/43039513:12
*** wolverineav has joined #openstack-infra13:13
*** vsaienko has joined #openstack-infra13:13
*** dprince has joined #openstack-infra13:15
openstackgerritZara proposed openstack-infra/storyboard master: Hacky WIP to provide postgresql support for StoryBoard  https://review.openstack.org/43039513:16
jheskethAJaeger: the patch is loaded in and I've restarted the static node.. I'm not sure if restarting it more is necessary as it's still picking up work, but it's very late so I don't have time to look more sorry13:16
*** wolverineav has quit IRC13:17
AJaegerjhesketh: sleep well, I expect fungi or pabelanger to be around any time now. THANKS a lot!13:18
*** lucas-hungry is now known as lucasagomes13:18
*** vsaienko has quit IRC13:18
*** trown|outtypewww is now known as trown13:19
*** salv-orlando has joined #openstack-infra13:19
*** vsaienko has joined #openstack-infra13:19
*** jheroux has joined #openstack-infra13:21
AJaegerfungi, pabelanger, if you're around: Please online the proposal node again - see above.13:22
*** flepied has quit IRC13:24
*** wolverineav has joined #openstack-infra13:26
*** vsaienko has quit IRC13:28
*** vsaienko has joined #openstack-infra13:28
*** lbragstad_ is now known as lbragstad13:29
openstackgerritMonty Taylor proposed openstack-infra/shade master: Add test of attaching a volume at boot time  https://review.openstack.org/43046813:30
*** wolverineav has quit IRC13:31
*** vsaienko has quit IRC13:32
*** jcoufal has joined #openstack-infra13:32
AJaegermordred: can you online the proposal node again, please? ^13:32
fungiAJaeger: done13:34
AJaegerthanks, fungi.13:34
mordredfungi beat me to it13:34
*** gouthamr has joined #openstack-infra13:34
*** rbuzatu has quit IRC13:34
AJaegerI think we added 10 ocata translation jobs, so only a few left ;/13:34
fungii'll try to keep hitting it some but also need to go make breakfast13:34
*** gouthamr has quit IRC13:35
AJaegerfungi, it will now run 10 or so translation jobs if we're lucky ;)13:35
AJaegerenjoy breakfast!13:35
*** sdake has joined #openstack-infra13:36
openstackgerritMerged openstack-infra/devstack-gate master: Ansibly the copy of mirror config  https://review.openstack.org/40373213:37
*** fguillot has joined #openstack-infra13:37
*** yujunz has joined #openstack-infra13:37
*** links has quit IRC13:39
*** rlandy has joined #openstack-infra13:41
*** eharney has joined #openstack-infra13:42
*** Goneri has joined #openstack-infra13:43
*** matrohon has quit IRC13:44
*** srobert has joined #openstack-infra13:45
*** srobert has quit IRC13:46
*** srobert has joined #openstack-infra13:46
fungiolaph: you're the community app catalog ptl? awesome!13:47
*** rkukura has quit IRC13:47
fungiand tonyb is double-booked for ptl positions again13:48
*** yamamoto has joined #openstack-infra13:49
*** salv-orl_ has joined #openstack-infra13:50
*** matrohon has joined #openstack-infra13:50
*** trown is now known as trown|brb13:50
AJaegerfungi, another ocata job just run - I expect it's time to take care of the proposal node again, please13:50
fungidone13:51
openstackgerritMonty Taylor proposed openstack-infra/jeepyb master: Cache acl application and skip if necessary  https://review.openstack.org/43032713:51
fungiAJaeger: thanks for the heads up13:51
mordredclarkb, fungi: ^^ updated based on clarkb's reviews ... bugging people just because it's easy to forget about jeepyb patches :)13:52
AJaegerfungi, so, horizon now - first master, then newton, then ocata - next bring in 10 mins ;)13:52
*** salv-orlando has quit IRC13:52
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT MERGE: WIP: Multinode periodic tempest  https://review.openstack.org/37623713:52
*** faizy has joined #openstack-infra13:52
fungimordred: thanks, i have it starred but yesterday got away from me13:53
*** flepied has joined #openstack-infra13:53
mordredfungi: tell me about it13:54
mordredfungi: tuesdays kinda blow13:54
* fungi nods13:54
*** yamamoto has quit IRC13:55
*** gordc has joined #openstack-infra13:55
*** zhurong has joined #openstack-infra13:56
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: use upstream code of tempest  https://review.openstack.org/43093313:57
*** tongli has joined #openstack-infra13:59
*** rlandy has quit IRC14:00
*** bexelbie has quit IRC14:01
*** baoli has joined #openstack-infra14:03
*** mdrabe has joined #openstack-infra14:03
*** esikachev has joined #openstack-infra14:04
*** mdrabe_ has joined #openstack-infra14:04
*** ihrachys has joined #openstack-infra14:05
*** mdrabe has quit IRC14:08
olaphfungi: yup! thanks!14:08
openstackgerritMerged openstack-infra/shade master: pass -1 for boot_index of non-boot volumes  https://review.openstack.org/43046514:08
*** esikachev has quit IRC14:08
AJaegerfungi, horizon finished -time to online the proposal node again, please14:08
*** rlandy has joined #openstack-infra14:09
*** esikachev has joined #openstack-infra14:12
*** rbuzatu has joined #openstack-infra14:13
*** mriedem has joined #openstack-infra14:13
fungiAJaeger: done again14:14
*** bexelbie has joined #openstack-infra14:14
fungifor some reason the launcher's initscript seems to successfully restart half the time, and leaves the daemon stopped the other half the time with nothing reported to stderr explaining why14:15
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config master: Normalize projects.yaml  https://review.openstack.org/43095314:17
*** mdrabe_ is now known as mdrabe14:19
*** nadya has joined #openstack-infra14:20
*** ccamacho|lunch is now known as ccamacho14:21
*** dtardivel has joined #openstack-infra14:22
*** rbuzatu has quit IRC14:22
*** nadya has quit IRC14:22
*** nadya has joined #openstack-infra14:23
*** pradk has quit IRC14:23
*** matt-borland has joined #openstack-infra14:24
*** Julien-zte has joined #openstack-infra14:26
AJaegerfungi, time again, please14:27
*** Guest44492 is now known as dansmith14:28
*** zz_dimtruck is now known as dimtruck14:28
*** zz_ja has quit IRC14:28
*** ekhugen has quit IRC14:28
*** patriciadomin has quit IRC14:28
*** dansmith is now known as Guest9404614:28
*** kgiusti has joined #openstack-infra14:29
*** markmcd has joined #openstack-infra14:31
openstackgerritMonty Taylor proposed openstack/os-client-config master: Add support for indicating required floating IPs  https://review.openstack.org/43096714:34
*** Guest94046 is now known as dansmith14:35
*** trown|brb is now known as trown14:35
*** tlian has joined #openstack-infra14:35
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Re-enable test_timer  https://review.openstack.org/43096814:35
*** Julien-zte has quit IRC14:36
*** kgiusti has left #openstack-infra14:36
*** Julien-zte has joined #openstack-infra14:37
*** dimtruck is now known as zz_dimtruck14:37
*** Julien-zte has quit IRC14:37
*** Julien-zte has joined #openstack-infra14:39
fungiAJaeger: done, sorry for the delay14:40
*** yujunz has quit IRC14:40
*** kgiusti has joined #openstack-infra14:40
*** rbuzatu has joined #openstack-infra14:41
*** Swami has joined #openstack-infra14:41
*** yujunz has joined #openstack-infra14:42
AJaegerfungi: thanks, no worries...14:42
*** john-dav_ has joined #openstack-infra14:45
*** john-davidge has quit IRC14:45
AJaegerfungi, and again ;(14:45
fungidone14:45
*** rbuzatu has quit IRC14:45
AJaegerthanks - only 3 more times if I counted correctly14:46
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: Conntainers with multi-nic and net-iso  https://review.openstack.org/42244814:47
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: Conntainers with multi-nic and net-iso  https://review.openstack.org/42244814:47
*** dosaboy_ is now known as dosaboy14:50
openstackgerritMonty Taylor proposed openstack-infra/shade master: Add test of attaching a volume at boot time  https://review.openstack.org/43046814:52
AJaegerfungi, and again ;(14:55
*** john-dav_ is now known as john-davidge14:56
fungidone yet once more14:56
*** hurgleburgler has joined #openstack-infra14:57
*** mtanino has joined #openstack-infra14:58
openstackgerritDavanum Srinivas (dims) proposed openstack-infra/project-config master: Step #3 - Retiring nova-docker  https://review.openstack.org/43088214:59
*** rbuzatu_ has joined #openstack-infra14:59
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Re-enable test_timer_smtp  https://review.openstack.org/43098315:01
*** martinkopec has joined #openstack-infra15:01
*** rbuzatu_ has quit IRC15:01
*** rbuzatu has joined #openstack-infra15:02
*** markus_z has quit IRC15:04
*** srobert has quit IRC15:04
SpamapSjeblair: btw, started playing with boartty today. :-D15:05
* SpamapS goes off to wake family15:05
*** zz_dimtruck is now known as dimtruck15:05
*** adrian_otto1 has joined #openstack-infra15:06
Zaranext step: pygame interface with stories as moving platforms15:08
mordredZara: zomg15:11
Zarawatch this space! :D15:11
*** rossella_s has quit IRC15:13
*** rossella_s has joined #openstack-infra15:14
openstackgerritDavanum Srinivas (dims) proposed openstack-infra/project-config master: Step #3 - Retiring nova-docker  https://review.openstack.org/43088215:15
openstackgerritSean Dague proposed openstack-infra/devstack-gate master: update grenade settings for stable/ocata  https://review.openstack.org/42791615:16
*** tongli has quit IRC15:16
*** yujunz has quit IRC15:17
*** rbrndt has joined #openstack-infra15:18
*** markus_z has joined #openstack-infra15:18
*** baoli has quit IRC15:20
*** dave-mccowan has quit IRC15:20
andreafclarkb: I still haven't fixed my "two accounts in gerrit" issue - do you have time to help me with that?15:20
*** adrian_otto1 is now known as adrian_otto15:21
andreafor anyone else here can help me with losing an un-used gerrit account?15:22
*** gouthamr has joined #openstack-infra15:22
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Add new Node ZK model attributes.  https://review.openstack.org/43099215:23
*** pradk has joined #openstack-infra15:24
fungiandreaf: i can help. what's one of the account ids?15:25
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Add new Node ZK model attributes.  https://review.openstack.org/43099215:25
*** hongbin has joined #openstack-infra15:26
andreaffungi: thanks! the good one is 192115:26
*** Guest97404 is now known as gtmanfred15:26
andreaffungi: the bad one is 1072315:26
*** gtmanfred has quit IRC15:26
*** gtmanfred has joined #openstack-infra15:26
*** patriciadomin has joined #openstack-infra15:26
*** zz_ja has joined #openstack-infra15:26
fungiandreaf: that makes it easy... just a sec and i'll get that straightened out15:27
*** ekhugen has joined #openstack-infra15:27
andreaffungi: cool thank you!15:27
*** srobert has joined #openstack-infra15:28
openstackgerritSean Dague proposed openstack-infra/devstack-gate master: update grenade settings for stable/ocata  https://review.openstack.org/42791615:28
*** rossella_s has quit IRC15:28
*** krtaylor has quit IRC15:28
fungiandreaf: all done, see if it continues to give you trouble and let us know if you need help with anything else15:29
*** Swami_ has joined #openstack-infra15:30
*** gouthamr has quit IRC15:30
*** EricGonczer_ has joined #openstack-infra15:30
*** tongli has joined #openstack-infra15:31
*** mriedem1 has joined #openstack-infra15:31
andreaffungi: thanks for your help!15:32
*** Swami has quit IRC15:33
*** mriedem has quit IRC15:34
*** EricGonc_ has joined #openstack-infra15:34
*** EricGonczer_ has quit IRC15:35
mtreinishfungi: I'm still seeing 2 users for andreaf in the gerrit UI, and I get "Account Not Found" when trying to add either to a gerrit group15:36
*** burgerk has joined #openstack-infra15:36
openstackgerritDan Radez proposed openstack-infra/tripleo-ci master: DO NOT MERGE: testing congress package  https://review.openstack.org/43099915:36
*** wolverineav has joined #openstack-infra15:37
fungimtreinish: i may need to flush gerrit's caches. or you may need to force reload your browser window/tab if you haven't already15:37
*** krtaylor has joined #openstack-infra15:37
*** dave-mccowan has joined #openstack-infra15:38
*** mriedem1 is now known as mriedem15:38
fungisome of the lookups it does are cached browser-side, but then the service also maintains an in-memory query cache which doesn't know to update automatically when changes are made directly in the db15:38
*** srobert has quit IRC15:39
mtreinishfungi: I did force reload it (and tried a separate browser too)15:39
*** pradk has quit IRC15:39
fungiokay, i'll tell gerrit to flush its caches. it's a bit of a performance hit but doesn't last too long at least15:39
*** srobert has joined #openstack-infra15:39
mordredfungi: it occurs to me that we have not added Shrews to infra-core in gerrit - should I do that?15:40
fungimordred: go ahead. i was going to today but hadn't gotten to it yet15:40
*** martinkopec has quit IRC15:40
mordredfungi: done! (he just +1'd a nodepool patch and I was like - dude, why the +1?)15:40
mtreinishfungi: ok cool, let me know when to try again15:41
*** pradk has joined #openstack-infra15:41
mordredShrews: you have the powers now15:41
*** wolverineav has quit IRC15:41
* Shrews ponders what to break first15:41
fungimtreinish: andreaf: oh, hah. i missed the subtle one-line error message the gerrit api spat back at me saying i got the syntax wrong. now the extra account should actually be disabled :/15:42
mtreinishhah, ok. Let me try again15:42
mtreinishfungi: ok it worked, thanks15:43
funginp, apologies for the false start there15:43
andreafmtreinish, fungi: thanks!15:43
openstackgerritMerged openstack-infra/nodepool master: Start using dogpile caching in devstack tests  https://review.openstack.org/29969015:45
*** florianf has quit IRC15:47
*** Swami has joined #openstack-infra15:47
*** yamahata has joined #openstack-infra15:48
*** jcoufal_ has joined #openstack-infra15:48
jeblairfungi, mordred, Shrews: we may need to run the wacky infra groups update script if it hasn't been recently15:49
*** pradk has quit IRC15:49
fungijeblair: yeah, i should rerun it again, but keep in mind it only updates which groups are included in other groups and not individual group memberships15:49
fungiso we shouldn't need to run it every time we alter membership in a group (even infra-core)15:50
*** Swami_ has quit IRC15:50
*** florianf has joined #openstack-infra15:50
*** jcoufal has quit IRC15:50
fungirerunning now anyway for good measure15:51
*** pradk has joined #openstack-infra15:51
*** Sukhdev has joined #openstack-infra15:51
jeblairfungi: oh right, i thought it had an individual membership component, but i guess i remembered wrongly.  i suppose it's only as urgent if we've added new projects.15:51
fungiwhich, in your defense, we have since the last time i remembered to run it ;)15:51
fungipuppet-lpmqtt-core, err2d2-core, puppet-ethercalc-core and lpmqtt-core just got updated15:52
fungialso it hit a 404 from the api on at least one call, so i'm running it again just in case15:53
*** adrianofr_ has joined #openstack-infra15:54
*** jlibosva has joined #openstack-infra15:55
jlibosvaclarkb: hi15:56
*** scottda is now known as scottda_phone15:56
jlibosvaclarkb: can you please help us to get some traction with https://review.openstack.org/#/c/383827/14 ?15:57
jlibosvait's been there already for a while and it blocks Neutron testing15:57
*** wolverineav has joined #openstack-infra15:57
*** inc0 has joined #openstack-infra15:57
*** adrian_otto has quit IRC15:57
*** yuval has quit IRC16:00
*** xarses has joined #openstack-infra16:00
*** EricGonc_ has quit IRC16:00
jlibosvaalso any other devstack-core is very welcome to provide feedback too :)16:00
*** andreww has joined #openstack-infra16:01
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh  https://review.openstack.org/43102916:01
*** andreww has quit IRC16:01
*** rbuzatu_ has joined #openstack-infra16:01
*** john-davidge has quit IRC16:01
*** vhosakot has joined #openstack-infra16:01
*** wolverineav has quit IRC16:01
*** andreww has joined #openstack-infra16:01
anilvenkatasdague, ^ regarding review of https://review.openstack.org/#/c/383827/1416:01
*** mptacekx has quit IRC16:02
jlibosvafungi: ^^ any help is appreciated :)16:02
*** armax has joined #openstack-infra16:03
*** annegentle has joined #openstack-infra16:04
*** xarses has quit IRC16:04
*** Rockyg has joined #openstack-infra16:04
*** annegentle has quit IRC16:04
*** rbuzatu has quit IRC16:04
*** marst has joined #openstack-infra16:07
*** kzaitsev_mb has quit IRC16:08
*** nadya has quit IRC16:08
*** rajinir_ has joined #openstack-infra16:11
*** skelso has joined #openstack-infra16:11
*** scottda_phone is now known as scottda16:12
*** bexelbie_ has joined #openstack-infra16:13
*** kzaitsev_mb has joined #openstack-infra16:15
*** matrohon has quit IRC16:15
*** adrian_otto has joined #openstack-infra16:15
*** florianf has quit IRC16:16
*** baoli has joined #openstack-infra16:16
*** bexelbie has quit IRC16:16
*** florianf has joined #openstack-infra16:17
*** annegentle has joined #openstack-infra16:18
*** rbuzatu_ has quit IRC16:18
*** salv-orl_ has quit IRC16:19
*** rbuzatu has joined #openstack-infra16:20
*** dimtruck is now known as zz_dimtruck16:23
*** david-lyle has joined #openstack-infra16:23
*** zz_dimtruck is now known as dimtruck16:24
*** zhurong has quit IRC16:25
*** srobert_ has joined #openstack-infra16:25
*** tphummel has joined #openstack-infra16:26
*** abregman has quit IRC16:26
*** srobert__ has joined #openstack-infra16:28
*** srobert has quit IRC16:28
*** trown is now known as trown|lunch16:30
*** srobert_ has quit IRC16:30
*** anilvenkata has quit IRC16:32
*** vsaienko has joined #openstack-infra16:32
*** mhickey has quit IRC16:34
openstackgerritZara proposed openstack-infra/storyboard master: Hacky WIP to provide postgresql support for StoryBoard  https://review.openstack.org/43039516:34
*** dimtruck is now known as zz_dimtruck16:35
*** vsaienko has quit IRC16:35
*** sdake_ has joined #openstack-infra16:36
*** rossella_s has joined #openstack-infra16:36
*** sdake has quit IRC16:38
*** rcernin has quit IRC16:39
openstackgerritMerged openstack-infra/tripleo-ci master: Improve console readability for quickstart jobs  https://review.openstack.org/42476816:39
*** vsaienko has joined #openstack-infra16:40
*** kzaitsev_mb has quit IRC16:41
*** vsaienko has quit IRC16:43
*** markus_z has quit IRC16:44
*** lennyb has quit IRC16:44
*** vsaienko has joined #openstack-infra16:44
*** rbuzatu has quit IRC16:45
*** mhickey has joined #openstack-infra16:47
*** jimbaker has joined #openstack-infra16:48
*** rbuzatu has joined #openstack-infra16:48
*** jimbaker is now known as Guest3555216:49
*** yamahata has quit IRC16:49
*** vsaienko has quit IRC16:49
*** megm has quit IRC16:50
*** andreas_s has quit IRC16:50
*** Guest35552 is now known as jimbaker`16:51
*** nadya has joined #openstack-infra16:51
*** vsaienko has joined #openstack-infra16:52
*** david-lyle has quit IRC16:52
*** Guest66676 is now known as Guest666616:53
*** nadya has quit IRC16:53
*** megm has joined #openstack-infra16:53
*** Guest86102 has quit IRC16:53
*** sree has joined #openstack-infra16:54
*** salv-orlando has joined #openstack-infra16:54
*** jimbaker` has quit IRC16:55
*** jimbaker` has joined #openstack-infra16:55
*** jimbaker` is now known as jimbaker16:55
*** vsaienko has quit IRC16:56
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: Installs openssh packages that try to workaround sshd start problem  https://review.openstack.org/43068816:56
*** panda is now known as panda|bbl16:56
*** skelso has quit IRC16:58
*** ociuhandu has quit IRC16:58
*** Daisy has joined #openstack-infra16:59
*** sree has quit IRC16:59
clarkbinfra-root would anyone be willing to put a second set of eyes on ethercalc01 backups? They don't appear to have run and I am currently confused by this16:59
*** EricGonczer_ has joined #openstack-infra16:59
*** knangia has joined #openstack-infra17:01
*** rossella_s has quit IRC17:02
*** baoli has quit IRC17:03
*** marst_ has joined #openstack-infra17:03
*** marst has quit IRC17:03
*** Daisy has quit IRC17:03
*** tesseract has quit IRC17:05
*** tqtran has joined #openstack-infra17:06
*** bh526r has joined #openstack-infra17:06
*** sandanar has quit IRC17:07
*** zz_dimtruck is now known as dimtruck17:07
openstackgerritDaniel Mellado proposed openstack-infra/project-config master: Add docs gate to interop-workloads  https://review.openstack.org/43107217:08
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh  https://review.openstack.org/43102917:10
*** tqtran has quit IRC17:10
clarkbjeblair: there shouldn't be any harm in running the cron tar | bup command in the foreground ya? maybe run it in screen to prevent session dying causing problems?17:11
*** david-lyle has joined #openstack-infra17:12
*** mriedem has quit IRC17:12
*** ihrachys has quit IRC17:12
*** ihrachys has joined #openstack-infra17:13
*** eroux has quit IRC17:13
*** mriedem has joined #openstack-infra17:14
*** vsaienko has joined #openstack-infra17:15
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: Conntainers with multi-nic and net-iso  https://review.openstack.org/42244817:16
*** john-davidge has joined #openstack-infra17:17
*** skelso has joined #openstack-infra17:20
*** jaosorior has joined #openstack-infra17:21
*** Hal has joined #openstack-infra17:22
*** Hal is now known as Guest2581017:22
*** armax has quit IRC17:23
*** ihrachys has quit IRC17:24
JayFGeneral question: Is it a safe assumption to think etherpads will always exist? Or should old content we'd like to keep around be archived somewhere else?17:24
*** ihrachys has joined #openstack-infra17:24
*** armax has joined #openstack-infra17:24
*** ihrachys has quit IRC17:24
fungiJayF: i recommend keeping a copy of anything important17:25
clarkbits "safe" in that we have backups performed daily. But "unsafe" in that its a public resource that can easily be vandalized17:25
clarkb(making it difficult to reconstruct what you really want out of what ever is currently there)17:25
*** ihrachys has joined #openstack-infra17:25
fungiwe have had pads get corrupted before, and it's nontrivial to "restore" a single pad from database backups17:25
*** mhickey has quit IRC17:26
fungiright, defacement is a little easier to deal with, though still not trivial17:26
fungiif you can get the slider back to a known good state for the pad you can export its contents at least17:26
JayFthat's a useful answer, thanks!17:26
JayFlets say we had old ehterpads we wanted to keep, for historical context17:27
JayFlike notes from a design summit for mitaka or newton17:27
JayFis there a particular place you'd suggest archiving them that'd be better than leaving them in the etherpad?17:27
*** Swami has quit IRC17:27
*** vsaienko has quit IRC17:28
*** EricGonczer_ has quit IRC17:28
*** Apoorva has joined #openstack-infra17:28
fungiwe haven't set anything up for that, no. probably pasting into a wiki page is the closest thing we have17:28
* clarkb wanders off to get breakfast and caffeine. Ideas welcome on the "why is ethercalc01 not doing backups" front17:30
*** apetrich has quit IRC17:30
JayFfungi: I think I'm going to advocate for moving forward, after the session, mailing the etherpad content to the ML17:31
*** apetrich has joined #openstack-infra17:31
*** Sukhdev has quit IRC17:31
fungiJayF: oh! yes, that's far better than wiki actually17:32
*** baoli has joined #openstack-infra17:32
fungino idea why that didn't cross my mind17:32
jeblairclarkb: o/17:32
*** vsaienko has joined #openstack-infra17:32
*** armax has quit IRC17:32
*** topol_ is now known as topol17:33
jeblairclarkb, fungi: hrm, i'm not sure we should have called the bup user ethercalc01... my guess is that when we replace the host with ethercalc02, we will want to maintain the same backups...17:34
jeblairmaybe not?17:34
jeblairat any rate, it's a grey area in our procedures now :)17:34
fungiyeah, that's a good question and i don't have a suitable answer right now17:35
clarkbI actually thoughy about that a little17:35
clarkband thought it might givr us a good way to "expire" old backups17:35
fungiit has bitten us before if we boot replacement servers using the same backup account name before switching them into production and then they fight it out with the current production server too17:35
*** bexelbie_ is now known as bexelbie17:36
*** vsaienko has quit IRC17:36
clarkbsince with apend only expiration/deletion is tricky17:36
jeblairok17:36
jeblairit's a change, but we can make it :)17:36
*** vsaienko has joined #openstack-infra17:37
*** lucasagomes is now known as lucas-afk17:37
*** dtantsur is now known as dtantsur|afk17:37
*** skelso has quit IRC17:38
*** adrian_otto1 has joined #openstack-infra17:39
dhellmanninfra-core: we need someone with +2 on devstack-gate to take a look at https://review.openstack.org/#/c/427916 as part of the end-of-cycle process, please17:39
*** armax has joined #openstack-infra17:39
*** vsaienko has quit IRC17:39
*** jkilpatr has quit IRC17:40
jeblairfungi, clarkb: i believe we may not getting root mail because of this https://review.openstack.org/42588617:40
jeblairclarkb: the cron mail for backups on ethercalc expired as undeliverable (because of that ^), so i can't read it off disk17:41
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE - testing TLS everywhere job  https://review.openstack.org/42686317:41
fungijeblair: oh, ouch17:41
clarkboh good I have reviewed the fix for the email problem17:42
fungidid you happen to pickaxe back and find when we introduced that issue?17:42
*** adrian_otto has quit IRC17:42
*** Swami has joined #openstack-infra17:42
fungicurious how long it's been going on17:42
jeblairfungi: i don't recall17:42
*** ccamacho has quit IRC17:43
jeblairdhellmann: +3 and rechecked17:43
dhellmannjeblair : rechecked?17:43
fungii noticed a little while back i wasn't getting sslcheck warnings i expected, but hadn't had time to dig in and find a cause. betting this was it17:43
dhellmannoh, it looks like one job still failed17:44
*** ociuhandu has joined #openstack-infra17:44
dhellmannjeblair : thanks17:45
jeblairdhellmann: well, i think the current error was that its depends on failed17:45
*** vsaienko has joined #openstack-infra17:45
*** gouthamr has joined #openstack-infra17:45
*** wolverineav has joined #openstack-infra17:47
openstackgerritMerged openstack-infra/puppet-exim master: Fix local_domains domain list  https://review.openstack.org/42588617:47
*** jcoufal has joined #openstack-infra17:48
*** dizquierdo has quit IRC17:48
*** esikachev has quit IRC17:49
clarkbjeblair: do you think a good next step on debugging bup on ethercalc would be to run the cron command as root in screen?17:49
*** derekh has quit IRC17:50
*** gouthamr has quit IRC17:50
clarkbsdague: did you see my question about addressing my old comments on the local.conf d-g change?17:51
sdagueclarkb: apparently not17:51
*** jcoufal_ has quit IRC17:51
*** adrian_otto1 has quit IRC17:51
*** wolverineav has quit IRC17:52
clarkbsdague: patchset 12 I left some commends about making localrc vs local.conf clearer17:52
clarkber comments17:52
*** jpich has quit IRC17:53
*** adrian_otto has joined #openstack-infra17:53
openstackgerritMerged openstack-infra/project-config master: Update CI for monasca-log-api  https://review.openstack.org/42401717:53
*** nadya has joined #openstack-infra17:54
jeblairclarkb: yes17:54
sdagueclarkb: I'm happy to do that on follow on, this get merge conflicted enough I'd like to chunk through it17:54
clarkbjeblair: ok I will do that now17:55
*** jkilpatr has joined #openstack-infra17:56
clarkbjeblair: error: '/root/.bup/' is not a bup repository; run "bup init"17:57
* clarkb is reading man page17:57
clarkbdid I miss a step of setup?17:57
clarkbnot according to our docs17:58
clarkbjeblair: looks like bup init initializes the local side of things and you don't need to run it against the remote if using the default blank path there (which we do)18:00
clarkbjeblair: is this a new step?18:00
clarkbsdague: ok I will take a look again once backups for ethercalc are happier18:00
*** jamielennox is now known as jamielennox|away18:00
*** hashar has quit IRC18:00
*** ccamacho has joined #openstack-infra18:00
jeblairclarkb: shouldn't be accessing /root/.bup18:00
jeblairclarkb: maybe it's not logging into the right user?18:01
clarkbjeblair: well its running as root on the local side and default is to look at ~/.bup18:01
jeblairoh, that's a local error?18:01
clarkbjeblair: ya I think so based on my reading of bup-init's man page18:02
*** esikachev has joined #openstack-infra18:02
*** ralonsoh has quit IRC18:02
clarkbseems like it wants a local thing configured then the remote stuff is automatic if using blank path there18:02
jeblairclarkb: that doesn't sound right to me18:02
jeblairi don't think we've done anything like that anywhere else18:02
clarkbhttps://bup.github.io/man/bup-init.html is sparse though18:02
*** bexelbie has quit IRC18:03
clarkbjeblair: ya reading the puppet we don't appaer to do it anywhere18:03
*** tqtran has joined #openstack-infra18:04
*** dprince has quit IRC18:04
*** bexelbie has joined #openstack-infra18:04
*** bexelbie has joined #openstack-infra18:04
jeblairclarkb: i have joined your screen18:04
jeblaircan you run the command again?18:04
clarkbsure18:05
jeblairsame bup version on review.o.o18:06
clarkbjeblair: https://github.com/bup/bup/blob/master/README.md#using-bup shows bup init for things local and remote18:07
jeblairthen again, maybe backups have been broken for some time and we haven't noticed because mail has been broken for some time18:07
*** esikachev has quit IRC18:08
*** nadya has quit IRC18:09
*** trown|lunch is now known as trown18:10
openstackgerritDan Smith proposed openstack-infra/devstack-gate master: Run discover_hosts.sh in post_stack.sh for grenade  https://review.openstack.org/43109518:10
jeblairclarkb: "Even though the destination is remote, a local bup repository is still required. "18:11
*** wolverineav has joined #openstack-infra18:11
clarkbjeblair: where'd you track that down?18:11
openstackgerritSean Dague proposed openstack-infra/devstack-gate master: update grenade settings for stable/ocata  https://review.openstack.org/42791618:11
clarkbjeblair: we can probably just add taht to the steps on our manual setup doc18:12
clarkbright when running ssh-keygen18:12
jeblairclarkb: that's in 0.29.  it's not in the man page for 0.25 which we are running18:12
*** lseki has joined #openstack-infra18:12
jeblairclarkb: but maybe that was an omission18:12
clarkboh huh18:12
jeblairlooking at bup-split doc history18:12
*** rbuzatu has quit IRC18:13
jeblairhttps://github.com/bup/bup/commit/854cc30a8cb260f2e67f8784eda4a68ce2fccee518:13
jeblairso it was a doc-only commit18:13
*** dizquierdo has joined #openstack-infra18:15
*** wolverineav has quit IRC18:16
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Run tempest in the same playbook to save time  https://review.openstack.org/43109818:16
*** marst_ has quit IRC18:16
*** rbuzatu has joined #openstack-infra18:17
*** wolverineav has joined #openstack-infra18:18
openstackgerritKhai Do proposed openstack/python-jenkins master: Extend support for views in folder(s)  https://review.openstack.org/41435818:18
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Add new Node ZK model attributes.  https://review.openstack.org/43099218:18
clarkbjeblair: should I go ahead and run bup init, rerun manually and see if it works? then push doc update on our side if happy?18:19
clarkbjeblair: don't want to step on your toes if you are debugging more18:19
jeblairclarkb: i think so18:19
openstackgerritEvgeny Antyshev proposed openstack-infra/zuul master: Fix setting of GIT_SSH for timer merge jobs  https://review.openstack.org/43087218:20
clarkbok doing that now18:20
clarkbwill run with your -vv instead of -q so we get more info (maybe)18:20
*** fbo has quit IRC18:21
jeblairmore action at least18:21
clarkbthere it goes18:21
jeblairthe bup-ethercalc01 dir on the backup server is filling up18:21
jeblairclarkb: we may need to do this on all our hosts which backup18:22
clarkbjeblair: ya18:22
clarkbunless they got grandfathered in some how18:22
*** bexelbie has quit IRC18:23
clarkbjeblair: there is no /root/.bup on etherpad.o.o18:23
*** jcoufal has quit IRC18:24
*** marst has joined #openstack-infra18:25
*** wolverineav has quit IRC18:25
*** wolverineav has joined #openstack-infra18:25
*** sree has joined #openstack-infra18:26
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: test  https://review.openstack.org/43110318:26
*** EricGonczer_ has joined #openstack-infra18:26
*** skelso has joined #openstack-infra18:27
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs  https://review.openstack.org/35921518:27
clarkbjeblair: I am going to try a manual bup run on etherpad.o.o now18:29
openstackgerritMonty Taylor proposed openstack-infra/shade master: Add test of attaching a volume at boot time  https://review.openstack.org/43046818:29
jeblairclarkb: ++18:29
*** dhinesh has joined #openstack-infra18:30
clarkbsame bup init deal18:30
*** rkukura has joined #openstack-infra18:30
clarkbgah and ssh doesn't work either (I think must've been lost when we replaced hosts :( )18:31
*** florianf has quit IRC18:31
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh  https://review.openstack.org/43102918:31
clarkbso uh going to get that working on etherpad.o.o as well. Maybe others can poke at other servers to double check them too18:31
*** rockstar has quit IRC18:35
*** Kevin_Zheng has quit IRC18:35
*** MarkAtwood has quit IRC18:35
*** ggherdov-__ has quit IRC18:35
*** sripriya has quit IRC18:35
*** mwhahaha has quit IRC18:35
*** portdirect has quit IRC18:35
*** robcresswell has quit IRC18:35
*** lin_yang has quit IRC18:35
*** tomhambleton has quit IRC18:35
*** adrianofr_ has quit IRC18:35
*** erlon has quit IRC18:35
*** larainema has quit IRC18:35
clarkbok backups running on etherpad.o.o now too18:35
*** tosky has quit IRC18:35
*** kmARC has quit IRC18:35
*** kozhukalov has quit IRC18:35
*** dtardivel has quit IRC18:35
*** yonglihe has quit IRC18:35
*** sslypushenko_ has quit IRC18:35
*** igorbelikov has quit IRC18:35
*** jaranovich_ has quit IRC18:35
*** rajinir_ has quit IRC18:35
*** jpmaxman has quit IRC18:35
*** jamesmcarthur has quit IRC18:35
*** amit213 has quit IRC18:35
*** fmccrthy has quit IRC18:35
*** ogelbukh has quit IRC18:35
*** madhuvishy has quit IRC18:35
*** auggy has quit IRC18:35
*** rguillebert has quit IRC18:35
*** evgenyl has quit IRC18:35
*** dham1 has quit IRC18:35
*** nikhil has quit IRC18:35
*** monester_ has quit IRC18:35
*** zxiiro has quit IRC18:35
clarkbI am going to write the docs update before I forget, but crowd sourcing getting these all working again would be great if others can help18:35
*** vsaienko has quit IRC18:36
AJaegermriedem: could you review https://review.openstack.org/#/c/427874/ and https://review.openstack.org/429391 (and it's backports) ,please? These move the db setup from project-config to the repos so that you can easily change it.18:36
*** vsaienko has joined #openstack-infra18:37
*** patrickeast has quit IRC18:37
*** ediardo has quit IRC18:37
*** DuncanT has quit IRC18:37
*** asaprykin has quit IRC18:37
*** cargonza has quit IRC18:37
*** geekinutah has quit IRC18:37
*** asingh_ has quit IRC18:37
*** iremizov has quit IRC18:37
*** csatari has quit IRC18:37
*** raddaoui has quit IRC18:37
*** kong has quit IRC18:37
*** zaro has quit IRC18:37
*** ildikov has quit IRC18:37
AJaegerproject-config cores, could you review https://review.openstack.org/429031 and  https://review.openstack.org/428851 to move from python-db to python, please?18:38
openstackgerritClark Boylan proposed openstack-infra/system-config master: Clean up backups documentation  https://review.openstack.org/43110618:39
clarkbok there are the docs updates18:39
clarkbask, groups, lists, review (maybe review-dev), storyboard and wiki are the things we think we have backups for that we need to be checkign in on18:41
clarkbI will do ask as soon as etherpad is done18:41
openstackgerritPaul Belanger proposed openstack-infra/puppet-nodepool master: Create nodepool::launcher  https://review.openstack.org/43032918:41
clarkband we might be able to delete /opt/backups/bup-jenkins-dev18:41
jeblairi'd be down with that18:41
*** jkilpatr has quit IRC18:42
clarkbalso I think we should be backing up translate.o.o (came up as part of upgrade work) and eavesdrop.o.o and still ahve to finish puppetmaster things18:42
*** jkilpatr has joined #openstack-infra18:42
clarkbfungi: re puppet master ^ now that you have the artifact signing stuff in place are we in better shape to do encryption of some sort on the puppet master things that should be backed up?18:43
*** dizquierdo has quit IRC18:44
*** morgan has quit IRC18:45
*** jraim has quit IRC18:45
*** berendt has quit IRC18:45
*** serverascode has quit IRC18:45
*** ashtokolov has quit IRC18:45
*** gmann has quit IRC18:45
*** bh526r has quit IRC18:45
*** betherly has quit IRC18:45
*** sridhar_ram has quit IRC18:45
*** aimeeu has quit IRC18:45
*** johnsom has quit IRC18:45
*** vdrok has quit IRC18:45
*** scottda has quit IRC18:45
*** rpodolyaka has quit IRC18:45
*** NobodyCam has quit IRC18:45
*** hogepodge has quit IRC18:45
*** aglarendil has quit IRC18:45
*** tdurakov has quit IRC18:45
*** simonmcc has quit IRC18:45
*** TheJulia has quit IRC18:45
*** ameade has quit IRC18:45
*** fdegir has quit IRC18:45
openstackgerritDan Smith proposed openstack-infra/devstack-gate master: Run discover_hosts.sh in post_stack.sh for grenade  https://review.openstack.org/43109518:45
*** vhosakot has quit IRC18:46
AJaegerinfra-root, http://git.openstack.org/cgit/openstack/scientific-wg is set up - but https://github.com/openstack/scientific-wg not yet. Could somebody fix github, please?18:46
*** yamahata has joined #openstack-infra18:46
*** sambetts is now known as sambetts|afk18:47
clarkbAJaeger: github errors on me when trying to add the gerrit team18:48
mordredI just added the gerrit team18:48
mordredclarkb: we probably conflicted with each other :)18:48
clarkbah ok18:48
mordredclarkb: relatedly, I believe I addressed your review: https://review.openstack.org/#/c/430327/ ... and thanks!18:49
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Add nl01.openstack.org server  https://review.openstack.org/43032418:49
*** vhosakot has joined #openstack-infra18:50
openstackgerritPaul Belanger proposed openstack-infra/puppet-nodepool master: Create nodepool::launcher  https://review.openstack.org/43032918:51
AJaegerclarkb, mordred, thanks18:51
openstackgerritDan Smith proposed openstack-infra/devstack-gate master: update grenade settings for stable/ocata  https://review.openstack.org/42791618:52
pabelangerclarkb: mordred: have a moment to +3 the 'break out' changes for https://review.openstack.org/#/q/topic:infra-ptg-pike18:52
*** mriedem1 has joined #openstack-infra18:52
*** mriedem has quit IRC18:53
AJaegerfungi, FYI, proposal slave is currently working on post queue - once it will return to periodic queue, it might need one more restart since there's one ocata job.18:53
SpamapSclarkb: you should just setup an etherpad for a team dinner and ask everyone to RSVP by Wednesday of next week.18:53
SpamapSthen reservations can be made, and funding sources secured. ;)18:53
fungiAJaeger: yep, i assumed the same and have been keeping an eye on the post and periodic pipelines in the zuul status18:54
AJaegerthanks, fungi.18:54
*** Sukhdev has joined #openstack-infra18:54
AJaegerSpamapS: not etherpad - ethercalc ;)18:54
*** esikachev has joined #openstack-infra18:54
fungiclarkb: i'm not immediately seeing a means of leveraging our artifact signing automation to improve the backup possibilities for the puppetmaster server18:55
clarkbSpamapS: did you just volunteer? :P but tahts actually not a bad idea18:55
clarkbfungi: I think at one time we were thinking we could use a similar key setup but instead of signing we could encrypt a tarball of the hieras18:55
fungiclarkb: i _think_ the current risky files are in /etc/puppet/hieradata/production and /root/certs18:56
clarkbfungi: yes, also certs may not be anymore now that we puppet apply?18:56
SpamapSOh is calc up now?18:56
SpamapSclarkb: I'm happy to facilitate.18:56
SpamapSYou just spoke up first. :)18:56
fungiclarkb: /root/certs has a copy of our ssl/tls key files (though yes those should also be available in hiera)18:56
SpamapSand I'm also just a team-lurker ;)18:57
clarkbSpamapS: I can send a response to list with etherpad link but want to get handle on bup stuff first18:57
fungilurky mclurkerson18:57
clarkb(also if other infra-roots can/want to help it would be much appreciated :) )18:57
pabelangerclarkb: what is it you need help with?18:58
*** ihrachys_ has joined #openstack-infra18:58
clarkbpabelanger: fixing bup on the list of servers above.18:58
pabelangerokay18:58
pabelangerI can poke at it18:58
*** dharinic is now known as dharinic|lunch18:58
*** john-davidge has quit IRC18:59
clarkbinfra-root basic process I have been using is su to root, run crontab command for bup to confirm we need bup init, run bup init, run cron again (do all this in screen or similar as it may take some time and we want to be resilient to broken sessions)18:59
clarkbalso let others know here if you are doing this on a host so we don't step on toes. I am currently doing etherpad.o.o19:00
pabelangerokay, doing lists.o.o19:00
*** ihrachys has quit IRC19:00
*** Guest25810 has quit IRC19:03
*** mdrabe has quit IRC19:04
clarkbarg and it just failed on etherpad.o.o due to no space on local device (it writes things into /root/.bup19:04
clarkbthis seems less than ideal :/19:04
*** mriedem1 is now known as mriedem19:04
*** mdrabe has joined #openstack-infra19:04
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh  https://review.openstack.org/43102919:04
clarkblooks like it spools things locally in that dir19:05
openstackgerritPaul Belanger proposed openstack-infra/puppet-nodepool master: Create nodepool::launcher  https://review.openstack.org/43032919:06
*** vsaienko has quit IRC19:07
fungifile at a time, or all in one go?19:07
*** faizy has quit IRC19:08
clarkbnot sure. But I have discovrred that when you reduce logrotate retention logrotate doesn't clear out the old stuff so we still have these old db backups we decided we didn't need anymore19:08
clarkbso thats an easy thing to clean up to make room19:08
fungiyeah, i think whoever approved the review.o.o db backup retention change said they were going to do the manual cleanup19:09
fungibut it's possible that didn't happen everywhere19:09
clarkbI am deleting those older backups on etherpad.o.o now then will restart the bup process19:09
*** mdrabe has quit IRC19:10
pabelangertar: /etc/puppet/modules/nova/.git: file changed as we read it19:10
*** madhuvishy has joined #openstack-infra19:10
pabelangerI'm surprised we are backing up /etc/puppet/modules19:11
clarkbpabelanger: we have a list of excludes that we can edit if we don't want to back those up19:11
clarkb(I'm on board with that fwiw)19:12
pabelangerk, let me update19:12
*** madhuvishy has quit IRC19:12
*** nadya has joined #openstack-infra19:13
*** ijw has joined #openstack-infra19:13
*** ijw has quit IRC19:13
openstackgerritPaul Belanger proposed openstack-infra/puppet-bup master: Excluded /etc/puppet/modules/*  https://review.openstack.org/43111619:14
*** vsaienko has joined #openstack-infra19:14
fungioh, yeah we should probably also exclude the local hiera files on servers from backups19:15
fungii doubt we revisited that after we switched to masterless puppet19:15
clarkband possibly we also need to exclude /root/.bup now19:15
clarkbso that its not backing up itself19:15
fungithen again, most anything secret in those local hiera files are also embedded in random conffiles on those systems, so maybe no need to exclude19:16
clarkbfungi: good point19:16
fungiunless we can come up with a good way to identify and exclude the conffiles too19:16
*** dprince has joined #openstack-infra19:16
openstackgerritBen Kero proposed openstack-infra/tripleo-ci master: WIP: Add scenario testing for multinode-oooq jobs  https://review.openstack.org/42746819:17
openstackgerritPaul Belanger proposed openstack-infra/puppet-nodepool master: Create nodepool::launcher  https://review.openstack.org/43032919:17
fungiprobably a better approach is to whitelist just what we want backed up (static data, et cetera) since things configmanagement will put back for us aren't really in need of backing up anyway19:17
openstackgerritDan Smith proposed openstack-infra/devstack-gate master: Run discover_hosts.sh in post_stack.sh for grenade  https://review.openstack.org/43109519:18
openstackgerritDan Smith proposed openstack-infra/devstack-gate master: update grenade settings for stable/ocata  https://review.openstack.org/42791619:18
pabelangerfungi: good idea19:18
*** mdrabe has joined #openstack-infra19:20
*** MasterOfBugs has joined #openstack-infra19:22
*** jraim has joined #openstack-infra19:23
clarkbwatching it backup on etherpad.o.o its using a lot of disk but I can't find where that disk is. du -hs /* shows about 50% of disk used which is where we were before starting bup19:24
clarkband df -h now reports we are using 100% of all disk but bup is still running... weird19:25
*** erlon has joined #openstack-infra19:25
clarkboh there it failed19:26
clarkband now I have a full 17GB again19:26
clarkbso weird19:26
*** ggherdov-__ has joined #openstack-infra19:27
*** simonmcc has joined #openstack-infra19:27
openstackgerritBen Kero proposed openstack-infra/tripleo-ci master: WIP: Add scenario testing for multinode-oooq jobs  https://review.openstack.org/42746819:28
*** madhuvishy has joined #openstack-infra19:28
fungimaybe it's using unlinked inodes as buffers?19:29
fungithat's not entirely uncommon, since it keeps the application from needing to handle cleanup when something catastrophic happens19:29
fungiand is also slightly more secure against certain local types of attacks19:30
*** trown is now known as trown|brb19:30
clarkbso I guess the question now is are partial updates happening to the server side and eventually the delta will get small enough that we can manage backups properly?19:30
*** dimtruck is now known as zz_dimtruck19:30
pabelangerlists.o.o still running crontab command19:30
clarkbI am going to try another run then compare the index numbers between them19:30
*** rajinir_ has joined #openstack-infra19:31
*** serverascode has joined #openstack-infra19:31
*** kgiusti has quit IRC19:31
*** DuncanT has joined #openstack-infra19:31
fungihttp://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=181 is disappointingly useless19:32
*** rockstar has joined #openstack-infra19:32
*** ameade has joined #openstack-infra19:32
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: vmware-nsx: Use normal python jobs  https://review.openstack.org/43112019:32
clarkbthe byte count is increasing on the remote side's objects dir19:33
clarkbso I think maybe eventually it will converge?19:33
*** kozhukalov has joined #openstack-infra19:33
*** evgenyl has joined #openstack-infra19:34
*** jordanP has quit IRC19:34
*** jaranovich_ has joined #openstack-infra19:34
*** pvaneck has joined #openstack-infra19:35
*** DuncanT has quit IRC19:35
fungior consume increasing amounts of space on the backup server until it runs out ;)19:35
openstackgerritDavanum Srinivas (dims) proposed openstack-infra/project-config master: Step #3 - Retiring nova-docker  https://review.openstack.org/43088219:37
*** MarkAtwood has joined #openstack-infra19:37
openstackgerritPaul Belanger proposed openstack-infra/puppet-bup master: Exclude /etc/puppet/hieradata/*  https://review.openstack.org/43112219:37
*** zxiiro has joined #openstack-infra19:37
pabelangerclarkb: fungi: ^19:37
*** bh526r has joined #openstack-infra19:38
*** adrianofr_ has joined #openstack-infra19:39
*** sridhar_ram has joined #openstack-infra19:39
*** rguillebert has joined #openstack-infra19:39
*** dtardivel has joined #openstack-infra19:40
AJaegermordred: https://github.com/openstack/scientific-wg is still empty ;( Is there anything else needed to change?19:41
*** kgiusti has joined #openstack-infra19:41
mordredAJaeger: oh - we should probably tell gerrit to replicate ... one se19:41
mordredsec19:41
*** larainema has joined #openstack-infra19:42
mordredAJaeger: done19:43
mordredAJaeger: sorry bout that19:43
*** Kevin_Zheng has joined #openstack-infra19:43
*** fdegir has joined #openstack-infra19:43
AJaegermordred: thanks! LGTM now19:43
*** EricGonczer_ has quit IRC19:44
*** monester_ has joined #openstack-infra19:44
clarkbbup just failed again on etherpad19:44
openstackgerritDan Smith proposed openstack-infra/devstack-gate master: Run discover_hosts.sh in post_stack.sh for grenade  https://review.openstack.org/43109519:44
openstackgerritDan Smith proposed openstack-infra/devstack-gate master: update grenade settings for stable/ocata  https://review.openstack.org/42791619:44
*** scottda has joined #openstack-infra19:44
mordredclarkb: can I blame node somehow?19:44
*** EricGonczer_ has joined #openstack-infra19:45
clarkbmordred: no I think git/bup19:45
mordredI'm still going to choose to blame node19:45
*** krtaylor has quit IRC19:45
* clarkb polls the audience. Should we keep trying on therpad to see if it will eventually converge?19:46
*** ogelbukh has joined #openstack-infra19:47
fungiwhat's the behavior exactly? it's buddering into / and fills up the filesystem, then dies?19:47
fungier, buffering]19:47
openstackgerritSean Dague proposed openstack-infra/devstack-gate master: Run discover_hosts.sh in post_stack.sh for grenade  https://review.openstack.org/43109519:47
openstackgerritSean Dague proposed openstack-infra/devstack-gate master: update grenade settings for stable/ocata  https://review.openstack.org/42791619:47
fungibuttering too, sure19:47
*** trown|brb is now known as trown19:47
*** lin_yang has joined #openstack-infra19:47
clarkbfungi: ya thats what it seems to be19:48
clarkb"You can back up directly to a remote bup server, without needing tons of temporary disk space on the computer being backed up." not sure how true that is based on this experience19:48
*** salv-orlando has quit IRC19:48
clarkbbut just after that it does say interrupted backups will pick up from where it left off so I think it should eventually converge19:48
fungithe / partition only has something like 22gb of data on it, so wouldn't think it would take more than a couple passes to complete if it was managing to do this incrementally19:49
clarkb"bup is overly optimistic about mmap." may be part of the issue here?19:49
*** dharinic|lunch is now known as dharinic19:49
*** yonglihe has joined #openstack-infra19:49
mordredhah. that's a great sentence19:50
mordredI think I'd like to be overly optimistic about mmap too19:50
clarkbI am going to run it again19:50
fungii'm overly optimistic about lots of syscalls19:50
*** aglarendil has joined #openstack-infra19:51
*** ashtokolov has joined #openstack-infra19:51
*** betherly has joined #openstack-infra19:51
*** jpmaxman has joined #openstack-infra19:51
*** auggy has joined #openstack-infra19:51
*** tomhambleton_ has joined #openstack-infra19:51
openstackgerritMerged openstack-infra/project-config master: Normalize projects.yaml  https://review.openstack.org/43095319:52
*** DuncanT has joined #openstack-infra19:52
*** sekelso has joined #openstack-infra19:52
*** mwhahaha has joined #openstack-infra19:53
*** skelso has quit IRC19:53
*** hashar has joined #openstack-infra19:54
*** sripriya has joined #openstack-infra19:55
Rockygcan you be underly optimistic about syscalls?19:55
*** sslypushenko_ has joined #openstack-infra19:55
jeblairclarkb: if bup has somehow changed so it's doing things locally, i think we need to rething things19:55
openstackgerritClint 'SpamapS' Byrum proposed openstack/boartty master: Add interactive search to story view  https://review.openstack.org/43113019:56
*** robcresswell has joined #openstack-infra19:56
SpamapSjeblair: ^ \o/19:56
clarkbjeblair: its definitely spooling something to disk in ways that du can't identify19:56
*** dham1 has joined #openstack-infra19:56
jeblairclarkb: because none of this is supposed to be necessary19:56
jeblairthat's part of why we chose it and set it up this way19:56
SpamapSnow that I figured it out I think I can easily add searching of other elements, but searching task titles was my main goal ;)19:56
*** fmccrthy has joined #openstack-infra19:57
*** portdirect has joined #openstack-infra19:57
clarkbits not all going into .bup according to du, though a small portion is used there19:57
*** david-lyle has quit IRC19:57
clarkb(I think its using .bup to sync index data between remote and local19:57
*** vsaienko has quit IRC19:57
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: taskflow: Use normal python jobs  https://review.openstack.org/43113419:57
clarkbjeblair: but ya I would agree seems like we should consider alternatives if bup can't backup a server with ~50% disk free19:58
*** igorbelikov_ has joined #openstack-infra19:58
*** panda|bbl is now known as panda19:58
*** kong has joined #openstack-infra19:59
*** spzala has joined #openstack-infra19:59
*** spzala has quit IRC19:59
*** tdurakov has joined #openstack-infra19:59
*** vdrok has joined #openstack-infra20:00
*** rpodolyaka has joined #openstack-infra20:00
openstackgerritMerged openstack-infra/shade master: Add test of attaching a volume at boot time  https://review.openstack.org/43046820:01
*** ildikov_ has joined #openstack-infra20:01
*** ildikov_ is now known as ildikov20:01
*** sekelso has quit IRC20:01
*** jaosorior has quit IRC20:02
*** mriedem has quit IRC20:02
*** NobodyCam has joined #openstack-infra20:02
*** sekelso has joined #openstack-infra20:02
fungiwhat's the error it's returning?20:02
pabelangerlists.o.o still running splitting data20:02
clarkbIOError: [Errno 28] No space left on device20:02
*** aimeeu has joined #openstack-infra20:02
clarkbfungi: with a traceback if you want more details20:03
*** nikhil has joined #openstack-infra20:03
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Re-enable test_timer  https://review.openstack.org/43096820:03
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Re-enable test_timer_smtp  https://review.openstack.org/43098320:03
*** arxcruz has quit IRC20:03
*** jcoufal has joined #openstack-infra20:04
*** rkukura has quit IRC20:04
*** mriedem has joined #openstack-infra20:04
*** electrofelix has quit IRC20:04
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Add action plugins to restrict untrusted execution  https://review.openstack.org/42879820:05
*** apetrich has quit IRC20:05
*** TheJulia has joined #openstack-infra20:05
*** patrickeast has joined #openstack-infra20:05
*** apetrich has joined #openstack-infra20:05
*** johnsom has joined #openstack-infra20:05
fungiclarkb: can i scroll up in your screen session to look at the tb?20:06
clarkbfungi: yup20:06
fungidon't want to interrupt if you're in the middle of something20:06
clarkbnah its fine20:06
clarkb(I am reading borg docs which is very bup like but seems to be a bit more fully featured and is 1.0 software)20:07
*** ijw has joined #openstack-infra20:07
mordredclarkb: link?20:07
*** asaprykin has joined #openstack-infra20:08
mordredclarkb: nevermind. google is my friend20:08
clarkbhttp://borgbackup.readthedocs.io/en/stable/deployment.html20:08
clarkbthe big downside to borg seems to be they recommend no compression by default20:08
*** iremizov has joined #openstack-infra20:08
clarkbbut compression is possible you just have to think about it a bit20:09
*** jamesmcarthur has joined #openstack-infra20:09
clarkb(not that I think borg is something we should switch to, I just found it beacuse its very similar to bup)20:09
clarkbhttp://borgbackup.readthedocs.io/en/stable/faq.html#the-borg-cache-eats-way-too-much-disk-space-what-can-i-do it does however have a workaround for needing all the disk20:10
*** trown is now known as trown|brb20:10
*** cargonza has joined #openstack-infra20:10
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: tap-as-a-service: Use normal python jobs  https://review.openstack.org/43113820:11
*** raddaoui has joined #openstack-infra20:11
*** amit213 has joined #openstack-infra20:11
*** salv-orlando has joined #openstack-infra20:11
*** morgan has joined #openstack-infra20:12
*** ediardo has joined #openstack-infra20:12
*** vsaienko has joined #openstack-infra20:12
clarkbfungi: I am noticing that disk use on etherpad.o.o has significantly dropped so maybe we are over the "hump"20:12
clarkband since its incremental we may be good at this point?20:12
jeblairclarkb: one thing that led me to bup was ability to do append-only, which is useful in the case of host compromises.20:13
*** jordanP has joined #openstack-infra20:13
*** zaro has joined #openstack-infra20:14
clarkbjeblair: borg supports append only repos20:15
clarkbjeblair: you have to specify that on repo init though20:15
fungithat's good news at least20:15
jeblairneato20:15
*** csatari has joined #openstack-infra20:15
clarkbthat said bup + etherpad looks much happier right now20:15
*** david-lyle has joined #openstack-infra20:15
*** berendt has joined #openstack-infra20:15
clarkbso I think we will converge and maybe we are ok?20:15
*** geekinutah has joined #openstack-infra20:16
*** rfolco has quit IRC20:16
*** asingh_ has joined #openstack-infra20:16
*** gmann has joined #openstack-infra20:16
clarkbwas looking at table at https://wiki.archlinux.org/index.php/Synchronization_and_backup_programs#Single_machine fwiw20:17
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: mistral: Change tox-db to tox jobs  https://review.openstack.org/43114820:18
clarkband bup and borg appear to be the most fully featured incremental append only systems ased on that info20:18
*** hogepodge has joined #openstack-infra20:19
*** kmARC has joined #openstack-infra20:19
*** jordanP has quit IRC20:19
*** arxcruz has joined #openstack-infra20:20
*** david-lyle has quit IRC20:20
*** spzala has joined #openstack-infra20:21
*** spzala has quit IRC20:21
*** mgkwill has quit IRC20:21
*** mgkwill has joined #openstack-infra20:21
*** trown|brb is now known as trown20:21
*** cody-somerville has joined #openstack-infra20:25
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: tuning-box: Use normal python jobs  https://review.openstack.org/43115420:25
*** cody-somerville has quit IRC20:25
*** cody-somerville has joined #openstack-infra20:25
clarkbhttp://borgbackup.readthedocs.io/en/stable/usage.html#append-only-mode so a neat thing about that is apparently append only only affects the client side20:25
clarkbso you can prune things as the admin user20:25
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Re-enable test_mutex  https://review.openstack.org/42912220:26
clarkbwhich I think is something that bup says is very scary to do with bup (you can rm/gc but use at your own risk)20:27
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Re-enable test_json_status  https://review.openstack.org/42914620:27
*** krtaylor has joined #openstack-infra20:28
*** matrohon has joined #openstack-infra20:28
*** kzaitsev_mb has joined #openstack-infra20:29
sdagueclarkb / jeblair - we discovered a last thing that needed to change in devstack-gate for the branches to get set - https://review.openstack.org/#/c/431095/5/devstack-vm-gate.sh20:30
pabelangerup to a 14GB tar for bup on lists.o.o, no failure yet20:31
sdaguethe critical tests on multinode grenade have passed on the dependant patch now20:31
*** nadya has quit IRC20:31
pabelangerclarkb: fungi: mordred: do you have a moment to +3 the stack of changes for system-config: https://review.openstack.org/#/c/43042720:35
clarkbpabelanger: ya will take a look20:38
clarkbheh that change shows up as a related change to itself20:40
clarkbsilly gerrit20:40
*** jaosorior has joined #openstack-infra20:42
clarkbas a heads up I will have to pop out this afternoon to watch kids.20:43
*** pcaruana has quit IRC20:43
clarkbetherpad backups still going so I think we crossed whatever threshold that was a problem20:43
*** eharney has quit IRC20:44
pabelangerya, lists.o.o still going, 15GB now20:44
monester_can someone help tell me what should be done to get patch with jenkins pipeline support get merged - https://review.openstack.org/#/c/323744/ it already has +2 on it and jenkins +1, but not workflow +120:45
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Add zuulv3-dev.openstack.org server  https://review.openstack.org/43041820:45
clarkbpabelanger: 15GB free?20:45
pabelangerclarkb: ya, 112GB free20:45
clarkbpabelanger: what is 15GB?20:46
clarkbmonester_: typically we require two core reviewers to +2 and the second will give the +1 workflow which triggers the merge machinery20:46
pabelangerclarkb: the splitting processes20:46
pabelangerSplitting: 15398912 kbytes20:46
clarkbpabelanger: ah20:46
*** gouthamr has joined #openstack-infra20:46
openstackgerritMerged openstack-infra/system-config master: Break out openstack_project::server from zuul_merger.pp  https://review.openstack.org/43040820:47
*** gouthamr has quit IRC20:47
clarkbmonester_: zaro has already +2'd. zxiiro is another reviewer20:47
clarkbthe other cores don't seem to be on irc at the moment20:47
*** sekelso has quit IRC20:47
*** sekelso has joined #openstack-infra20:47
openstackgerritMerged openstack-infra/system-config master: Break out openstack_project::server from zuul_dev.pp  https://review.openstack.org/43041920:47
clarkbpabelanger: I hadn't noticed that that was a running tally20:47
openstackgerritMerged openstack-infra/system-config master: Break out openstack_project::server from zuul_prod.pp  https://review.openstack.org/43042720:48
monester_clarkb: thanks, I'll try to reach zxiiro via irc and ask it to review the patch20:48
clarkbpabelanger: and ya just noticed bup trying to backup its .bup dir and a file was removed before it could read it. So thinking we should add taht to the excludes list too20:50
clarkbpabelanger: I will push that patch shortly20:50
pabelangerk20:51
*** jlibosva has quit IRC20:51
clarkbfungi: re whitelist vs blacklist the nice thing about the blacklist is we have a single list for all the servers rather than needing to curate a whitelist for each server (granted we might be able to make a whitelist that works for all too)20:51
*** esikachev has quit IRC20:51
fungidownside is we have lots of sensitive plaintext credentials scattered all over those filesystems, so anyone who gets access to our backups likely has (most of?) the keys to the kingdom20:54
fungiunless we do some very specific exclusions for each and every system to avoid copying them20:54
*** sdake has joined #openstack-infra20:54
clarkbya20:55
openstackgerritClark Boylan proposed openstack-infra/puppet-bup master: Exclude bups local /root/.bup dir  https://review.openstack.org/43116120:56
clarkbpabelanger: ^ based it on your change20:56
*** salv-orlando has quit IRC20:56
*** EricGonczer_ has quit IRC20:57
*** sdake_ has quit IRC20:58
* clarkb wanders off to find lunch20:58
*** chem has quit IRC20:59
*** wolverineav has quit IRC20:59
pabelanger++20:59
*** wolverineav has joined #openstack-infra20:59
*** wolverineav has quit IRC20:59
*** EricGonczer_ has joined #openstack-infra21:00
*** trown is now known as trown|brb21:00
*** wolverineav has joined #openstack-infra21:01
zxiiromonester_: merged21:02
monester_zxiiro: thanks!21:03
*** jamielennox|away is now known as jamielennox21:03
openstackgerritDoug Hellmann proposed openstack-infra/release-tools master: remove list-repos command  https://review.openstack.org/43116921:04
openstackgerritDoug Hellmann proposed openstack-infra/release-tools master: move propose-final-releases command to releases repo  https://review.openstack.org/43117021:04
openstackgerritDoug Hellmann proposed openstack-infra/release-tools master: move list_unreleased_changes.sh to the releases repo  https://review.openstack.org/43117121:04
openstackgerritDoug Hellmann proposed openstack-infra/release-tools master: move list_library_unreleased_changes.sh to releases repo  https://review.openstack.org/43117221:04
*** sekelso has quit IRC21:04
openstackgerritDoug Hellmann proposed openstack-infra/release-tools master: remove list_oslo_unreleased_changes.sh  https://review.openstack.org/43117321:04
openstackgerritDoug Hellmann proposed openstack-infra/release-tools master: move list_stable_unreleased_changes.sh to releases repo  https://review.openstack.org/43117421:04
openstackgerritDoug Hellmann proposed openstack-infra/release-tools master: remove branching scripts  https://review.openstack.org/43117521:04
openstackgerritDoug Hellmann proposed openstack-infra/release-tools master: remove ptl command  https://review.openstack.org/43117621:04
openstackgerritDoug Hellmann proposed openstack-infra/release-tools master: remove highest_semver.py  https://review.openstack.org/43117721:04
openstackgerritDoug Hellmann proposed openstack-infra/release-tools master: remove latest-deliverable-versions command  https://review.openstack.org/43117821:04
*** jcoufal_ has joined #openstack-infra21:05
openstackgerritMerged openstack-infra/jenkins-job-builder master: Add Pipeline job type and deprecate Workflow  https://review.openstack.org/32374421:05
*** sekelso has joined #openstack-infra21:06
*** sree has quit IRC21:07
*** jcoufal has quit IRC21:08
*** cody-somerville has quit IRC21:09
*** eharney has joined #openstack-infra21:09
*** sree has joined #openstack-infra21:10
openstackgerritMerged openstack-infra/puppet-bup master: Excluded /etc/puppet/modules/*  https://review.openstack.org/43111621:10
openstackgerritMerged openstack-infra/puppet-bup master: Exclude bups local /root/.bup dir  https://review.openstack.org/43116121:10
*** trown|brb is now known as trown21:13
*** hashar has quit IRC21:14
*** askb has joined #openstack-infra21:15
*** cody-somerville has joined #openstack-infra21:16
*** cody-somerville has joined #openstack-infra21:16
*** zz_dimtruck is now known as dimtruck21:18
*** adriant has joined #openstack-infra21:20
*** matrohon has quit IRC21:22
clarkbwoot etherpad.o.o finished buping21:24
clarkbI will move on to the next server as soon as lunch is done21:24
*** dave-mccowan has quit IRC21:26
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE - testing TLS everywhere job with RabbitMQ  https://review.openstack.org/42630721:29
*** adrian_otto has quit IRC21:29
*** esikachev has joined #openstack-infra21:32
*** Jeffrey4l has quit IRC21:34
*** Jeffrey4l has joined #openstack-infra21:34
*** esikachev has quit IRC21:36
*** salv-orlando has joined #openstack-infra21:40
*** jordanP has joined #openstack-infra21:40
*** marst has quit IRC21:41
*** marst has joined #openstack-infra21:45
*** vsaienko has quit IRC21:45
*** dave-mccowan has joined #openstack-infra21:46
*** vhosakot has quit IRC21:46
*** annegentle has quit IRC21:46
*** vhosakot has joined #openstack-infra21:46
pabelanger20gb for lists.o.o now21:47
openstackgerritJohn L. Villalovos proposed openstack-infra/project-config master: ironic: Put initial GRENADE_PLUGINRC in ironic-vars-early  https://review.openstack.org/43119221:47
*** vhosakot has quit IRC21:47
*** vhosakot has joined #openstack-infra21:48
*** adrian_otto has joined #openstack-infra21:48
*** bh526r has quit IRC21:48
*** esikachev has joined #openstack-infra21:49
openstackgerritStephen Watson proposed openstack-infra/project-config master: Run magnum migration on non-doc changes  https://review.openstack.org/43119321:50
*** dtardivel has quit IRC21:50
*** raildo has quit IRC21:51
*** thorst_ has quit IRC21:53
*** esikachev has quit IRC21:53
*** thorst_ has joined #openstack-infra21:53
*** fguillot has quit IRC21:54
*** jaosorior has quit IRC21:56
dims_sdague : hi, need a nail for a coffin please :) https://review.openstack.org/#/c/430882/21:56
*** thorst_ has quit IRC21:58
*** rfolco has joined #openstack-infra21:58
*** rfolco has quit IRC21:58
*** vhosakot has quit IRC22:01
*** vhosakot has joined #openstack-infra22:01
*** sdake has quit IRC22:02
*** sdake has joined #openstack-infra22:02
*** ihrachys_ has quit IRC22:05
openstackgerritPaul Belanger proposed openstack-infra/puppet-openstackci master: Add openstackci::zuul_launcher  https://review.openstack.org/43120122:09
*** dprince has quit IRC22:09
*** vhosakot_ has joined #openstack-infra22:09
clarkbok I am back. Going to do ask next22:10
*** yamamoto has joined #openstack-infra22:10
*** vhosakot_ has quit IRC22:10
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Switch to openstackci::zuul_launcher  https://review.openstack.org/43120222:11
*** vhosakot_ has joined #openstack-infra22:11
*** vhosakot has quit IRC22:13
clarkbask is running now22:14
*** vhosakot_ has quit IRC22:14
*** vhosakot has joined #openstack-infra22:16
*** gyee has joined #openstack-infra22:17
*** annegentle has joined #openstack-infra22:18
*** gouthamr has joined #openstack-infra22:20
*** ccamacho has quit IRC22:21
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Remove unused variables for zuul_launcher  https://review.openstack.org/43120622:22
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Remove unused variables for zuul_launcher  https://review.openstack.org/43120622:23
*** trown is now known as trown|outtypewww22:23
*** rhallisey has quit IRC22:26
*** panda is now known as panda|Zz22:26
*** jordanP has quit IRC22:27
*** cody-somerville has quit IRC22:28
*** jkilpatr has quit IRC22:28
mriedemAJaeger: https://review.openstack.org/#/c/429391/ seems fine, i noticed you cleaned up some nits in cinder's version of this,22:29
mriedemnot sure how much you care about keeping this consistent between repos22:29
*** matt-borland has quit IRC22:29
*** EricGonczer_ has quit IRC22:30
openstackgerritBrian Haley proposed openstack-infra/project-config master: neutron/tempest: make tempest dvr-multinode voting  https://review.openstack.org/41097322:31
*** burgerk has quit IRC22:31
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci master: Add postci metric  https://review.openstack.org/43120922:31
*** dave-mccowan has quit IRC22:31
*** sdake_ has joined #openstack-infra22:32
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs  https://review.openstack.org/35921522:32
*** baoli has quit IRC22:33
*** erlon has quit IRC22:35
*** srobert__ has quit IRC22:35
*** vhosakot has quit IRC22:35
*** sdake has quit IRC22:36
*** ccamacho has joined #openstack-infra22:36
*** jheroux has quit IRC22:36
*** Hal has joined #openstack-infra22:39
*** Hal is now known as Guest2543422:40
clarkbask.o.o bup is still running22:43
clarkbI have to largely afk but will keep an eye on ask as the day goes on22:43
clarkbgroups, review, storyboard and wiki are the other servers that will need attention. I will pick it back up tomorrow if others dont manage to get them done today22:44
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Add zuulv3-dev.openstack.org server  https://review.openstack.org/43041822:45
*** mriedem has quit IRC22:46
pabelangerheh, starting to get crossed eyed looking at all the puppet layers22:46
*** Goneri has quit IRC22:49
mordredpabelanger: just cover them with icing22:50
*** esikachev has joined #openstack-infra22:50
*** inc0 has quit IRC22:52
*** cody-somerville has joined #openstack-infra22:54
*** esikachev has quit IRC22:55
tonybfungi: possible for the last time (re: double booking)22:56
*** jkilpatr has joined #openstack-infra22:56
openstackgerritKendall Nelson proposed openstack-infra/storyboard master: Handle Branches in Migration Scripts  https://review.openstack.org/43121522:57
pabelanger26gb for lists.o.o and climbing22:57
pabelangerre: bup22:58
*** Swami_ has joined #openstack-infra22:58
*** lseki has quit IRC23:00
*** aeng has quit IRC23:00
*** Swami has quit IRC23:01
*** mdrabe has quit IRC23:01
*** wolverineav has quit IRC23:02
*** wolverineav has joined #openstack-infra23:03
*** esikachev has joined #openstack-infra23:04
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Add wait_for from tripleo-incubator  https://review.openstack.org/42533523:04
*** Swami_ has quit IRC23:04
*** jcoufal_ has quit IRC23:05
*** wolverineav has quit IRC23:09
ianwwe're still getting really frequent static.openstack.org disconnects, here's a sample -> http://paste.openstack.org/show/598152/23:09
*** esikachev has quit IRC23:10
ianwthink i should try out https://review.openstack.org/#/c/426639/ ?23:10
*** wolverineav has joined #openstack-infra23:11
*** jaypipes has quit IRC23:13
*** annegentle has quit IRC23:13
*** aeng has joined #openstack-infra23:18
*** admcleod_ has joined #openstack-infra23:18
*** admcleod has quit IRC23:20
*** denisra has quit IRC23:20
*** rkukura has joined #openstack-infra23:21
*** cody-somerville has quit IRC23:22
*** cody-somerville has joined #openstack-infra23:22
*** cody-somerville has joined #openstack-infra23:22
*** wolverineav has quit IRC23:22
*** krtaylor has quit IRC23:25
*** ajo has quit IRC23:26
*** _ari_ has quit IRC23:26
*** ajo has joined #openstack-infra23:27
*** _ari_ has joined #openstack-infra23:27
*** flepied has quit IRC23:27
*** pradk has quit IRC23:29
*** zhurong has joined #openstack-infra23:30
*** rkukura has quit IRC23:32
*** dimtruck is now known as zz_dimtruck23:35
*** zhurong has quit IRC23:36
*** skelso has joined #openstack-infra23:36
*** sekelso has quit IRC23:37
*** aeng has quit IRC23:42
*** manjeets has joined #openstack-infra23:42
*** dingyichen has joined #openstack-infra23:45
*** vsaienko has joined #openstack-infra23:46
manjeetshi recently I was seeing the tempest test failure issue over an grenade job in networking-odl23:46
manjeetswhich was basically because of permissions23:46
manjeetsand this patch fixes it23:46
manjeetshttps://review.openstack.org/#/c/425841/3/devstack-vm-gate.sh23:46
manjeets can this patch be accepted ?23:46
manjeets23:46
*** Swami has joined #openstack-infra23:46
*** gordc has quit IRC23:46
*** marst has quit IRC23:46
manjeetshere is log of failing tests http://logs.openstack.org/43/425843/2/check/gate-grenade-dsvm-networking-odl-nv/51dfb82/testr_results.html.gz23:46
*** vsaienko has quit IRC23:47
clarkbmanjeets: I dont think that will affect grenade jobs will it?23:47
clarkbmanjeets: grenade runs its own tempest and takes ocer the job once it starts23:48
manjeetsbut the permissions are being handled in devstack-gate23:48
*** sshnaidm is now known as sshnaidm|off23:48
clarkbmanjeets: not for grenade I dont think so23:49
clarkbonce grenade starts it handles all test exwcution so we never hit that part of devstack gate23:50
*** andreww has quit IRC23:50
*** gildub has joined #openstack-infra23:50
clarkbmanjeets: see the else n line 64023:51
clarkbabove that is grenade, below that is not grenade we are below in the else block for not grenade23:51
manjeetshttp://logs.openstack.org/43/425843/4/check/gate-grenade-dsvm-networking-odl-nv/9abf3cf/console.html#_2017-02-06_19_08_24_42207023:52
manjeetsclarkb, ^^ these tests are being run after grenade.sh is completed23:54
manjeetshttp://logs.openstack.org/43/425843/4/check/gate-grenade-dsvm-networking-odl-nv/9abf3cf/console.html#_2017-02-06_19_25_23_51708523:54
*** aeng has joined #openstack-infra23:55
*** baoli has joined #openstack-infra23:55
clarkboh huh I thought grenade ran temoest seems yhe block ends earlier than I though23:55
clarkbmanjeets: and you nees tempest to be able to read the images for some reason? maybe you can just set them to be globally readablr when copied instead?23:57
clarkbthat way all tempest invocations can do the reads23:57
*** tojuvone has quit IRC23:58
manjeetsclarkb you mean set them to be readable globally in devstack-gate ?23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!