Tuesday, 2018-04-03

*** pahuang has quit IRC00:00
*** germs has quit IRC00:01
*** jistr has quit IRC00:02
*** jistr has joined #openstack-infra00:03
*** olaph1 has joined #openstack-infra00:11
*** HeOS has quit IRC00:11
*** olaph has quit IRC00:11
*** odyssey4me has quit IRC00:14
*** odyssey4me has joined #openstack-infra00:14
*** pahuang has joined #openstack-infra00:17
*** caphrim007 has joined #openstack-infra00:24
*** jroll has quit IRC00:24
*** caphrim007 has quit IRC00:28
*** sree has joined #openstack-infra00:29
*** sree has quit IRC00:34
*** andreww has quit IRC00:35
*** shu-mutow-AWAY is now known as shu-mutow00:36
*** camunoz has quit IRC00:38
openstackgerritKendall Nelson proposed openstack-infra/project-config master: Adds Cyborg to StoryBoard  https://review.openstack.org/55832700:40
*** bobh_ has joined #openstack-infra00:45
*** pahuang has quit IRC00:45
*** jamesmcarthur has joined #openstack-infra00:48
*** annp has quit IRC00:50
*** annp has joined #openstack-infra00:51
*** bobh_ has quit IRC00:52
*** yamamoto has joined #openstack-infra00:53
*** pahuang has joined #openstack-infra00:58
*** yamamoto has quit IRC00:59
*** markvoelker_ has joined #openstack-infra01:02
*** hongbin has joined #openstack-infra01:05
*** gouthamr has joined #openstack-infra01:06
*** markvoelker has quit IRC01:06
*** ameliac has joined #openstack-infra01:08
*** zhangfei has joined #openstack-infra01:09
*** annp has quit IRC01:12
*** annp has joined #openstack-infra01:13
*** gyankum has joined #openstack-infra01:14
pabelangerclarkb: ianw: https://review.openstack.org/558237/ configures ubuntu-bionic to use AFS mirrors for nodepool-builders01:21
*** salv-orl_ has quit IRC01:21
dmsimardinfra-root: I have two held fedora nodes to help troubleshoot an Ansible docker module issue (they're labelled after https://review.openstack.org/#/c/557933/), let me know if it's a problem .. issue is https://github.com/ansible/ansible/issues/2049201:21
*** salv-orlando has joined #openstack-infra01:22
clarkbdmsimard: should be fine just make sure the reason field is set so we can figure it out later01:22
dmsimardcan we update the reason ?01:22
clarkbya I think yoi can?01:23
clarkbBut maybe not once the node is actually held01:23
dmsimard{list,image-list,dib-image-list,image-build,alien-image-list,delete,image-delete,dib-image-delete,config-validate,request-list,info,erase}01:23
dmsimardah, in the autohold request01:24
openstackgerritJohn Hinman proposed openstack-infra/project-config master: Add python35 job to neutron-classifier.  https://review.openstack.org/55832901:24
dmsimardBTW I really love the cliff framework, the same as openstackclient. Is there a particular reason we're not using it ?01:24
dmsimardah, a question for #zuul01:24
clarkbdmsimard: its slow01:25
clarkbit incurs pkg_resources overhead01:25
pabelangerclarkb: mordred: I think we may have leaked volumes again in vexxhost. Maybe in the morning we can brain storm how to have nodepool / shade help detect that and do some auto clean up01:26
*** caphrim007 has joined #openstack-infra01:27
pabelangermnaser: ^01:28
clarkbpabelanger: we should also investigate why nova delete doesnt delete the volumes01:29
ianwclarkb: mind if i update the comment on 557842 as resolvconf isn't involved01:29
clarkbianw: sure01:29
dmsimardpabelanger: are we setting terminate on destroy ? (or whatever it is called)01:30
pabelangerclarkb: yah, can ask mnaser for help with that01:30
pabelangerdmsimard: yes, nodepool does01:32
pabelangerjust not properly getting cleaned up on openstack side I think01:32
pabelangerdmsimard: are you still using the boot-from-volume in rdoproject?01:33
dmsimardhmmm, so in glance you know how we can't delete the image if there's still a cinder volume around for that image ? I wonder if we're hitting some kind of loop01:34
*** jamesmcarthur has quit IRC01:34
dmsimardpabelanger: not since they changed the spindles for ssds01:34
*** jamesmcarthur has joined #openstack-infra01:34
pabelangerdmsimard: I take it never had a leaked volume?01:34
dmsimardit was mostly working fine for us afaik01:34
dmsimardwell, yeah.. but for other reasons :)01:34
*** cshastri_ has joined #openstack-infra01:35
*** felipemonteiro__ has joined #openstack-infra01:36
*** jamesmcarthur has quit IRC01:37
*** jamesmcarthur has joined #openstack-infra01:37
*** gouthamr has quit IRC01:37
*** gouthamr has joined #openstack-infra01:42
dmsimardhuh, I just got an idea.01:43
*** dayou has quit IRC01:44
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: mqtt: add basic reporter  https://review.openstack.org/53554301:44
*** diablo_rojo has quit IRC01:47
*** harlowja has quit IRC01:50
pabelangerclarkb: ianw: etcd fix for nodepool-builder just landed01:51
*** anteaya has quit IRC01:51
*** anteaya has joined #openstack-infra01:53
*** yamamoto has joined #openstack-infra01:55
*** jamesmcarthur has quit IRC01:56
*** germs has joined #openstack-infra01:57
*** germs has quit IRC01:57
*** germs has joined #openstack-infra01:57
*** dayou has joined #openstack-infra01:58
ianwok, well yeah, that was queens right?  we might still look on pike?01:58
*** yamamoto has quit IRC02:00
*** ramishra has joined #openstack-infra02:01
*** germs has quit IRC02:01
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Setup AFS mirrors for ubuntu-xenial-arm64  https://review.openstack.org/55833102:01
dmsimardhey it looks like we have some leaked nodes in rax-iad http://paste.openstack.org/raw/718218/02:01
dmsimardI noticed when checking for something else02:02
pabelangerianw: no, we made it past the error it looks like02:02
pabelangerianw: clarkb: ^also updates nb03.o.o to use AFS mirrors from linaro02:02
pabelangerianw: but ya, just queens02:03
ianwok; maybe we didn't have that extra-cache code in pike02:03
*** gouthamr_ has joined #openstack-infra02:04
ianwclarkb / pabelanger : i'm wondering if the disablement of systemd-resolvd is, long term, going to cause problems.  if things are querying dns via it's bus-api stuff, and it's not there02:04
ianwit actually pretty much seems to do the right thing wrt to not touching resolv.conf if it is there02:05
clarkbianw: will it lookup over dbus to our unbound?02:07
clarkbif not maybe failing is correct?02:07
clarkbthe dbus stuff is :( though02:07
pabelangerlast I looked, we don't install dbus in DIB. But that might have changed with systemd things02:08
*** gouthamr has quit IRC02:08
ianwclarkb: well i mean presumably at some point, apps will come along that use this dbus name query stuff, and i think the only thing that will answer is systemd-resolvd02:08
pabelangeryah, dbus looks to be a hard dependency for bionic now02:09
pabelangerxenial it isn't02:09
clarkbianw: right but does it respect resolv.conf and use unbound for its lookups or just do its own conifg?02:09
ianwpableanger: yeah, i htink that's a failure though.  i'm sure i've built systems that you can't log into02:09
ianwclarkb: from what i understand, if /etc/resolv.conf is present and a file (more accurately, not a symlink pointing to one of the "compatability" resolv.conf files that systemd creates), it will obey it02:10
ianwwhich seems right02:10
clarkbcool02:11
clarkbso my hack should work then in a reaosnable way?02:11
clarkbwe can also emable resolved?02:11
pabelangersystemd-resolved?02:11
clarkbthen we have to sort out the fight fir localhost:53?02:11
clarkbpabelanger: ya02:11
ianwit's also fairly reasonable for the systemd postinst scripts to assume if there's no /etc/resolv.conf file there, that it should point it to the systemd-resolvd created files02:11
clarkbheh02:12
pabelangerclarkb: when I looked, it will force all dns requests to 127.0.53, and by pass unbound02:12
ianwclarkb: on that, my reading was that it binds to 127.0.0.53 ?02:12
ianwnot 127.0.0.1:5302:12
clarkbgotcha02:12
*** bobh_ has joined #openstack-infra02:12
pabelangeryah02:12
pabelangernameserver 127.0.0.53 was the original issue02:12
clarkbpabelanger: how does it do that? if resolv.conf says use 127.0.0.1 worst case systemd resolved should just proxy to unbound02:13
clarkbso evwm if rwsolved is used that should be ok if unbound is its server02:13
ianwright, and that would be because /etc/resolv.conf was pointing to systemd-resolvd's files02:13
clarkbah02:13
ianwin that case, when /etc/resolv.conf is a pointer to systemd-resolvd's provided files, it's not actually providing any configuration to systemd-resolvd, if that makes sense02:15
clarkbya its on resolved to dhcp or whatevwr02:15
ianwit's just being put in by systemd-resolvd as a compatibility thing, because people expect the file to be there02:15
clarkbdoes this mean we can hard set localhost in the image build then just use rc.local to avoid rax doing special stuff?02:16
clarkbthrn resolved will lookup against unbound and dbus will work and we get caching and all that02:16
ianwquite possibly.  dhcpd is the only thing that might overwrite it i think02:17
pabelangerno, I think we disable that still02:18
pabelangerin dhclient.conf02:18
*** zhenguo has joined #openstack-infra02:18
pabelangerhttp://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/elements/nodepool-base/finalise.d/89-unbound#n15202:18
pabelangerwe supersede dns02:19
pabelangerianw: clarkb: corvus: what are your thoughts on https://review.openstack.org/557845/ to help resolve git clone hanging for diskimage-builder, try in nodepool.yaml first then push into diskimage-builder if works?02:21
pabelangerthat is the same logic we use on zuul mergers for same issue02:21
clarkbI'll have to look in the morning02:22
pabelangernp02:22
ianwseems ok, will read up on it ... so many options :)02:25
*** armaan_ has quit IRC02:30
*** armaan has joined #openstack-infra02:31
*** harlowja has joined #openstack-infra02:31
*** anteaya has quit IRC02:32
*** daidv has joined #openstack-infra02:32
*** daidv_ has joined #openstack-infra02:33
*** xinliang3 has joined #openstack-infra02:34
*** anteaya has joined #openstack-infra02:36
*** jamesmcarthur has joined #openstack-infra02:36
*** psachin has joined #openstack-infra02:36
ianwclarkb / pabelanger : now i'm wondering if we set /etc/resolv.conf to 127.0.0.1 during the build, if we break name resolution in the chroot02:40
dmsimardshould we open a rackspace ticket to get rid of those leaked instances ?02:40
dmsimardI'm trying an "openstack server show" on one of them and it's not coming back..02:40
clarkbianw: oh ha probably02:41
dmsimardoh, it eventually returned... this one is up since 2018-02-0202:41
pabelangerclarkb: ianw: it's finalize.d we shouldn't need to install anything else right?02:41
pabelangerclarkb: ianw or, guess we make that the last things we do02:41
ianwpabelanger: you know, now i think about it, we setup the resolv.conf in the chroot from the host machine02:42
ianwhmm, we're missing a lot of debug output from http://nb01.openstack.org/ubuntu-xenial-0000001385.log02:47
*** liujiong has joined #openstack-infra02:49
*** daidv_ has quit IRC02:49
dmsimardHuh, it's interesting that the gnu.org git server provides different options to browse git repos: http://git.savannah.gnu.org/02:52
*** zhurong has joined #openstack-infra02:53
*** yamamoto has joined #openstack-infra02:56
*** gouthamr_ has quit IRC03:02
*** yamamoto has quit IRC03:02
*** gyan_ has joined #openstack-infra03:04
*** chkumar246 has joined #openstack-infra03:04
*** chandankumar has quit IRC03:04
*** gyankum has quit IRC03:04
*** cshastri has joined #openstack-infra03:04
*** gyan__ has joined #openstack-infra03:05
*** cshastri_ has quit IRC03:08
*** gyan_ has quit IRC03:09
*** chkumar246 has quit IRC03:09
*** zhangfei has quit IRC03:10
*** bobh_ has quit IRC03:11
*** agopi has quit IRC03:14
*** agopi has joined #openstack-infra03:15
*** chkumar246 has joined #openstack-infra03:22
*** zhangfei has joined #openstack-infra03:22
*** spzala has joined #openstack-infra03:24
*** spzala has quit IRC03:24
*** iyamahat has quit IRC03:25
*** links has joined #openstack-infra03:32
*** yamamoto has joined #openstack-infra03:33
*** jamesmcarthur has quit IRC03:35
*** nicolasbock has quit IRC03:40
*** caphrim007 has quit IRC03:42
*** felipemonteiro__ has quit IRC03:43
*** ykarel|away has joined #openstack-infra03:49
*** zhurong has quit IRC03:52
*** udesale has joined #openstack-infra03:53
*** harlowja has quit IRC03:55
*** hongbin has quit IRC03:56
*** annp has quit IRC03:57
*** daidv has quit IRC03:57
*** germs has joined #openstack-infra03:57
*** germs has quit IRC03:57
*** germs has joined #openstack-infra03:58
*** daidv has joined #openstack-infra03:58
*** annp has joined #openstack-infra03:58
*** germs has quit IRC04:02
ianwhmm, we loose dib debugging output soon after running one of the dib utilities in nodepool.  interesting ...04:03
*** ykarel|away is now known as ykarel04:05
*** jamesmcarthur has joined #openstack-infra04:10
*** markvoelker_ has quit IRC04:16
*** eernst has joined #openstack-infra04:22
*** xinliang3 has quit IRC04:26
*** zhenguo has quit IRC04:27
*** eernst has quit IRC04:27
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Restore tracing on exit points of block_device_create_config_file  https://review.openstack.org/55835804:35
* ianw breathes a sigh of relief that it's not due to some weird thread/fd/outfilter.py interaction04:35
*** dhajare__ has joined #openstack-infra04:37
*** andreas_s has joined #openstack-infra04:39
*** andreas_s has quit IRC04:43
*** jamesmcarthur has quit IRC04:45
*** yamamoto has quit IRC04:45
*** yamamoto has joined #openstack-infra04:46
*** xinni9e_ has joined #openstack-infra04:49
*** lpetrut has joined #openstack-infra04:50
*** jaosorior has joined #openstack-infra04:57
*** agopi has quit IRC05:04
*** pgadiya has joined #openstack-infra05:08
*** garyk has quit IRC05:10
*** liujiong has quit IRC05:11
xinni9e_Hi there, I am working on several newly added xstatic-* repositories. I already have verified openstack/xstatic-* repos and pypi packages tagged 0.0.0, and want to publish the first useful release. I am reading this doc https://docs.openstack.org/infra/manual/creators.html , and wondering should I just follow the section `Tagging a Release` and `Update the Global Requirements List` to proceed?05:16
*** e0ne has joined #openstack-infra05:18
*** e0ne has quit IRC05:22
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Formalise saving of /etc/resolv.conf  https://review.openstack.org/55836205:29
ianwxinni9e_: yes, it should just be a matter of pushing a signed tag as in that document05:29
*** jtomasek has joined #openstack-infra05:33
*** e0ne has joined #openstack-infra05:34
*** jtomasek has quit IRC05:35
*** snapiri has joined #openstack-infra05:37
clarkbunless they are release team managed05:38
clarkbnot sure in the case if xstatic if they are05:38
xinni9e_ianw: thank you, I will have a try.05:38
openstackgerritIan Wienand proposed openstack-infra/project-config master: Revert "Stop systemd-resolve from booting on our images"  https://review.openstack.org/55836705:39
*** ameliac has quit IRC05:41
*** olaph has joined #openstack-infra05:47
*** lpetrut has quit IRC05:48
*** lpetrut has joined #openstack-infra05:48
*** garyk has joined #openstack-infra05:48
*** olaph1 has quit IRC05:49
*** e0ne has quit IRC05:50
*** jtomasek has joined #openstack-infra05:51
*** lpetrut has quit IRC05:54
*** apetrich has joined #openstack-infra05:54
*** germs has joined #openstack-infra05:58
*** germs has quit IRC05:58
*** germs has joined #openstack-infra05:58
*** germs has quit IRC06:02
*** olaph1 has joined #openstack-infra06:03
*** olaph has quit IRC06:04
*** iranzo has joined #openstack-infra06:04
*** iranzo has quit IRC06:04
*** iranzo has joined #openstack-infra06:04
*** garyk has quit IRC06:10
*** daidv has quit IRC06:13
*** pcichy has quit IRC06:17
*** markvoelker has joined #openstack-infra06:17
*** lpetrut has joined #openstack-infra06:21
*** rfolco|off has quit IRC06:27
*** rfolco|off has joined #openstack-infra06:27
*** aviau has quit IRC06:30
*** aviau has joined #openstack-infra06:30
*** aojea has joined #openstack-infra06:31
*** lpetrut has quit IRC06:33
*** namnh has joined #openstack-infra06:34
*** olaph has joined #openstack-infra06:38
*** olaph1 has quit IRC06:40
*** florianf has joined #openstack-infra06:42
*** markvoelker has quit IRC06:51
*** armaan_ has joined #openstack-infra06:53
*** armaan has quit IRC06:53
*** pcaruana has joined #openstack-infra06:53
*** tesseract has joined #openstack-infra07:00
*** afazekas has quit IRC07:01
*** afazekas has joined #openstack-infra07:01
xinliangianw, please take a look this patch, if you have time. https://review.openstack.org/#/c/557902/07:02
xinliangianw: it might need to setup debian arm64 nodes for linaro cloud.07:04
*** ykarel_ has joined #openstack-infra07:04
*** gyan_ has joined #openstack-infra07:04
*** pgadiya_ has joined #openstack-infra07:04
*** imacdonn has quit IRC07:04
*** cshastri_ has joined #openstack-infra07:04
*** chandankumar has joined #openstack-infra07:04
*** imacdonn has joined #openstack-infra07:04
xinliangnow the lacking part maybe AFS debian mirror07:04
*** gyan__ has quit IRC07:04
*** ykarel has quit IRC07:04
*** links has quit IRC07:05
*** pgadiya has quit IRC07:05
*** links has joined #openstack-infra07:05
*** chkumar246 has quit IRC07:05
*** gyan__ has joined #openstack-infra07:05
*** ykarel__ has joined #openstack-infra07:06
*** yolanda__ is now known as yolanda07:06
*** cshastri has quit IRC07:08
*** pgadiya_ has quit IRC07:09
*** cshastri_ has quit IRC07:09
*** gyan_ has quit IRC07:09
*** chandankumar has quit IRC07:09
*** ykarel_ has quit IRC07:09
*** tosky has joined #openstack-infra07:11
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Formalise saving of /etc/resolv.conf  https://review.openstack.org/55836207:18
*** aviau has quit IRC07:19
*** aviau has joined #openstack-infra07:19
ianwxinliang: has anyone actually built debian-stretch with dib on arm64?07:20
*** cshastri_ has joined #openstack-infra07:20
*** pgadiya_ has joined #openstack-infra07:20
xinliangianw: yes, I buit it locally. without project-configs' elemets.07:21
*** chandankumar has joined #openstack-infra07:21
xinliangianw: with debian-minimal vm efi elements07:22
xinliangianw: now the lacking part maybe AFS debian mirror.07:26
ianwxinliang: yeah, that might result in some instability07:26
*** e0ne has joined #openstack-infra07:26
ianwxinliang: if you clone project-config theres a file tools/build_image.sh that is a pretty good description of how to build, with project-config elements too07:27
ianwbut, if debian-minimal works, that is the majority of the issues sorted out07:27
ianwlet's sync up with what pabelanger thinks about the mirror in the meeting; i can help bring up what we need07:28
xinliangianw: yeah, will look into project-config for that.07:29
*** hashar has joined #openstack-infra07:29
*** ykarel__ is now known as ykarel07:30
*** amoralej|off is now known as amoralej07:34
*** jpena|off is now known as jpena07:37
*** jaosorior has quit IRC07:38
*** andreas_s has joined #openstack-infra07:39
*** yamamoto_ has joined #openstack-infra07:42
*** andreas_s has quit IRC07:44
*** jaosorior has joined #openstack-infra07:44
*** yamamoto has quit IRC07:45
*** markvoelker has joined #openstack-infra07:47
*** alexchadin has joined #openstack-infra07:51
*** electrofelix has joined #openstack-infra07:54
*** dhajare__ has quit IRC07:58
*** germs has joined #openstack-infra07:59
*** germs has quit IRC07:59
*** germs has joined #openstack-infra07:59
*** germs has quit IRC08:03
*** links has quit IRC08:04
*** jpich has joined #openstack-infra08:06
*** dmellado has quit IRC08:08
*** dmellado has joined #openstack-infra08:08
*** bmace has quit IRC08:11
*** shardy has joined #openstack-infra08:11
*** lucas-pto is now known as lucasagomes08:11
*** zhangfei has quit IRC08:13
*** links has joined #openstack-infra08:14
*** zhurong has joined #openstack-infra08:14
*** gcb has quit IRC08:14
*** zoliXXL is now known as zoli08:17
*** andreas_s has joined #openstack-infra08:20
*** markvoelker has quit IRC08:21
*** electrofelix has quit IRC08:21
*** bmace has joined #openstack-infra08:22
*** xinni9e_ has quit IRC08:22
*** gcb has joined #openstack-infra08:22
*** electrofelix has joined #openstack-infra08:26
*** ralonsoh has joined #openstack-infra08:29
*** gcb has quit IRC08:30
*** efoley has joined #openstack-infra08:30
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Fix sql reporting with postgres  https://review.openstack.org/53572008:34
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Add tests for postgresql  https://review.openstack.org/54951808:34
*** lpetrut has joined #openstack-infra08:35
*** HeOS has joined #openstack-infra08:35
*** dklyle has quit IRC08:40
*** dklyle has joined #openstack-infra08:41
*** kdas__ has joined #openstack-infra08:43
*** dhajare__ has joined #openstack-infra08:44
*** kdas__ is now known as kushal08:46
*** gcb has joined #openstack-infra08:46
*** kushal has quit IRC08:46
*** kushal has joined #openstack-infra08:46
*** cshastri_ is now known as cshastri08:50
*** kushal has quit IRC08:54
*** claudiub has joined #openstack-infra08:58
*** pcichy has joined #openstack-infra09:00
*** gfidente has joined #openstack-infra09:02
*** gfidente has joined #openstack-infra09:02
*** namnh has quit IRC09:04
*** annp has quit IRC09:04
*** namnh has joined #openstack-infra09:05
*** annp has joined #openstack-infra09:05
*** efoley_ has joined #openstack-infra09:05
*** efoley has quit IRC09:08
*** sambetts|afk is now known as sambetts09:14
*** markvoelker has joined #openstack-infra09:18
*** gcb has quit IRC09:21
*** dmellado has quit IRC09:23
*** dmellado has joined #openstack-infra09:23
*** gcb has joined #openstack-infra09:28
openstackgerritAndrea Frittoli proposed openstack-infra/zuul-jobs master: Remove support for extensions as lists  https://review.openstack.org/54089009:30
*** armaan_ has quit IRC09:31
*** armaan has joined #openstack-infra09:31
*** garyk has joined #openstack-infra09:33
*** armaan has quit IRC09:36
*** alexchadin has quit IRC09:45
*** markvoelker has quit IRC09:52
*** garyk has quit IRC09:56
*** alexchadin has joined #openstack-infra09:57
openstackgerritFabien Boucher proposed openstack-infra/zuul master: Make Zuul able to start with a broken config  https://review.openstack.org/53551110:04
*** cshastri has quit IRC10:06
*** zoli is now known as zoli|lunch10:09
*** dtantsur|afk is now known as dtantsur10:18
*** florianf has quit IRC10:19
*** alexchadin has quit IRC10:20
*** florianf has joined #openstack-infra10:21
*** zhurong has quit IRC10:21
*** alexchadin has joined #openstack-infra10:22
*** alexchadin has quit IRC10:22
*** alexchadin has joined #openstack-infra10:23
*** armaan has joined #openstack-infra10:26
*** alexchadin has quit IRC10:27
*** zhurong has joined #openstack-infra10:33
*** nicolasbock has joined #openstack-infra10:33
*** garyk has joined #openstack-infra10:36
openstackgerritMatthieu Huin proposed openstack-infra/infra-specs master: Tenant-scoped Zuul admin web API  https://review.openstack.org/55846810:36
*** namnh has quit IRC10:38
*** pbourke has quit IRC10:47
*** pbourke has joined #openstack-infra10:47
*** markvoelker has joined #openstack-infra10:48
*** alexchadin has joined #openstack-infra10:51
openstackgerritMatthieu Huin proposed openstack-infra/infra-specs master: Tenant-scoped Zuul admin web API  https://review.openstack.org/55846810:52
*** florianf has quit IRC10:54
*** alexchadin has quit IRC10:56
*** alexchadin has joined #openstack-infra10:56
*** shu-mutow is now known as shu-mutow-AWAY11:01
*** lucasagomes is now known as lucas-hungry11:03
openstackgerritMatthieu Huin proposed openstack-infra/infra-specs master: Tenant-scoped Zuul admin web API  https://review.openstack.org/55846811:05
*** ianychoi_ has joined #openstack-infra11:11
*** florianf has joined #openstack-infra11:12
*** ianychoi has quit IRC11:14
*** cshastri has joined #openstack-infra11:15
*** rcernin has quit IRC11:20
*** markvoelker has quit IRC11:22
*** zoli|lunch is now known as zoli11:29
*** xinliang has quit IRC11:29
*** olaph has quit IRC11:31
*** olaph has joined #openstack-infra11:31
*** bobh has joined #openstack-infra11:33
*** dprince has joined #openstack-infra11:34
*** bobh has quit IRC11:35
*** rfolco|off is now known as rfolco|ruck11:37
*** yamamoto_ has quit IRC11:39
*** xinliang has joined #openstack-infra11:42
*** xinliang has quit IRC11:42
*** xinliang has joined #openstack-infra11:42
*** jamesmcarthur has joined #openstack-infra11:42
*** ldnunes has joined #openstack-infra11:42
*** jpena is now known as jpena|lunch11:46
*** jamesmcarthur has quit IRC11:47
*** pgadiya_ has quit IRC11:53
*** lucas-hungry is now known as lucasagomes11:59
*** ralonsoh_ has joined #openstack-infra12:01
*** ralonsoh has quit IRC12:04
*** panda|off is now known as panda12:05
*** amoralej is now known as amoralej|lunch12:06
*** dtantsur is now known as dtantsur|brb12:06
zigoxinliang: It'd be nice if you could modify infra to do BOTH amd64 and arm64.12:07
zigoxinliang: I will need amd64 for testing puppet-openstack with Debian.12:07
zigoIs it what your patch does?12:08
zigoIt looks like it...12:08
*** claudiub|2 has joined #openstack-infra12:08
*** jamesmcarthur has joined #openstack-infra12:10
*** claudiub has quit IRC12:11
*** ldnunes has quit IRC12:12
*** odyssey4me has quit IRC12:15
*** odyssey4me has joined #openstack-infra12:15
*** trown|outtypewww is now known as trown12:16
*** markvoelker has joined #openstack-infra12:19
*** gyan__ has quit IRC12:21
*** psachin has quit IRC12:22
*** alexchadin has quit IRC12:23
*** cshastri has quit IRC12:23
*** markvoelker has quit IRC12:23
*** markvoelker has joined #openstack-infra12:24
*** jamesmcarthur has quit IRC12:24
*** ldnunes has joined #openstack-infra12:25
*** alexchadin has joined #openstack-infra12:27
*** edmondsw has quit IRC12:28
*** edmondsw has joined #openstack-infra12:28
*** edmondsw has quit IRC12:32
openstackgerritFabien Boucher proposed openstack-infra/zuul master: Tenant config can be read from an external script  https://review.openstack.org/53587812:33
*** fried_bunny is now known as efried12:35
*** jroll has joined #openstack-infra12:37
*** dkehn has quit IRC12:39
*** yamamoto has joined #openstack-infra12:40
*** hashar has quit IRC12:42
*** VW has joined #openstack-infra12:43
*** hashar has joined #openstack-infra12:44
*** camunoz has joined #openstack-infra12:44
*** yamamoto has quit IRC12:45
*** gouthamr has joined #openstack-infra12:47
*** eharney has joined #openstack-infra12:47
*** eharney has quit IRC12:47
*** eharney has joined #openstack-infra12:48
*** lpetrut has quit IRC12:48
*** jamesmcarthur has joined #openstack-infra12:50
*** udesale has quit IRC12:50
*** jpena|lunch is now known as jpena12:50
*** lbragstad has joined #openstack-infra12:50
*** udesale has joined #openstack-infra12:50
*** lpetrut has joined #openstack-infra12:52
openstackgerritBharat Kunwar proposed openstack/diskimage-builder master: Fix yaml dependency issue with dracut-regenerate for python3  https://review.openstack.org/55849712:54
*** edmondsw has joined #openstack-infra12:58
*** gouthamr has quit IRC12:59
openstackgerritBharat Kunwar proposed openstack/diskimage-builder master: Fix yaml dependency issue with dracut-regenerate for python3  https://review.openstack.org/55849712:59
*** trown is now known as trown|brb12:59
*** gouthamr has joined #openstack-infra12:59
*** Goneri has joined #openstack-infra13:00
*** trown|brb is now known as trown13:01
*** jamesmcarthur has quit IRC13:02
*** jcoufal has joined #openstack-infra13:07
*** amoralej|lunch is now known as amoralej13:08
*** rosmaita has joined #openstack-infra13:09
*** anteaya has quit IRC13:11
*** ykarel is now known as ykarel|away13:20
*** VW has quit IRC13:22
*** VW has joined #openstack-infra13:23
*** ykarel|away has quit IRC13:25
*** dkehn has joined #openstack-infra13:26
*** VW has quit IRC13:27
*** dklyle has quit IRC13:27
*** felipemonteiro__ has joined #openstack-infra13:29
*** yamamoto has joined #openstack-infra13:32
*** yamamoto has quit IRC13:36
*** boden has joined #openstack-infra13:40
*** eernst has joined #openstack-infra13:40
*** d0ugal has quit IRC13:41
bodenhi, as of recent I’ve been running into failures in the requirements-check job for neutron-lib “Incompatible requirement”. for example http://logs.openstack.org/09/557809/2/check/requirements-check/a7cad6c/job-output.txt.gz#_2018-04-02_15_20_52_63964213:41
bodenanyone have a tips on this? it’s not clear to me what the issue is13:41
*** d0ugal has joined #openstack-infra13:44
*** d0ugal has quit IRC13:44
*** d0ugal has joined #openstack-infra13:44
*** neiloy has joined #openstack-infra13:44
*** snapiri has quit IRC13:46
*** caphrim007 has joined #openstack-infra13:46
*** caphrim007 has quit IRC13:46
*** zhurong has quit IRC13:48
*** ykarel|away has joined #openstack-infra13:51
*** ykarel|away is now known as ykarel13:51
*** esberglu has joined #openstack-infra13:53
*** VW has joined #openstack-infra13:55
*** dtantsur|brb is now known as dtantsur13:59
*** efoley_ is now known as efoley14:00
*** germs has joined #openstack-infra14:01
*** germs has quit IRC14:01
*** germs has joined #openstack-infra14:01
*** agopi has joined #openstack-infra14:04
*** bobh has joined #openstack-infra14:05
*** germs has quit IRC14:06
*** r-daneel has joined #openstack-infra14:06
*** yamamoto has joined #openstack-infra14:09
*** snapiri has joined #openstack-infra14:10
*** ralonsoh_ has quit IRC14:11
*** alexchadin has quit IRC14:14
*** alexchadin has joined #openstack-infra14:14
*** alexchadin has quit IRC14:15
*** alexchadin has joined #openstack-infra14:15
*** alexchadin has quit IRC14:15
*** alexchadin has joined #openstack-infra14:16
*** alexchadin has quit IRC14:16
*** hongbin has joined #openstack-infra14:16
*** alexchadin has joined #openstack-infra14:16
*** alexchadin has quit IRC14:17
*** alexchadin has joined #openstack-infra14:17
*** cdent has joined #openstack-infra14:17
*** alexchadin has quit IRC14:18
*** alexchadin has joined #openstack-infra14:18
cdentis anyone aware of a replacement for yasfb? It seems like it used to do the job of creating rss feeds for specs.o.o but doesn't work in python3. I see the dhellmann made a bug about back in 2015 :) on yasfb14:19
*** gouthamr has quit IRC14:19
*** gouthamr has joined #openstack-infra14:20
clarkbcdent: looks like yasfb is a dhellmann project, maybe just updating it to support python3?14:21
clarkbcdent: https://github.com/sphinx-contrib/yasfb/commit/34b6d54b88a1fd9e34bf6fa8575728eaecae161014:21
clarkbwhich appears to be released14:21
*** ldnunes has quit IRC14:21
cdenthmmm, then something weird is up14:22
cdentthanks clarkb14:22
* cdent digs harder14:22
*** alexchadin has quit IRC14:22
*** shoogz has quit IRC14:24
sshnaidmpabelanger, dmsimard hi, I have problem with zuul on RDO SF not cloning repo dependencies correctly. Can we look into it in your time?14:26
*** trown is now known as trown|brb14:27
*** alexchadin has joined #openstack-infra14:27
*** mattt has left #openstack-infra14:27
dhellmanncdent, clarkb : yeah, we should just fix it to work for py314:28
cdentdhellmann: it appears to almost work but is trying to output bytes when it wants a string14:28
dhellmannI threw that together pretty quickly ages ago, iirc14:28
dhellmanndo you have enough of it loaded into your head to find a fix? or should I put it on my todo list?14:28
cdentdhellmann: I think I got it14:29
dhellmanncool14:29
dmsimardsshnaidm: if it's on the RDO gerrit/zuul, let's move that to #rdo :)14:29
dhellmannI'm going to be in and out a lot this week. My folks are moving.14:29
sshnaidmdmsimard, already there :)14:29
*** trown|brb is now known as trown14:30
cdentdhellmann: there's already a pull request that out to fix it: https://github.com/sphinx-contrib/yasfb/pull/314:31
cdentought14:31
dhellmannaha14:31
dhellmannhey, I know that guy :-)14:31
cdent:)14:31
dhellmanngosh, who maintains this project. that pr has been open since june14:32
*** alexchadin has quit IRC14:32
cdentdhellmann: since we're looking at yasfb, how do I make it look for files outside the top dir?14:32
dhellmanniirc it just picks up whatever files sphinx reads14:32
dhellmannok, I merged that pR14:32
dhellmannlet me do a release14:32
dhellmanncdent : 0.7.0 released14:33
cdentwoot!14:34
cdentyay opensource14:34
dhellmannwe probably need to update the constraint. I can vote +2 on that if you propose it.14:34
*** ldnunes has joined #openstack-infra14:34
dhellmanncdent : https://github.com/sphinx-contrib/yasfb/blob/master/yasfb/__init__.py#L29 causes create_feed_item to be called for each HTML page output. So if you can make sphinx find those files outside of the source dir and emit them as a page, they should just show up in the feed.14:35
cdentyeah, I'm just reading through the event docs now14:36
dhellmannthe logic to build URLs is probably not going to be right: https://github.com/sphinx-contrib/yasfb/blob/master/yasfb/__init__.py#L11414:36
dhellmannthough that depends on whether 'current_page_name' is right or not14:36
*** links has quit IRC14:37
*** alexchadin has joined #openstack-infra14:47
*** jamielennox has quit IRC14:47
*** gmann_ has quit IRC14:47
*** olivierbourdon38 has quit IRC14:47
*** olivierbourdon38 has joined #openstack-infra14:48
*** trown has quit IRC14:48
*** kuromagi has quit IRC14:48
*** felipemonteiro_ has joined #openstack-infra14:48
*** kuromagi has joined #openstack-infra14:48
*** rosmaita has quit IRC14:48
*** greghaynes has quit IRC14:48
cdentdhellmann: I figure out my issue, and have made a fix: https://github.com/sphinx-contrib/yasfb/pull/414:49
*** greghaynes has joined #openstack-infra14:49
*** gmann_ has joined #openstack-infra14:50
*** trown has joined #openstack-infra14:50
*** jamielennox has joined #openstack-infra14:50
pabelangerclarkb: mind a review on https://review.openstack.org/558237 and https://review.openstack.org/558331/ setups on nodepool-builders to use AFS mirrors for arm64 and bionic images14:51
*** david-lyle has joined #openstack-infra14:52
*** felipemonteiro__ has quit IRC14:52
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Upgrade to webpack 4  https://review.openstack.org/55198714:56
*** rosmaita has joined #openstack-infra14:56
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Upgrade from angularjs (v1) to angular (v5)  https://review.openstack.org/55198914:56
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Use glyphicons for status balls  https://review.openstack.org/55199214:56
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Stop falling back to job name for missing url  https://review.openstack.org/55405614:56
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Remove dashboard workaround for missing log_url  https://review.openstack.org/55406614:56
openstackgerritMonty Taylor proposed openstack-infra/zuul master: web: add /{tenant}/jobs/{job_name} route  https://review.openstack.org/55097814:56
openstackgerritMonty Taylor proposed openstack-infra/zuul master: web: add /{tenant}/projects routes  https://review.openstack.org/55097914:56
openstackgerritMonty Taylor proposed openstack-infra/zuul master: web: add /{tenant}/pipelines route  https://review.openstack.org/54152114:56
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Add zuul-build-dashboard-multi-tenant  https://review.openstack.org/55771314:56
openstackgerritMonty Taylor proposed openstack-infra/zuul master: dashboard: add /{tenant}/job.html page to display job details  https://review.openstack.org/53554514:56
openstackgerritBharat Kunwar proposed openstack/diskimage-builder master: Fix yaml dependency issue with dracut-regenerate for python3  https://review.openstack.org/55849714:57
clarkbpabelanger: is https://review.openstack.org/#/c/558237/2/nodepool/elements/nodepool-base/root.d/50-apt-allow-unauthenticated that safe for older apt on trusty and xenial builds too?14:57
*** garyk has quit IRC14:58
*** diablo_rojo has joined #openstack-infra15:00
pabelangerclarkb: yes, but only enforced in bionic15:00
*** felipemonteiro_ has quit IRC15:02
*** felipemonteiro_ has joined #openstack-infra15:02
*** jamesmcarthur has joined #openstack-infra15:03
*** iyamahat has joined #openstack-infra15:03
*** jamesmcarthur has quit IRC15:04
*** alexchadin has quit IRC15:04
*** jamesmcarthur has joined #openstack-infra15:04
*** olaph has quit IRC15:06
*** olaph has joined #openstack-infra15:06
*** felipemonteiro__ has joined #openstack-infra15:07
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Add regex support to project stanzas  https://review.openstack.org/53571315:07
openstackgerritMerged openstack-infra/project-config master: Use AFS mirrors to build ubuntu-bionic DIB  https://review.openstack.org/55823715:09
openstackgerritMerged openstack-infra/project-config master: Setup AFS mirrors for ubuntu-xenial-arm64  https://review.openstack.org/55833115:09
*** garyk has joined #openstack-infra15:10
*** felipemonteiro_ has quit IRC15:11
dhellmanncdent : merged and released as 0.7.115:13
cdentthanks dhellmann, sorry for making you need to release twice15:13
*** pblaho has joined #openstack-infra15:15
openstackgerritAndrea Frittoli proposed openstack-infra/zuul-jobs master: Remove support for extensions as lists  https://review.openstack.org/54089015:16
*** pblaho has quit IRC15:17
*** ramishra has quit IRC15:17
*** felipemonteiro has joined #openstack-infra15:20
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Add winrm certificate handling  https://review.openstack.org/53571715:22
*** felipemonteiro__ has quit IRC15:23
*** eernst has quit IRC15:23
*** felipemonteiro has quit IRC15:25
*** iranzo has quit IRC15:25
prometheanfireclarkb: tag for glean?15:25
*** salv-orlando has quit IRC15:26
* clarkb reads commit log for glean15:26
clarkbprometheanfire: looks like it adds a new feature, networkd for gentoo? so next tag would be 1.11.0? Then at some point I assume we add it as option for other distros that support it?15:28
*** andreas_s has quit IRC15:29
clarkbprometheanfire: I can do that as soon as I have a gap in meetings15:30
*** tesseract has quit IRC15:32
*** andreas_s has joined #openstack-infra15:34
*** gouthamr has quit IRC15:34
fungi1.11.0 sounds good15:34
*** gouthamr has joined #openstack-infra15:35
clarkbI lied as soon as I have tea and a gap in meetings :) /me grabs tea15:35
prometheanfireclarkb: ya, 1.11.015:35
*** eernst has joined #openstack-infra15:36
*** gouthamr has quit IRC15:36
prometheanfireI don't know any other distro that goes all in on the networkd stuff15:36
*** andreas_s has quit IRC15:38
clarkbprometheanfire: ya I dont either but may be nice option to select for it as a user. Future work15:39
*** alexchadin has joined #openstack-infra15:41
slaweqhi infra team, today I wrote some additional guide how to debug neutron jobs directly on compute node if there is no other possibility, can You check it https://review.openstack.org/558537 and tell me if that is "acceptable" from Your point of view to make such tips as an "official" in docs?15:41
slaweqthx in advance15:41
*** cdent has left #openstack-infra15:42
prometheanfireclarkb: yarp15:42
*** gouthamr has joined #openstack-infra15:43
*** iranzo has joined #openstack-infra15:46
clarkbprometheanfire: glean 1.11.0 pushed15:46
clarkbhttps://git.openstack.org/cgit/openstack-infra/glean/log/ shows it now too15:47
*** chandankumar has quit IRC15:47
openstackgerritTobias Henkel proposed openstack-infra/nodepool master: Don't gather host keys for non ssh connections  https://review.openstack.org/53568515:49
openstackgerritTobias Henkel proposed openstack-infra/nodepool master: Add connection-port to provider diskimage  https://review.openstack.org/53568615:49
*** e0ne has quit IRC15:50
prometheanfire:q15:52
*** shoogz- has joined #openstack-infra15:54
clarkbslaweq: I've left some thoughts on the change15:56
*** shoogz- has quit IRC15:56
slaweqclarkb: thx15:59
*** jcoufal has quit IRC15:59
*** chandankumar has joined #openstack-infra16:00
*** VW_ has joined #openstack-infra16:00
*** VW_ has quit IRC16:01
bmaceclarkb: i sent a message to the mailing list last weds regarding the kolla-cli repository issue, which just needs someone that can do a force push into the repo to avoid going through gerrit, but thus far no response, so I thought it was about time for me to bring it back up in here again, or should I keep asking on the mailing list until someone becomes interested?16:01
*** VW_ has joined #openstack-infra16:01
*** hemna_ has joined #openstack-infra16:02
clarkbbmace: I think it may be our least favorite activity :/ I was traveling last wednesday so not sure I saw it. Looking for email now16:02
*** germs has joined #openstack-infra16:02
*** germs has quit IRC16:02
*** germs has joined #openstack-infra16:02
*** inc0 has joined #openstack-infra16:03
bmaceclarkb: sure, you had been the one to suggest I post on the mailing list about it when I brought it up in here last week :)16:03
*** inc0_ has joined #openstack-infra16:03
clarkbah derp. Its been a long couple weeks16:03
*** VW has quit IRC16:04
clarkbreading the error with the merge that is annoying16:04
bmacei completely understand :)16:04
*** ykarel is now known as ykarel|afk16:04
*** Sukhdev has joined #openstack-infra16:04
*** manjeets has joined #openstack-infra16:04
clarkbbmace: can we just make master point to o3l_4.0.1 ?16:05
clarkbI think we may be able to ninja that in the gerrit web ui fairly easily16:05
bmacewhatever works, i don't care how it happens, as long as master is pointing at the right commit and we can move forward as normal from there.16:05
bmaceclarkb: i trust your gerrit fu is considerably more impressive than mine :)16:06
clarkbheh gerrit doesn't let you delete master to move it16:06
clarkbbmace: I've got to context switch back to a meeting but will look more after16:07
*** germs has quit IRC16:07
bmaceclarkb: thanks kindly.  I will be around all day if I can be of any assistance.16:07
*** alexchadin has quit IRC16:08
*** jcoufal has joined #openstack-infra16:09
*** mgagne_ has quit IRC16:11
*** mgagne_ has joined #openstack-infra16:11
*** udesale has quit IRC16:12
*** olaph1 has joined #openstack-infra16:13
*** zoli is now known as zoli|gone16:14
*** zoli|gone is now known as zoli16:14
*** iranzo has quit IRC16:14
*** olaph has quit IRC16:15
*** alexchadin has joined #openstack-infra16:17
*** rpioso|afk is now known as rpioso16:18
*** panda is now known as panda|bbl16:19
*** spzala has joined #openstack-infra16:21
*** dtantsur is now known as dtantsur|brb16:21
*** timothyb89_ is now known as timothyb8916:22
*** garyk has quit IRC16:24
*** caphrim007 has joined #openstack-infra16:26
*** caphrim007 has quit IRC16:26
*** links has joined #openstack-infra16:26
*** alexchadin has quit IRC16:26
*** gouthamr has quit IRC16:26
*** caphrim007 has joined #openstack-infra16:27
*** alexchadin has joined #openstack-infra16:28
*** Sukhdev has quit IRC16:30
*** jpich has quit IRC16:33
*** alexchadin has quit IRC16:33
*** inc0 has quit IRC16:37
*** lucasagomes is now known as lucas-afk16:38
*** zoli has quit IRC16:38
*** lpetrut has quit IRC16:38
*** Krenair has quit IRC16:39
*** afazekas has quit IRC16:39
*** zoli has joined #openstack-infra16:39
*** afazekas has joined #openstack-infra16:39
*** sshnaidm is now known as sshnaidm|bbl16:39
*** alexchadin has joined #openstack-infra16:41
*** Swimingly_ has quit IRC16:42
*** Swimingly has joined #openstack-infra16:43
*** Swimingly has quit IRC16:43
*** Swimingly has joined #openstack-infra16:43
*** d0ugal_ has joined #openstack-infra16:44
*** alexchadin has quit IRC16:45
*** Krenair has joined #openstack-infra16:46
*** gfidente has quit IRC16:46
*** d0ugal has quit IRC16:47
*** kgiusti has joined #openstack-infra16:49
jlvillalgerritbot review request: https://review.openstack.org/#/c/545469/  Some cleanup/refactoring and adding unit tests. Has one +2 Thank you.16:51
clarkbcorvus: can we abandon https://review.openstack.org/#/c/555063/1 at this point?16:52
clarkbmordred: pabelanger did we reach a conclusion on https://review.openstack.org/#/c/558256/216:52
clarkbdmsimard: before I approve https://review.openstack.org/#/c/555478/3 can you read my most recent comments on that change?16:52
dmsimardclarkb: replied16:53
*** garyk has joined #openstack-infra16:53
corvusclarkb: done16:53
clarkbI think this week si going to be a spring cleaning week. Rereview all the things \o/16:54
*** alexchadin has joined #openstack-infra16:55
clarkbbmace: Jeffrey4l looks like it will be a fast forward. Since kolla-cli is a kolla project can we get Jeffrey4l to ack the fast forward of master to o3l_4.0.1 kolla-cli then I can push that when I get a moment16:58
*** trown is now known as trown|lunch17:00
*** jpena is now known as jpena|off17:00
dmsimardbtw openstack-ansible has already managed to retool their log collection to use the new ara middleware instead of html reports \o/ https://review.openstack.org/#/c/557921/17:01
*** dhajare__ has quit IRC17:02
bmaceclarkb: sure, thanks, i'll send him a message.  he is pretty hit or miss this time of day.17:03
pabelangerclarkb: I was able to reproduce the issue and think mordred was looking more into it with my launchnode-env17:04
clarkbpabelanger: cool so still in wait before merging that change I guess17:04
*** inc0_ has quit IRC17:04
*** david-lyle has quit IRC17:04
pabelangerclarkb: yah, I had to revert the patch dmsimard had and use old shade to launch revew01.o.o and get dns info17:04
openstackgerritMerged openstack-infra/system-config master: Update signing docs for Zuul v3  https://review.openstack.org/55137617:07
*** camunoz has quit IRC17:11
mordredpabelanger: I'm working on reproducing the issue right now17:12
pabelangerack17:12
*** jtomasek has quit IRC17:13
*** links has quit IRC17:13
*** inc0 has joined #openstack-infra17:14
*** pcichy has quit IRC17:14
*** jamesmcarthur has quit IRC17:16
*** annp has quit IRC17:17
pabelangerclarkb: I haven't pushed on it for a couple weeks, but trying to remove fedora-26 images from nodepool. I've announce on ML we wanted to remove them 2 weeks ago, but jobs are still using them.  Any suggestions on how to encourage those projects to move to fedora-27? Given that we've already at 6 weeks heads up, I'm unsure more time is going to help.17:17
mordredpabelanger: perhaps we should submit patches to those projects shifting to 27?17:18
*** annp has joined #openstack-infra17:18
pabelangeranother thought is to update all jobs to fedora-latest, with the note that we announce on ML when the next release of fedora is out, and just update the label in nodepool17:18
pabelangermordred: I did, however the jobs fail on fedora-2717:18
mordredpabelanger: AWESOME17:18
pabelangerhttps://review.openstack.org/#/q/topic:fedora-2717:19
pabelangermordred: so, should the patch be updated to make then non-voting or do we remove and just have jobs remove NODE_FAILURE, since image is done17:19
clarkbpabelanger: in the past we've just had to switch them17:19
pabelangergone*17:19
mordredpabelanger: oh - at least there's only two projects outstanding17:19
pabelangerclarkb: yah, little harder before they are in-tree jobs17:19
pabelangermordred: yah, that is the bright side :)17:20
clarkbpabelanger: when does 26 drop support?17:20
clarkbI think as logn as that lines up with our timeline we just stop building them17:20
*** sambetts is now known as sambetts|afk17:20
clarkbthen their jobs won't work and you can point them to the emails and the changes17:20
mordredpabelanger: the bifrost error looks to be a python3 compat issue17:21
pabelangerclarkb: soon, but usually we've only tried to keep the latest image of fedora in nodepool, to keep things easier to manage17:21
mordredraise DistutilsError(\"This package requires Python 2.4 or higher versions 2.x not Python3\") <-- from UcsSdk==0.8.1.917:22
clarkbpabelanger: but in this case 26 goes away at like end of may right?17:22
clarkbpabelanger: because 28 releases first week of may17:22
pabelangerclarkb: yah17:23
pabelangerin fact, I'd like to start working to bring fedora-28 online and remove fedora-27 too.  So, maybe starting with fedora-28, we call it fedora-latest17:23
pabelangerthen, send out warning on ML about changes17:23
clarkbor current or something17:25
*** jamesmcarthur has joined #openstack-infra17:26
clarkbI think with 26 the switch is easier because there is a hard upstream this is no longer supported date17:26
*** camunoz has joined #openstack-infra17:26
pabelangeryah, with zuul2 we could just cut everybody to fedora-27 via project-config too.  But, intree would mean we update jobs directly to non-voting I guess, then approve ourself?17:27
*** aojea has quit IRC17:28
mordredpabelanger, TheJulia: I updated remote:   https://review.openstack.org/547119 Update to fedora-27 for testing ... not sure if what I did was completely correct17:29
TheJuliamordred: I'll take a a look *hopes*17:29
clarkbpabelanger: no I think we just delete the image17:30
clarkbpabelanger: then their jobs will block and they won't get any ci until they fix it.17:30
TheJuliahmm. merge conflict17:31
clarkb(that may not be tenable long term but hopefully is a good enough carrot to get things moving17:31
*** amoralej is now known as amoralej|off17:31
pabelangerclarkb: okay, yah, that means they will fail with NODE_FAILURE17:31
mordredpabelanger: AHA - I see what the launch_node issue is17:32
Jeffrey4lclarkb, re kolla-cli, yeah, please update the master, thanks.17:33
*** Nil_ has quit IRC17:33
pabelangermordred: yah17:34
pabelangeryay*17:34
*** lpetrut has joined #openstack-infra17:34
mordredpabelanger: patching coming - testing it out now17:34
mordredpabelanger: tl;dr - the dns.py fix updated how dns.py creates the cloud object ... for when dns.py is called independently - but launch_node.py was creating the object differently and passing it as an argument to dns17:34
mordredpabelanger: thus why I was running code and saying "it works for me" and you were hitting the errors when you ran launch_node17:35
pabelangerack17:36
mordredpabelanger: so I'm testing via a full launch_node run - just to make sure :)17:36
openstackgerritMonty Taylor proposed openstack-infra/zuul master: web: add /{tenant}/projects routes  https://review.openstack.org/55097917:38
openstackgerritMonty Taylor proposed openstack-infra/zuul master: web: add /{tenant}/pipelines route  https://review.openstack.org/54152117:38
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Add zuul-build-dashboard-multi-tenant  https://review.openstack.org/55771317:38
openstackgerritMonty Taylor proposed openstack-infra/zuul master: dashboard: add /{tenant}/job.html page to display job details  https://review.openstack.org/53554517:38
openstackgerritMonty Taylor proposed openstack-infra/zuul master: dashboard: add /{tenant}/projects.html web page  https://review.openstack.org/53787017:38
*** eernst has quit IRC17:39
*** eernst has joined #openstack-infra17:40
*** ldnunes has quit IRC17:41
clarkbmordred: oh it doesn't fork it it imports it?17:41
openstackgerritMonty Taylor proposed openstack-infra/zuul master: dashboard: add /{tenant}/projects.html web page  https://review.openstack.org/53787017:42
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Remove ansible-role-functional-jobs-fedora-26 for windmill  https://review.openstack.org/55857817:42
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Update zuul-jobs to use fedora-27 for testing  https://review.openstack.org/55857917:42
*** felipemonteiro has joined #openstack-infra17:43
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: Make fedora-27 jobs voting  https://review.openstack.org/55858017:44
mordredclarkb: yah17:45
*** ykarel|afk has quit IRC17:46
*** anteaya has joined #openstack-infra17:47
openstackgerritNguyen Hai proposed openstack-infra/project-config master: Change from py27 to pep8 job in tacker-specs  https://review.openstack.org/55858117:48
dmsimardinfra-root: fyi it looks like new ARA reports out of ara-report doesn't always fully load on the first attempt (but a reload/refresh works)17:48
mordredyay17:49
dmsimardI'm not sure why that is yet, I'll have to look.. my first guess would be something with caching17:49
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: Switch openstack-tox-py36 to fedora-27  https://review.openstack.org/55858217:49
openstackgerritNguyen Hai proposed openstack-infra/project-config master: Change from py27 to pep8 job in tacker-specs  https://review.openstack.org/55858117:49
*** dhinesh has joined #openstack-infra17:49
dmsimardSometimes it will load partially, like missing icons/fonts or results, it's odd.17:50
openstackgerritMonty Taylor proposed openstack-infra/system-config master: Add error messages to dns.py about old sdk  https://review.openstack.org/55825617:50
openstackgerritMonty Taylor proposed openstack-infra/system-config master: Update connection construction in launch_node  https://review.openstack.org/55858317:50
*** r-daneel has quit IRC17:50
*** alexchadin has quit IRC17:50
*** r-daneel has joined #openstack-infra17:50
mordredclarkb, pabelanger, dmsimard: that should actually fix the dns.py issues17:50
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: Remove ansible-role-functional-fedora-26 jobs  https://review.openstack.org/55858417:51
pabelangermordred: yay, thanks17:52
*** ldnunes has joined #openstack-infra17:52
prometheanfireclarkb: tagged?17:53
clarkbprometheanfire: ya forever ago17:53
clarkbprometheanfire: I pinged you when I did it :)17:53
clarkbprometheanfire: https://pypi.python.org/pypi/glean/1.11.017:54
prometheanfirejust wanted to make sure since the project-config thing just got workflowed17:54
*** olaph has joined #openstack-infra17:54
*** olaph1 has quit IRC17:55
bodenis there any known bugs with the requirements-check job W/R/T calculating versions on more complex version strings (!=1.6.6,!=1.6.7,>=1.6.2)?17:57
bodenhttp://logs.openstack.org/38/553838/3/check/requirements-check/b25a52f/job-output.txt.gz#_2018-04-03_15_02_38_52622717:58
clarkbboden: no, I think it uses the actual pip/setuptools parser and resolution for that17:58
bodenclarkb: I’m not understanding why ^^ is failing to be honest17:58
clarkbthough maybe the new less strict rules changed some of that17:59
clarkbprometheanfire: ^17:59
bodenwe constrain with: sphinx!=1.6.6,!=1.6.7,>=1.6.2 # BSD    but its complaining: constrained to 1.6.5 which does not match the minimum version specifier 1.6.2 in test-requirements.txt17:59
*** trown|lunch is now known as trown18:01
*** iyamahat has quit IRC18:01
clarkbboden: I understand it now. It is saying you ahve to increase your minimum18:01
clarkbmaybe?18:01
fungiboden: sounds like a lower-constraints test failure? dhellmann may also have some background on it18:01
clarkbI think what it is saying is that since the lower constraint is 1.6.5 that is the lowest value you can have and 1.6.2 is lower than that18:02
clarkbI don't know if that is the intent with the new tooling but I believe that is why it failed18:02
*** germs has joined #openstack-infra18:03
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: Remove ansible-role-functional-fedora-26 jobs  https://review.openstack.org/55858418:03
*** VW_ has quit IRC18:03
bodenclarkb ack, seems odd given our contraint says >=1.6.2 and does not cap it below 1.6.518:03
clarkbboden: the issue is that elsewhere the constraint is higher I think18:04
*** david-lyle has joined #openstack-infra18:04
*** VW has joined #openstack-infra18:04
clarkbboden: so you have to be between the minimum and the maximum18:04
clarkbbut flexibility within those is fine18:04
bodenclarkb: so I should increase our >=1.6.2  to  >=1.6.5  ??18:05
clarkbboden: yes I think that is what the test is trying to tell you18:08
*** germs has quit IRC18:08
openstackgerritMerged openstack-infra/project-config master: make Gentoo use the systemd profile  https://review.openstack.org/55665218:08
*** diablo_rojo has quit IRC18:09
*** diablo_rojo has joined #openstack-infra18:12
*** efried has quit IRC18:14
*** efried has joined #openstack-infra18:14
*** efoley has quit IRC18:14
*** bhavik1 has joined #openstack-infra18:16
*** iyamahat has joined #openstack-infra18:16
*** efoley has joined #openstack-infra18:17
prometheanfireboden: clarkb: there's a problem with the job where comments will do a false positive, that's this patch series https://review.openstack.org/55850418:18
prometheanfireboden: your lower-constraints.txt should have sphinx===1.6.218:18
clarkbprometheanfire: the lower constraints is 1.6.5 though18:19
clarkbprometheanfire: which is why its unhappy aiui18:19
prometheanfireclarkb: of 1.6.2 doesn't work then the requirement should be raised to 1.6.518:19
clarkboh interesting lower constrainst does say 1.6.2 so now I'm extra confused18:20
prometheanfireclarkb: looks like the patch raises the sphinx version without raising the lower-constraint version18:20
clarkbaha18:20
*** electrofelix has quit IRC18:21
clarkbyup18:21
prometheanfirehttps://review.openstack.org/#/c/55383818:21
smcginnisJust noticed some of the backlog. There's a glance lower-constraints patch that has some odd results.18:21
*** gouthamr has joined #openstack-infra18:21
smcginnisIn case it's related - https://review.openstack.org/#/c/555483/18:21
prometheanfirewould be nice to get some more reviews on the https://review.openstack.org/558504 series of patches18:21
*** gouthamr has quit IRC18:21
*** olaph1 has joined #openstack-infra18:22
prometheanfire2018-04-03 02:14:19.068616 | ubuntu-xenial | Package u'qpid-python' is used in test-requirements.txt but not in lower-constraints.txt18:22
smcginnisprometheanfire: Oh, I looked at that this morning but was waiting on zuul results. Any reason I should hold off on +A?18:22
prometheanfire2018-04-03 02:14:19.061845 | ubuntu-xenial | Package u'alembic' is constrained to 0.9.8 which does not match the minimum version specifier 0.8.10 in requirements.txt18:22
prometheanfirebunch of stuff like that18:22
prometheanfiresmcginnis: I don't think so, did you look at the entire series?18:22
smcginnisprometheanfire: The first few at least. I can try to get through the whole set shortly.18:23
prometheanfirethanks18:23
*** gouthamr has joined #openstack-infra18:23
*** olaph has quit IRC18:23
clarkbprometheanfire: we sort of explicitly avoid reviewing those changes around here unless its gate breaking. So is it gate breaking and if not do we need to chagne that practice?18:23
*** hemna_ has quit IRC18:24
prometheanfireclarkb: ?18:25
*** dtantsur|brb is now known as dtantsur18:25
clarkbbmace: Jeffrey4l I've got to eat and prep for the infra meeating in 35 minutes but after that fixing kolla-cli is next on my list. Ping me in a couple hours if you don't hear from me before that18:25
prometheanfireclarkb: oh, I was asking smcginnis in with his requirements hat18:25
clarkbprometheanfire: re would be nice to get some more reviews on the https://review.openstack.org/558504 series of patches18:25
clarkbprometheanfire: oh gotcha18:26
dhellmannboden : your lower constraint value must match the value of the version set to >= in the requirements file, otherwise one of them isn't valid18:33
bmaceclarkb: will do, danke18:36
*** dhinesh has quit IRC18:36
*** dhinesh has joined #openstack-infra18:37
*** isssp has joined #openstack-infra18:38
*** armaan has quit IRC18:38
*** bobh has quit IRC18:38
*** armaan has joined #openstack-infra18:39
*** kuromagi^ has joined #openstack-infra18:40
*** gouthamr has quit IRC18:40
*** wendar_ has joined #openstack-infra18:41
*** mugsie_ has joined #openstack-infra18:41
*** tinwood_ has joined #openstack-infra18:41
*** dprince has quit IRC18:41
*** dhajare__ has joined #openstack-infra18:42
*** mikap has joined #openstack-infra18:42
*** anteaya has quit IRC18:42
*** dhinesh has quit IRC18:42
*** Roamer`_ has joined #openstack-infra18:42
*** anteaya has joined #openstack-infra18:43
*** pcaruana has quit IRC18:44
*** kencjohnston_ has joined #openstack-infra18:45
*** numans_ has joined #openstack-infra18:46
*** bgmccollum_ has joined #openstack-infra18:46
*** EmilienM_ has joined #openstack-infra18:46
*** _bmjen has joined #openstack-infra18:46
*** gouthamr has joined #openstack-infra18:46
*** rook_ has joined #openstack-infra18:47
*** kuromagi has quit IRC18:47
*** numans has quit IRC18:47
*** EmilienM has quit IRC18:47
*** tinwood has quit IRC18:47
*** bmjen has quit IRC18:47
*** mordred has quit IRC18:47
*** Roamer` has quit IRC18:47
*** masayukig[m] has quit IRC18:47
*** aspiers[m] has quit IRC18:47
*** bgmccollum has quit IRC18:47
*** ispp has quit IRC18:47
*** wendar has quit IRC18:47
*** mika has quit IRC18:47
*** Shrews has quit IRC18:47
*** rook has quit IRC18:47
*** kencjohnston has quit IRC18:47
*** mugsie has quit IRC18:47
*** mikap is now known as mika18:47
*** EmilienM_ is now known as EmilienM18:48
*** inc0 has quit IRC18:48
*** dtantsur is now known as dtantsur|afk18:49
*** EmilienM has quit IRC18:49
*** EmilienM has joined #openstack-infra18:49
*** olivierbourdon38 has quit IRC18:49
pabelangerclarkb: prometheanfire: do we need a new release of DIB?18:51
pabelangerhttp://nb01.openstack.org/gentoo-17-0-systemd-0000000213.log18:51
pabelangergentoo project not valid18:51
*** Sukhdev has joined #openstack-infra18:51
*** olivierbourdon38 has joined #openstack-infra18:52
prometheanfirepabelanger: if we are using dib releases, probably18:53
*** inc0 has joined #openstack-infra18:53
*** masayukig[m] has joined #openstack-infra18:53
pabelangeryah we are18:53
pabelangerokay, lets pause that build for now18:53
pabelangerand confirm with ianw about doing a release18:53
*** stevebaker has quit IRC18:53
*** mordred has joined #openstack-infra18:54
*** Shrews has joined #openstack-infra18:54
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Pause gentoo-17-0-systemd DIBs for nodepool  https://review.openstack.org/55859718:55
pabelangerclarkb: ianw: ^lets pause so we don't filling HDDs with logs18:55
*** aspiers[m] has joined #openstack-infra18:55
*** shardy has quit IRC18:55
*** dmellado has quit IRC18:56
*** shardy has joined #openstack-infra18:56
*** bhavik1 has quit IRC18:57
ianwpabelanger: ok ... i think we want a release with the resolv.conf stuff to when i sort it out18:57
pabelangerianw: wfm18:58
openstackgerritKendall Nelson proposed openstack-infra/project-config master: Adds Cyborg to StoryBoard  https://review.openstack.org/55832718:58
*** dmellado has joined #openstack-infra18:58
clarkbinfra meeting time is now19:00
*** Goneri has quit IRC19:00
*** stevebaker has joined #openstack-infra19:01
*** stevebaker has quit IRC19:01
ianw[[ 5365760000 -lt 5366611968 ]] ... echo 'Root device does not appear to have grown: 5365760000' ... hmm, i wonder if the calculations are askew19:04
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul master: WIP: Add Gerrit docs to Zuul From Scratch  https://review.openstack.org/55860019:05
*** bobh has joined #openstack-infra19:10
*** dmellado has quit IRC19:12
*** jtomasek has joined #openstack-infra19:13
*** hongbin has quit IRC19:14
openstackgerritPaul Belanger proposed openstack-infra/zuul master: WIP: Fix issue when using _blockPluginDirs  https://review.openstack.org/55860519:14
*** bobh has quit IRC19:15
*** jamesmcarthur has quit IRC19:18
*** inc0 has quit IRC19:20
*** jamesmcarthur has joined #openstack-infra19:20
*** dmellado has joined #openstack-infra19:21
*** hongbin has joined #openstack-infra19:21
*** olaph has joined #openstack-infra19:23
*** inc0 has joined #openstack-infra19:23
*** olaph1 has quit IRC19:23
*** lpetrut has quit IRC19:23
*** harlowja has joined #openstack-infra19:23
*** rook_ is now known as rook19:25
*** shardy has quit IRC19:25
*** Hal has joined #openstack-infra19:26
*** Hal has quit IRC19:26
*** shardy has joined #openstack-infra19:27
*** eernst has quit IRC19:27
*** yamamoto has quit IRC19:30
*** eharney has quit IRC19:30
openstackgerritJohn L. Villalovos proposed openstack-infra/storyboard master: DNM/WIP: Testing  https://review.openstack.org/55860719:30
*** dmellado has quit IRC19:31
*** jtomasek has quit IRC19:36
*** yamamoto has joined #openstack-infra19:36
*** agopi has quit IRC19:37
*** ianychoi__ has joined #openstack-infra19:38
*** pcaruana has joined #openstack-infra19:38
*** e0ne has joined #openstack-infra19:39
*** ianychoi_ has quit IRC19:41
*** yamamoto has quit IRC19:42
*** eharney has joined #openstack-infra19:43
*** Goneri has joined #openstack-infra19:45
*** dhajare__ has quit IRC19:45
*** dprince has joined #openstack-infra19:45
*** e0ne has quit IRC19:47
*** wendar_ is now known as wendar19:48
openstackgerritColleen Murphy proposed openstack-infra/openstack-zuul-jobs master: [debug]Convert infra-puppet-apply jobs to non-legacy  https://review.openstack.org/55701119:49
*** pcaruana has quit IRC19:50
*** yamamoto has joined #openstack-infra19:52
*** dmellado has joined #openstack-infra19:55
*** yamamoto has quit IRC19:57
*** yamamoto has joined #openstack-infra19:59
*** yamamoto has quit IRC19:59
*** yamamoto has joined #openstack-infra20:01
*** david-lyle is now known as dklyle20:02
openstackgerritmegan guiney proposed openstack-infra/subunit2sql master: Add ability to grab stream processing variables from kwargs  https://review.openstack.org/54383620:02
*** Sukhdev has quit IRC20:02
*** e0ne has joined #openstack-infra20:05
pabelangerinfra-root: could I get a +3 on https://review.openstack.org/558597/ to stop gentoo DIBs, we need a new release of diskimage-builder but not ready yet20:05
*** yamamoto has quit IRC20:06
clarkbpabelanger: done20:06
pabelangerthanks20:06
clarkbfor logging purposes I am going to add myself to the project bootstrappers group in gerrit then `git push gerrit master` with o3l_4.0.1 tag checked out in kolla-cli locally. My reading of the git history is this is a fast forward so shouldn't even need to force push20:07
clarkbwill status log as soon as I am done20:07
*** VW_ has joined #openstack-infra20:08
*** bobh has joined #openstack-infra20:09
clarkbhrm that told me "Everything up-to-date" which is not quite what I expected20:10
*** hemna_ has joined #openstack-infra20:11
clarkboh is that beacuse push gerrit master implies local master to remote master? I have to say HEAD:master don't I?20:11
clarkbya20:11
*** VW has quit IRC20:11
*** efried is now known as efried_schoolrun20:12
clarkbbmace: Jeffrey4l ok I think it is done now. Can you please confirm it is as you expected?20:12
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add some more fudge factor for growroot test  https://review.openstack.org/55861720:12
*** VW_ has quit IRC20:12
*** florianf has quit IRC20:13
openstackgerritColleen Murphy proposed openstack-infra/openstack-zuul-jobs master: Convert infra-puppet-apply jobs to non-legacy  https://review.openstack.org/55701120:13
*** agopi has joined #openstack-infra20:13
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Upgrade from angularjs (v1) to angular (v5)  https://review.openstack.org/55198920:15
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Translate zuulStartStream into typescript  https://review.openstack.org/55861820:15
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Shift log streaming code into StreamComponent  https://review.openstack.org/55861920:15
pabelangerand email sent to ML about gerrit replacement20:16
*** yamamoto has joined #openstack-infra20:16
openstackgerritMerged openstack-infra/project-config master: Pause gentoo-17-0-systemd DIBs for nodepool  https://review.openstack.org/55859720:16
*** gagehugo has quit IRC20:17
bmaceclarkb: thanks a lot, that looks good!  going forward, since the upstream was just for the initial seed of the repo, should i update the project-config to remove the upstream: association?20:18
*** kgiusti has quit IRC20:18
*** gagehugo has joined #openstack-infra20:18
clarkbbmace: ya that can be removed at this point. I want to say there may even be an automated script aht proposes those periodically. Or maybe ajaeger is just that awesome20:18
clarkbbmace: I'll respond to the infra list thread with deatils too just so that we have record keeping done20:19
*** camunoz has quit IRC20:19
bmaceclarkb: perfect, thanks again, and i'll make the project-config change and submit it for review.20:19
*** yamamoto has quit IRC20:21
ianwinfra-root: rounding in growroot sizes ring any bells?  --> https://review.openstack.org/#/c/558617/ .  reviews would be good because it's making dib gate unstable, and we've got a few dib things to get through before a release20:21
openstackgerritMerged openstack-infra/infra-specs master: Mark zuul v3 work as done  https://review.openstack.org/55777220:22
*** stevebaker has joined #openstack-infra20:23
*** yamamoto has joined #openstack-infra20:23
*** yamamoto has quit IRC20:23
openstackgerritMerged openstack-infra/infra-specs master: Amend top-level project hosting spec  https://review.openstack.org/55510420:25
openstackgerritJohn L. Villalovos proposed openstack-infra/storyboard master: Fix PEP8 failures for H236, remove unneeded ignores, use newer hacking  https://review.openstack.org/55860720:27
openstackgerritIan Wienand proposed openstack-infra/project-config master: Add debian stretch nodes to arm64/amd64 clouds  https://review.openstack.org/55790220:28
ianwpabelanger: just a rebase/merge on ^ ... what's your thoughts re it?20:29
ianwre mirrors i mean20:29
*** bobh has quit IRC20:30
*** bobh_ has joined #openstack-infra20:30
pabelangerianw: yah, I think we are going to need mirrors, given the network on nb03.o.o. we haven't done debian before20:30
*** HeOS has quit IRC20:30
pabelangerianw: but, I think we can do that in a follow up20:31
dmsimardpabelanger: I didn't even know "with_inventory_hostnames" was a thing20:31
*** rfolco|ruck is now known as rfolco|brb20:31
pabelangerianw: lets confirm with clarkb but +220:32
pabelangerdmsimard: yah, I've used it a few times20:32
*** agopi has quit IRC20:33
*** felipemonteiro_ has joined #openstack-infra20:33
dmsimardTIL20:33
ianwpabelanger: yeah, i just had a thought that we should use deb.debian.org which is the cdn ... which probably gets you a server closers to nb0320:33
*** VW has joined #openstack-infra20:34
pabelangerianw: in the past it hasn't been reliable that's why we pinged to ftp.debian.org20:34
pabelangerpinned*20:34
*** inc0 has quit IRC20:35
ianwthat's relatively new, however, i think in the past we were using httpredir or whatever it was called, which was more a round-robbin thing that sent you to various mirrors, which were in the unreliable state20:35
clarkbinfra-root https://etherpad.openstack.org/p/irc-trolling draft on email to dev list about irc things20:35
clarkbdmsimard: ^ you probably want ot look that over and corvus fungi can you double check the section on freenode things?20:35
ianwthis is the cloudfront one which i think is generally consistent20:35
*** felipemonteiro has quit IRC20:36
pabelangerstill think having a local cache would be helpful for linaro, if bandwidth is an issue there20:37
ianwpabelanger: agree, but we'll have to sort out architecture mirroring20:38
bmaceclarkb: interesting, looks like the upstream was yanked from the kolla-cli already, perhaps automagically via some means.  guess my work is done here :)  thanks again for your time.20:38
clarkbianw: pabelanger +2, marked the other place things would have to be updated re mirror location20:38
*** Nil_ has joined #openstack-infra20:39
clarkbianw: pabelanger not approving in case you want to quickly make that change, but also happy to try it using this mirror (I'm not up to date on the most relaibel debian mirror options)20:39
corvusclarkb: thanks, i replaced the freenode/wall paragraph with a suggested alternative, how's that look?20:39
pabelangerianw: I thought we had something in place today20:39
clarkbcorvus: lgtm thanks20:39
corvusactually i'm not sure about the accuracy of 'wall' so i think maybe just remove it20:40
*** e0ne has quit IRC20:40
ianwlet's make it the cdn mirror20:40
clarkbdmsimard: was that you that removed the listing? I thoguht it read better if you want to add it back20:41
clarkbdmsimard: the explicitness of what the bots are in particular was a good addition I thought20:41
*** jamesmcarthur has quit IRC20:42
dmsimardclarkb: It was me but I thought I went too far into details20:42
dmsimardI can add it back20:42
pabelangerianw: sure if we want to test20:42
ianwpabelanger: well i've not looked at how to get reprepro to mirror arm64 debs ... with ubuntu it's just in ubuntu-ports so that makes it easier in some ways as it's separate20:42
clarkbdmsimard: I think they help in this case because goal is to remove any confusion20:42
pabelangerianw: I thought fungi said it was easier on debian20:42
pabelangerjust a sub folder in same repo20:42
pabelangerstretch debian-arm64 http://ftp.debian.org/debian20:44
pabelangerI think20:44
openstackgerritIan Wienand proposed openstack-infra/project-config master: Add debian stretch nodes to arm64/amd64 clouds  https://review.openstack.org/55790220:44
*** e0ne has joined #openstack-infra20:45
*** inc0 has joined #openstack-infra20:45
dmsimardclarkb: added that back if you want to take another look20:45
*** iyamahat_ has joined #openstack-infra20:45
clarkbdmsimard: looks good thanks20:46
pabelangerianw: missed nb03.o.o.yaml mirror20:46
dmsimardI also did an edit on the channel operator part20:46
clarkbya lgtm, I'll let that stew a little longer then send it to the dev list20:47
*** mrhillsman is now known as mrhillsman-irc20:47
fungipabelanger: ianw: yeah, for debian all architecture-specific binary packages are served from a common pool while ubuntu splits i386/amd64 out from their other supported architectures20:47
*** iyamahat has quit IRC20:47
*** mrhillsman-irc is now known as mrhillsman20:48
fungijust add arm64 to the list of debian architectures you want reprepro to mirror and it will include those20:48
openstackgerritIan Wienand proposed openstack-infra/project-config master: Add debian stretch nodes to arm64/amd64 clouds  https://review.openstack.org/55790220:49
*** HeOS has joined #openstack-infra20:49
pabelanger+220:50
pabelangerfungi: +120:50
*** hashar has quit IRC20:50
*** eernst has joined #openstack-infra20:52
openstackgerritIan Wienand proposed openstack-infra/system-config master: Add arm64 to Debian mirror  https://review.openstack.org/55862420:53
ianwfungi: ^ like that maybe?20:53
pabelangerOh, right, we already have debian mirrors.20:53
fungiclarkb: ianw yes, exactly20:54
pabelangerianw: so, we should not be using the cdn, but AFS here. To avoid the issue when reprepro stops working and DIB installs newer packages that AFS has20:54
fungier, sorry clarkb, stray tab complete there20:55
ianwpabelanger: ahh, ok, so i thought it was weird you were putting that in there :)20:55
*** HeOS has quit IRC20:55
pabelangerianw: yah, totally confused. my fault20:56
pabelangerI guess we did mirror them afterall20:56
pabelanger:)20:56
pabelangerit also looks like we have stretch packages too20:56
ianwright, but not for arm6420:58
pabelangerianw: feel free to +3 on 558624 but we should manually hold lock to avoid timeout20:58
ianwyeah, i can do and i also need to check we have enough volume space20:58
ianwprobably a safe assumption it will double20:58
pabelangerianw: yah, so we could split 557902 into amd64, bring them online first. then do arm64 once reprepro is done20:58
pabelangerI'd be okay wit that20:59
pabelangerianw: think so20:59
*** priteau has joined #openstack-infra20:59
fungidmsimard: so i'm struggling with "The Infra team does not use the meetbot to update channel topics." since we run the meetbot and it _does_ temporarily update irc channel topics. trying to figure out a less confusing way to word that20:59
clarkbok sending irc message to dev list as soon as I can get it copied from one location to another21:00
clarkboh wait last message above is re ^ I'll wait21:00
*** trown is now known as trown|outtypewww21:01
clarkbfungi: that came from my original paragraph, I'm open to a more better way of saying that too21:01
ianwdebian$ fs quota21:01
ianw82% of quota used.21:01
clarkbfungi: Maybe: "Should the infra team need to notify of pending maintenance work, that notification will come via the status bot and not the meetbot"21:01
fungiclarkb: thanks, much better21:02
dmsimardclarkb: +121:02
* fungi skims remainder of message real fast21:02
fungiyou're all green so i was having a hard time telling everyone apart21:03
clarkbfungi: well the color changed at least once there too21:04
clarkbsince it doesn't do the right thing with copy/cut and paste21:04
fungiclarkb: i've made some very minor edits, but looks good and hopefuly nonconfusing21:06
fungier, hopefully21:06
clarkbcool I'll work on sending it now. Thank you for helping make it more clear21:06
fungithanks for drafting it!21:07
dmsimardI changed my shade of green :P21:07
dmsimard</troll>21:07
fungiheh21:07
fungisorry i wasn't faster to read through it, got waylaid with dinner prep21:08
*** edmondsw has quit IRC21:09
*** sshnaidm|bbl is now known as sshnaidm21:09
clarkbfungi: I should be better at getting waylaid. I've completely failed at doing things like eating proper lunch and have been swamped by meetings all day. THe good news is once I send this email I can actually take a few minutes to recombobulate my day21:09
*** edmondsw has joined #openstack-infra21:09
fungipabelanger: were you planning to send a copy of that gerrit ip address change announcement to the third-party-announce ml as well?21:10
*** ldnunes has quit IRC21:13
fungii expect they're the most likely ones to be impacted by the change as most normal gerrit users can rely on the rest api and git over https, but people who run ci systems need the gerrit event stream which is only accessible through the ssh api21:13
clarkbalright with email sent I'm going to afk for a bit21:14
*** edmondsw has quit IRC21:14
clarkbfungi: ++21:14
*** e0ne has quit IRC21:17
pabelangerfungi: good idea!21:21
*** yamamoto has joined #openstack-infra21:24
openstackgerritPaul Belanger proposed openstack-infra/nodepool master: Use AFS mirrors for debian-stretch testing  https://review.openstack.org/55863321:24
*** shardy has quit IRC21:25
*** shardy has joined #openstack-infra21:25
*** ldnunes has joined #openstack-infra21:25
openstackgerritMerged openstack-infra/system-config master: Add arm64 to Debian mirror  https://review.openstack.org/55862421:26
openstackgerritColleen Murphy proposed openstack-infra/openstack-zuul-jobs master: Convert infra-puppet-apply jobs to non-legacy  https://review.openstack.org/55701121:26
*** aeng has joined #openstack-infra21:28
pabelangerclarkb: fungi: ML post sent, thanks21:29
*** yamamoto has quit IRC21:29
*** ldnunes has quit IRC21:31
clarkbdirk: when you get a chance can you review https://review.openstack.org/#/c/558260/ ?21:32
*** gouthamr has quit IRC21:37
ianw2018-04-03 21:14:56.055411 | ubuntu-xenial | kazoo.exceptions.ConnectionLoss ... http://logs.openstack.org/17/558617/1/gate/tox-py35/3fc86ed/job-output.txt.gz21:39
ianwis zookeeper issues something we know about in tox-py35 jobs for nodepool?21:39
pabelangerianw: which provider, I think we've seen that when the compute node is a little slow21:40
pabelangerlimestone it looks like21:40
ianwcloud: limestone21:41
ianwfrom what can be seen, it does seem like the connection goes up and down21:41
*** eharney has quit IRC21:42
*** HeOS has joined #openstack-infra21:42
clarkbusing all the cpus for python may be starving the jvm?21:46
*** Goneri has quit IRC21:47
pabelangerhow many vcpus did we do again in limestone?21:47
logan-8/8gb/80gb21:48
*** e0ne has joined #openstack-infra21:52
*** rcernin has joined #openstack-infra21:53
logan-i will be home in a few, can check for a local compute node issue where that instance was located. havent seen much trend of failures on there while monitoring logstash, but i have noticed the jobs are primarily cpu heavy so i turned one compute off in nova an hour or so ago so we can roll thru and upgrade cpus from 2620 to 2650 so there are more physical cores to work with21:54
*** efried_schoolrun is now known as efried21:54
logan-if thats causing too much resource contention ill add more computes asap to spread things out more21:56
*** rfolco|brb is now known as rfolco|away21:56
clarkbwe can also reduce max servers count to reduce contention while you do that21:59
*** jcoufal has quit IRC21:59
logan-as soon as centos-7-limestone-regionone-0003316910 i will upgrade the compute and turn it back on, then we can add more capacity before doing any more upgrades22:04
*** spzala has quit IRC22:04
logan-as soon as centos-7.. finishes and deletes*22:05
logan-which it just did as i typed that22:05
openstackgerritIan Wienand proposed openstack-infra/system-config master: Add debian arm64 to updates  https://review.openstack.org/55864122:06
ianwclarkb / logan-: http://logs.openstack.org/17/558617/1/check/tox-py35/d8b51a0/ another failure, seems to be about the same22:10
openstackgerritMerged openstack-infra/release-tools master: Make getopt and eol_branch.sh agree on the name for the dry-run argument  https://review.openstack.org/55785822:10
*** inc0 has quit IRC22:11
logan-yea so recent22:11
logan-should resolve shortly once this node is back up22:12
openstackgerritColleen Murphy proposed openstack-infra/openstack-zuul-jobs master: Convert infra-puppet-apply jobs to non-legacy  https://review.openstack.org/55701122:13
*** priteau has quit IRC22:15
openstackgerritJames E. Blair proposed openstack-infra/zuul master: WIP: Check out more appropriate branches of role and playbook repos  https://review.openstack.org/55864322:22
*** bobh_ has quit IRC22:22
*** jlabarre has quit IRC22:25
*** yamamoto has joined #openstack-infra22:26
*** agopi has joined #openstack-infra22:28
*** rpioso is now known as rpioso|afk22:28
*** yamamoto has quit IRC22:30
*** inc0 has joined #openstack-infra22:31
*** shardy has quit IRC22:31
*** shardy has joined #openstack-infra22:31
*** anteaya has quit IRC22:31
*** anteaya has joined #openstack-infra22:32
logan-back up ^ will keep an eye out for any more issues but it should taper off now. i think we'll be able to avoid a recurrence without modifying max-servers22:35
*** felipemonteiro_ has quit IRC22:35
*** e0ne has quit IRC22:36
logan-good motivation to add more capacity :D22:36
*** lbragstad has quit IRC22:36
*** boden has quit IRC22:43
openstackgerritMonty Taylor proposed openstack-infra/zuul master: WIP Make websocket streaming more event-driven  https://review.openstack.org/55864622:44
*** shardy has quit IRC22:44
openstackgerritMerged openstack-infra/system-config master: Add debian arm64 to updates  https://review.openstack.org/55864122:46
*** shardy has joined #openstack-infra22:49
*** inc0 has quit IRC22:51
*** lbragstad has joined #openstack-infra22:53
*** inc0 has joined #openstack-infra22:53
*** jamesmcarthur has joined #openstack-infra22:53
*** hongbin has quit IRC22:58
*** jamesmcarthur has quit IRC22:59
*** VW has quit IRC23:01
*** inc0 has quit IRC23:12
*** r-daneel has quit IRC23:18
*** yamamoto has joined #openstack-infra23:26
*** neiloy has quit IRC23:29
*** inc0 has joined #openstack-infra23:29
*** kjackal has quit IRC23:30
*** kjackal has joined #openstack-infra23:31
*** yamamoto has quit IRC23:32
pabelangerclarkb: ianw: logan-: http://logs.openstack.org/33/558133/2/gate/tempest-full/0b0e82a/job-output.txt.gz looks to be another timeout on limestone, that was from top of gate.23:33
*** jamesmcarthur has joined #openstack-infra23:34
*** threestrands has joined #openstack-infra23:36
prometheanfireinfra experiencing longer gate times due to clounds patching for spectre/meltdown?23:36
* prometheanfire is wondering if we should increase our gate23:37
*** threestrands has quit IRC23:37
*** threestrands has joined #openstack-infra23:38
*** threestrands has quit IRC23:38
*** threestrands has joined #openstack-infra23:38
ianwhttp://logs.openstack.org/33/558133/2/gate/tempest-full/0b0e82a/controller/logs/screen-n-cpu.txt.gz#_Apr_03_22_43_50_66925323:38
ianwBuild of instance 34281e21-2749-408f-a31e-52a8aa0442d4 was re-scheduled:23:39
*** threestrands has quit IRC23:39
*** jamesmcarthur has quit IRC23:39
*** threestrands has joined #openstack-infra23:39
*** threestrands has quit IRC23:39
*** threestrands has joined #openstack-infra23:39
ianwalthough, i guess http://logs.openstack.org/33/558133/2/gate/tempest-full/0b0e82a/controller/logs/screen-n-cpu.txt.gz#_Apr_03_22_43_48_898261 is a timeout waiting for rpc stuff23:40
ianwwhich is actually probably pretty similar to zookeeper dropping out23:40
*** threestrands has quit IRC23:40
*** threestrands has joined #openstack-infra23:41
*** spzala has joined #openstack-infra23:45
openstackgerritMerged openstack-infra/nodepool master: Add some more fudge factor for growroot test  https://review.openstack.org/55861723:51
*** dsariel has quit IRC23:53
*** inc0 has quit IRC23:53
openstackgerritMerged openstack-dev/pbr master: Better Sem-Ver header handling  https://review.openstack.org/55473023:55
*** dprince has quit IRC23:56
*** efoley has quit IRC23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!