Wednesday, 2018-04-11

*** jklare has joined #openstack-infra00:00
*** priteau has joined #openstack-infra00:03
*** priteau has quit IRC00:03
*** zer0c00l has joined #openstack-infra00:04
zer0c00lHow does the CI picks up a node from nodepool? Is there a nodepool command to request for a node?00:05
zer0c00lnodepool --help doesn't have anycommand to pick/lock a node from the pool http://paste.openstack.org/show/718868/00:06
pabelangerzer0c00l: there is no command to manually do it, nodepool-launcher has the logic built internal to look for NodeRequests00:06
*** yamahata has joined #openstack-infra00:07
pabelangerzer0c00l: the interface is zookeeper, there have been few people building tools around that to support things like jenkins00:07
zer0c00lpabelanger: i see, so CI/some script in CI talks to zookeeper directly to create a "nodeRequest" ?00:08
*** aeng has joined #openstack-infra00:09
zer0c00li am trying to setup a downstream nodepool for our internal CI. I am able to bring up a nodepool-launcher which spawns some nodes, but i have no idea on how to 'create' a noderequest00:09
pabelangerzer0c00l: Yah, we don't recommend doing it yourself, that's why we wrote nodepool-launcher. Is there a use case where that does not work?00:09
pabelangerzer0c00l: usually zuul will create them as demand is required00:09
zer0c00lpabelanger: i see, zuul launches nodepool-launcher?00:10
zer0c00lWhich in turn creates pool of VMs?00:10
zer0c00lIs that a correct understanding?00:10
zer0c00lI was under the impression that nodepool is setup seperately runs on nodepool.openstack.org00:11
*** digurh has joined #openstack-infra00:11
zer0c00lManages bunch of Nodes for CI/devstack use00:11
pabelangerzer0c00l: zuul will create a NodeRequest, based on the nodesets a job uses. In turn, nodepool-launcher will then query zookeeper to see what needs to be done, they launch VMs as needed. Updating the nodeRequest so zuul-executor can then use said node00:11
pabelangerzer0c00l: yes, it is a separate application, but really is connected to zuul today00:12
zer0c00lIs there a script/code that creates this NodeRequest?00:12
pabelangerzer0c00l: yes, as I said, there are some people just using nodepool without zuul, but we don't really have a public api to handle noderequests right now00:12
zer0c00lin zull00:13
zer0c00l*zuul00:13
zer0c00lokay00:13
pabelangerzer0c00l: yes, you can look at zuul source, there is also the following etherpad: https://etherpad.openstack.org/p/zuulv3-jenkins-integration00:13
clarkbzer0c00l: are you trying to use nodepool for something specific?00:13
pabelangerI know somebody has written a basic jenkins driver for zookeeper, but unsure who ATM00:13
*** olaph1 has joined #openstack-infra00:14
zer0c00lclarkb: yes, we have a downstream CI for openstack00:14
*** olaph has quit IRC00:14
zer0c00lclarkb: want to run some devstack/tempest tests on pull requests/gerrit reviews downstream00:14
zer0c00lnodepool with bunch of VMs seemed like a good idea00:14
pabelangersounds like you are wanting to do 3rd party CI00:15
pabelangerwhich, works well today using zuul and nodepool00:15
clarkbya for that use case I'd consider using zuul with nodepool00:15
zer0c00li see00:15
*** smarcet has joined #openstack-infra00:16
*** jcoufal has joined #openstack-infra00:20
*** zhurong has joined #openstack-infra00:21
*** dpawlik has joined #openstack-infra00:33
*** gyee has quit IRC00:33
ianwfull-stack engineers is so 2017, now it's "polyglot engineers"00:34
*** smarcet has quit IRC00:36
*** smarcet has joined #openstack-infra00:37
*** dpawlik has quit IRC00:37
*** smarcet has quit IRC00:38
*** digurh has quit IRC00:38
*** dhill_ has quit IRC00:40
openstackgerritIan Wienand proposed openstack-infra/puppet-askbot master: Don't require virtualenv package  https://review.openstack.org/55962200:41
openstackgerritIan Wienand proposed openstack-infra/puppet-askbot master: Ensure more correct ordering  https://review.openstack.org/55962300:41
openstackgerritIan Wienand proposed openstack-infra/puppet-askbot master: Use earlier six release  https://review.openstack.org/55962400:41
openstackgerritIan Wienand proposed openstack-infra/puppet-askbot master: Add rspec test  https://review.openstack.org/55899100:41
*** yamamoto has joined #openstack-infra00:43
*** dave-mccowan has joined #openstack-infra00:47
*** odyssey4me has quit IRC00:47
*** odyssey4me has joined #openstack-infra00:47
*** yamamoto has quit IRC00:48
*** jcoufal has quit IRC00:49
*** mriedem has quit IRC00:49
*** aeng has quit IRC00:49
*** felipemonteiro__ has joined #openstack-infra00:50
*** germs has quit IRC00:51
*** germs has joined #openstack-infra00:52
*** germs has quit IRC00:52
*** germs has joined #openstack-infra00:52
*** germs has quit IRC00:56
*** agopi has quit IRC01:01
*** shu-mutow-AWAY is now known as shu-mutow01:02
*** rosmaita has quit IRC01:04
*** wolverineav has quit IRC01:13
*** wolverin_ has joined #openstack-infra01:13
*** harlowja has quit IRC01:14
*** salv-orl_ has joined #openstack-infra01:14
*** kiennt26 has joined #openstack-infra01:14
*** salv-orlando has quit IRC01:17
*** olaph has joined #openstack-infra01:18
openstackgerritIan Wienand proposed openstack-infra/system-config master: Update to latest puppet-python  https://review.openstack.org/56023401:19
openstackgerritIan Wienand proposed openstack-infra/puppet-askbot master: Don't require virtualenv package  https://review.openstack.org/55962201:20
openstackgerritIan Wienand proposed openstack-infra/puppet-askbot master: Ensure more correct ordering  https://review.openstack.org/55962301:20
openstackgerritIan Wienand proposed openstack-infra/puppet-askbot master: Use earlier six release  https://review.openstack.org/55962401:20
openstackgerritIan Wienand proposed openstack-infra/puppet-askbot master: Add rspec test  https://review.openstack.org/55899101:20
*** olaph1 has quit IRC01:20
*** sihan_ has quit IRC01:23
*** zhangfei has joined #openstack-infra01:23
*** iyamahat has quit IRC01:24
*** iyamahat has joined #openstack-infra01:24
*** felipemonteiro__ has quit IRC01:25
*** wolverin_ has quit IRC01:32
*** wolverineav has joined #openstack-infra01:32
*** ihrachys has quit IRC01:33
*** wolverin_ has joined #openstack-infra01:35
*** wolverineav has quit IRC01:36
*** berndbausch has joined #openstack-infra01:38
*** hongbin has joined #openstack-infra01:40
*** wolverin_ has quit IRC01:40
logan-zer0c00l: i'm using https://plugins.jenkins.io/nodepool-agents to consume nodepool from jenkins01:43
*** germs has joined #openstack-infra01:44
*** germs has quit IRC01:44
*** germs has joined #openstack-infra01:44
*** yamamoto has joined #openstack-infra01:44
*** cshastri has joined #openstack-infra01:45
*** agopi has joined #openstack-infra01:48
*** yamamoto has quit IRC01:50
*** wolverineav has joined #openstack-infra01:54
*** wolverineav has quit IRC01:58
*** zhurong has quit IRC02:00
*** wolverineav has joined #openstack-infra02:01
*** gagehugo has quit IRC02:01
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: mqtt: add basic reporter  https://review.openstack.org/53554302:09
zer0c00llogan-: will take a look02:10
*** s-shiono has joined #openstack-infra02:10
*** bhavik1 has joined #openstack-infra02:13
*** nicolasbock has quit IRC02:14
*** bhavik1 has quit IRC02:16
*** claudiub has quit IRC02:19
*** ramishra has joined #openstack-infra02:29
*** germs has quit IRC02:32
*** ianychoi has quit IRC02:35
*** yamamoto has joined #openstack-infra02:47
*** r-daneel has joined #openstack-infra02:47
*** andreas_s has joined #openstack-infra02:49
*** yamamoto has quit IRC02:52
*** sambetts|afk has quit IRC02:53
*** andreas_s has quit IRC02:54
*** sambetts_ has joined #openstack-infra02:54
*** dave-mccowan has quit IRC02:56
*** psachin has joined #openstack-infra03:00
*** imacdonn has quit IRC03:08
*** imacdonn has joined #openstack-infra03:08
*** psachin has quit IRC03:09
*** ianychoi has joined #openstack-infra03:19
openstackgerritKendall Nelson proposed openstack-infra/storyboard master: Reorganize Docs  https://review.openstack.org/55982703:21
*** psachin has joined #openstack-infra03:26
*** kei-ichi has joined #openstack-infra03:28
kei-ichiHello infra-team members. thank you for your cooperation for creating slogging project! I'd like to ask you one thing. Cloud you plz add me (Keiichi HIkita / keiichi.hikita@gmail.com)03:33
kei-ichi as slogging-core (https://review.openstack.org/#/admin/groups/1892,members)  member?03:33
*** vipul has quit IRC03:33
*** spiffxp has quit IRC03:34
*** udesale has joined #openstack-infra03:34
*** agopi has quit IRC03:37
*** kiennt26 has quit IRC03:41
openstackgerritIan Wienand proposed openstack-infra/system-config master: Update to latest puppet-python  https://review.openstack.org/56023403:41
*** yamamoto has joined #openstack-infra03:48
*** VW has quit IRC03:50
*** yamamoto has quit IRC03:53
*** caphrim007 has joined #openstack-infra03:54
prometheanfireetherpad rce, fun03:54
*** hongbin has quit IRC03:59
*** Qiming_ is now known as Qiming04:02
*** felipemonteiro__ has joined #openstack-infra04:13
*** neiloy has joined #openstack-infra04:20
*** jaosorior has joined #openstack-infra04:21
*** harlowja has joined #openstack-infra04:23
*** sree has joined #openstack-infra04:26
*** udesale has quit IRC04:26
AJaegerclarkb: tox-cover should stay in project-config, shouldn't it? And thus I would not push back on that change. But we should start pushing back on legacy changes....04:27
AJaegerianw, frickler, could you put infra-manual on your review list, please?04:28
*** rlandy has quit IRC04:31
openstackgerritMerged openstack-infra/project-config master: Create 'Review-Priority' for designate repos  https://review.openstack.org/29525304:32
*** dpawlik has joined #openstack-infra04:33
*** dpawlik has quit IRC04:38
*** udesale has joined #openstack-infra04:45
*** stakeda has joined #openstack-infra04:46
*** udesale has quit IRC04:46
*** udesale has joined #openstack-infra04:47
*** neiloy has quit IRC04:47
*** yamamoto has joined #openstack-infra04:50
*** felipemonteiro_ has joined #openstack-infra04:51
*** felipemonteiro__ has quit IRC04:54
*** yamamoto has quit IRC04:55
*** felipemonteiro_ has quit IRC05:02
*** pcichy has joined #openstack-infra05:08
*** d0ugal has quit IRC05:16
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Also prefix the alembic_version table  https://review.openstack.org/56026305:18
*** aeng has joined #openstack-infra05:24
fricklerkei-ichi: done05:26
*** d0ugal has joined #openstack-infra05:26
*** dhajare has joined #openstack-infra05:28
*** links has joined #openstack-infra05:29
*** dhajare has quit IRC05:31
*** dhajare has joined #openstack-infra05:31
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Also prefix the alembic_version table  https://review.openstack.org/56026305:31
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Add postgresql release note  https://review.openstack.org/56007905:33
kei-ichifrickler Thanks!05:34
*** harlowja has quit IRC05:35
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Add regex support to project stanzas  https://review.openstack.org/53571305:36
*** markvoelker_ has quit IRC05:39
*** iranzo has joined #openstack-infra05:40
openstackgerritTobias Henkel proposed openstack-infra/zuul master: DNM: test  https://review.openstack.org/56026505:41
*** armaan has quit IRC05:42
*** armaan has joined #openstack-infra05:44
*** kjackal has joined #openstack-infra05:46
*** wolverineav has quit IRC05:46
*** neiloy has joined #openstack-infra05:48
*** iranzo has quit IRC05:49
*** yamamoto has joined #openstack-infra05:52
*** iranzo has joined #openstack-infra05:53
openstackgerritMerged openstack-infra/zuul master: Return CORS headers on all requests  https://review.openstack.org/55502705:56
*** yamamoto has quit IRC05:57
openstackgerritMerged openstack-infra/infra-manual master: Add code-block for formatting to creators.rst  https://review.openstack.org/55990806:00
openstackgerritMerged openstack-infra/infra-manual master: Add missing -f to cookiecutter example  https://review.openstack.org/55990906:00
openstackgerritMerged openstack-infra/infra-manual master: Mention the availability of other cookiecutter templates  https://review.openstack.org/54990406:00
*** quite has quit IRC06:02
AJaegerfrickler: check https://governance.openstack.org/tc/reference/projects/release-management.html - I changed it because of that reference06:05
AJaegerfrickler: But I can leave it as is, if you think I'm confused...06:06
*** trown|outtypewww has quit IRC06:06
AJaegerfrickler: Ah, "RElease Managers" as Gerrit group - will change...06:08
*** trown has joined #openstack-infra06:08
openstackgerritAndreas Jaeger proposed openstack-infra/infra-manual master: Update ACL sections with release team policy change  https://review.openstack.org/55994606:09
AJaegerfrickler: fixed, thanks! ^06:10
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config master: Normalize projects.yaml  https://review.openstack.org/56027206:10
*** sree has quit IRC06:11
*** sree has joined #openstack-infra06:12
*** neiloy has quit IRC06:12
*** sree has quit IRC06:13
*** sree has joined #openstack-infra06:13
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: gerrit: recognize project-created event  https://review.openstack.org/56027406:14
openstackgerritIan Wienand proposed openstack-infra/system-config master: Update to latest puppet-python  https://review.openstack.org/56023406:14
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Get rid of Jenkins slave scripts  https://review.openstack.org/54840406:14
*** armaan has quit IRC06:19
*** armaan has joined #openstack-infra06:20
*** sree has quit IRC06:22
*** sree has joined #openstack-infra06:22
*** sree has quit IRC06:24
*** armaan has quit IRC06:24
*** sree has joined #openstack-infra06:25
*** armaan has joined #openstack-infra06:25
*** quite has joined #openstack-infra06:25
*** sree has quit IRC06:26
*** sree has joined #openstack-infra06:27
*** hemna_ has quit IRC06:29
*** jhesketh_ is now known as jhesketh06:29
*** sree has quit IRC06:31
openstackgerritMerged openstack-infra/project-config master: Normalize projects.yaml  https://review.openstack.org/56027206:32
*** dbecker has quit IRC06:32
*** priteau has joined #openstack-infra06:32
*** hashar has joined #openstack-infra06:36
*** olaph1 has joined #openstack-infra06:37
*** olaph has quit IRC06:38
*** dsariel has joined #openstack-infra06:39
*** dpawlik has joined #openstack-infra06:39
*** dsariel has quit IRC06:40
*** chkumar|off is now known as chandankumar06:41
*** sree has joined #openstack-infra06:43
*** dpawlik has quit IRC06:44
*** dbecker has joined #openstack-infra06:44
*** dingyichen has joined #openstack-infra06:46
*** slaweq has joined #openstack-infra06:47
*** sree has quit IRC06:48
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: mqtt: add basic reporter  https://review.openstack.org/53554306:49
*** yamamoto has joined #openstack-infra06:53
*** alexchadin has joined #openstack-infra06:53
*** janki has joined #openstack-infra06:54
*** florianf has joined #openstack-infra06:58
*** florianf has joined #openstack-infra06:58
*** yamamoto has quit IRC06:59
*** dpawlik has joined #openstack-infra06:59
openstackgerritIan Wienand proposed openstack-infra/system-config master: Update to latest puppet-python  https://review.openstack.org/56023406:59
*** krenczew1 has quit IRC07:01
*** krenczewski has joined #openstack-infra07:01
AJaegerianw, frickler, could you review these simple job removals from openstack-zuul-jobs, please? https://review.openstack.org/558205 https://review.openstack.org/55937807:04
AJaegerand two project-config changes: https://review.openstack.org/#/c/555722/ and https://review.openstack.org/55505307:04
*** sree has joined #openstack-infra07:05
*** rcernin has quit IRC07:06
*** yamamoto has joined #openstack-infra07:08
*** amoralej|off is now known as amoralej07:08
*** zhurong has joined #openstack-infra07:09
*** sree has quit IRC07:10
*** shardy has joined #openstack-infra07:14
*** HeOS has joined #openstack-infra07:14
openstackgerritIan Wienand proposed openstack-infra/system-config master: Update to latest puppet-python  https://review.openstack.org/56023407:23
*** tesseract has joined #openstack-infra07:28
*** claudiub has joined #openstack-infra07:29
*** iyamahat has quit IRC07:29
*** gcb has joined #openstack-infra07:30
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: [rally] Remove unused jobs  https://review.openstack.org/55820507:33
*** andreas_s has joined #openstack-infra07:34
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: [rally] remove legacy-rally-dsvm-magnum-rally  https://review.openstack.org/55937807:36
openstackgerritMerged openstack-infra/project-config master: Set up python-cyborgclient repository  https://review.openstack.org/55572207:36
*** xinliang has quit IRC07:42
*** xinliang has joined #openstack-infra07:42
*** yamamoto has quit IRC07:49
*** dsariel has joined #openstack-infra07:51
*** ralonsoh has joined #openstack-infra07:54
*** jpena|off is now known as jpena07:55
*** tosky has joined #openstack-infra07:55
*** jpich has joined #openstack-infra07:58
*** sree has joined #openstack-infra07:58
*** Hal has joined #openstack-infra08:01
*** markvoelker has joined #openstack-infra08:01
*** rossella_s has joined #openstack-infra08:02
*** arxcruz|off is now known as arxcruz|ruck08:04
*** sree has quit IRC08:08
*** zhurong has quit IRC08:08
*** dhajare has quit IRC08:10
*** lucas-afk is now known as lucasagomes08:11
*** pcaruana has joined #openstack-infra08:13
*** yamamoto has joined #openstack-infra08:13
*** yamamoto has quit IRC08:15
*** calbers has joined #openstack-infra08:16
*** yamamoto has joined #openstack-infra08:19
*** aojea has joined #openstack-infra08:29
*** ramineni_ has joined #openstack-infra08:34
*** s-shiono has quit IRC08:35
*** pcichy has quit IRC08:35
*** annp has quit IRC08:35
*** pcichy has joined #openstack-infra08:36
*** derekh has joined #openstack-infra08:40
*** pcichy has quit IRC08:40
*** pcichy has joined #openstack-infra08:40
*** electrofelix has joined #openstack-infra08:40
*** annp has joined #openstack-infra08:42
*** ianychoi has quit IRC08:42
*** aeng has quit IRC08:50
*** zhurong has joined #openstack-infra08:50
*** dhajare has joined #openstack-infra08:51
*** yamamoto has quit IRC08:54
*** yamamoto has joined #openstack-infra08:56
*** yamamoto has quit IRC08:58
*** yamamoto has joined #openstack-infra08:58
*** stakeda has quit IRC08:59
*** dingyichen has quit IRC09:03
*** olaph has joined #openstack-infra09:04
*** olaph1 has quit IRC09:05
openstackgerritIan Wienand proposed openstack-infra/puppet-askbot master: Don't require virtualenv package  https://review.openstack.org/55962209:07
openstackgerritIan Wienand proposed openstack-infra/puppet-askbot master: Ensure more correct ordering  https://review.openstack.org/55962309:07
openstackgerritIan Wienand proposed openstack-infra/puppet-askbot master: Use earlier six release  https://review.openstack.org/55962409:07
openstackgerritIan Wienand proposed openstack-infra/puppet-askbot master: Add rspec test  https://review.openstack.org/55899109:07
*** shu-mutow is now known as shu-mutow-AWAY09:07
openstackgerritWitold Bedyk proposed openstack-infra/project-config master: Enable voting Python 3 unit tests for monasca-ui  https://review.openstack.org/56032309:07
*** pblaho has joined #openstack-infra09:08
*** gfidente has joined #openstack-infra09:14
*** gfidente has joined #openstack-infra09:14
*** lpetrut has joined #openstack-infra09:16
openstackgerritThierry Carrez proposed openstack-infra/project-config master: Remove direct tag / branch ACLs for official projects  https://review.openstack.org/55773009:18
mugsiehey could I get someone to add me (gr@ham.ie) to https://review.openstack.org/#/admin/groups/1893,members ?09:27
mugsiethe group was created by https://review.openstack.org/#/c/295253/ which I proposed09:27
mugsiethanks !09:27
*** dpawlik has quit IRC09:39
*** annp has quit IRC09:40
*** dpawlik has joined #openstack-infra09:40
*** annp has joined #openstack-infra09:40
fricklermugsie: done09:41
mugsiefrickler: ty09:41
*** olaph1 has joined #openstack-infra09:42
*** yamamoto has quit IRC09:43
*** yamamoto has joined #openstack-infra09:44
AJaegermugsie: wow, took you 2 years to get that merged ;(09:44
mugsieAJaeger: yeah - just a little bit of time :)09:44
*** olaph has quit IRC09:45
AJaegermugsie: I'd be interested to learn how you'll use it and whether the experiment is successfull - in a couple of weeks09:45
mugsieyeah - I think the real test will be around branch freezes and releases09:46
openstackgerritIan Wienand proposed openstack-infra/project-config master: Create puppet-solr  https://review.openstack.org/56033609:46
*** ianychoi has joined #openstack-infra09:46
*** alexchadin has quit IRC09:47
*** alexchadin has joined #openstack-infra09:48
*** hashar is now known as hasharAway09:48
*** alexchadin has quit IRC09:48
*** alexchadin has joined #openstack-infra09:49
*** alexchadin has quit IRC09:49
*** alexchadin has joined #openstack-infra09:50
*** alexchadin has quit IRC09:50
*** alexchadin has joined #openstack-infra09:50
openstackgerritAndreas Scheuring proposed openstack-infra/project-config master: Networking-dpm now uses in-repo zuul configuration  https://review.openstack.org/56033909:50
*** alexchadin has quit IRC09:51
*** alexchadin has joined #openstack-infra09:51
openstackgerritIan Wienand proposed openstack-infra/project-config master: Create puppet-solr  https://review.openstack.org/56033609:51
openstackgerritIan Wienand proposed openstack-infra/project-config master: Add puppet-solr jobs  https://review.openstack.org/56034009:51
*** alexchadin has quit IRC09:51
*** alexchadin has joined #openstack-infra09:52
*** lewo has quit IRC09:52
*** Hal has quit IRC09:54
*** gfidente has quit IRC09:55
*** alexchadin has quit IRC09:56
openstackgerritIan Wienand proposed openstack-infra/project-config master: Create puppet-solr  https://review.openstack.org/56033609:57
openstackgerritIan Wienand proposed openstack-infra/project-config master: Add puppet-solr jobs  https://review.openstack.org/56034009:57
openstackgerritMerged openstack/diskimage-builder master: Don't only install python3-virtualenv  https://review.openstack.org/55987110:00
AJaegermugsie: ok, let's see...10:04
*** sree has joined #openstack-infra10:05
*** katkapilatova has joined #openstack-infra10:07
*** katkapilatova has left #openstack-infra10:07
*** sree has quit IRC10:09
openstackgerritMerged openstack/diskimage-builder master: Set the dhclient timeout to match DIB_DHCP_TIMEOUT  https://review.openstack.org/55908410:12
*** pbourke has joined #openstack-infra10:14
*** annp has quit IRC10:14
*** annp has joined #openstack-infra10:15
*** sree has joined #openstack-infra10:15
*** cshastri has quit IRC10:19
*** gfidente has joined #openstack-infra10:20
*** gfidente has joined #openstack-infra10:20
*** lewo has joined #openstack-infra10:23
*** alexchadin has joined #openstack-infra10:23
*** rajinir has quit IRC10:25
odyssey4meodd, is zuul dead in some way? we +2 a patch 25 mins ago and it's not in the queue: https://review.openstack.org/#/c/560168/10:26
*** lpetrut has quit IRC10:27
*** zoli is now known as zoli|lunch10:29
*** lpetrut has joined #openstack-infra10:30
*** armaan has quit IRC10:36
*** annp has quit IRC10:36
*** shardy is now known as shardy_afk10:36
*** armaan has joined #openstack-infra10:36
*** nicolasbock has joined #openstack-infra10:37
*** alexchadin has quit IRC10:45
*** neiloy has joined #openstack-infra10:47
*** alexchadin has joined #openstack-infra10:50
*** e0ne has joined #openstack-infra10:54
fricklerodyssey4me: that looks strange indeed, I'll try to dig into zuul logs10:57
*** neiloy has quit IRC11:00
*** neiloy has joined #openstack-infra11:01
*** zhangfei has quit IRC11:01
*** alexchadin has quit IRC11:03
*** shardy_afk is now known as shardy11:04
*** e0ne has quit IRC11:04
fricklerodyssey4me: the +2+A at 10:01 by evrardjp is missing in the zuul logs, seems this got lost somehow on the way from gerrit to zuul. could one of you please try to re-apply the +A ?11:05
odyssey4medone11:05
AJaegerand queued...11:06
odyssey4meodd, I guess the event just got missed somewhere11:06
odyssey4methanks for looking into it11:06
*** xinliang has quit IRC11:11
*** neiloy has quit IRC11:11
fricklerodyssey4me: gerrit has logged an exception for your event, but I think that's unrelated to your patch. seems to just happen occasionally, found two other occurrences in 48h. http://paste.openstack.org/show/718909/11:12
fricklerinfra-root: ^^ not sure whether someone will want to investigate further or whether to accept this as sporadic known issue11:12
*** dpawlik_ has joined #openstack-infra11:13
*** dpawlik has quit IRC11:13
*** yamamoto has quit IRC11:18
*** e0ne has joined #openstack-infra11:20
*** panda|off is now known as panda|lunch11:21
openstackgerritWitold Bedyk proposed openstack-infra/project-config master: Enable voting Python 3 unit tests for monasca-ui  https://review.openstack.org/56032311:22
*** xinliang has joined #openstack-infra11:22
*** xinliang has quit IRC11:22
*** xinliang has joined #openstack-infra11:22
openstackgerritWitold Bedyk proposed openstack-infra/project-config master: Enable voting Python 3 unit tests for monasca-ui  https://review.openstack.org/56032311:24
*** yamamoto has joined #openstack-infra11:25
*** sambetts_ is now known as sambetts11:28
*** yamamoto has quit IRC11:30
*** yamamoto has joined #openstack-infra11:34
*** zoli|lunch is now known as zoli11:36
*** armaan has quit IRC11:39
*** yamamoto has quit IRC11:39
*** armaan has joined #openstack-infra11:39
*** rfolco|off is now known as rfolco|rover11:44
*** jpena is now known as jpena|lunch11:46
*** olaph1 has quit IRC11:47
*** olaph has joined #openstack-infra11:48
*** Hal has joined #openstack-infra11:48
*** ldnunes has joined #openstack-infra11:50
*** aojea has quit IRC11:50
*** ramineni_ has left #openstack-infra11:57
pabelangerinfra-root: I'm traveling most of the day today, and will catch up on backscroll later this evening11:57
AJaegerpabelanger: safe travels!11:58
dmelladoodyssey4me: extremely slow for me at least11:59
dmelladoAJaeger: is that slowness something that is due to capacity or is there something else?12:00
fricklerdmellado: haven't confirmed slowness yet, where do you see that? the earlier issue was something different12:01
*** amoralej is now known as amoralej|lunch12:01
*** dsariel has quit IRC12:01
dmelladohey frickler for example https://review.openstack.org/55360512:01
dmelladotaking waaaay longer than it should12:02
*** neiloy has joined #openstack-infra12:04
*** hasharAway has quit IRC12:05
*** jcoufal has joined #openstack-infra12:07
fricklerdmellado: I don't see that, seems the recheck was at 10:28, now it is 12:05 and zuul estimates 26 min remaining, which isn't optimal, but in my expected range for tempest-full or similar jobs12:07
dmelladoheh, well, I'll take a look at another runs lately12:08
dmelladowe're just running around +-10 tempest jobs, but lets see12:08
dmellados/jobs/tests12:08
*** edmondsw has joined #openstack-infra12:08
*** rosmaita has joined #openstack-infra12:11
*** odyssey4me has quit IRC12:14
*** odyssey4me has joined #openstack-infra12:14
*** alexchadin has joined #openstack-infra12:15
*** panda|lunch is now known as panda12:16
*** tpsilva has joined #openstack-infra12:16
*** yamamoto has joined #openstack-infra12:21
*** yamamoto has quit IRC12:25
*** gcb has quit IRC12:27
*** neiloy_ has joined #openstack-infra12:28
*** rlandy has joined #openstack-infra12:30
*** neiloy has quit IRC12:32
*** yamamoto has joined #openstack-infra12:34
*** dave-mccowan has joined #openstack-infra12:35
*** olaph1 has joined #openstack-infra12:39
*** olaph has quit IRC12:41
rosmaitaeasy review if someone has a few minutes: https://review.openstack.org/#/c/559839/ (will save some resources, too)12:45
*** dpawlik_ has quit IRC12:50
*** jpena|lunch is now known as jpena12:50
*** VW has joined #openstack-infra12:50
*** VW has quit IRC12:51
*** VW has joined #openstack-infra12:52
*** dpawlik has joined #openstack-infra12:53
*** salv-orl_ has quit IRC12:54
*** felipemonteiro has joined #openstack-infra12:54
*** kgiusti has joined #openstack-infra12:55
*** VW has quit IRC12:56
*** dpawlik has quit IRC12:57
*** armaan has quit IRC12:58
*** armaan has joined #openstack-infra12:58
*** amoralej|lunch is now known as amoralej13:01
*** bobh has joined #openstack-infra13:01
*** armaan has quit IRC13:03
*** olaph1 is now known as olaph13:04
*** dsariel has joined #openstack-infra13:04
*** psachin has quit IRC13:05
*** mriedem has joined #openstack-infra13:06
*** felipemonteiro_ has joined #openstack-infra13:07
*** zoli is now known as zoli|afk-TPB13:09
*** felipemonteiro has quit IRC13:10
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck master: Add query for nova functional race KeyError bug 1762876  https://review.openstack.org/56040913:12
openstackbug 1762876 in OpenStack Compute (nova) "KeyError during move operation functional tests" [Medium,Confirmed] https://launchpad.net/bugs/176287613:12
*** cshastri has joined #openstack-infra13:13
*** Goneri has joined #openstack-infra13:14
*** felipemonteiro_ has quit IRC13:15
*** felipemonteiro__ has joined #openstack-infra13:15
openstackgerritWitold Bedyk proposed openstack-infra/project-config master: Enable voting Python 3 unit tests for monasca-ui  https://review.openstack.org/56032313:16
*** _ari_|conf is now known as _ari_13:17
openstackgerritWitold Bedyk proposed openstack-infra/project-config master: Enable voting Python 3 unit tests for monasca-ui  https://review.openstack.org/56032313:21
*** dhill_ has joined #openstack-infra13:22
*** dsariel has quit IRC13:22
*** dpawlik has joined #openstack-infra13:25
*** cshastri has quit IRC13:26
*** dhill_ has quit IRC13:26
*** dhill_ has joined #openstack-infra13:26
*** gcb has joined #openstack-infra13:33
*** pblaho has quit IRC13:35
*** esberglu has joined #openstack-infra13:35
*** jamesmcarthur has joined #openstack-infra13:37
openstackgerritMerged openstack-infra/elastic-recheck master: Add query for nova functional race KeyError bug 1762876  https://review.openstack.org/56040913:37
openstackbug 1762876 in OpenStack Compute (nova) "KeyError during move operation functional tests" [Medium,Confirmed] https://launchpad.net/bugs/176287613:37
*** VW has joined #openstack-infra13:38
*** VW has quit IRC13:40
*** VW has joined #openstack-infra13:41
*** david-lyle has joined #openstack-infra13:41
*** alexchadin has quit IRC13:44
*** dklyle has quit IRC13:45
*** hongbin has joined #openstack-infra13:47
*** AR2018 has joined #openstack-infra13:47
*** david-lyle has quit IRC13:47
*** udesale has quit IRC13:51
*** AR2018 has quit IRC13:51
*** armaan has joined #openstack-infra13:51
*** felipemonteiro_ has joined #openstack-infra13:54
*** ttx has quit IRC13:55
*** ttx has joined #openstack-infra13:57
*** links has quit IRC13:58
*** zhipeng has joined #openstack-infra13:58
*** felipemonteiro__ has quit IRC13:58
*** eharney has joined #openstack-infra14:00
*** udesale has joined #openstack-infra14:00
*** jistr is now known as jistr|mtg14:02
*** kiennt26 has joined #openstack-infra14:12
*** salv-orlando has joined #openstack-infra14:13
*** dsariel has joined #openstack-infra14:17
*** iyamahat has joined #openstack-infra14:19
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck master: Amend query for nova functional race bug 1762876  https://review.openstack.org/56043514:19
openstackbug 1762876 in OpenStack Compute (nova) "KeyError during move operation functional tests" [Medium,Confirmed] https://launchpad.net/bugs/176287614:19
*** eernst has joined #openstack-infra14:20
*** agopi has joined #openstack-infra14:22
openstackgerritRaoul Scarazzini proposed openstack-infra/project-config master: Create tripleo-ha-utils project (gerrit)  https://review.openstack.org/55583214:23
*** bobh has quit IRC14:25
corvusdirk: are you still using https://review.openstack.org/547779 ?14:28
corvuspabelanger: are you still using https://review.openstack.org/549867 ?14:29
*** iyamahat has quit IRC14:29
dirkcorvus: sorry, no14:30
*** iyamahat has joined #openstack-infra14:30
*** dklyle has joined #openstack-infra14:35
*** dklyle has quit IRC14:36
*** david-lyle has joined #openstack-infra14:36
AJaegercorvus: could you have a look at https://review.openstack.org/540890 (zuul-jobs change), please?14:36
*** dhajare has quit IRC14:37
AJaegercorvus: and if you could review again https://review.openstack.org/#/c/559946/ (infra-manual), it would be great as well...14:37
corvusAJaeger: since we didn't merge that before release, we may want to send out a deprecation notice and wait a while.  normally i'd say 4 weeks.  maybe that change could be shorter?14:38
*** hemna_ has joined #openstack-infra14:39
*** gagehugo has joined #openstack-infra14:39
AJaegercorvus: I can send that notice out and make it two weeks. Ok?14:39
*** dsariel has quit IRC14:39
*** berndbausch has quit IRC14:39
corvusAJaeger: sounds good; please send it to (at least) zuul-announce14:40
AJaegerok14:41
openstackgerritJeremy Stanley proposed openstack-infra/zuul master: Add instructions for reporting vulnerabilities  https://review.openstack.org/55435214:42
openstackgerritsebastian marcet proposed openstack-infra/puppet-openstackid master: Updated puppet scripts to use webpack instead of bower ( deprecated)  https://review.openstack.org/56002814:46
AJaegercorvus: send out14:47
*** dklyle has joined #openstack-infra14:47
*** caphrim007 has quit IRC14:47
*** david-lyle has quit IRC14:47
*** caphrim007 has joined #openstack-infra14:48
openstackgerritMerged openstack-infra/elastic-recheck master: Amend query for nova functional race bug 1762876  https://review.openstack.org/56043514:48
openstackbug 1762876 in OpenStack Compute (nova) "test_resize_with_reschedule_then_live_migrate intermittently failing; migration is not yet complete" [High,In progress] https://launchpad.net/bugs/1762876 - Assigned to Matt Riedemann (mriedem)14:48
*** efoley has joined #openstack-infra14:49
openstackgerritMerged openstack-infra/infra-manual master: Update ACL sections with release team policy change  https://review.openstack.org/55994614:50
*** felipemonteiro_ has quit IRC14:50
*** felipemonteiro_ has joined #openstack-infra14:50
*** cshastri has joined #openstack-infra14:51
*** rajinir has joined #openstack-infra14:51
*** dhajare has joined #openstack-infra14:52
*** markvoelker has quit IRC14:55
*** markvoelker has joined #openstack-infra14:56
*** ldnunes has quit IRC14:56
*** bobh has joined #openstack-infra14:57
*** dhajare has quit IRC14:59
*** markvoelker has quit IRC15:00
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: Add installation/deprecation documentation  https://review.openstack.org/56046315:04
corvusAJaeger: ^15:05
AJaegerthanks15:06
corvusAJaeger: and approved your msg to announce15:07
*** dsariel has joined #openstack-infra15:07
AJaegerthanks, corvus15:07
*** armaan has quit IRC15:08
*** armaan has joined #openstack-infra15:09
*** Hal has quit IRC15:09
*** jistr|mtg is now known as jistr15:10
*** ihrachys has joined #openstack-infra15:11
*** germs has joined #openstack-infra15:11
*** germs has quit IRC15:11
*** germs has joined #openstack-infra15:11
*** dhajare has joined #openstack-infra15:12
*** germs has quit IRC15:16
*** germs has joined #openstack-infra15:17
*** germs has quit IRC15:17
*** germs has joined #openstack-infra15:17
*** andreas_s has quit IRC15:24
*** andreas_s has joined #openstack-infra15:24
*** zhipeng has quit IRC15:27
AJaegercorvus: commented15:29
*** iyamahat has quit IRC15:29
ssbarneai got a openstack-tox-pypy failing to find pypy interpreter. is this something new?  https://review.openstack.org/#/c/559141/15:33
*** yamahata has quit IRC15:33
*** belliott has joined #openstack-infra15:33
*** lpetrut_ has joined #openstack-infra15:34
*** olaph1 has joined #openstack-infra15:35
*** bobh has quit IRC15:36
*** olaph has quit IRC15:36
*** bobh_ has joined #openstack-infra15:36
*** lpetrut has quit IRC15:36
*** iranzo has quit IRC15:37
*** andreas_s has quit IRC15:38
fungissbarnea: seems like either should drop the openstack-tox-pypy job or add pypy to bindep.txt15:38
*** edmondsw has quit IRC15:39
funginot sure what would have changed recently to trigger that though15:39
clarkbI think there was a change to add the pypy jobs because pypy was passing locally?15:39
clarkbI would just update bindep.txt if that is a desireable thing to test15:39
ssbarneafungi: no problem adding it to bindep.txt but I had the impression that bindep was only informational, not used to install packages.15:40
*** zoli|afk-TPB is now known as zoli15:40
belliottHello, can someone please tell me what the purpose of the “hold” state is in NodePool? I can see that Zuul sets this on a failed build, but I would like to know what differentiaties a “hold” from the node just being “in-use”.15:41
*** kiennt26 has quit IRC15:41
Shrewsbelliott: a held node will not be deleted by nodepool (has to be done manually). an "in-use" node is a node being used by zuul for a job15:42
*** cshastri has quit IRC15:42
Shrewsbelliott: we sometimes hold a node so that we can examine it after the job fails15:42
*** andreas_s has joined #openstack-infra15:43
belliottShrews: gotcha, that makes sense, thanks.15:43
*** r-daneel has quit IRC15:44
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck master: Add query for ReadTimeoutError bug 1763070  https://review.openstack.org/56047415:44
openstackbug 1763070 in OpenStack-Gate "urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool in tempest jobs" [Undecided,New] https://launchpad.net/bugs/176307015:44
clarkbssbarnea: no it is very much used in every "default" job to set up your test environment beacuse nova will need libvirt and python-jenkins will need pypy15:45
*** iyamahat has joined #openstack-infra15:45
clarkbssbarnea: and ratehr than try and install and curate a global list (which we've done in the past to much painful experience) we ask that individual projects manage their own local bindep files which we can install from during the jobs15:46
ssbarneaclarkb: that's good to know, i was wondering where is the code that installs these because last time I checked bindep was unable to install deps, only to check them.15:46
Shrewshttp://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/roles/bindep/tasks/packages.yaml15:46
Shrewsssbarnea: ^^15:46
clarkbssbarnea: we do a sudo apt-get install -y $(bindep -b) or similar15:47
dmsimardclarkb, fungi: The timestamps in the "OpenStack Summit Speaker - ACTION REQUIRED!" email are kind of weird. I think they're on local time instead of PST for Vancouver ? https://imgur.com/a/F5oBR15:47
clarkblooks like its the ansible equivalent now15:47
dmsimard(I know you can't do anything about that, but you probably know who might)15:47
fungibelliott: basically, nodepool is _so_ efficient at deleting nodes as soon as jobs complete, there's no way to race it and check out the state of the node unless we have it refrain from deletion until we're done looking it over15:47
ssbarneaclarkb: thanks.15:47
*** dsariel has quit IRC15:48
fungissbarnea: to clarify, bindep doesn't install dependencies, but we use it to tell our jobs what dependencies they should install15:48
belliottfungi: ok, so basically it’s a means for zuul to release the lock on the node, forget about it, but still retain the node on the nodepool side15:48
*** dhajare has quit IRC15:50
ssbarneaclarkb: is this going to be ok? https://review.openstack.org/#/c/560475/1/bindep.txt15:51
ssbarneai mentioned the "test" profile, as pypy is not needed for runtime.15:51
*** ramishra has quit IRC15:52
clarkbssbarnea: ya that should be fine, you also probably don't need to list pypy twice if it is the same package name on dpkg and rpm systems15:52
clarkbssbarnea: just have a single entry for test taht should apply to rpm and dpkg15:52
ssbarnealike "pypy [platform:rpm platform:dpkg test]" ?15:53
ssbarneaor with commas15:53
clarkbssbarnea: pypy [test]15:54
clarkbthat should match everything when test profile is used15:54
fungiyeah, a package without any platform profiles matches all platforms15:54
fungiwhich we've generally taken to mean "all platforms you plan to support (running tests on in this case)"15:54
*** hamzy has quit IRC15:55
*** gyee has joined #openstack-infra15:56
ssbarneaclarkb: done. i was afraid that it may break my darwin platform but it happens to be the same name, even for brew.15:56
*** andreas_s has quit IRC15:57
*** lucasagomes is now known as lucas-afk15:59
*** patriciadomin has quit IRC16:00
openstackgerritDoug Hellmann proposed openstack-dev/hacking master: uncap eventlet  https://review.openstack.org/56049516:00
*** patriciadomin has joined #openstack-infra16:01
*** pblaho has joined #openstack-infra16:01
*** olaph has joined #openstack-infra16:05
*** yamahata has joined #openstack-infra16:05
*** harlowja has joined #openstack-infra16:05
*** olaph1 has quit IRC16:07
*** bobh_ has quit IRC16:09
fungiugh, i just noticed i haven't been keeping up with moderating the infra ml16:10
fungisince we had to start rejecting owner e-mails i no longer get reminders of posts from non-subscribers16:11
fungiso i'm going through them now and will attempt to discard any from people who eventually subscribed and re-asked things16:12
*** andreas_s has joined #openstack-infra16:12
fungibut there will be at least a few from as far back as january getting delivered here shortly16:12
*** armaan has quit IRC16:12
openstackgerritTobias Henkel proposed openstack-infra/zuul master: WIP: Fix implicit role for repos named ansible  https://review.openstack.org/56055416:12
*** armaan has joined #openstack-infra16:12
*** jmorgan1 has quit IRC16:14
*** neiloy_ has quit IRC16:17
*** florianf has quit IRC16:23
*** janki has quit IRC16:23
*** andreas_s has quit IRC16:25
*** masayukig has joined #openstack-infra16:29
*** sree has quit IRC16:29
*** andreas_s has joined #openstack-infra16:30
*** jpich has quit IRC16:31
*** Hal1 has joined #openstack-infra16:33
clarkbssbarnea: ya its weird how some things end up being universally named and others just don't come close16:34
clarkbssbarnea: just wait for pypy2 and pypy316:34
*** udesale has quit IRC16:35
fungiclarkb: i've tried adding 'RewriteRule ^/api/ - [F,L]' to /etc/apache2/sites-enabled/50-etherpad-dev.openstack.org.conf on etherpad-dev.o.o and that switched queries to /api/(|anything) from 200 ok to 403 forbidden while not appearing to break general pad usage16:35
clarkbfungi: cool16:36
clarkbfungi: do you want to push that change up to puppet-etherpad?16:36
clarkb(since you've tested it)16:36
fungii'm still poking around some of the less used features to make sure they're functioning16:36
*** felipemonteiro_ has quit IRC16:36
clarkbsounds good. Let me know if you want help with the testing. I'm currently following up on the pycodestyle debugging from yesterday since I helped with some of that and now release team is looking at it16:36
*** zoli is now known as zoli|gone16:37
*** zoli|gone is now known as zoli16:37
*** ihrachys_ has joined #openstack-infra16:37
*** r-daneel has joined #openstack-infra16:37
fungiyeah, stuff like the history and exporting content also still seem to work16:37
*** agopi has quit IRC16:37
dmsimardfungi: I suppose we can't hook etherpad-dev to etherpad.o.o's database ? (Somewhat like how we have logs/logs-dev pointing to the same content)16:39
clarkbdmsimard: since logs.o.o is read only that is relatively safe, but since etherpad-lite is read write I don't think we can count on that particularly with differing db migration levels16:39
*** ihrachys has quit IRC16:40
fungidmsimard: no, i think the risk of the two of them fighting over writes could be disasterous16:40
dmsimardah, that makes sense.16:40
dmsimardI was interested in looking at some of the same pads between etherpad and etherpad-dev to see if there was any difference16:40
*** wolverineav has joined #openstack-infra16:40
fungier, disastrous16:40
ssbarneaclarkb: not sure what to do now, make pypy gate an optional check sounds like one approach, other ideas? http://logs.openstack.org/75/560475/2/check/openstack-tox-pypy/669c6fa/job-output.txt.gz16:41
clarkbssbarnea: that particular issue is why most of openstack basically said well fine nevermind pypy :/16:43
clarkbssbarnea: however, if pypy is important to you what I would maybe try is running the job on bionic since that should have newer pypy16:43
openstackgerritMikhail S Medvedev proposed openstack-infra/puppet-openstackci master: Fix puppet syntax for nodepool builder logic  https://review.openstack.org/56056616:48
*** olaph1 has joined #openstack-infra16:49
*** andreas_s has quit IRC16:49
openstackgerritSorin Sbarnea proposed openstack-infra/project-config master: python-jenkins: make pypy gate nonvoting  https://review.openstack.org/56056716:49
*** jamesmcarthur has quit IRC16:50
*** olaph has quit IRC16:51
*** lpetrut has joined #openstack-infra16:51
*** lpetrut_ has quit IRC16:51
ssbarneaclarkb: ^^ this should be safe. i didn't know how to make this one alone to run on bionic.16:53
*** electrofelix has quit IRC16:53
*** andreas_s has joined #openstack-infra16:54
clarkbssbarnea: yup +2'd16:54
clarkbssbarnea: as for making it run on bionic let me find an example16:54
ssbarneai did a grep (ag) but didn't find anything relevant for me.16:54
*** lpetrut has quit IRC16:57
clarkbssbarnea: https://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/projects.yaml#n5344 you do something like that with the nodeset16:58
openstackgerritJeremy Stanley proposed openstack-infra/puppet-etherpad_lite master: Refuse external connections to the API  https://review.openstack.org/56057016:58
clarkbssbarnea: you would remove the template and replace it with explicit check and or gate jobs that override the nodeset as a variant16:58
*** iyamahat has quit IRC16:59
*** tesseract has quit IRC16:59
*** iyamahat has joined #openstack-infra16:59
*** e0ne has quit IRC16:59
clarkbfungi: thanks!16:59
*** sree has joined #openstack-infra17:03
*** andreas_s has quit IRC17:03
*** agopi has joined #openstack-infra17:04
*** dhajare has joined #openstack-infra17:06
*** harlowja has quit IRC17:07
*** sree has quit IRC17:07
*** andreas_s has joined #openstack-infra17:08
openstackgerritSorin Sbarnea proposed openstack-infra/project-config master: python-jenkins: make pypy gate optional check  https://review.openstack.org/56056717:10
*** kjackal has quit IRC17:15
*** jpena is now known as jpena|off17:16
*** hamzy has joined #openstack-infra17:18
*** panda is now known as panda|bbl17:22
clarkbhttp://pyfound.blogspot.com/2018/03/warehouse-all-new-pypi-is-now-in-beta.html I don't expect this will cause us any problems17:22
clarkbhowever I'm not sure how new a distro release we need for tls 1.217:22
*** andreas_s has quit IRC17:22
*** efoley has quit IRC17:22
*** _ari_ has quit IRC17:23
*** hemna_ has quit IRC17:25
*** hemna_ has joined #openstack-infra17:25
*** iranzo has joined #openstack-infra17:27
*** iranzo has quit IRC17:27
*** iranzo has joined #openstack-infra17:27
*** andreas_s has joined #openstack-infra17:27
*** iranzo has quit IRC17:28
*** jamesmcarthur has joined #openstack-infra17:28
*** iranzo has joined #openstack-infra17:28
*** iranzo has quit IRC17:28
*** iranzo has joined #openstack-infra17:28
*** sree has joined #openstack-infra17:28
*** hemna__ has joined #openstack-infra17:29
*** kjackal has joined #openstack-infra17:29
*** yamamoto has quit IRC17:29
*** wolverineav has quit IRC17:29
*** wolverineav has joined #openstack-infra17:30
*** _ari_ has joined #openstack-infra17:32
*** jamesmcarthur has quit IRC17:33
*** sree has quit IRC17:33
*** wolverineav has quit IRC17:34
fungilooks like centos 7 and ubuntu trusty should be fine17:35
fungitry `openssl ciphers -v TLSv1.2` on systems you're concerned about17:36
*** agopi has quit IRC17:38
*** hashar has joined #openstack-infra17:38
clarkbfungi: thanks17:38
fungipython builds against libssl, so in theory it should match whatever openssl on that platform supports17:38
clarkbyup17:39
*** ralonsoh has quit IRC17:39
*** lpetrut has joined #openstack-infra17:40
*** olaph1 has quit IRC17:40
*** trown is now known as trown|lunch17:40
*** andreas_s has quit IRC17:42
*** dtruong_ has joined #openstack-infra17:45
*** olaph has joined #openstack-infra17:45
*** dtruong_ has quit IRC17:45
*** dtruong_ has joined #openstack-infra17:46
*** dtruong_ has quit IRC17:46
*** dtruong_ has joined #openstack-infra17:47
*** dtruong_ has quit IRC17:47
*** ihrachys__ has joined #openstack-infra17:47
*** dtruong_ has joined #openstack-infra17:48
*** dtruong has quit IRC17:48
*** yamamoto has joined #openstack-infra17:49
*** e0ne has joined #openstack-infra17:50
*** ihrachys_ has quit IRC17:50
*** yamamoto has quit IRC17:50
clarkbdmsimard: following up on that email the html source doesn't do any conversions its actually hardcoded to that in the email. I also see the asme shifted times17:50
clarkbdmsimard: so I don't think it is relative to the recipient's timezone but possibly some server running the scripts? I will ask17:50
dmsimardyeah I have no idea, just thought I would mention it17:50
*** yamamoto has joined #openstack-infra17:50
ssbarneaclarkb: please revote/approve https://review.openstack.org/#/c/560567/ so i can unlock jjb queue17:50
*** arxcruz|ruck is now known as arxcruz17:50
*** harlowja has joined #openstack-infra17:52
clarkbssbarnea: done17:53
clarkbdmsimard: I've pinged about it17:53
dmsimard++17:53
*** armaan has quit IRC17:54
*** armaan has joined #openstack-infra17:54
*** andreas_s has joined #openstack-infra17:55
*** sambetts is now known as sambetts|afk17:55
*** yamamoto has quit IRC17:57
openstackgerritMerged openstack-infra/project-config master: python-jenkins: make pypy gate optional check  https://review.openstack.org/56056717:58
*** wolverineav has joined #openstack-infra17:58
*** edmondsw has joined #openstack-infra17:59
*** dklyle has quit IRC18:06
*** andreas_s has quit IRC18:10
*** cburgess_ is now known as cburgess18:13
*** ihrachys__ has quit IRC18:14
*** andreas_s has joined #openstack-infra18:14
*** iyamahat has quit IRC18:14
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul master: Reorganize "Zuul From Scratch" document  https://review.openstack.org/55698818:14
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul master: Add Gerrit docs to Zuul From Scratch  https://review.openstack.org/55860018:14
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul master: Add static driver doc to Zuul From Scratch  https://review.openstack.org/55880218:14
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul master: Add sample systemd service files.  https://review.openstack.org/55883018:14
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul master: Move zuul jobs config to zuul config section  https://review.openstack.org/55886718:14
*** sree has joined #openstack-infra18:14
*** yamamoto has joined #openstack-infra18:16
*** ihrachys has joined #openstack-infra18:16
*** iyamahat has joined #openstack-infra18:16
*** yamamoto has quit IRC18:18
*** felipemonteiro_ has joined #openstack-infra18:18
*** sree has quit IRC18:19
*** Goneri has quit IRC18:22
*** iranzo has quit IRC18:23
*** yamamoto has joined #openstack-infra18:25
*** andreas_s has quit IRC18:28
*** claudiub has quit IRC18:31
*** armaan has quit IRC18:31
*** armaan has joined #openstack-infra18:31
*** trown|lunch is now known as trown18:32
*** jamesmcarthur has joined #openstack-infra18:33
*** yamamoto has quit IRC18:34
*** kjackal has quit IRC18:35
*** rosmaita has quit IRC18:35
*** yamamoto has joined #openstack-infra18:38
*** armaan has quit IRC18:38
*** armaan has joined #openstack-infra18:39
*** yamamoto has quit IRC18:40
clarkbany other infra-root willing to review https://review.openstack.org/#/c/560570/ I brought it up with fungi when thinking about the risk we expose for etherpad service18:41
*** agopi has joined #openstack-infra18:41
*** tjgresha has joined #openstack-infra18:42
dmsimardclarkb: makes sense to me, do we have it on etherpad-dev.o.o to test or something ?18:42
clarkbdmsimard: ya fungi tested it on -dev according to irc scrollback18:43
dmsimardoh, slightly before I jumped in, I see it now.18:43
fungialso mentioned that in a review comment18:44
mtreinishI didn't even realize there was an etherpad api18:47
clarkbmtreinish: ya its entirely admin only so not useful to users18:49
clarkbsince there is a single key that gives you everything to use it18:49
fungithe only unauthenticated function i could find was the one to return the api version string18:53
*** dhajare has quit IRC18:54
*** dklyle has joined #openstack-infra18:55
*** pcaruana has quit IRC18:55
*** priteau has quit IRC19:00
*** jamesmcarthur has quit IRC19:01
openstackgerritMerged openstack-infra/puppet-etherpad_lite master: Refuse external connections to the API  https://review.openstack.org/56057019:04
*** salv-orlando has quit IRC19:07
*** salv-orlando has joined #openstack-infra19:07
*** salv-orlando has quit IRC19:11
openstackgerritMerged openstack-infra/zuul master: Switch implicit role not found message to debug  https://review.openstack.org/55686619:14
*** rpioso|afk is now known as rpioso19:17
*** panda|bbl is now known as panda|off19:19
*** pcichy has quit IRC19:21
*** Goneri has joined #openstack-infra19:24
*** krenczew1 has joined #openstack-infra19:25
*** wolverineav has quit IRC19:25
*** krenczewski has quit IRC19:28
*** jamesmcarthur has joined #openstack-infra19:29
*** jamesmcarthur has quit IRC19:31
*** jamesmcarthur has joined #openstack-infra19:31
*** wolverineav has joined #openstack-infra19:31
*** iyamahat has quit IRC19:35
*** olaph1 has joined #openstack-infra19:40
*** olaph has quit IRC19:40
dirkinfra-root: is it possible that the opensuse / tumbleweed mirroring job is hanging or not finishing successfully again? it seems there is some inconsistent data on the mirrors19:44
*** eernst has quit IRC19:44
*** eernst has joined #openstack-infra19:44
*** iyamahat has joined #openstack-infra19:44
*** gfidente has quit IRC19:45
*** edmondsw has quit IRC19:47
*** eharney has quit IRC19:49
*** kjackal has joined #openstack-infra19:50
*** dbecker has quit IRC19:54
*** eernst has quit IRC19:54
*** salv-orlando has joined #openstack-infra19:56
*** bobh has joined #openstack-infra19:58
openstackgerritJesse Keating proposed openstack-infra/project-config master: Add a zuul test to run on github3.py changes  https://review.openstack.org/55832319:59
*** eernst has joined #openstack-infra19:59
*** claudiub has joined #openstack-infra20:02
openstackgerritMerged openstack-infra/elastic-recheck master: Add query for ReadTimeoutError bug 1763070  https://review.openstack.org/56047420:06
openstackbug 1763070 in OpenStack-Gate "urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool in tempest jobs" [Undecided,Confirmed] https://launchpad.net/bugs/176307020:06
clarkbdirk: I can take a look20:08
clarkbdirk: it successfully updated at 2018-04-11T18:48:59,898002903+00:0020:12
fungi#status log added review-dev01.openstack.org to emergency disable list in preparation for manually experimenting with some configuration changes in an attempt to further diagnose the its-storyboard plugin20:12
openstackstatusfungi: finished logging20:12
clarkbdirk: updated several hudnred packages20:12
clarkbdirk: possible the upstream was out of sync and we synced that?20:12
fungiclarkb: i think we discovered that for some reason the rsync wrapper (or rsync itself?) is not exiting nonzero when it gets killed by the timeout, and so we end up running vos release on an incomplete sync20:14
* fungi thinks he may have written a fix for this but now can't remember20:14
clarkbfungi: this appears to be a complete sync20:14
clarkbjust based on the output from rsync itself20:14
fungiahh20:14
clarkbsent 6,419 bytes  received 4,118,372,644 bytes  12,498,874.24 bytes/sec20:15
openstackgerritMerged openstack-infra/zuul master: Also prefix the alembic_version table  https://review.openstack.org/56026320:15
clarkbit did the whole summary in the log if you want to see it20:15
funginah, that's alright20:17
dirkclarkb: thanks20:17
clarkbdirk: my best guess at this point is syncing some bad state20:18
clarkbI think we'll try again in a couple hours automatically or we can trigger a sync manually if it is really bad20:18
fungiis it still broken or was it maybe only broken prior to the most recent sync and vos release?20:19
*** markvoelker has joined #openstack-infra20:20
clarkbthats a good question20:20
clarkbdirk: ^20:20
*** berendt_ has joined #openstack-infra20:21
*** geekinutah_ has joined #openstack-infra20:21
*** amito_ has joined #openstack-infra20:21
*** NobodyCam_ has joined #openstack-infra20:21
*** ameade_ has joined #openstack-infra20:21
*** sweston_ has joined #openstack-infra20:21
*** dirk__ has joined #openstack-infra20:21
*** icey_ has joined #openstack-infra20:21
*** evgenyl_ has joined #openstack-infra20:21
*** v1k0d3n_ has joined #openstack-infra20:21
*** kmalloc_ has joined #openstack-infra20:21
*** electrical__ has joined #openstack-infra20:21
*** patricku_ has joined #openstack-infra20:21
openstackgerritMerged openstack-infra/zuul master: Add postgresql release note  https://review.openstack.org/56007920:22
fungishould be serving a timestamp file from the root of that mirror subtree which indicates when the most recent vos release occurred20:23
*** dirk has quit IRC20:24
*** dirk__ is now known as dirk20:24
clarkbI'll double check that matches my timestamp above20:24
dirktsts, someone stole my nick20:24
*** dtruong_ has quit IRC20:24
clarkbhttp://mirror.dfw.rax.openstack.org/opensuse/timestamp.txt20:25
dirkclarkb: I'm just rerunning a DNM review, and this one gets further. so whatever the issue was it seems fixed20:25
*** dtruong has joined #openstack-infra20:25
*** kmalloc_ has quit IRC20:25
clarkbthe timestamp is written ebfore the sync which explains the 5 minute difference but it matches up otherwise20:25
*** kmalloc_ has joined #openstack-infra20:25
dirkare we stopping the mirroring when nodepool images are not rebuilding?20:25
*** rosmaita has joined #openstack-infra20:25
dirkthe nodepool image and the repo might not be matching at some points20:25
clarkbno the mirror continuously updates unless something breaks20:25
*** kmalloc_ is now known as notmorgan20:26
clarkbdirk: we can probably build the image off of our mirror to address that20:26
clarkb(I think we do this with ubuntu now)20:26
*** notmorgan has quit IRC20:26
*** notmorgan has joined #openstack-infra20:26
*** notmorgan has quit IRC20:26
*** notmorgan has joined #openstack-infra20:26
dirkyeah, I think that makes sense. we don't want the image to be newer than the mirror20:26
dirkthat is bound to cause fallout20:26
*** kmalloc has quit IRC20:27
dirkbecause installing packages will ask about "do you really wanna downgrade package foo?"20:27
*** andreaf_ has joined #openstack-infra20:27
*** tobberydberg_ has joined #openstack-infra20:27
*** xhku_ has joined #openstack-infra20:27
*** notmorgan is now known as kmalloc20:27
dirkwhere's the plumbing that needs to be adjusted for that?20:27
clarkbdirk: let me find the example for how we do it with ubuntu20:28
*** johnthetubaguy_ has joined #openstack-infra20:28
*** aspiers[m] has quit IRC20:28
*** mordred has quit IRC20:28
*** vkmc has quit IRC20:28
*** icey has quit IRC20:28
*** v1k0d3n has quit IRC20:28
*** andreaf has quit IRC20:28
*** NobodyCam has quit IRC20:28
*** fbouliane has quit IRC20:28
*** evgenyl has quit IRC20:28
*** patricku has quit IRC20:28
*** fkautz has quit IRC20:28
*** electrical_ has quit IRC20:28
*** geekinutah has quit IRC20:28
*** johnsom has quit IRC20:28
*** ameade has quit IRC20:28
*** berendt has quit IRC20:28
*** amito has quit IRC20:28
*** sweston has quit IRC20:28
*** dkliban has quit IRC20:28
*** tobberydberg has quit IRC20:28
*** johnthetubaguy has quit IRC20:28
*** geekinutah_ is now known as geekinutah20:28
*** v1k0d3n_ is now known as v1k0d3n20:28
*** amito_ is now known as amito20:28
*** icey_ is now known as icey20:28
*** evgenyl_ is now known as evgenyl20:28
*** sweston_ is now known as sweston20:28
*** berendt_ is now known as berendt20:28
*** NobodyCam_ is now known as NobodyCam20:28
*** electrical__ is now known as electrical_20:28
*** ameade_ is now known as ameade20:28
*** patricku_ is now known as patricku20:28
clarkbdirk: https://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/nodepool.yaml#n321 and then the dib element for that installation has to support configuring the source to that mirror20:29
*** johnsom has joined #openstack-infra20:29
*** andreaf_ is now known as andreaf20:29
dirkwhy is it hardocding rax?20:29
clarkbdirk: because our image builders are in that datacenter20:29
dirkare all nb builders running there?20:29
dirkok20:30
dirkso thats easy then20:30
clarkbdirk: looks like zypper-minimal supports that env var too20:30
*** dkliban has joined #openstack-infra20:30
*** mordred has joined #openstack-infra20:30
clarkbthe exception to that is the nb03 config in that same dir but it should have different values set20:31
*** vkmc has joined #openstack-infra20:31
*** bandini has quit IRC20:31
*** aspiers[m] has joined #openstack-infra20:33
*** bandini has joined #openstack-infra20:33
ianwso i'm hitting a puppet issue i'm not sure how to get around.  i've forked puppet-solr and it seems to have exposed a dependency issue between system-config and puppet-python20:51
ianwDuplicate declaration: Package[virtualenv] is already declared in file /etc/puppet/modules/openstack_project/manifests/server.pp:293; cannot redeclare at /etc/puppet/modules/python/manifests/install.pp:5720:51
*** yamamoto has joined #openstack-infra20:52
ianwhttps://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/server.pp#n28920:52
ianwas i've found out, in puppet3 you can not have two packages with different providers (openstack_pip and system packages)20:52
clarkbianw: ya this is one of the biggest gripes with puppets uniqueness requirement, it doesn't allow duplicate package defs even though they would be harmless20:52
clarkbianw: we may want to remove it from server.pp if nothing is using it20:52
*** HeOS has quit IRC20:53
clarkband then use puppet-python to manage it where we want it20:53
*** HeOS has joined #openstack-infra20:53
ianwclarkb: ok cool, that was actually the change i had queued so glad we had the same idea :)20:54
ianwthe slight difference is that in server.pp we install virtualenv via pip at a given version20:54
*** jtomasek has quit IRC20:54
*** edmondsw has joined #openstack-infra20:54
*** sree has joined #openstack-infra20:55
ianwbut it's not a safe operation anyway -- we haven't ensure python-virtualenv package is installed first, so if some other module comes along and pulls that in after the pip install, it just overwrites it with crap anyway20:55
clarkbI'm trying to think where we'd use virtualenv and need it at a specific version20:55
clarkband can't come up with any answers quickly20:55
ianwyeah, not on control plane20:56
fungiprobably a holdover from the bad old days of puppeting test nodes20:56
fungior later relying on puppet in our image builds20:56
ianwof course, it's still kind of wrong to use "class { '::python' }" in openstack-health/askbot/etc as it makes them all uninstallable together20:56
*** yamamoto has quit IRC20:56
ianwas i've found out, pyhton should be a virtual package for that to work20:57
ianwbut i'm not going to rewrite the world20:57
clarkbif you use include python it would work20:57
clarkbsince you can have duplicate includes20:57
ianwunfortunately it has some rather silly defaults20:58
ianwlike installing pip, but *not* installing -dev packages20:58
ianwyou can pip install things, but only things that don't need to build20:58
ianw(sorry, not even just not installing, but actively marking "absent")20:58
clarkbwow20:59
clarkbso you can't manage it easily from another module20:59
*** edmondsw has quit IRC20:59
*** sree has quit IRC21:00
ianwnot really; i've been trying and trying various things :)21:00
clarkbthat is annoying21:00
*** trown is now known as trown|outtypewww21:01
ianwaskbot is maybe the only module that uses python::virtualenv we currently have21:01
ianwbut i figured we might want this a bit more in the future21:01
clarkbI think openstack health did at one time but migrated way from it to be more like the others21:01
clarkbianw: ya I think that is likely given the pip direction21:01
ianwmy other thought, and i don't know how to do it because it's pretty ruby heavy, was to fiddle openstack_pip so it can take a virtualenv argument21:02
ianwhttps://git.openstack.org/cgit/openstack-infra/puppet-pip/tree/lib/puppet/provider/package/openstack_pip.rb21:02
ianwsomewhere in there?  all it really needs to do is use "/path/to/virtualenv/pip install"; then we could largely get rid of puppet-python21:03
ianwit's trivial to make a virtualenv in an exec ... it's just all the packages it sucks to install in execs; might as well write a shell script at that point21:03
openstackgerritMerged openstack-infra/project-config master: Remove direct tag / branch ACLs for official projects  https://review.openstack.org/55773021:05
*** kjackal has quit IRC21:05
clarkbianw: ya iirc openstack_pip does a which lookup you'd replace that with a heres your argument for where to find pip21:05
clarkbif it is set21:05
clarkber the puppet pip resource does that21:06
clarkbso we'd ahve to find a way ot override it in that child class21:06
*** sfeole has joined #openstack-infra21:06
ianwclarkb: i guess that's https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/package/pip.rb ?21:07
clarkbianw: https://github.com/puppetlabs/puppet/blob/3.8.7/lib/puppet/provider/package/pip.rb#L2921:07
clarkbya21:07
*** slaweq has quit IRC21:08
clarkbianw: also https://github.com/puppetlabs/puppet/blob/3.8.7/lib/puppet/provider/package/pip.rb#L12121:08
*** slaweq has joined #openstack-infra21:09
clarkbianw: looks like proper which works against rooted paths21:09
clarkbso if we overrode def self.cmd we might be able to hack thati n21:10
ianwhmm, i wonder if you just set self.pip_cmd?21:10
clarkbya21:10
rosmaitasorry to interrupt, but can someone tell me why build-openstack-releasenotes is not running on https://review.openstack.org/#/c/560439/ ? it's configured in https://github.com/openstack-infra/project-config/blob/b8f5aecf0c3d66e2a95a246e690727698af548b6/zuul.d/projects.yaml#L666721:10
clarkbrosmaita: without looking at the code my guess is because it doesn't match the file matchers21:11
rosmaitathat would be bad, because the only thing on the patch is a release note21:11
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck master: Add query for nova functional race bug 1763181  https://review.openstack.org/56064021:12
openstackbug 1763181 in OpenStack Compute (nova) "test_parallel_evacuate_with_server_group intermittently fails" [Medium,Confirmed] https://launchpad.net/bugs/176318121:12
*** slaweq has quit IRC21:13
*** rfolco|rover is now known as rfolco|off21:13
clarkbrosmaita: confirmed the problem is you have prefexed your releasenotes with etc/ and the file matcher only looks for ^releasenotes not ^etc/releasenotes21:13
rosmaitaclarkb: good eye! i completely missed the /etc21:14
rosmaitathank you21:14
*** r-daneel has quit IRC21:15
*** r-daneel has joined #openstack-infra21:15
openstackgerritKendall Nelson proposed openstack-infra/project-config master: Adds OpenStackClient to StoryBoard  https://review.openstack.org/56065321:15
corvusi'd like to restart zuul now.  we're mildly backlogged, but i'd like to get it done today.21:16
clarkbtehre were a few important bug fixes so seems reasonable to me21:17
corvusi want to try to improve the playbook i've been working on to stop all of zuul.  waiting for the pidfile (which we used to do) doesn't work anymore; i guess we clean it up early now?21:18
clarkbcorvus: to signal when it is safe to start services again?21:18
corvusclarkb: yep21:19
clarkbcorvus: I think there was a bug in the init scripts that prevented the rm that was in htem from removing the pidfile on stop. Which may have gotten fixed21:19
clarkb(it was rm'ing the wrong path)21:19
clarkband if that happens before its probably stopped that could explain it21:19
corvusshould they be removing the pidfile at all?21:19
clarkbcorvus: I think it does because kill -9 doesn't allow zuul to remove it itself21:19
corvusthere was some confusion because zuul-executor couldn't remove its own pidfile after dropping privileges21:20
clarkbif we can get all the zuul services to stop on a more graceful signal/command then we could remove that from the init script21:20
corvusclarkb: nothing should execute kill -921:20
corvusclarkb: they should all run "zuul-$service stop"21:20
clarkbcorvus: I think historically it did at least, may not be the case now21:20
clarkbso probably just a missing cleanup from that work21:20
corvusto be clear, i believe they all do run that now21:20
corvuswe may still have that issue with zuul-fingergw though21:20
eanderssonDoes anyone happen to have any good example configurations for haproxy and large scale deployments?21:23
openstackgerritKendall Nelson proposed openstack-infra/project-config master: Add OpenStack SDK to StoryBoard  https://review.openstack.org/56065821:23
clarkbeandersson: are you asking specifically for openstack? You might have better luck asking the openstack ops folks (probably on their mailing list?)21:24
clarkbeandersson: we run one haproxy to front git servers which is likely very different than fronting openstack services21:24
eanderssonI see21:24
eanderssonWhat is the ops irc channel in general?21:25
* clarkb looks21:25
eanderssonI assumed that was infra :D21:25
corvusclarkb: okay, as an experiment, i will comment out the pidfile removals from the init scripts on zuul01.  that will answer the question for 3 services at least21:25
clarkbcorvus: seems reasonable21:25
clarkbeandersson: #openstack-operators though logs make it looks quiet. mailing list tends to get traction if that channel doesn't21:26
openstackgerritMerged openstack-infra/zuul master: Update pypi metadata  https://review.openstack.org/56009821:26
clarkbeandersson: we run the openstack developer infrastructure but currently do not run any openstacks21:26
clarkbother than the ones that are deployed for testing21:26
corvuswe run zero production openstacks :)21:27
corvusokay, restarting zuul now21:30
ianwour main role is to find interesting ways to break production openstacks :)21:31
ianwclarkb: i think that overriding self.pip_cmd and looking for @resource[:virtualenv] in openstack-pip might actually work ... i just have to figure out how to test it now21:32
corvushrm, zuul-web may not shutdown cleanly (or quickly) if we stop zuul-scheduler first.  probably due to waiting on gearman.  this may be related to the async thing clarkb, Shrews.21:32
*** kgiusti has left #openstack-infra21:32
clarkbcorvus: that makes esnse if it is blocking on data there21:33
clarkb(and it can never get it)21:33
corvusthere should be exceptions, but i don't see anything logged.21:33
corvusit's probably worth some manual experimenting.21:33
corvusmy pidfile experiments were inconclusive.21:34
*** hashar has quit IRC21:36
corvusi've restarted the scheduler.  executors are still stopping.21:37
corvus(but we should be picking up gerrit events now)21:37
openstackgerritDirk Mueller proposed openstack-infra/project-config master: build opensuse images from the openstack mirror  https://review.openstack.org/56066221:37
clarkbrosmaita: swinging back around on your release notes thing is etc/releasenotes a common practice? if so we should probably support it in the job however I'm not sure what reno documents and supports (is it arbitrary or is it opinionated)21:38
corvusso it looks like when we need to do a full restart, we can stop everything, and once the services receive the stop signal, it's okay to restart the scheduler.  that can reduce the perceived downtime as far as missing gerrit events.21:38
rosmaitaclarkb: no it's not, someone made a mistake21:38
*** salv-orlando has quit IRC21:38
clarkbcorvus: thats nice21:38
*** salv-orlando has joined #openstack-infra21:38
clarkbssbarnea: heads up I rechecked https://review.openstack.org/#/c/560475/2 for you since the pypy on bionic chagne went in21:41
*** salv-orlando has quit IRC21:43
corvusokay that's really weird, something just told zuul to stop21:45
clarkbwasn't me21:45
corvus2018-04-11 21:42:57,176 DEBUG zuul.CommandSocket: Accepted socket connection <socket.socket fd=18, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, laddr=/var/lib/zuul/scheduler.socket>21:45
corvus2018-04-11 21:42:57,176 DEBUG zuul.CommandSocket: Received b'_stop' from socket21:45
*** esberglu has quit IRC21:47
corvusso much for reduced downtime :(21:47
*** rcernin has joined #openstack-infra21:49
clarkblet me know if I can help (but otherwise staying out of the way to avoid confusion)21:50
corvusclarkb: i've been unable to find out what caused that.  i'm pretty sure i didn't (none of my terminals suggest i accidentally re-ran the stop script or anything).  and i don't see anything in syslog.21:51
corvusnothing suggestive of an ansible command in auth.log21:52
clarkbmight've been systemd rerunning the init script for some reason?21:52
clarkbthough that seems more likely if we were using units with deps strongly defined and we don't do that21:53
corvusclarkb: i don't see anything additional in journalctl output... but maybe there's some other way to look i don't know about.21:53
clarkbsystemctl unit-status or something like that21:54
clarkbshould tell you recent actions /me looks21:54
clarkbsystemctl status21:55
*** lpetrut has quit IRC21:56
clarkbcorvus: thinking about loud here is it possible it ran twice because it hit zuul01.o.o and zuul.o.o as separate hosts but in reality they are the same? auth.log implies this isn't the case unless maybe control persist was used?21:57
corvusclarkb: my stop_all_zuul playbook was long finished by that point21:58
wolverineavhi! i’ve got a zuul gate failure for the newton branch in networking-bigswitch https://review.openstack.org/#/c/556657/7 - i believe there was a switch from testr to stestr. and mordred helped us fix that in master and other stable branches (https://review.openstack.org/#/c/550280/3).21:59
wolverineavhowever porting that to newton doesn’t fix. i see that in py27 tests it is actually using the latest neutron tag instead of the one corresponding to newton-eol and that causes the py27 to fail. has anyone seen something similar?22:00
clarkbwolverineav: chances are this is happening due to the tox-siblings feature in our tox jobs. Where if you ahve listed a required-project in the zuul job config it will run with the source of that project installed for you. Problem here is neutron won't have a newton branch so it defaults/fallsback on master22:01
clarkbwolverineav: you can address this by setting an override-ref for that job on that branch to newton-eol or whatever refs is appropriate22:01
*** felipemonteiro_ has quit IRC22:02
clarkbcorvus: Warning: zuul-scheduler.service changed on disk. Run 'systemctl daemon-reload' to reload units. related to inconclusive pid file checking if you didn't daemon-reload then used service/systemctl to stop start then that may explain it22:02
clarkbthe inconclusive testing, not the extra stop22:03
corvusclarkb: agree22:03
clarkbthat always trips me up too22:04
corvusoh i think i need to restart zuul-web22:05
corvusdone22:06
clarkbcorvus: reading journalctl -u zuul-scheduler it seems it noticed it started multiple times then stopped?22:06
corvusclarkb: that was me trying to start it22:06
clarkbah22:06
corvusclarkb: that actually suggests that whatever stopped it didn't go through systemd22:06
clarkband then the stop happens 24 seconds later which is odd22:06
corvusthat was also me22:06
corvusthat was me saying "start it" and systemd saying "ok" but doing nothing because it thought it was still running.22:07
clarkboh right thats all minutes after the log you have above22:07
clarkbyes that tells me systemd was likely not at fault22:07
corvusit typically takes me like 5 times of doing that before i realize that when systemd says "ok" it means "nope"22:07
*** hamzy has quit IRC22:07
wolverineavclarkb: setting override-ref would be in project-config repo, right?22:08
clarkbwolverineav: wherever that job is currently specified to run in check/gate for you22:08
clarkbwolverineav: you'll create a 'variant' of the job specific to your project htat does that22:08
clarkb(I think)22:09
wolverineavclarkb: ah, okay. yes, its project-config then. thanks for the pointer. let me take a look there and update it.22:09
wolverineavclarkb: appreciate the quick help!22:09
clarkbwolverineav: no problem feel free to ask more questions as you dig in22:10
*** VW_ has joined #openstack-infra22:10
*** belliott has quit IRC22:13
*** VW has quit IRC22:13
*** VW_ has quit IRC22:14
tonybwas zuul restarted in the mast 45mins? I have a chnage that I think shoudl have moved into the gate pipeline but it hasn't and sometimes that's a side effect of the restart ... but I don't see anything on the Infrastructure_Status page22:16
*** pcrews has quit IRC22:16
corvuswolverineav, clarkb: use "https://zuul-ci.org/docs/zuul/user/config.html#attr-job.override-checkout"  (override-ref is deprecated)22:17
corvustonyb: yep, i was bad and did not send out a notice22:17
corvusi also haven't re-enqueued yet22:17
wolverineavah, thanks corvus22:17
corvustonyb: re-enqueuing now; i'll send a notice when it's done22:18
*** pcrews has joined #openstack-infra22:18
tonybcorvus: Thanks22:19
corvusstatus notice zuul was restarted to updated to the latest code; you may need to recheck changes uploaded or approvals added between 21:30 and 21:4522:19
corvusthat's what i'll send22:19
tonybcorvus: cool once I see that I'll check the changes I care about have done the right thing22:20
corvusoh, and now that i'm doing that, i see where the load spike came from22:20
*** e0ne has quit IRC22:20
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Updated Summit TimeZone  https://review.openstack.org/56067222:21
corvusthere's a bunch of cinder backports all at once22:21
*** tpsilva has quit IRC22:24
*** andreas_s has joined #openstack-infra22:26
*** jcoufal has quit IRC22:27
openstackgerritMerged openstack-infra/openstackid-resources master: Updated Summit TimeZone  https://review.openstack.org/56067222:28
corvus#status notice zuul was restarted to updated to the latest code; you may need to recheck changes uploaded or approvals added between 21:30 and 21:4522:28
openstackstatuscorvus: sending notice22:28
-openstackstatus- NOTICE: zuul was restarted to updated to the latest code; you may need to recheck changes uploaded or approvals added between 21:30 and 21:4522:30
openstackgerritJohn L. Villalovos proposed openstack-infra/project-config master: gerritbot: Announce stable branch changes of openstack/requirements  https://review.openstack.org/56067722:31
*** andreas_s has quit IRC22:31
jlvillalgerritbot review request: https://review.openstack.org/#/c/545469/  Some cleanup/refactoring and adding unit tests. Has one +2 Thank you.22:31
*** jcoufal has joined #openstack-infra22:31
openstackstatuscorvus: finished sending notice22:31
*** jcoufal has quit IRC22:33
*** jamesmcarthur has quit IRC22:35
*** dsariel has joined #openstack-infra22:36
*** mriedem is now known as mriedem_afk22:36
*** salv-orlando has joined #openstack-infra22:39
*** derekh has quit IRC22:40
*** bobh has quit IRC22:40
openstackgerritMerged openstack-infra/zuul master: Add test for builds/ web route  https://review.openstack.org/56011722:41
*** bobh has joined #openstack-infra22:42
*** felipemonteiro_ has joined #openstack-infra22:42
*** edmondsw has joined #openstack-infra22:43
*** d0ugal has quit IRC22:43
*** salv-orlando has quit IRC22:43
*** Goneri has quit IRC22:44
*** yamamoto has joined #openstack-infra22:46
*** d0ugal has joined #openstack-infra22:47
*** edmondsw has quit IRC22:48
*** yamamoto has quit IRC22:50
*** yamamoto has joined #openstack-infra22:51
*** hongbin has quit IRC22:53
*** yamamoto has quit IRC22:56
*** yamamoto has joined #openstack-infra22:57
*** yamamoto has quit IRC23:01
*** yamamoto has joined #openstack-infra23:03
*** jamesmcarthur has joined #openstack-infra23:04
*** jamesmcarthur has quit IRC23:09
*** slaweq has joined #openstack-infra23:09
*** dsariel has quit IRC23:09
*** sree has joined #openstack-infra23:11
*** olaph has joined #openstack-infra23:14
*** slaweq has quit IRC23:14
*** olaph1 has quit IRC23:14
*** sree has quit IRC23:16
*** claudiub has quit IRC23:35
*** salv-orlando has joined #openstack-infra23:40
*** felipemonteiro_ has quit IRC23:42
pabelangercorvus: no, I have abandoned 54986723:43
*** tjgresha has quit IRC23:44
*** salv-orlando has quit IRC23:45
*** yamamoto has quit IRC23:47
*** owalsh is now known as owalsh_afk23:47
*** rpioso is now known as rpioso|afk23:54
*** eernst has quit IRC23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!