Tuesday, 2018-04-24

*** gyee has quit IRC00:02
*** hemna_ has quit IRC00:02
*** pcaruana has quit IRC00:02
*** dingyichen has joined #openstack-infra00:10
*** smarcet has joined #openstack-infra00:12
*** jcoufal_ has quit IRC00:23
*** dhill_ has joined #openstack-infra00:27
*** tonyb has quit IRC00:33
*** tonyb has joined #openstack-infra00:35
*** mriedem is now known as mriedem_away00:37
*** kjackal has quit IRC00:41
*** namnh has joined #openstack-infra00:42
*** tjgresha has quit IRC00:42
*** kjackal has joined #openstack-infra00:42
*** spzala has joined #openstack-infra00:49
*** caphrim007 has joined #openstack-infra00:50
*** rlandy has quit IRC00:52
*** caphrim007 has quit IRC00:53
*** caphrim007 has joined #openstack-infra00:53
*** caphrim007 has quit IRC00:53
*** caphrim007 has joined #openstack-infra00:55
*** Douhet has joined #openstack-infra00:55
*** caphrim007 has quit IRC00:56
*** mriedem_away has quit IRC01:05
*** caphrim007 has joined #openstack-infra01:06
*** jamesmcarthur has joined #openstack-infra01:11
*** spzala has quit IRC01:13
*** zeus has quit IRC01:15
*** MasterofJOKers has quit IRC01:16
*** MasterofJOKers has joined #openstack-infra01:16
*** zeus has joined #openstack-infra01:18
*** zeus is now known as Guest6275101:18
*** smarcet has quit IRC01:23
*** yamahata has quit IRC01:26
*** salv-orl_ has joined #openstack-infra01:28
*** salv-orlando has quit IRC01:31
*** hongbin_ has joined #openstack-infra01:32
*** jamesmcarthur has quit IRC01:40
*** shardy has quit IRC01:42
*** shardy has joined #openstack-infra01:43
*** ramishra has joined #openstack-infra01:52
*** spzala has joined #openstack-infra01:52
*** esberglu has joined #openstack-infra01:53
*** spzala has quit IRC01:57
*** jamesmcarthur has joined #openstack-infra01:59
*** zhipeng has joined #openstack-infra02:03
*** jamesmcarthur has quit IRC02:04
*** jamesmcarthur has joined #openstack-infra02:05
*** smarcet has joined #openstack-infra02:10
*** zhipeng has quit IRC02:17
*** jamesmcarthur has quit IRC02:29
*** spzala has joined #openstack-infra02:34
*** annp has quit IRC02:42
*** gongysh has joined #openstack-infra02:45
*** psachin has joined #openstack-infra02:46
clarkbcorvus: did you see 2001 a space odyssey is getting a 70mm rerelease?02:47
*** spzala has quit IRC02:48
*** yamamoto has quit IRC02:53
*** annp has joined #openstack-infra02:54
*** yamamoto has joined #openstack-infra02:54
*** yamamoto has quit IRC02:59
*** esberglu has quit IRC03:01
*** jamesmcarthur has joined #openstack-infra03:06
*** jamesmcarthur has quit IRC03:10
*** nicolasbock has quit IRC03:26
*** zhangfei has joined #openstack-infra03:28
*** hwoarang has quit IRC03:37
*** eernst has joined #openstack-infra03:38
*** lpetrut has joined #openstack-infra03:39
*** hwoarang has joined #openstack-infra03:43
*** janki has joined #openstack-infra03:44
*** shu-mutow-AWAY is now known as shu-mutow03:57
*** hongbin_ has quit IRC03:58
*** caphrim007_ has joined #openstack-infra03:59
*** smarcet has quit IRC04:00
*** ianychoi has quit IRC04:02
*** caphrim007 has quit IRC04:03
*** ianychoi has joined #openstack-infra04:04
*** lpetrut has quit IRC04:11
openstackgerritMerged openstack-infra/irc-meetings master: Change Dragonflow meetings to be bi-weekly  https://review.openstack.org/56359604:12
*** evrardjp has quit IRC04:15
*** dklyle has joined #openstack-infra04:17
ianwtfw you go to write a spec, but are ending up debugging sphinx :/04:17
ianwor should i say pbr04:25
*** ianychoi has quit IRC04:28
*** markvoelker has quit IRC04:29
*** aeng has quit IRC04:30
*** ianychoi has joined #openstack-infra04:31
openstackgerritIan Wienand proposed openstack-dev/pbr master: Fix builddoc with sphinx <= 1.6  https://review.openstack.org/56381304:34
ianwnow, why does infra-specs have a too low sphinx ...04:34
*** dklyle has quit IRC04:37
*** slaweq has joined #openstack-infra04:39
*** janki has quit IRC04:39
*** eernst has quit IRC04:41
openstackgerritIan Wienand proposed openstack-infra/infra-specs master: Update to Sphinx 1.6  https://review.openstack.org/56381504:43
*** edmondsw has joined #openstack-infra04:43
*** edmondsw_ has joined #openstack-infra04:44
*** slaweq has quit IRC04:47
*** edmondsw has quit IRC04:48
*** lpetrut has joined #openstack-infra04:58
*** edmondsw_ has quit IRC04:59
shu-mutowHi, can we set "non-voting" for lower-constraints job?05:10
*** gongysh has quit IRC05:11
*** dhill_ has quit IRC05:15
*** rajinir has quit IRC05:18
*** e0ne has joined #openstack-infra05:19
*** evrardjp has joined #openstack-infra05:21
*** lpetrut has quit IRC05:27
*** yolanda_ has joined #openstack-infra05:27
*** yolanda has quit IRC05:28
*** dhill_ has joined #openstack-infra05:29
*** CrayZee has joined #openstack-infra05:31
*** dhill_ has quit IRC05:35
*** dhill_ has joined #openstack-infra05:36
*** quiquell|off is now known as quiquell|rover05:40
*** e0ne has quit IRC05:40
*** edmondsw has joined #openstack-infra05:41
*** edmondsw has quit IRC05:45
*** janki has joined #openstack-infra05:49
*** garyk has joined #openstack-infra05:53
*** hrubi has quit IRC05:54
*** shu-mutow is now known as shu-mutow-AWAY05:59
*** iranzo has joined #openstack-infra06:01
*** edmondsw has joined #openstack-infra06:02
AJaegershu-mutow-AWAY: best discuss on #openstack-requirements - but non-voting defeats the purpose of that job06:03
*** edmondsw has quit IRC06:06
*** gongysh has joined #openstack-infra06:07
*** dingyichen has quit IRC06:13
*** links has joined #openstack-infra06:16
*** bhavik1 has joined #openstack-infra06:17
*** quite has quit IRC06:22
*** bhavik1 has quit IRC06:23
*** quite has joined #openstack-infra06:24
*** armaan has joined #openstack-infra06:24
*** armaan has quit IRC06:24
*** jtomasek has joined #openstack-infra06:25
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Use publish-openstack-releasenotes-python3 for python3 release notes  https://review.openstack.org/56377306:25
*** stakeda has joined #openstack-infra06:26
*** lpetrut has joined #openstack-infra06:29
*** markvoelker has joined #openstack-infra06:30
*** slaweq has joined #openstack-infra06:32
*** threestrands has quit IRC06:32
*** aojea has joined #openstack-infra06:33
*** gfidente has joined #openstack-infra06:35
*** gfidente has joined #openstack-infra06:35
*** slaweq has quit IRC06:42
*** salv-orl_ has quit IRC06:45
*** salv-orlando has joined #openstack-infra06:46
*** tesseract has joined #openstack-infra06:49
*** threestrands has joined #openstack-infra06:50
*** threestrands has quit IRC06:50
*** threestrands has joined #openstack-infra06:50
*** salv-orlando has quit IRC06:50
openstackgerritAndreas Florath proposed openstack/diskimage-builder master: WIP: Docker matrix build  https://review.openstack.org/41434706:57
*** pcaruana has joined #openstack-infra06:58
*** gongysh has quit IRC06:59
*** ianychoi has quit IRC07:00
*** salv-orlando has joined #openstack-infra07:00
*** caphrim007 has joined #openstack-infra07:03
*** markvoelker has quit IRC07:04
*** armaan has joined #openstack-infra07:06
*** caphrim007_ has quit IRC07:07
*** jaosorior has joined #openstack-infra07:08
*** quiquell|rover is now known as quique|rover|afk07:09
*** gfidente has quit IRC07:09
*** threestrands has quit IRC07:10
*** ianychoi has joined #openstack-infra07:11
*** amoralej|off is now known as amoralej07:11
openstackgerritColleen Murphy proposed openstack-infra/system-config master: Stop managing infracloud things  https://review.openstack.org/56363407:13
*** sshnaidm|off is now known as sshnaidm07:14
*** slaweq has joined #openstack-infra07:16
*** rcernin has quit IRC07:16
*** gfidente has joined #openstack-infra07:17
*** gfidente has joined #openstack-infra07:17
*** jchhatbar has joined #openstack-infra07:21
*** janki has quit IRC07:23
*** edmondsw has joined #openstack-infra07:24
*** agopi is now known as agopi|zzz07:24
*** ralonsoh has joined #openstack-infra07:25
*** armaan has quit IRC07:28
*** edmondsw has quit IRC07:28
*** florianf has joined #openstack-infra07:29
*** dpawlik has joined #openstack-infra07:36
andreaftobiash frickler I replied to your comments on https://review.openstack.org/#/c/56358407:37
*** zhipeng has joined #openstack-infra07:40
sshnaidmcan I get http://zuul.openstack.org/builds.html information in json? Is there any docs about it?07:42
*** derekjhyang has quit IRC07:44
*** edmondsw has joined #openstack-infra07:45
*** ethfci has quit IRC07:45
*** ethfci has joined #openstack-infra07:45
*** ethfci has quit IRC07:47
*** ethfci has joined #openstack-infra07:47
*** jpena|off is now known as jpena07:48
*** quique|rover|afk is now known as quiquell|rover07:49
*** edmondsw has quit IRC07:49
openstackgerritIan Wienand proposed openstack-infra/infra-specs master: Direction setting for 3rd Party CI  https://review.openstack.org/56384907:49
*** yolanda_ has quit IRC07:49
*** chrisyang_0660 has joined #openstack-infra07:49
*** hamzy has quit IRC07:51
*** vivsoni_ has joined #openstack-infra07:51
*** hamzy has joined #openstack-infra07:51
*** vivsoni has quit IRC07:51
tobiashandreaf: responded07:51
*** spzala has joined #openstack-infra07:53
*** electrofelix has joined #openstack-infra07:54
*** pblaho has joined #openstack-infra07:57
*** spzala has quit IRC07:57
*** jpich has joined #openstack-infra07:58
*** markvoelker has joined #openstack-infra08:01
*** yolanda_ has joined #openstack-infra08:02
*** yolanda_ is now known as yolanda08:02
*** edmondsw has joined #openstack-infra08:05
*** armaan has joined #openstack-infra08:08
*** edmondsw has quit IRC08:09
*** zhipeng has quit IRC08:10
*** tosky has joined #openstack-infra08:13
*** gongysh has joined #openstack-infra08:15
toskycorvus, clarkb: please ignore my yesterday issue: it was of course on my side, I did the math wrong and the wrong execution happened before the test was fixed; sorry for that08:16
*** hashar has joined #openstack-infra08:20
*** dsariel has joined #openstack-infra08:20
*** ssbarnea_ has joined #openstack-infra08:21
*** priteau has joined #openstack-infra08:25
*** edmondsw has joined #openstack-infra08:26
*** jistr is now known as jistr|mtgs08:28
*** edmondsw has quit IRC08:30
*** lucas-afk is now known as lucasagomes08:31
*** markvoelker has quit IRC08:34
*** witek_ has quit IRC08:34
*** efoley has joined #openstack-infra08:37
*** derekh has joined #openstack-infra08:41
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Bail on common devuser usernames  https://review.openstack.org/56386008:45
openstackgerritWitold Bedyk proposed openstack-infra/project-config master: Add monasca-transform to Monasca ACL config  https://review.openstack.org/56386108:46
fricklersshnaidm: not sure about docs but http://zuul.openstack.org/status gives you the raw json data08:48
*** yolanda has quit IRC08:48
openstackgerritWitold Bedyk proposed openstack-infra/project-config master: Add monasca-transform to Monasca ACL config  https://review.openstack.org/56386108:49
ssbarneaif we want to move the issue tracker of jenkins-job-builder from storyboard to github, who/where do we have to ask this? someone with admin access over openstack-infra github organization would need to do few click to enable the issue tracker.08:52
*** ianw is now known as ianw_pto08:54
*** e0ne has joined #openstack-infra08:56
*** armaan has quit IRC08:58
*** armaan has joined #openstack-infra08:58
*** yolanda has joined #openstack-infra09:01
*** armaan has quit IRC09:03
openstackgerritWitold Bedyk proposed openstack-infra/project-config master: Add monasca-transform to Monasca ACL config  https://review.openstack.org/56386109:09
*** gfidente has quit IRC09:09
*** dtantsur|afk is now known as dtantsur09:09
openstackgerritMerged openstack-infra/zuul-jobs master: Deploy ssh key as root for non-root users  https://review.openstack.org/56358409:15
*** jaosorior has quit IRC09:18
*** zhangfei has quit IRC09:23
*** panda|bbl is now known as panda|ruck09:25
*** gfidente has joined #openstack-infra09:27
*** gfidente has joined #openstack-infra09:27
*** wolverineav has joined #openstack-infra09:29
*** markvoelker has joined #openstack-infra09:31
*** jaosorior has joined #openstack-infra09:44
*** gongysh has quit IRC09:48
*** wolverineav has quit IRC09:49
*** wolverineav has joined #openstack-infra09:49
*** efoley has quit IRC09:50
*** wolverineav has quit IRC09:54
*** namnh has quit IRC09:54
*** armaan has joined #openstack-infra10:02
*** markvoelker has quit IRC10:06
*** dbecker has joined #openstack-infra10:07
*** sambetts_ is now known as sambetts10:11
*** priteau has quit IRC10:11
*** priteau has joined #openstack-infra10:11
*** quiquell|rover has quit IRC10:14
*** gfidente has quit IRC10:14
*** zoli is now known as zoli|lunch10:15
*** efoley has joined #openstack-infra10:17
slaweqAJaeger: hi, will You maybe have few minutes to take a look once again at https://review.openstack.org/#/c/561930/ ?10:18
slaweqthx in advace :)10:18
*** wolverineav has joined #openstack-infra10:18
*** armaan has quit IRC10:28
*** armaan has joined #openstack-infra10:29
*** annp has quit IRC10:29
*** annp has joined #openstack-infra10:30
*** boden has joined #openstack-infra10:33
*** armaan has quit IRC10:33
*** armaan has joined #openstack-infra10:33
*** dpawlik has quit IRC10:34
*** lxkong has quit IRC10:35
*** dpawlik has joined #openstack-infra10:36
*** dbecker has quit IRC10:36
*** jaosorior has quit IRC10:43
openstackgerritLuigi Toscano proposed openstack-infra/openstack-zuul-jobs master: Remove a migrated Sahara job (multinode-spark)  https://review.openstack.org/56388210:43
openstackgerritLuigi Toscano proposed openstack-infra/project-config master: Remove a migrated Sahara job (multinode-spark)  https://review.openstack.org/56388310:43
*** dbecker has joined #openstack-infra10:45
openstackgerritLuigi Toscano proposed openstack-infra/openstack-zuul-jobs master: Remove a migrated Sahara job (multinode-spark)  https://review.openstack.org/56388210:45
openstackgerritLuigi Toscano proposed openstack-infra/project-config master: Remove a migrated Sahara job (multinode-spark)  https://review.openstack.org/56388310:45
*** annp has quit IRC10:46
*** ssbarnea_ has quit IRC10:50
AJaegerslaweq: there're others that can review as well...10:54
openstackgerritFelix Schmidt proposed openstack-infra/zuul master: Add start and end timestamp to task result in zuul_json callback  https://review.openstack.org/56388810:56
openstackgerritAndrea Frittoli proposed openstack-infra/zuul-jobs master: Run authorized_keys as root  https://review.openstack.org/56388910:59
*** efoley has quit IRC11:02
*** markvoelker has joined #openstack-infra11:02
*** nicolasbock has joined #openstack-infra11:05
*** jcoufal has joined #openstack-infra11:07
*** dbecker has quit IRC11:13
*** pbourke has quit IRC11:18
*** pbourke has joined #openstack-infra11:18
*** stakeda has quit IRC11:18
*** ssbarnea_ has joined #openstack-infra11:19
*** ggherdov- has joined #openstack-infra11:20
*** ggherdov- has quit IRC11:23
*** armaan has quit IRC11:26
*** armaan has joined #openstack-infra11:26
*** armaan has quit IRC11:33
*** armaan has joined #openstack-infra11:33
*** zoli|lunch is now known as zoli11:35
*** markvoelker has quit IRC11:36
*** rh-jelabarre has joined #openstack-infra11:40
*** ldnunes has joined #openstack-infra11:41
*** lucasagomes is now known as lucas-hungry11:41
*** jaosorior has joined #openstack-infra11:48
zulhow do you depends on with git review again?11:49
openstackgerritJavier Peña proposed openstack-infra/irc-meetings master: Change rpm-packaging meeting schedule to Wednesdays  https://review.openstack.org/56301011:49
*** jpena is now known as jpena|lunch11:50
fricklerzul: have a line "Depends-On: https://review.openstack.org/12345" in your commit message11:50
zulfrickler: cool thanks11:51
*** spzala has joined #openstack-infra11:53
*** liuyulong has joined #openstack-infra11:54
*** spzala has quit IRC11:57
pabelangerhave a few errands to run this morning12:00
*** zhangfei has joined #openstack-infra12:08
*** zhongjun_ has quit IRC12:09
*** zhongjun_ has joined #openstack-infra12:09
*** larainema has quit IRC12:10
*** larainema has joined #openstack-infra12:10
*** Kevin_Zheng has quit IRC12:11
*** Kevin_Zheng has joined #openstack-infra12:12
*** wolverineav has quit IRC12:12
*** wolverineav has joined #openstack-infra12:12
*** wolverineav has quit IRC12:13
*** wolverineav has joined #openstack-infra12:14
*** yamamoto has joined #openstack-infra12:14
*** smarcet has joined #openstack-infra12:14
*** efoley has joined #openstack-infra12:15
*** jaosorior has quit IRC12:15
*** jaosorior has joined #openstack-infra12:16
*** efried has joined #openstack-infra12:16
*** markvoelker has joined #openstack-infra12:16
*** wolverineav has quit IRC12:18
*** tomhambleton_ has quit IRC12:19
*** tomhambleton_ has joined #openstack-infra12:20
*** amoralej is now known as amoralej|lunch12:22
*** smarcet has quit IRC12:25
*** LinuxJedi has quit IRC12:26
*** LinuxJedi has joined #openstack-infra12:26
*** mwhahaha has quit IRC12:26
*** mwhahaha has joined #openstack-infra12:26
*** edmondsw has joined #openstack-infra12:28
*** lucas-hungry is now known as lucasagomes12:29
*** lbragstad has joined #openstack-infra12:29
*** tpsilva has joined #openstack-infra12:29
*** edmondsw_ has joined #openstack-infra12:29
*** rosmaita has joined #openstack-infra12:31
*** Guest46098 has quit IRC12:32
*** Guest46098 has joined #openstack-infra12:32
*** gfidente has joined #openstack-infra12:33
*** gfidente has joined #openstack-infra12:33
*** edmondsw has quit IRC12:33
*** rlandy has joined #openstack-infra12:35
*** trown|outtypewww is now known as trown12:36
*** portdirect has quit IRC12:36
*** portdirect has joined #openstack-infra12:36
*** psachin has quit IRC12:37
*** panda|ruck is now known as panda|ruck|lunch12:37
*** Dmitrii-Sh has quit IRC12:37
*** zhangfei has quit IRC12:37
*** Dmitrii-Sh has joined #openstack-infra12:37
*** Goneri has joined #openstack-infra12:39
*** edmondsw_ has quit IRC12:44
*** edmondsw has joined #openstack-infra12:48
*** VW has joined #openstack-infra12:50
*** felipemonteiro__ has joined #openstack-infra12:52
*** edmondsw has quit IRC12:52
*** spzala has joined #openstack-infra12:53
*** jpena|lunch is now known as jpena12:54
*** eernst has joined #openstack-infra12:55
*** wolverineav has joined #openstack-infra12:56
openstackgerritDirk Mueller proposed openstack-infra/irc-meetings master: Change rpm-packaging meeting schedule to Wednesdays  https://review.openstack.org/56301012:56
*** spzala has quit IRC12:57
*** kgiusti has joined #openstack-infra12:58
*** edmondsw has joined #openstack-infra12:59
*** edmondsw has quit IRC13:01
*** alexchadin has joined #openstack-infra13:02
*** eernst has quit IRC13:04
*** garyk has quit IRC13:11
*** armaan has quit IRC13:12
*** armaan has joined #openstack-infra13:13
openstackgerritMatthieu Huin proposed openstack-infra/os-loganalyze master: Support ANSI coloring in logs  https://review.openstack.org/56394713:18
*** salv-orlando has quit IRC13:18
*** salv-orlando has joined #openstack-infra13:19
*** armaan has quit IRC13:20
*** armaan has joined #openstack-infra13:21
smcginnisIt appears the openstack/cinder repo mirror on github has not updated for some time.13:22
smcginnisWould someone be able to check on that?13:22
*** dave-mccowan has joined #openstack-infra13:22
smcginnisLooking at http://git.openstack.org/cgit/openstack/cinder/log/doc/source/configuration/block-storage/drivers/nimble-volume-driver.rst vs https://github.com/openstack/cinder/commits/a11cb46578afc013881cfc2860913f5779f143e2/doc/source/configuration/block-storage/drivers/nimble-volume-driver.rst13:22
*** caphrim007 has quit IRC13:24
*** salv-orlando has quit IRC13:24
*** spzala has joined #openstack-infra13:26
*** edmondsw has joined #openstack-infra13:27
*** liuyulong has quit IRC13:27
*** hrubi has joined #openstack-infra13:28
*** yamamoto has quit IRC13:28
fricklersmcginnis: https://github.com/openstack/cinder/commits/master/doc/source/configuration/block-storage/drivers/nimble-volume-driver.rst looks fine to me. no errors in the replication log, too13:28
*** yamamoto has joined #openstack-infra13:28
smcginnisfrickler: Ah, I think I see what happened.13:29
smcginnisI think I did a search, which led me to a specific commit, which when viewing history only shows up to that point.13:29
smcginnisSorry, user error. Web sites are hard.13:29
rosmaitafrickler: if you have a minute, could you hit https://review.openstack.org/#/c/527263/ ? you +2d it a while back but were waiting for the gate to stabilize before +A13:29
fricklerrosmaita: ack, thanks for the reminder13:31
*** edmondsw has quit IRC13:31
rosmaitanp, ty13:31
*** armaan has quit IRC13:32
*** armaan has joined #openstack-infra13:32
fricklerinfra-root: if someone could do another review on https://review.openstack.org/562381 we could proceed with setting proper URLs on storyboard emails13:36
*** witek has joined #openstack-infra13:37
*** jbryce has quit IRC13:37
*** jbryce has joined #openstack-infra13:37
witekhello, could someone please help on this one http://logs.openstack.org/60/543560/2/check/build-openstack-sphinx-docs/5035292/job-output.txt.gz#_2018-04-24_11_19_19_90130813:39
witekneutron-horizon-hack role fails and the sphinx-docs job stopped working for us on stable13:39
*** amoralej|lunch is now known as amoralej13:40
*** esberglu has joined #openstack-infra13:40
openstackgerritMatthieu Huin proposed openstack-infra/os-loganalyze master: Support ANSI coloring in logs  https://review.openstack.org/56394713:41
fricklerwitek: you need to fix your tools/tox_install.sh, because pip 10 turned the previous warning into an error. see e.g. https://review.openstack.org/#/c/562386/1/tools/tox_install.sh13:42
*** panda|ruck|lunch is now known as panda|ruck13:43
witekthanks frickler!13:43
*** zul has quit IRC13:48
*** zul has joined #openstack-infra13:48
*** alexchadin has quit IRC13:50
openstackgerritTobias Henkel proposed openstack-infra/zuul master: DNM: test tox-cover  https://review.openstack.org/56319913:50
*** armaan has quit IRC13:51
*** armaan has joined #openstack-infra13:51
*** jaosorior has quit IRC13:55
*** jaosorior has joined #openstack-infra13:55
*** mrmartin has quit IRC13:56
*** mrmartin has joined #openstack-infra13:56
*** felipemonteiro__ has quit IRC13:57
*** felipemonteiro__ has joined #openstack-infra13:57
*** sdoran has quit IRC13:58
*** sdoran has joined #openstack-infra13:58
*** hongbin_ has joined #openstack-infra13:59
*** edmondsw has joined #openstack-infra13:59
*** annp has joined #openstack-infra14:00
*** zhangfei has joined #openstack-infra14:03
*** edmondsw has quit IRC14:04
*** yamahata has joined #openstack-infra14:04
*** edmondsw has joined #openstack-infra14:04
*** jaosorior has quit IRC14:07
*** edmondsw has quit IRC14:09
*** edmondsw has joined #openstack-infra14:09
*** r-daneel has joined #openstack-infra14:10
*** edmondsw has quit IRC14:14
*** rajinir has joined #openstack-infra14:14
*** annp has quit IRC14:16
*** annp has joined #openstack-infra14:16
*** bobh has joined #openstack-infra14:16
*** wolverineav has quit IRC14:18
*** dhill_ has quit IRC14:18
*** wolverineav has joined #openstack-infra14:18
*** iyamahat has joined #openstack-infra14:19
*** zhongjun_ has quit IRC14:19
*** rpioso|afk is now known as rpioso14:19
*** wolverineav has quit IRC14:24
*** wolverineav has joined #openstack-infra14:24
*** dhill_ has joined #openstack-infra14:25
*** alexchadin has joined #openstack-infra14:25
*** mriedem has joined #openstack-infra14:26
openstackgerritMerged openstack-infra/zuul-website-media master: Add simulation video  https://review.openstack.org/56200114:26
*** iyamahat has quit IRC14:26
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Don't try to delete non-existing local refs  https://review.openstack.org/56398214:26
*** iyamahat has joined #openstack-infra14:26
*** zoli is now known as zoli|brb14:28
*** wolverineav has quit IRC14:28
openstackgerritMerged openstack-infra/puppet-storyboard master: Clean up OpenStack-isms  https://review.openstack.org/56238114:30
openstackgerritMerged openstack-infra/puppet-storyboard master: Set default_url for email notification  https://review.openstack.org/55523714:31
openstackgerritMerged openstack-infra/system-config master: Override the default_url for StoryBoard servers  https://review.openstack.org/56238214:33
*** edmondsw has joined #openstack-infra14:35
corvusinfra-root: can you take a look at and +2 https://review.openstack.org/562670 ?14:36
*** efoley has quit IRC14:36
*** r-daneel has quit IRC14:38
*** r-daneel has joined #openstack-infra14:39
*** edmondsw has quit IRC14:40
*** hemna_ has joined #openstack-infra14:40
*** links has quit IRC14:42
*** rossella_s has quit IRC14:42
*** edmondsw has joined #openstack-infra14:42
*** eharney has joined #openstack-infra14:44
*** edmondsw has quit IRC14:46
*** alexchadin has quit IRC14:48
*** eharney has quit IRC14:49
*** ajo has quit IRC14:50
*** ajo has joined #openstack-infra14:51
*** jamesmcarthur has joined #openstack-infra14:51
*** eernst has joined #openstack-infra14:51
*** yolanda_ has joined #openstack-infra14:55
*** smarcet has joined #openstack-infra14:56
*** efoley has joined #openstack-infra14:57
*** annp has quit IRC14:57
*** yolanda has quit IRC14:58
*** yamamoto has quit IRC14:58
*** rossella_s has joined #openstack-infra14:59
*** dpawlik has quit IRC15:00
openstackgerritMerged openstack-infra/system-config master: Add docs on mailman lock files  https://review.openstack.org/56267015:02
*** smarcet has quit IRC15:02
*** esberglu has quit IRC15:04
*** jaosorior has joined #openstack-infra15:05
*** esberglu has joined #openstack-infra15:05
*** smarcet has joined #openstack-infra15:06
*** dbecker has joined #openstack-infra15:06
*** tesseract has quit IRC15:06
clarkbssbarnea: we explicitly disable that feature via jeepyb iirc15:06
*** tesseract has joined #openstack-infra15:06
clarkbssbarnea: at the very least we'd have to stop doing that but chances are someone would also have to enable the bug tracker (as I don't know that the automation toggles that binary state in the other direction)15:07
clarkbtosky: ya I had done that math shortly after you went afk yesterday15:07
toskyclarkb: I read that in the logs, yes; thanks again15:08
clarkbssbarnea: it would probably be helpful to understand why a change wants to be made as well (understand the deficiencies you are fascing in general and to evaluate if github issue tracker would address them)15:09
ssbarneaclarkb: true, i have an TODO item to write about that, comparison of storyboard, launchpad and github tracker.15:12
*** felipemonteiro_ has joined #openstack-infra15:13
*** alexchadin has joined #openstack-infra15:13
*** armaan has quit IRC15:14
openstackgerritAde Lee proposed openstack-infra/irc-meetings master: Change barbican meeting time  https://review.openstack.org/56400015:14
*** armaan has joined #openstack-infra15:14
*** felipemonteiro__ has quit IRC15:16
openstackgerritMerged openstack-infra/infra-specs master: Update to Sphinx 1.6  https://review.openstack.org/56381515:16
*** eharney has joined #openstack-infra15:18
*** zoli|brb is now known as zoli15:19
*** lyaaaaaaaaarwood is now known as lyarwood15:19
ssbarneaclarkb: thanks for the links to the jeepyb as i didn't know about it. i looked at it and it seems that "has-issues=False" is the flag that would have to be switched.15:20
*** dbecker has quit IRC15:20
ssbarneastill i was not able to find any specific repository info, so I don't know if is possible to change its value for one specific repo, or not.15:20
clarkbI believe it is toggleable for a single repo. our file is at openstack-infra/project-config/gerrit/projects.yaml iirc15:21
clarkbssbarnea: ^ I want to say you put has-issues: false under the project list entry15:21
fungiclarkb: ssbarnea: i think we would want to retire it in gerrit so that it could be moved to another github org where the maintainers have control over whatever github features are needed?15:22
fungior someone could run a little mirroring daemon to perform pull/push replication to another gh org15:23
fungithough using gh issues without supporting gh pull requests for contribution would probably be really confusing for people15:23
ssbarneai was hopping that this would not be needed, as I have nothing against gerrit and zuul usage. no plans to open PRs.15:24
fungigh isn't particularly designed for hosting issue tracking independent from code contribution15:24
ssbarneais only about issue tracker.15:24
*** caphrim007 has joined #openstack-infra15:24
fungithe gh api goes so far as to treat issues and pull requests as the same object15:24
fungitype15:25
clarkbfungi: I think our PR closer bot handles it properly at least15:25
fungiyeah, you can differentiate based on some of the attributes15:26
clarkbfungi: it does a leap of faith api transition that is totally undocumented and does PR queries against PR api but modifications against issue apio15:26
fungianyway, just suggesting we may not want to be hosting issue tracking within the openstack-infra gh org15:26
fungifor consistency's sake15:27
*** smarcet has quit IRC15:27
clarkbfungi: ya it would also fly in the face of github is a read only git mirror stance we'ev had15:27
* SotK would also be interested in hearing about the deficiencies which have led to wanting to use gh issues instead of storyboard/launchpad15:27
clarkbSotK: ^^ ++ I think that is what we should talk about before deciding github is the answer15:27
*** dklyle has joined #openstack-infra15:29
*** smarcet has joined #openstack-infra15:31
*** spsurya has quit IRC15:31
*** spsurya has joined #openstack-infra15:32
*** pblaho has quit IRC15:32
*** eharney has quit IRC15:32
*** zhangfei has quit IRC15:32
*** edmondsw has joined #openstack-infra15:33
*** salv-orl_ has joined #openstack-infra15:35
*** gyee has joined #openstack-infra15:38
*** e0ne has quit IRC15:39
openstackgerritMerged openstack-infra/irc-meetings master: Change rpm-packaging meeting schedule to Wednesdays  https://review.openstack.org/56301015:40
*** thorst has joined #openstack-infra15:43
pabelangernotmyname: It looks like you have merge permissions on pyeclib, mind a +3 on https://review.openstack.org/563247/ or suggest somebody who can15:44
pabelangerTheJulia: do you mind reviewing https://review.openstack.org/563252/ too please for bifrost15:45
*** alexchadin has quit IRC15:46
*** felipemonteiro_ has quit IRC15:49
*** felipemonteiro_ has joined #openstack-infra15:49
*** pcaruana has quit IRC15:51
*** smarcet has quit IRC15:54
*** hashar is now known as hasharAway15:55
*** jbadiapa has quit IRC15:55
*** rossella_s has quit IRC15:59
*** yamamoto has joined #openstack-infra15:59
*** r-daneel_ has joined #openstack-infra15:59
*** rossella_s has joined #openstack-infra15:59
*** r-daneel has quit IRC16:00
*** r-daneel_ is now known as r-daneel16:00
*** iranzo has quit IRC16:02
*** zoli is now known as zoli|gone16:03
*** zoli|gone is now known as zoli16:03
*** armaan has quit IRC16:04
*** pbourke has quit IRC16:04
pabelangerfungi: corvus: could I get a review on https://review.openstack.org/563673/ removes debian-jessie from bindep (and eventually nodepool)16:04
*** yamamoto has quit IRC16:04
*** pbourke has joined #openstack-infra16:05
*** armaan has joined #openstack-infra16:05
*** esberglu has quit IRC16:05
*** ramishra has quit IRC16:05
*** esberglu has joined #openstack-infra16:05
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: Remove debian-jessie nodesets  https://review.openstack.org/56287016:06
*** lucasagomes is now known as lucas-afk16:06
*** armaan_ has joined #openstack-infra16:07
*** armaan has quit IRC16:08
*** armaan_ has quit IRC16:09
*** esberglu has quit IRC16:10
*** iyamahat has quit IRC16:10
*** rossella_s has quit IRC16:10
*** armaan has joined #openstack-infra16:12
*** armaan has quit IRC16:13
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: Remove debian-jessie nodesets  https://review.openstack.org/56287016:14
*** rossella_s has joined #openstack-infra16:15
*** yamahata has quit IRC16:15
pabelangerdmsimard: do you mind reviewing https://review.openstack.org/564019/ to update feature/1.0 to debian-stable, thanks16:16
pabelangerfor ARA16:16
*** jamesmcarthur has quit IRC16:19
*** jamesmcarthur has joined #openstack-infra16:20
*** jamesmcarthur has quit IRC16:24
*** bhavik1 has joined #openstack-infra16:25
*** rossella_s has quit IRC16:28
*** rossella_s has joined #openstack-infra16:30
clarkbfungi: cmurphy I'm rereviewing https://review.openstack.org/#/c/562068/ now but that would probably be a good one for post infra meeting merging and babysitting (I can help with that)16:30
*** thorst has quit IRC16:30
clarkbalso I think we should restart zuul when we have opportunity16:30
clarkbanother post infra meeting thing16:30
pabelangerShrews: mnaser: clarkb: I think vexxhost is wedged. I haven't seen any new nodes launched in a few days16:30
cmurphyclarkb: sounds good, i'll be around16:30
pabelangerlooks like we have some VMs stuck in deleting state16:30
pabelangerand 3 ready locked16:31
*** jchhatbar has quit IRC16:32
*** jpich has quit IRC16:32
clarkbcmurphy: and one clarification, basemodulepath defaults to /etc/puppet/modules and /etc/puppetlabs/code/modules in puppet3 and 4 respectively if not explicitly set?16:40
cmurphyclarkb: https://puppet.com/docs/puppet/4.9/configuration.html#basemodulepath technically /etc/puppetlabs/code/modules:/opt/puppetlabs/puppet/modules16:43
cmurphyfor puppet 416:43
*** bobh has quit IRC16:43
*** pcaruana has joined #openstack-infra16:43
clarkbcmurphy: thanks. I've +2'd it and left a note on how we might role that out safely and do a sanity check before using it everywhere16:43
cmurphypuppet 3 also defaults to /etc/puppet/modules and maybe something else, can check in a sec16:43
*** bobh has joined #openstack-infra16:44
openstackgerritMatthew Thode proposed openstack/diskimage-builder master: Install sudo on Gentoo images by deault  https://review.openstack.org/56402816:45
*** trown is now known as trown|lunch16:45
clarkbs/role/roll/ ETOOMUCHANSIBLE16:45
cmurphylol16:46
*** armaan has joined #openstack-infra16:46
jlvillalgerritbot review request: https://review.openstack.org/#/c/545469/  Some cleanup/refactoring and adding unit tests. Has one +2 Thank you.16:47
pabelangerdo we know anybody else on the pyeclib team?16:48
*** salv-orl_ has quit IRC16:48
clarkbpabelanger: its the swift team I'd ask in their channel16:48
pabelangerack16:48
clarkbjlvillal: I'm half tempted to just make you core on gerritbot16:49
clarkbinfra-root ^ thoughts16:49
clarkbjlvillal has revived the project and worked to add features and testing, I'm happy to let interested parties have core if they are interested.16:50
*** e0ne has joined #openstack-infra16:51
Shrewspabelanger: out for an appointment now. Can look later if you don't find anything16:51
tdasilvapabelanger: hello! what's your question on pyeclib?16:52
pabelangertdasilva: just posted in #openstack-swift, but looking to get https://review.openstack.org/563247/ merged16:53
jlvillalclarkb, Cool. I'd be willing to do that. Thanks for considering me :)16:53
dmsimardpabelanger: I will be retiring that branch soon but sure16:54
corvusclarkb: i'm interested in gerritbot changes (i'm the original author) but i think development should head in the limnoria direction16:54
corvusjlvillal: ^16:54
pabelangerdmsimard: thanks, will allow us to remove nodes from nodepool16:54
clarkbcorvus: thats fair, I guess the point there being lets not spend effort on refactoring to make gerritbot easier to maintain16:55
corvusclarkb, jlvillal: do we really want to continue developing the existing gerritbot?16:55
*** sambetts is now known as sambetts|afk16:55
jlvillalcorvus, I guess the issue is we have something that exists and then we have this idea of something that would like to do but nothing exists at the moment.16:56
corvussure, but the thing that exists is fundamentally flawed16:57
jlvillalcorvus, But the thing that isn't flawed doesn't exist ;)16:57
jlvillalUp to others if they want to just say no development/improvements. Just security fixes to gerritbot.16:58
corvusand (i know you won't believe this) while i'm normally a fan of incremental development, this really does feel like a zero-sum situation, where every bit of work spent on old gerritbot could instead be invested in the new one.  the same people/skills are involved.16:58
*** felipemonteiro__ has joined #openstack-infra16:59
*** yamamoto has joined #openstack-infra17:00
*** felipemonteiro__ has quit IRC17:01
corvusi'm not sure what to do here, because it's nice to improve the current thing, but our community really needs the new thing (we're out of channels), and work on the old one won't port over.17:01
*** derekh has quit IRC17:01
*** felipemonteiro__ has joined #openstack-infra17:01
jlvillalcorvus, I'm willing to help on the new one. But kind of need someone to setup a framework to get started.17:02
jlvillalI for sure can't work on that in the next 1-2 weeks as have other obligations.17:02
*** felipemonteiro_ has quit IRC17:02
corvusjlvillal: maybe i can do that later in the week17:02
jlvillalcorvus, Sweet :)17:03
*** felipemonteiro has joined #openstack-infra17:03
corvusmaybe on friday i can set aside some time to get limnoria installed.  then maybe we can make a limnoria feature branch of gerritbot, and get that installed as a plugin17:03
jlvillalcorvus, I'll be mostly out of town over the next 1-2 weeks. But will check IRC in the mornings.17:03
*** iyamahat has joined #openstack-infra17:03
corvusmaybe that's enough of a framework to get things started?17:03
jlvillalcorvus, Sounds like a good starting point to me.17:04
jlvillalcorvus, I've actually recently created a Limnoria plugin. So I have a tiny bit of knowledge in that area.17:04
*** edmondsw has quit IRC17:04
corvuswe could run it in parallel with the current 'openstack' supybot for a while, get gerritbot and meetbot added to it, then switch it over17:04
*** panda|ruck is now known as panda|ruck|off17:04
corvusjlvillal: great!  i'll focus on getting things to the point where you can do that.17:05
clarkbas a thought here, it should be fairly straightforward to run such a bot locally (eg it doesn't have to be hosted to bootstrap)17:05
*** yamamoto has quit IRC17:06
clarkbsince it just needs read access to gerrit events and irc connection to freenode, it can basically run anywhere17:06
jlvillalcorvus, I have also heard of Sopel: https://github.com/sopel-irc/sopel17:06
jlvillalBut never tried it.17:06
corvustrue, maybe we should go ahead and make a gerritbot feature branch?17:06
*** felipemonteiro__ has quit IRC17:06
clarkbcorvus: ya maybe start there and then tox can run a test connection if you provide an ssh key or something17:06
corvusjlvillal: limnoria was chosen because of its multi-connection support; that seemed pretty unique at the time.17:06
*** rossella_s has quit IRC17:06
jlvillalcorvus, Ah. Okay.17:07
corvusclarkb, jlvillal: should we go ahead and branch gerritbot "feature/limnoria" ?17:07
clarkbwfm17:07
jlvillalSounds reasonable to me17:07
*** ralonsoh has quit IRC17:07
*** dizquierdo has joined #openstack-infra17:08
corvusclarkb, jlvillal: https://git.openstack.org/cgit/openstack-infra/gerritbot/log/?h=refs/heads/feature/limnoria  exists17:08
corvusjlvillal: so whenever you have time, feel free to do whatever you want to that branch to turn it into a limnoria plugin17:08
corvus(i imagine that may start with deleting a bunch of stuff :)17:09
jlvillalcorvus, Okay. I leave in a few hours. So no promises...17:09
corvusjlvillal: understood :)17:09
clarkbjlvillal: do you get to sit on a beach somewhere?17:09
corvusjlvillal: later in the week, i'll work on the hosting bits, so that hopefully once there's a basic plugin in there, we can have it deployed.  but as clarkb says, none of the python coding needs to wait on that.17:09
*** jtomasek has quit IRC17:10
jlvillalclarkb, Uh no. Get to go to Corvallis and Lebanon. Help work on the house my sister bought. And also help my mother with her stuff...17:10
clarkbjlvillal: weather is nice for it at least, have fun17:10
clarkbjlvillal: block 15 in corvallis is good too :)17:10
*** rossella_s has joined #openstack-infra17:11
*** jpena is now known as jpena|off17:12
*** dsariel has quit IRC17:13
jlvillalclarkb, Thanks!17:14
openstackgerritTrevor McCasland proposed openstack-infra/subunit2sql master: Add attachments flag to get_test_runs_by_status  https://review.openstack.org/55215217:15
*** udesale has joined #openstack-infra17:15
*** j3p0uk has joined #openstack-infra17:16
*** dtantsur is now known as dtantsur|afk17:17
*** felipemonteiro_ has joined #openstack-infra17:17
*** efoley has quit IRC17:18
* clarkb pops out for late breakfast then will prep for the meeting17:19
*** felipemonteiro has quit IRC17:20
*** j3p0uk has quit IRC17:21
*** jaosorior has quit IRC17:24
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Don't try to delete non-existing local refs  https://review.openstack.org/56398217:26
*** dhill_ has quit IRC17:27
*** smarcet has joined #openstack-infra17:29
*** armaan has quit IRC17:33
*** smarcet has quit IRC17:34
*** dhill_ has joined #openstack-infra17:34
*** lpetrut_ has joined #openstack-infra17:34
*** lpetrut has quit IRC17:38
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Don't try to delete non-existing local refs  https://review.openstack.org/56398217:39
*** agopi|zzz has quit IRC17:41
*** agopi|zzz has joined #openstack-infra17:41
*** dpawlik has joined #openstack-infra17:45
*** gfidente has quit IRC17:46
*** bhavik1 has left #openstack-infra17:48
*** dpawlik has quit IRC17:50
*** smarcet has joined #openstack-infra17:50
*** trown|lunch is now known as trown17:55
*** edmondsw has joined #openstack-infra17:56
*** smarcet has quit IRC17:56
*** udesale has quit IRC17:57
*** eernst has quit IRC18:01
*** wolverineav has joined #openstack-infra18:01
*** eernst has joined #openstack-infra18:01
*** dizquierdo has quit IRC18:02
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: Stop using slave_scripts/install-distro-packages.sh  https://review.openstack.org/56403818:02
*** yamamoto has joined #openstack-infra18:03
*** edmondsw_ has joined #openstack-infra18:03
pabelangerclarkb: fungi: AJaeger: large change to ozj to remove install-distro-packages.sh scripts^ if I could get a few reviews to avoid merge conflicts18:03
*** dhill_ has quit IRC18:04
*** eernst has quit IRC18:04
*** eernst has joined #openstack-infra18:04
pabelangerstill need a review on https://review.openstack.org/563673/ to remove debian-jessie from bindep18:05
*** edmonds__ has joined #openstack-infra18:05
*** edmonds__ is now known as edmondsw__18:06
*** edmondsw__ is now known as edmondsw___18:06
*** edmondsw has quit IRC18:06
*** edmondsw___ is now known as edmondsw18:06
*** wolverineav has quit IRC18:06
*** electrofelix has quit IRC18:07
*** eernst has quit IRC18:07
*** eernst has joined #openstack-infra18:08
*** yamamoto has quit IRC18:08
*** edmondsw_ has quit IRC18:09
*** smarcet has joined #openstack-infra18:10
*** dhill_ has joined #openstack-infra18:11
clarkbwoo had a bit of a hardware fault coming back from food18:13
Shrewspabelanger: back now. did you find anything with the vexxhost issue?18:14
clarkbI seem to have revived the desktop though (it did a hard lock up and I couldn't get a vty so it got alt sysrq b'd then didn't come back immediately)18:14
clarkbI'm prepping for meeting now18:14
fungiclarkb: computers18:14
* fungi feigns an ascii shrug18:15
clarkbfungi: the hardware is old enoguh I wouldn't be surprised if I need new hardware18:15
pabelangerShrews: I haven't looked to much into it yet18:16
fungithis is why i always buy two of anything. that way when one starts to die i can swap it out while i take my time looking for two of something else18:16
corvusfungi: especially helpful with shoes18:16
corvusi always buy two shoes18:16
*** eernst has quit IRC18:17
*** esberglu has joined #openstack-infra18:17
*** lpetrut_ has quit IRC18:17
*** esberglu has quit IRC18:17
*** esberglu has joined #openstack-infra18:18
fungime too. then if one wears out i can still hop to the shoe store18:19
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: Stop using slave_scripts/install-distro-packages.sh  https://review.openstack.org/56403818:22
pabelangerfungi: thanks, updated18:23
clarkbI lied not back. This may require me to retire the desktop and use laptop and find a new desktop18:23
*** eernst has joined #openstack-infra18:23
*** amoralej is now known as amoralej|off18:24
*** ianychoi has quit IRC18:26
*** iyamahat has quit IRC18:26
*** ianychoi has joined #openstack-infra18:26
clarkbok maybe now it will be stable18:27
clarkb(I really want to use my real keyboard...)18:27
*** eernst has quit IRC18:27
*** smarcet has quit IRC18:28
*** gyee has quit IRC18:28
*** pcaruana has quit IRC18:32
*** sshnaidm has quit IRC18:32
*** eernst has joined #openstack-infra18:32
*** smarcet has joined #openstack-infra18:32
*** eernst_ has joined #openstack-infra18:33
*** eernst has quit IRC18:33
*** eernst_ has quit IRC18:34
*** VW has quit IRC18:36
*** VW has joined #openstack-infra18:38
*** felipemonteiro__ has joined #openstack-infra18:39
*** felipemonteiro_ has quit IRC18:39
*** iranzo has joined #openstack-infra18:39
pabelangersmcginnis: can I get you to review https://review.openstack.org/563717/ again please, I think brian (don't know his IRC) is happy with it now18:42
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Fix awkward merging of code  https://review.openstack.org/56404518:44
*** VW has quit IRC18:47
*** VW has joined #openstack-infra18:49
openstackgerritKendall Nelson proposed openstack-infra/project-config master: Add Barbican to StoryBoard  https://review.openstack.org/56404718:51
corvuspabelanger: meet rosmaita!  :)18:52
*** smarcet has quit IRC18:52
rosmaitacorvus: ty for the introduction!18:53
openstackgerritKendall Nelson proposed openstack-infra/project-config master: Add Barbican to StoryBoard  https://review.openstack.org/56404718:53
pabelangerrosmaita: hello!18:53
rosmaitapabelanger: thanks for putting up that patch18:54
pabelangernp, will help us remove some legacy scripts from jenkins18:54
corvusthere is no... nevermind18:55
*** ociuhandu has joined #openstack-infra18:55
*** xinliang has quit IRC18:56
clarkbinfra meeting time19:00
*** smarcet has joined #openstack-infra19:01
*** smarcet has quit IRC19:02
*** dims has quit IRC19:02
openstackgerritKendall Nelson proposed openstack-infra/project-config master: Add Barbican to StoryBoard  https://review.openstack.org/56404719:02
*** aojea has quit IRC19:02
*** bobh has quit IRC19:04
*** yamahata has joined #openstack-infra19:04
*** yamamoto has joined #openstack-infra19:04
*** lpetrut_ has joined #openstack-infra19:04
*** iyamahat has joined #openstack-infra19:05
*** agopi|zzz has quit IRC19:07
*** dims has joined #openstack-infra19:09
*** yamamoto has quit IRC19:10
Shrewspabelanger: the vexxhost thread appears to be paused handling a request (which no longer exists, btw), but it can't do anything because vexxhost is at quota and deletes are failing there for some reason19:11
*** esberglu has quit IRC19:11
Shrewspabelanger: fix the delete situation and it should auto-fix19:11
pabelangerShrews: okay, cool. That is what I hoped19:11
*** xinliang has joined #openstack-infra19:11
*** xinliang has quit IRC19:11
*** xinliang has joined #openstack-infra19:11
pabelangermnaser: ^maybe you can look?19:11
mnaserso the cinder volumes not disappearing that is?19:12
pabelangermnaser: looks like deletes of compute nodes are failing19:12
pabelangerbut we don't see an error19:12
mnaserso it gets stuck in deleting state?19:13
*** imacdonn has quit IRC19:14
*** imacdonn has joined #openstack-infra19:14
*** esberglu has joined #openstack-infra19:14
pabelangermnaser: Yah, been a few days now19:15
*** yolanda__ has joined #openstack-infra19:15
*** sshnaidm has joined #openstack-infra19:16
*** salv-orlando has joined #openstack-infra19:16
pabelangerdmsimard: mind a +3 on https://review.openstack.org/564019/ is one of last patches to remove debian-jessie19:17
pabelangerthanks19:17
*** yolanda_ has quit IRC19:18
*** VW has quit IRC19:21
*** tesseract has quit IRC19:26
*** agopi|zzz has joined #openstack-infra19:31
openstackgerritJames E. Blair proposed openstack-infra/zuul-website master: Create a zuul-website gate queue  https://review.openstack.org/56405319:31
openstackgerritJames E. Blair proposed openstack-infra/zuul-website-media master: Join the zuul-website queue  https://review.openstack.org/56405419:32
openstackgerritMerged openstack-infra/zuul-website master: Add simulation video  https://review.openstack.org/56200519:33
*** bobh has joined #openstack-infra19:36
*** VW has joined #openstack-infra19:38
openstackgerritMerged openstack-infra/zuul-website master: Create a zuul-website gate queue  https://review.openstack.org/56405319:38
*** ianychoi has quit IRC19:39
*** iranzo has quit IRC19:39
openstackgerritMerged openstack-infra/zuul-website-media master: Join the zuul-website queue  https://review.openstack.org/56405419:41
*** bobh has quit IRC19:41
*** ianychoi has joined #openstack-infra19:41
clarkbalright I'm gonna take a short break post meeting but will be back to help restart zuul and do other things that need doing19:44
clarkbcorvus: ^ fyi19:44
corvusclarkb: thanks!19:45
fungii'll be around in an hour-ish after fried rice is made and then consumed19:45
*** florianf has quit IRC19:48
pabelangerfungi: corvus: mind reviewing https://review.openstack.org/563673/ the last project to remove debian-jessie from19:49
pabelangerwe can then remove the debian-jessie nodeset from ozj: https://review.openstack.org/562870/19:50
corvuspabelanger: do we not have any debian?19:50
corvusoh we have stretch...19:51
pabelangercorvus: we do, debian-stable, but for bindep-fallback jobs we are not going to run.  I'm going to propose a new job to test against debian-stable, using in-tree bindep.txt19:52
corvuspabelanger: okay, good -- it seems like all of our bindep testing right now is the fallback job, so of course if we eliminate the fallback file, we won't have anything to test with.  so having some kind of test on each platform would be good.19:52
corvuspabelanger: maybe even make a bindep.txt file specifically for a test19:53
corvuspabelanger: anyway, 673 lgtm; don't need to wait on that19:53
pabelangeryah, clarkb suggested that too, happy to make that job19:53
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Stop using slave_scripts/install-distro-packages.sh  https://review.openstack.org/56403819:56
*** rkukura_ has joined #openstack-infra20:03
*** rasca has quit IRC20:04
*** weshay has quit IRC20:05
*** ianychoi has quit IRC20:05
*** yamamoto has joined #openstack-infra20:06
*** rkukura has quit IRC20:06
*** rkukura_ is now known as rkukura20:06
*** lpetrut_ has quit IRC20:07
*** weshay has joined #openstack-infra20:07
*** rasca has joined #openstack-infra20:07
*** ianychoi has joined #openstack-infra20:09
*** yamamoto has quit IRC20:11
*** hemna_ has quit IRC20:11
openstackgerritMatthieu Huin proposed openstack-infra/os-loganalyze master: Support ANSI coloring in logs  https://review.openstack.org/56394720:12
*** tjgresha has joined #openstack-infra20:12
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Remove infracloud  https://review.openstack.org/53439520:12
*** smarcet has joined #openstack-infra20:12
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Fix on error processing for google cal sync  https://review.openstack.org/56405720:14
*** e0ne has quit IRC20:15
dhellmannis it normal for jobs to sit for >5 hours in the post queue?20:16
dhellmannI'm looking at the jobs for oslo.config 198f53920:17
dhellmannI have no idea how long that queue usually takes to process things20:17
*** dhill_ has quit IRC20:17
pabelangerdhellmann: usually happens when nodepool is running a capacity.  post piplines have the lowest priority20:17
dhellmannthere are other older sets of jobs20:17
*** dklyle has quit IRC20:18
dhellmannah, sure, that makes sense. is that the state we're in now?20:18
pabelangeryah, you can see at http://grafana.openstack.org/dashboard/db/zuul-status20:18
pabelangerour node requests are currently 520 and coming down20:18
pabelangerI imagine they will get picked up in the next few hours20:18
dhellmannok, cool, thanks for that20:19
pabelangerdhellmann: np20:19
pabelangerhmm, does look like inap is having issues deleting nodes again: http://grafana.openstack.org/dashboard/db/nodepool-inap20:19
openstackgerritMerged openstack-infra/openstackid-resources master: Fix on error processing for google cal sync  https://review.openstack.org/56405720:20
dhellmannthat's an odd thing to have trouble with20:20
pabelangerwe uploaded ubuntu-xenial 1hr ago too20:21
pabelangerlikely thundering hurd getting images from glance20:21
*** bobh has joined #openstack-infra20:22
mtreinishinfra-root: I think mosquitto went down again. I expect for the same websockets gp fault20:22
*** agopi|zzz has quit IRC20:22
pabelanger[4645569.867251] mosquitto[12039]: segfault at 8 ip 00007fb1ceee6072 sp 00007fff8c075380 error 4 in libwebsockets.so.7[7fb1ceed1000+1f000]20:23
pabelangermtreinish: like that?^20:23
*** bobh has quit IRC20:23
mtreinishyeah20:23
*** bobh has joined #openstack-infra20:23
pabelangersorry, that was old one20:23
*** pblaho has joined #openstack-infra20:23
pabelanger[Mon Apr 23 06:25:01 2018] traps: mosquitto[16907] general protection ip:7f5cac623a0c sp:7fffebb2ed80 error:0 in libwebsockets.so.7[7f5cac61b000+1f000]20:23
mtreinishpabelanger: https://github.com/eclipse/mosquitto/issues/27820:23
mtreinishI wonder if we should re-disable the tls websockets listener again, this all started after we turned it on20:24
*** dhill_ has joined #openstack-infra20:25
mtreinishit'll break the js I embedded in my blog post, but it's better than just randomly crashing20:25
pabelangerk20:25
mtreinishclarkb, fungi: ^^^ thoughts?20:25
pabelangermosquitto.service has be restarted20:25
*** slaweq has quit IRC20:25
mtreinishpabelanger: cool, thanks20:26
mgagnepabelanger: looking into it20:26
*** slaweq has joined #openstack-infra20:26
openstackgerritMerged openstack-infra/bindep master: Remove debian-jessie nodeset  https://review.openstack.org/56367320:26
pabelangerclarkb: https://review.openstack.org/562870/ could use a +3 we can now remove the debian-jessie nodeset from zuul20:27
mgagnepabelanger: I ran the SQL query to clear that up, lets wait if nodepool recovers by itself or not.20:28
*** iyamahat has quit IRC20:28
pabelangermgagne: ack, thanks20:29
*** yamahata has quit IRC20:29
mtreinishpabelanger: can you check the status of the germqtt and lpmqtt daemons on firehose01.o.o too? I expect at least one of them crashed because of the outage.20:31
mtreinishI have a proposed fix up for that in germqtt: https://review.openstack.org/#/c/561952/20:31
*** priteau_ has joined #openstack-infra20:31
*** agopi|zzz has joined #openstack-infra20:32
*** spzala has quit IRC20:32
*** spzala has joined #openstack-infra20:32
pabelangersure, checking20:32
pabelangermtreinish: germqtt.service was stopped20:34
pabelangerstarted now20:34
clarkbmtreinish: ya, would it be possible to terminate with apache or some other web server instead?20:34
clarkbmaybe even just a proxy like haproxy?20:34
mtreinishpabelanger: cool, thanks. Yeah I see the messages on it now20:34
*** jcoufal_ has joined #openstack-infra20:34
mtreinishclarkb: I looked into trying that before and had some issues getting the connection to establish20:34
*** priteau has quit IRC20:34
*** spzala has quit IRC20:34
*** kgiusti has left #openstack-infra20:34
mtreinishI'm sure I just screwed something up though20:34
mtreinishI didn't try that hard :p20:35
pabelangerrosmaita: mind a +3 on https://review.openstack.org/563717/ now20:35
clarkbmtreinish: maybe stunnel is a more appropriate in this case?20:35
clarkbhrm if dhellmann is waiting on post jobs now is maybe not the bset time to restart zuul20:36
dhellmannI'm waiting on some jobs to make sure the release notes job changes for oslo libs work right20:36
dhellmannwe approved some trivial patches to trigger release notes rebuilds20:37
*** stevebaker has quit IRC20:37
mtreinishclarkb: yeah stunnel would probably be a good choice, definitely at least worth another try20:37
pabelangerlooks like we might be caught up on check pipeline now20:37
*** hemna_ has joined #openstack-infra20:37
*** jcoufal has quit IRC20:37
*** gouthamr has quit IRC20:38
*** alexchadin has joined #openstack-infra20:38
*** dmellado has quit IRC20:38
*** alexchadin has quit IRC20:38
*** ianychoi has quit IRC20:39
*** ianychoi has joined #openstack-infra20:40
openstackgerritClark Boylan proposed openstack-infra/nodepool master: Fix awkward merging of code  https://review.openstack.org/56404520:40
*** eharney has joined #openstack-infra20:41
*** stevebaker has joined #openstack-infra20:42
*** gouthamr has joined #openstack-infra20:43
rosmaitapabelanger: done!20:43
*** dmellado has joined #openstack-infra20:43
pabelangerrosmaita: thanks20:45
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Cache configuration objects in addition to YAML dicts  https://review.openstack.org/56406120:48
*** iyamahat has joined #openstack-infra20:50
*** iyamahat_ has joined #openstack-infra20:50
*** sshnaidm is now known as sshnaidm|off20:52
*** iyamahat has quit IRC20:55
*** dklyle has joined #openstack-infra20:56
smcginnisHmm, looks like the Summit web site is down.20:58
openstackgerritKendall Nelson proposed openstack-infra/project-config master: Add Barbican to StoryBoard  https://review.openstack.org/56404720:59
*** trown is now known as trown|outtypewww21:02
clarkbsmcginnis: seems there for me but the about the summit content is missing?21:02
*** smarcet has quit IRC21:03
smcginnisclarkb: I got a weird CloudFlare connection error message, then I got the page but just the header graphics with no content.21:05
smcginnisYeah, https://www.openstack.org/summit/vancouver-2018/ is giving the tabs for the different options, but then blank white below for me.21:05
smcginnisAt least on the default About the Summit tab. Looks like the others are working now.21:06
*** yamamoto has joined #openstack-infra21:08
*** lxkong has joined #openstack-infra21:09
*** yamahata has joined #openstack-infra21:09
*** tjgresha has quit IRC21:10
clarkbsmcginnis: I've passed that info along21:10
*** edmondsw_ has joined #openstack-infra21:11
*** rajinir has quit IRC21:12
smcginnisclarkb: Thanks21:13
*** edmondsw has quit IRC21:13
*** yamamoto has quit IRC21:14
*** dklyle has quit IRC21:14
*** slaweq has quit IRC21:14
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: [WIP] Toggable colors in job-output.txt  https://review.openstack.org/56406621:15
*** ldnunes has quit IRC21:15
*** slaweq has joined #openstack-infra21:15
*** slaweq has quit IRC21:20
*** Goneri has quit IRC21:22
*** ianychoi has quit IRC21:24
*** ianychoi has joined #openstack-infra21:25
cmurphyclarkb: you still think you want to merge https://review.openstack.org/#/c/562068/ tonight?21:25
clarkbcmurphy: I was half expecing fungi to rereview it, but if not rereviewed then probalby not21:26
cmurphyclarkb: actually it will probably fail the gate anyway for the same reason this one did https://review.openstack.org/#/c/563634/21:26
*** VW has quit IRC21:27
cmurphyso will have to deal with that first21:27
*** jcoufal_ has quit IRC21:27
*** VW has joined #openstack-infra21:28
*** priteau_ has quit IRC21:32
*** ianychoi has quit IRC21:35
*** wolverineav has joined #openstack-infra21:35
*** ianychoi has joined #openstack-infra21:36
*** jbadiapa has joined #openstack-infra21:38
*** dbecker has joined #openstack-infra21:38
*** ociuhandu has quit IRC21:39
*** bobh has quit IRC21:40
clarkbcmurphy: hrm is that a pip10 failure?21:43
*** dklyle has joined #openstack-infra21:44
*** ociuhandu has joined #openstack-infra21:46
cmurphyclarkb: no i think it's just one of the libs dropped support for python 3, but actually i think it's been fixed since i looked at it this morning https://github.com/jaraco/keyring/blob/master/CHANGES.rst rechecking to see21:47
clarkbError: Could not update: Execution of '/usr/local/bin/pip install -q --upgrade keyring' returned 1: SecretStorage requires Python '>=3.5' but the running Python is 2.7.6 not pip1021:47
clarkbcmurphy: that dep is coming from keystoneclient via swiftclient ?21:48
clarkbhttps://pypi.org/project/SecretStorage/ says SecretStorage 3.x is python3.5 or newer only. And that unpin in the changelog did it I guess21:49
clarkbah the unpin is for python3 only so ya this should be fixed hopefully21:50
cmurphyclarkb: no it's coming from openstackci::logserver21:50
fungipabelanger: mtreinish: clarkb: cmurphy: i see i've been pinged. back at the 'puter now and catchnig up21:51
cmurphyyou're in high demand21:51
clarkbcmurphy: but I agree I think if we rerun now we'll get 12.1.0 https://pypi.org/project/keyring/ and it should work21:52
*** r-daneel has quit IRC21:52
*** agopi|zzz has quit IRC21:52
*** dbecker has quit IRC21:53
cmurphyclarkb: yep last recheck came out green21:54
cmurphyall good21:54
*** edmondsw_ has quit IRC21:55
*** agopi|zzz has joined #openstack-infra21:57
clarkbcorvus: pabelanger fungi: the post queue appears to have claered out21:58
clarkbwhat do we think about restarting zuul now?21:58
*** boden has quit IRC22:00
* clarkb goes to get ssh key loaded22:01
fungimtreinish: in theory we have a core dump from mosquitto (i think?)22:01
fungiroot@firehose01:~# mosquitto -v -c /etc/mosquitto/mosquitto.conf22:01
fungiSegmentation fault (core dumped)22:01
fungithat's what i found in my root screen session anyway22:02
fungihunting for it next22:02
*** ianychoi has quit IRC22:02
*** mriedem has quit IRC22:03
fungipabelanger: mtreinish: 06:25:01 looks like it once again corresponded to a log rotation event22:03
*** ianychoi has joined #openstack-infra22:04
*** esberglu has quit IRC22:04
*** edmondsw has joined #openstack-infra22:05
*** edmondsw has quit IRC22:05
fungimtreinish: clarkb: i used stunnel4 as an ssl proxy for my personal ircd for ages until i switched to inspircd which included its own ssl/tls socket implementation22:05
funginot that it's any sort of proper endorsement for using with mqtt websockets, just anecdata22:05
fungimtreinish: i'm not finding the coredump mosquitto claimed to have created :(22:06
corvusclarkb: ++22:08
clarkbok I'm going to review https://review.openstack.org/#/c/563982/ really quickly with the goal of being restart things with that in place22:08
fungiclarkb: zuul restart sounds fine to me22:09
*** jamesmcarthur has joined #openstack-infra22:09
*** yamamoto has joined #openstack-infra22:09
fungidhellmann: clarkb says post pipeline has cleared (not sure if you were still waiting)22:10
fungi563982 looks like the solution which was arrived at in #zuul, so seems sane22:12
fungiand with that, i seem to have caught up on fried rice scrollback22:13
*** hasharAway has quit IRC22:13
mnaserwould infra be opposed to pushing a patch which adds mirroring for percona? (OSA is hitting gate failures because of timeouts accessing percona.com)22:13
mnaserthey offer an rsync server (see https://www.percona.com/blog/2014/02/14/using-percona-rsync-repositories-set-local-centos-mirror/ )22:14
*** yamamoto has quit IRC22:14
mnaserseems like it consumes 2GB for centos 6 according to that post so i assume centos 7 cant be far away from it22:15
*** dave-mccowan has quit IRC22:15
clarkbok that change is approved22:16
clarkbnow we wait for that to merge then we restart the world22:16
clarkbcorvus: fungi we need to restart the mergers and executors to pull that one in right? so everything should be restarted?22:16
*** esberglu has joined #openstack-infra22:17
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Remove layout from ParseContext  https://review.openstack.org/56369522:17
corvusclarkb: yes22:17
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Remove 'base' from UnparsedAbideConfig  https://review.openstack.org/56375722:18
*** jamesmcarthur has quit IRC22:19
*** jamesmcarthur has joined #openstack-infra22:20
*** jamesmcarthur has quit IRC22:21
*** jamesmcarthur has joined #openstack-infra22:21
clarkbhrm 563982 is going to fail the gate22:22
clarkbcorvus: fungi do we want to hold off until we can get that in because it is a useful bug fix or just go ahead and restart and start testing stuff and worry about 563982 later?22:23
*** gyee has joined #openstack-infra22:23
clarkbhttp://logs.openstack.org/82/563982/3/gate/tox-py35/fc06597/ is the failure, looking at it now22:23
fungiseems like we should try to restart with it in place if we can get it through testing22:23
clarkblooks like a yarn failure22:24
fungibut if we can't, there are other bug fixes we could be taking advantage of with a restart22:24
fungiand there will always be another restart22:24
corvusah22:24
fungiso maybe let it merge when it can merge unless there's a quick fix for that22:24
corvusclarkb, fungi: i just spotted that same failure on a patch i submitted22:24
corvusi think yarn/nodeenv/something has wedged zuul22:25
corvusclarkb: i think it's fine to restart without that patch22:25
clarkbcorvus: ok I'll go ahead and work on that. General process is stop all the executors, then stop mergers, stop scheduler and friends, start shceduler and friends, start mergers, start executors and reapply saved queued?22:25
fungii'll buy that ;)22:25
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Cache configuration objects in addition to YAML dicts  https://review.openstack.org/56406122:26
clarkber no, mergers should be stopped after scheduler so t hat we don't report failed builds when I stop them22:26
*** jamesmcarthur has quit IRC22:26
corvusclarkb: don't we retry merges now?22:27
clarkbcorvus: maybe?22:27
corvushrm, maybe not.  we handle retrying jobs on executor disconnect, but i don't see special handling for merger failures22:28
clarkbcorvus: and when is best time to save queues? right before I stop the zuul-web and scheduler processes?22:28
corvusclarkb: yes22:28
clarkbok so plan will be stop executors, save queues, stop zuul-scheduler, zuul-fingergw, zuul-web, stop mergers, start mergers, start zuul-schduler,fingergw,web, start executors22:29
clarkbalright I'm going to run the ansible command to stop the executors. Can someone work on a status log?22:30
corvusstop executors, save queues, stop zuul-scheduler, zuul-fingergw, zuul-web, stop mergers, start zuul-scheduler, wait for merger stop, start mergers, start fingergw,web, wait for executor stop, start executors22:30
corvusclarkb: ^22:30
*** VW has quit IRC22:30
clarkbcorvus: oh neat I don't have to wait for them to synchronize thanks22:30
corvus(though i usually stop the mergers before stopping the scheduler; we should make it safe to do it my way cause it's faster)22:31
clarkb++22:31
*** VW has joined #openstack-infra22:31
clarkbalright I'm starting that process now22:31
clarkboh one last thing22:31
mtreinishfungi: :/ hmm yeah that's not really surprising. Although I don't think I've ever gotten systemd-coredump to work on my systems, so I might have a biased view22:31
clarkbsudo ansible -f 10 'zm*' -m shell -a 'systemctl stop zuul-merger' <- that command to stop the zuul mergers?22:31
clarkbrolling with that and starting now22:32
mtreinishfungi: I wonder if we could change the log rotation config to not cause this issue somehow22:32
corvusclarkb: yep, systemctl all around22:32
*** ociuhandu_ has joined #openstack-infra22:32
fungiclarkb: yeah, or `service blah stop` should also work with systend22:32
fungisystemd22:33
*** ociuhandu has quit IRC22:33
clarkbexecutors asked to stop, saving queues then stopping scheduler services22:33
corvus(i prefer service foo stop because it's easier to edit stop->start, but i'm afraid they're going to take it away from me)22:33
fungimtreinish: well, knowing that it somehow gets triggered by log rotation (so likely something to do with either signal handling or a missing log fd) may help22:33
clarkbzuul scheduler doesn't stop immediately22:35
clarkband now its gone22:35
clarkbstopping mergers now22:35
clarkbmergers asked to stop22:35
mtreinishfungi: yeah, I updated the bug with that insight last week. Hopefully ral will know where to look, I don't really have the time to debug it22:35
clarkbstarting zuul-scheduler now22:36
fungimtreinish: but sure, we could step down logging and turn off log rotation i suppose?22:36
mtreinishfungi: it also matched up to an earlier similar bug that someone else reported22:36
fungimtreinish: even just changing the signal logrotate uses to notify daemons to close and open new fds may help22:37
fungii think it defaults to sighup22:37
clarkbmergers starting now22:37
mtreinishfungi: yeah, I think it defaults to sending sighup22:37
clarkbjust waiting on the executors now22:38
clarkbreading executor logs they seem to be waiting for each ssh-agent to stop one by one?22:41
clarkbif so this could take a while22:42
*** efried has quit IRC22:42
clarkbI think everything is back up and happy otherwise22:42
*** ssbarnea_ has quit IRC22:43
mtreinishfungi: heh, now I kinda wish there was a crash dump somewhere. The reload code is pretty isolated in mosquitto: https://github.com/eclipse/mosquitto/blob/4f838e51611b9a71f2e3e98e6fddd02a5c1b4c03/src/loop.c#L391-L399 debugging it doesn't seem as daunting now22:44
clarkbok just waiting on ze01 at this point22:44
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Pin node to version 9  https://review.openstack.org/56407322:44
mtreinishfungi: oh, especially if you look at the loop in L406, I bet there is some interaction between reloading the config and the libwebsockets poll there22:45
clarkbexecutors starting back up now22:45
fungimtreinish: if you have an idea what the filename might be, i can scour the entire rootfs in search of it. there are tools for finding files22:45
clarkbwill requeue once that is done22:45
fungimtreinish: e.g., nothing matching "*.core" under /var/22:47
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Pin node to version 8: LTS stable  https://review.openstack.org/56407322:47
clarkbgate has been requeued and check is in progress22:47
fungithanks clarkb!!!22:48
mtreinishfungi: iirc on ubuntu they're .crash files (from apport)22:48
corvusclarkb: yes, it takes a while -- most of the time is deleting files22:48
mtreinishin /var/crash22:48
clarkbcorvus: 623533ab85ddf406bcac16452a638a706682775a is the sha1 in use for tagging purposes if we don't get around to testing something newer in the future22:48
fungimtreinish: bingo! i see one /var/crash/_usr_sbin_mosquitto.0.crash22:48
mtreinishbut it's been a while since I ran ubuntu on a box22:48
corvusclarkb: thanks!22:48
fungimtreinish: want a copy of that?22:48
mtreinishfungi: sure22:48
corvusfungi, mtreinish: not totally keeping up with convo -- but that won't have private key files, right?22:49
mtreinishcorvus: tbh, I don't know22:50
clarkbif it is in memory it likely will22:50
fungicorvus: it's what i'm checking on now22:50
clarkb(and chances are it will be in memory)22:50
mtreinishyeah, I don't know if it'll be in memory22:50
fungicorvus: worst case, i can load it in gdb and provide a bt22:50
clarkband check is reenqueued22:50
clarkblooks good so far22:51
openstackgerritKendall Nelson proposed openstack-infra/infra-manual master: Update Launchpad mentions to StoryBoard  https://review.openstack.org/56175322:51
fungiftr, the coredump itself is base64 encoded and stored as a field in the crashfile22:51
diablo_rojo_clarkb, I added some of the extra details you wanted ^^22:52
clarkbdiablo_rojo_: thanks, I'll take a look momentarily22:52
*** dizquierdo has joined #openstack-infra22:52
diablo_rojo_clarkb, sounds good, no rush22:52
openstackgerritDavid Ames proposed openstack-infra/project-config master: Add Neutron Dynamic Routing charm and BGP interface  https://review.openstack.org/56407422:52
mtreinishfungi: I think you can use http://manpages.ubuntu.com/manpages/xenial/en/man1/apport-retrace.1.html to interact with the .crash files instead of doing it by hand22:54
fungimtreinish: i don't see a apport-retrace but i used `apport-unpack _usr_sbin_mosquitto.0.crash foo` and (among other files) now have a foo/CoreDump which the file utility claims is a "ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from '/usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf'"22:55
fungiso getting closer22:56
fungiplaying around with `gdb /usr/sbin/mosquitto foo/CoreDump` next22:58
fungimtreinish: http://paste.openstack.org/show/719878 is the bt but i expect we're missing debugging symbols23:00
fungiat least, that doesn't look super useful for me23:01
fungii'll see if ubuntu has a -dbg package for mosquitto23:01
mtreinishfungi: oh, yeah I bet when we switched to ppa we probably didn't install that23:01
mtreinishnot that we would have installed it when we were using the ubuntu repo23:01
fungii have mosquitto-dbg (1.4.15-0mosquitto1~xenial1) installed now23:01
fungimuch better!23:02
fungimtreinish: http://paste.openstack.org/show/719879 now with symbols!23:02
fungistill a bit vague, but at least we have some source filenames and line numbers23:03
clarkband appears to be websocket disconnect that causedit to happen?23:04
fungithat it does23:04
clarkbwhich correlates to enabling tls on the websockets at least23:04
clarkbfungi: cmurphy at this rate I won't be able to do the puppet thing today23:04
mtreinishfungi: just curious are their libwebsockets debug symbols available too?23:04
clarkbI'm on the hook for dinner and wanted to be around for tc office hours23:05
clarkbfungi: maybe you can rereview it tomorrow and we make our attempt then?23:05
mtreinishfungi: it might be enough to get a start though, it at least confirms it's happening in the sighup handling23:05
clarkbzuul has now successfully reported job results to mysql and gerrit23:06
fungiclarkb: which change? i reviewed the latest patchset of 562068 earlier23:06
fungimtreinish: checking23:06
clarkbfungi: oh perfect that is the one23:07
clarkbI guess tomorrow morning then?23:07
fungifine by me23:07
fungimtreinish: headers (libwebsockets-dev) but no separate debugging symbols that i can find23:08
fungiinstalling libwebsockets-dev doesn't seem to change the gdb bt23:09
*** dizquierdo has quit IRC23:09
clarkbalright afk for a bit now to prep dinner before I have to be back at computer for tc office hours23:09
*** yamamoto has joined #openstack-infra23:10
mtreinishfungi: hmm, ok. I don't know then. I'm not sure it'll matter23:10
fungimtreinish: i don't see anything in the archive which looks like it might provide stripped debugging symbols corresponding to ubuntu's libwebsockets7 build anyway23:11
fungibut regardless, perhaps that last bt might still prove useful for the gh issue you're updating23:12
mtreinishyeah, I'm skimming through the call path in mosquitto right now just to see if I find anything obvious. I'll update the issue in a minute with the bt23:13
*** felipemonteiro__ has quit IRC23:16
*** yamamoto has quit IRC23:16
fungii've preserved a copy of the coredump, so if they want any additional details i'm happy to dig them out23:18
*** dhill_ has quit IRC23:18
*** caphrim007_ has joined #openstack-infra23:22
*** caphrim007_ has quit IRC23:23
*** dizquierdo has joined #openstack-infra23:25
*** rlandy has quit IRC23:25
openstackgerritDavid Ames proposed openstack-infra/project-config master: Add Neutron Dynamic Routing charm and BGP interface  https://review.openstack.org/56407423:25
*** caphrim007 has quit IRC23:25
openstackgerritDavid Ames proposed openstack-infra/project-config master: Add Neutron Dynamic Routing charm and BGP interface  https://review.openstack.org/56407423:27
openstackgerritMerged openstack-infra/zuul master: Pin node to version 8: LTS stable  https://review.openstack.org/56407323:27
mtreinishfungi: https://github.com/eclipse/mosquitto/issues/278#issuecomment-38411149223:28
fungithe saga continues23:30
mtreinishfungi: heh, I couldn't believe it's been over a year and half since we first started having this issue23:31
mtreinishit feels like it wasn't that long ago we first set firehose up23:32
*** klimber has joined #openstack-infra23:32
fungifeels like it wasn't that long ago i fell into the openstack community, and they were just setting up their own foundation23:32
fungitime is accelerating23:32
*** hongbin_ has quit IRC23:33
*** tpsilva has quit IRC23:36
*** dizquierdo has quit IRC23:39
*** stakeda has joined #openstack-infra23:49
fungithere goes my evening. happened across a dusty box containing a couple dozen very early backissues (circa '81-'82) of "80 microcomputing" https://en.wikipedia.org/wiki/80_Micro23:51
fungifor the modern equivalent of us$250 (after inflation) you too could install a serial port card to which to connect your 110 baud modem23:52
*** klimber has quit IRC23:53
fungialso all the best ways to make use of a 4-color palette from a 16-color display23:53
fungichoose your bits wisely!23:53
fungithe january '81 issue includes assembly code for a terminal emulator23:56
dhellmannfungi : thanks for the ping; it looks like the release notes jobs for those repos ran fine this time around, bnemec23:57
*** tosky has quit IRC23:58
*** yamamoto has joined #openstack-infra23:58
fungi"... The program, as it stands, is written for a 48K disk system (TRSDOS 2.3 values assumed), but can be run on a 16K Level II system, as long as a few addresses are changed. ..."23:58
fungithat takes me back23:58
mtreinishfungi: you should try to run it in a z80 emulator :p23:58
fungiif only i felt like typing in some 5 pages full of assembly23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!