Tuesday, 2018-01-02

*** andreww has joined #openstack-infra00:00
*** xarses_ has quit IRC00:03
*** lukebrowning has joined #openstack-infra00:19
*** lukebrowning has quit IRC00:23
*** lukebrowning has joined #openstack-infra00:26
*** markvoelker has joined #openstack-infra00:27
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Slack driver  https://review.openstack.org/53052100:29
*** lukebrowning has quit IRC00:31
*** markvoelker has quit IRC00:31
*** lukebrowning has joined #openstack-infra00:33
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Slack driver  https://review.openstack.org/53052100:36
*** lukebrowning has quit IRC00:38
*** lukebrowning has joined #openstack-infra00:39
*** zhurong has joined #openstack-infra00:43
*** lukebrowning has quit IRC00:44
*** lukebrowning has joined #openstack-infra00:45
*** lukebrowning has quit IRC00:50
*** lukebrowning has joined #openstack-infra00:52
*** lukebrowning has quit IRC00:57
*** lukebrowning has joined #openstack-infra00:58
*** namnh has joined #openstack-infra01:03
*** lukebrowning has quit IRC01:03
*** kiennt26 has joined #openstack-infra01:08
*** lukebrowning has joined #openstack-infra01:09
*** lukebrowning_ has joined #openstack-infra01:13
*** daidv has joined #openstack-infra01:13
*** lukebrowning has quit IRC01:13
*** cuongnv has joined #openstack-infra01:15
*** heyongli has joined #openstack-infra01:17
*** lukebrowning_ has quit IRC01:20
*** ijw has quit IRC01:21
*** kiennt26 has quit IRC01:21
*** lukebrowning has joined #openstack-infra01:22
*** cshastri has joined #openstack-infra01:23
*** lukebrowning has quit IRC01:26
*** heyongli has quit IRC01:27
*** heyongli has joined #openstack-infra01:27
*** markvoelker has joined #openstack-infra01:28
*** xinliang has joined #openstack-infra01:29
*** markvoelker has quit IRC01:32
*** lukebrowning has joined #openstack-infra01:33
*** numans has quit IRC01:35
*** heyongli has quit IRC01:37
*** numans has joined #openstack-infra01:37
*** heyongli has joined #openstack-infra01:37
*** lukebrowning has quit IRC01:37
*** lukebrowning has joined #openstack-infra01:39
openstackgerritTristan Cacqueray proposed openstack-infra/zuul-jobs master: DNM: negative test  https://review.openstack.org/52243801:43
*** lukebrowning has quit IRC01:44
*** niska has quit IRC01:45
*** heyongli has quit IRC01:47
openstackgerritTristan Cacqueray proposed openstack-infra/zuul-jobs master: DNM: negative test  https://review.openstack.org/52243801:47
*** heyongli has joined #openstack-infra01:47
*** niska has joined #openstack-infra01:50
*** lukebrowning has joined #openstack-infra01:52
openstackgerritTristan Cacqueray proposed openstack-infra/zuul-jobs master: unittests: make bindep role optional  https://review.openstack.org/52225501:54
*** lukebrowning has quit IRC01:57
*** heyongli has quit IRC01:57
*** heyongli has joined #openstack-infra01:58
*** chrisyang_0660 has joined #openstack-infra02:06
*** gcb has joined #openstack-infra02:06
*** heyongli has quit IRC02:08
*** heyongli has joined #openstack-infra02:08
*** zhenguo has joined #openstack-infra02:14
*** heyongli has quit IRC02:18
*** heyongli has joined #openstack-infra02:18
*** ijw has joined #openstack-infra02:21
*** pbourke_ has quit IRC02:22
*** lukebrowning has joined #openstack-infra02:22
*** pbourke_ has joined #openstack-infra02:24
*** ijw has quit IRC02:27
*** lukebrowning has quit IRC02:27
*** heyongli has quit IRC02:28
*** heyongli has joined #openstack-infra02:28
*** lukebrowning has joined #openstack-infra02:34
*** liujiong has joined #openstack-infra02:38
*** heyongli has quit IRC02:38
*** heyongli has joined #openstack-infra02:39
*** lukebrowning has quit IRC02:42
*** lukebrowning has joined #openstack-infra02:43
*** edmondsw has joined #openstack-infra02:45
*** lukebrowning has quit IRC02:48
*** heyongli has quit IRC02:49
*** heyongli has joined #openstack-infra02:49
*** lukebrowning has joined #openstack-infra02:50
*** kiennt26 has joined #openstack-infra02:51
*** lukebrowning has quit IRC02:54
*** ijw has joined #openstack-infra02:59
*** heyongli has quit IRC02:59
*** heyongli has joined #openstack-infra02:59
*** lukebrowning has joined #openstack-infra03:00
*** liujiong has quit IRC03:00
*** liujiong has joined #openstack-infra03:01
*** edmondsw has quit IRC03:03
*** lukebrowning has quit IRC03:05
*** bobh has joined #openstack-infra03:05
*** edmondsw has joined #openstack-infra03:05
*** edmondsw has quit IRC03:06
*** bobh has quit IRC03:06
*** heyongli has quit IRC03:09
*** heyongli has joined #openstack-infra03:09
*** annp has joined #openstack-infra03:14
*** liujiong has quit IRC03:17
*** liujiong has joined #openstack-infra03:18
*** heyongli has quit IRC03:19
*** heyongli has joined #openstack-infra03:20
*** yamamoto has quit IRC03:27
*** markvoelker has joined #openstack-infra03:29
*** heyongli has quit IRC03:30
*** heyongli has joined #openstack-infra03:30
*** xarses has joined #openstack-infra03:32
*** andreww has quit IRC03:33
*** links has joined #openstack-infra03:34
*** markvoelker has quit IRC03:34
*** ijw has quit IRC03:38
*** lukebrowning has joined #openstack-infra03:39
*** heyongli has quit IRC03:40
*** heyongli has joined #openstack-infra03:40
*** lukebrowning has quit IRC03:44
jbrycefrickler, clarkb We've sent it to our trademark lawyer to advise if it's a real thing or not. Thanks for the heads up!03:44
*** lukebrowning has joined #openstack-infra03:45
*** lukebrowning has quit IRC03:50
*** heyongli has quit IRC03:50
*** heyongli has joined #openstack-infra03:50
*** dave-mccowan has quit IRC03:51
*** lukebrowning has joined #openstack-infra03:52
*** coolsvap has joined #openstack-infra03:53
*** lukebrowning has quit IRC03:56
*** lukebrowning has joined #openstack-infra03:58
*** heyongli has quit IRC04:00
*** heyongli has joined #openstack-infra04:01
*** yamamoto has joined #openstack-infra04:01
*** lukebrowning has quit IRC04:03
*** lukebrowning has joined #openstack-infra04:04
*** ramishra has joined #openstack-infra04:08
*** lukebrowning has quit IRC04:09
*** heyongli has quit IRC04:11
*** lukebrowning has joined #openstack-infra04:11
*** heyongli has joined #openstack-infra04:11
*** vivsoni has quit IRC04:11
*** lukebrowning has quit IRC04:16
*** lukebrowning has joined #openstack-infra04:17
*** yamamoto has quit IRC04:17
*** heyongli has quit IRC04:21
*** heyongli has joined #openstack-infra04:21
*** lukebrowning has quit IRC04:22
*** vivsoni has joined #openstack-infra04:23
*** markvoelker has joined #openstack-infra04:30
*** heyongli has quit IRC04:31
*** heyongli has joined #openstack-infra04:31
*** markvoelker has quit IRC04:35
*** edmondsw has joined #openstack-infra04:35
*** yamamoto has joined #openstack-infra04:36
*** nicolasbock has quit IRC04:38
*** ijw has joined #openstack-infra04:39
*** edmondsw has quit IRC04:40
*** yamamoto has quit IRC04:41
*** heyongli has quit IRC04:41
*** heyongli has joined #openstack-infra04:42
*** ykarel has joined #openstack-infra04:42
*** udesale has joined #openstack-infra04:43
*** lukebrowning has joined #openstack-infra04:49
openstackgerritTristan Cacqueray proposed openstack-infra/zuul-jobs master: Add buildset-artifacts-location role  https://review.openstack.org/53067904:49
*** heyongli has quit IRC04:52
*** heyongli has joined #openstack-infra04:52
*** ijw has quit IRC04:55
*** lukebrowning has quit IRC04:55
*** lukebrowning has joined #openstack-infra04:57
*** lukebrowning has quit IRC05:02
*** heyongli has quit IRC05:02
*** heyongli has joined #openstack-infra05:02
*** lukebrowning has joined #openstack-infra05:03
*** lukebrowning has quit IRC05:08
*** heyongli has quit IRC05:12
*** heyongli has joined #openstack-infra05:12
*** lukebrowning has joined #openstack-infra05:20
*** heyongli has quit IRC05:22
*** heyongli has joined #openstack-infra05:23
*** zhurong has quit IRC05:23
*** andrewbogott has quit IRC05:24
*** andrewbogott has joined #openstack-infra05:24
*** lukebrowning has quit IRC05:25
*** bhavik1 has joined #openstack-infra05:31
*** psachin has joined #openstack-infra05:31
*** heyongli has quit IRC05:33
*** heyongli has joined #openstack-infra05:33
*** bhavik1 has quit IRC05:34
*** heyongli has quit IRC05:43
*** heyongli has joined #openstack-infra05:43
*** yamamoto has joined #openstack-infra05:44
*** lukebrowning has joined #openstack-infra05:47
*** yamamoto has quit IRC05:50
*** hongbin has joined #openstack-infra05:52
*** lukebrowning has quit IRC05:52
*** heyongli has quit IRC05:53
*** heyongli has joined #openstack-infra05:53
*** lukebrowning has joined #openstack-infra05:54
openstackgerritTristan Cacqueray proposed openstack-infra/zuul-jobs master: Add linter job and roles  https://review.openstack.org/53068205:55
*** ijw has joined #openstack-infra05:55
*** pgadiya has joined #openstack-infra05:56
*** lukebrowning has quit IRC05:59
*** lukebrowning has joined #openstack-infra06:01
*** heyongli has quit IRC06:03
*** heyongli has joined #openstack-infra06:04
*** lukebrowning_ has joined #openstack-infra06:04
*** lukebrowning has quit IRC06:06
*** jbadiapa has joined #openstack-infra06:07
*** lukebrowning_ has quit IRC06:09
*** lukebrowning has joined #openstack-infra06:10
*** heyongli has quit IRC06:14
*** heyongli has joined #openstack-infra06:14
*** lukebrowning has quit IRC06:15
openstackgerritTristan Cacqueray proposed openstack-infra/zuul-jobs master: Add buildset-artifacts-location role  https://review.openstack.org/53067906:20
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Slack driver  https://review.openstack.org/53052106:21
*** liujiong has quit IRC06:22
*** edmondsw has joined #openstack-infra06:23
*** heyongli has quit IRC06:24
*** heyongli has joined #openstack-infra06:24
*** jbadiapa has quit IRC06:26
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Slack driver  https://review.openstack.org/53052106:27
*** edmondsw has quit IRC06:28
*** armaan has quit IRC06:31
*** armaan has joined #openstack-infra06:31
*** jbadiapa has joined #openstack-infra06:31
openstackgerritTristan Cacqueray proposed openstack-infra/zuul-jobs master: Add buildset-artifacts-location role  https://review.openstack.org/53067906:32
*** heyongli has quit IRC06:34
*** lukebrowning has joined #openstack-infra06:34
*** heyongli has joined #openstack-infra06:34
*** e0ne has joined #openstack-infra06:36
*** e0ne has quit IRC06:36
*** lukebrowning has quit IRC06:38
*** lukebrowning has joined #openstack-infra06:40
openstackgerritTristan Cacqueray proposed openstack-infra/zuul-jobs master: Add linter job and role  https://review.openstack.org/53068206:41
*** ijw has quit IRC06:41
openstackgerritTristan Cacqueray proposed openstack-infra/zuul-jobs master: Fix pep8/yamllint errors  https://review.openstack.org/53068506:43
*** dhajare has joined #openstack-infra06:43
*** heyongli has quit IRC06:44
*** heyongli has joined #openstack-infra06:45
*** lukebrowning has quit IRC06:45
*** lukebrowning has joined #openstack-infra06:47
*** lukebrowning has quit IRC06:51
*** lukebrowning has joined #openstack-infra06:53
*** heyongli has quit IRC06:55
*** heyongli has joined #openstack-infra06:55
*** lukebrowning has quit IRC06:58
*** lukebrowning has joined #openstack-infra06:59
*** gmann__ has joined #openstack-infra07:00
*** lukebrowning has quit IRC07:04
*** dsariel has quit IRC07:05
*** heyongli has quit IRC07:05
*** heyongli has joined #openstack-infra07:05
*** jaosorior has joined #openstack-infra07:07
*** lukebrowning has joined #openstack-infra07:07
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove legacy jobs in Cyborg  https://review.openstack.org/51108707:09
openstackgerritMerged openstack-infra/project-config master: Remove legacy jobs in Cyborg  https://review.openstack.org/51108807:10
*** lukebrowning has quit IRC07:13
*** lukebrowning has joined #openstack-infra07:14
*** heyongli has quit IRC07:15
*** heyongli has joined #openstack-infra07:15
*** lukebrowning has quit IRC07:19
*** lukebrowning has joined #openstack-infra07:20
*** gmann__ is now known as gmann07:23
*** pcaruana has joined #openstack-infra07:24
*** lukebrowning has quit IRC07:25
*** pcaruana has quit IRC07:25
*** heyongli has quit IRC07:25
*** pcaruana has joined #openstack-infra07:26
*** heyongli has joined #openstack-infra07:26
*** lukebrowning has joined #openstack-infra07:26
*** pcaruana has quit IRC07:29
*** pcaruana has joined #openstack-infra07:29
*** lukebrowning has quit IRC07:31
*** markvoelker has joined #openstack-infra07:32
*** heyongli has quit IRC07:36
*** heyongli has joined #openstack-infra07:36
*** slaweq has joined #openstack-infra07:38
*** ijw has joined #openstack-infra07:42
*** pgadiya has quit IRC07:44
*** lukebrowning has joined #openstack-infra07:44
*** heyongli has quit IRC07:46
*** heyongli has joined #openstack-infra07:46
*** tosky has joined #openstack-infra07:48
*** lukebrowning has quit IRC07:49
*** lukebrowning has joined #openstack-infra07:51
*** e0ne has joined #openstack-infra07:51
*** larainema has quit IRC07:53
*** hongbin has quit IRC07:54
*** lukebrowning has quit IRC07:56
*** heyongli has quit IRC07:56
*** heyongli has joined #openstack-infra07:56
*** lukebrowning has joined #openstack-infra07:57
*** jaosorior has quit IRC07:58
*** jaosorior has joined #openstack-infra08:00
*** lukebrowning has quit IRC08:02
*** lukebrowning has joined #openstack-infra08:03
*** dhajare has quit IRC08:05
*** heyongli has quit IRC08:06
*** heyongli has joined #openstack-infra08:07
*** lukebrowning has quit IRC08:08
*** lukebrowning has joined #openstack-infra08:10
*** jtomasek has joined #openstack-infra08:10
*** edmondsw has joined #openstack-infra08:12
*** jtomasek has quit IRC08:12
*** jtomasek has joined #openstack-infra08:12
*** edmondsw has quit IRC08:16
*** heyongli has quit IRC08:17
*** heyongli has joined #openstack-infra08:17
*** lukebrowning has quit IRC08:19
*** tesseract has joined #openstack-infra08:20
*** armaan has quit IRC08:20
*** lukebrowning has joined #openstack-infra08:21
*** lukebrowning has quit IRC08:25
*** armaan has joined #openstack-infra08:26
*** heyongli has quit IRC08:27
*** heyongli has joined #openstack-infra08:27
*** hashar has joined #openstack-infra08:29
*** ijw has quit IRC08:29
*** ijw has joined #openstack-infra08:30
*** ijw has quit IRC08:31
*** ijw has joined #openstack-infra08:31
*** ykarel has quit IRC08:31
*** ykarel has joined #openstack-infra08:32
*** heyongli has quit IRC08:37
*** heyongli has joined #openstack-infra08:37
*** danpawlik has joined #openstack-infra08:39
*** heyongli has quit IRC08:47
*** heyongli has joined #openstack-infra08:48
*** lukebrowning has joined #openstack-infra08:49
*** lukebrowning has quit IRC08:53
*** lukebrowning has joined #openstack-infra08:57
*** heyongli has quit IRC08:58
*** heyongli has joined #openstack-infra08:58
*** jpich has joined #openstack-infra08:58
*** lukebrowning has quit IRC09:02
*** AJaeger_ has quit IRC09:05
*** heyongli has quit IRC09:08
*** heyongli has joined #openstack-infra09:08
*** lukebrowning has joined #openstack-infra09:09
*** yolanda has quit IRC09:13
*** yolanda has joined #openstack-infra09:16
*** lukebrowning has quit IRC09:18
*** heyongli has quit IRC09:18
*** heyongli has joined #openstack-infra09:18
*** lukebrowning has joined #openstack-infra09:19
*** lukebrowning has quit IRC09:24
*** e0ne has quit IRC09:27
*** e0ne has joined #openstack-infra09:28
*** heyongli has quit IRC09:28
*** heyongli has joined #openstack-infra09:29
*** shardy has joined #openstack-infra09:32
*** lucas-afk is now known as lucasagomes09:33
*** dbecker has joined #openstack-infra09:34
*** heyongli has quit IRC09:39
*** heyongli has joined #openstack-infra09:39
*** heyongli has quit IRC09:49
*** heyongli has joined #openstack-infra09:49
*** lukebrowning has joined #openstack-infra09:55
*** derekh has joined #openstack-infra09:55
*** armaan has quit IRC09:56
*** heyongli has quit IRC09:59
*** lukebrowning has quit IRC09:59
*** heyongli has joined #openstack-infra09:59
*** edmondsw has joined #openstack-infra10:00
*** dsariel has joined #openstack-infra10:00
*** pbourke_ has quit IRC10:02
*** edmondsw has quit IRC10:04
*** sambetts_ is now known as sambetts10:07
*** heyongli has quit IRC10:09
*** heyongli has joined #openstack-infra10:10
*** pbourke has joined #openstack-infra10:10
*** ianychoi_ is now known as ianychoi10:11
*** cuongnv has quit IRC10:13
*** namnh has quit IRC10:14
*** gmann has quit IRC10:19
*** heyongli has quit IRC10:20
*** heyongli has joined #openstack-infra10:20
*** kiennt26 has quit IRC10:21
*** liusheng has quit IRC10:23
*** liusheng has joined #openstack-infra10:23
*** pgadiya has joined #openstack-infra10:26
sambettsHey infra, there seems to have been *a lot* of spam occuring in openstack- channels over the christmas break and its even still happening at the moment in meeting-3 can we add a rule that auto-kick/bans anyone starting a message with â–„â–„â–„â–„â–„â–„â–„â–„â–„ ??10:27
jpichshinji257904 is spamming #openstack-mistral, if someone here has the power to do something about it (?)10:27
toskyjpich, sambetts : ping people on #freenode, they can add global k-line for those users10:27
jpichtosky: Thanks!10:27
*** heyongli has quit IRC10:30
*** heyongli has joined #openstack-infra10:30
*** markvoelker has quit IRC10:32
*** slaweq has quit IRC10:32
*** lukebrowning has joined #openstack-infra10:32
*** ramishra has quit IRC10:35
*** lukebrowning has quit IRC10:37
*** armaan has joined #openstack-infra10:38
*** lukebrowning has joined #openstack-infra10:39
*** armaan_ has joined #openstack-infra10:40
*** armaan has quit IRC10:40
*** heyongli has quit IRC10:40
*** heyongli has joined #openstack-infra10:40
*** lukebrowning has quit IRC10:44
*** lukebrowning has joined #openstack-infra10:45
*** ramishra has joined #openstack-infra10:48
*** lukebrowning has quit IRC10:50
toskysambetts, jpich: was the issue solved?10:50
jpichyep, cheers tosky!10:50
*** heyongli has quit IRC10:50
*** heyongli has joined #openstack-infra10:51
*** slaweq has joined #openstack-infra10:51
*** lukebrowning has joined #openstack-infra10:51
sambettstosky: seems to have been, however blocking by name doesn't seem to work, they just generate a new one and then pop up again, it would be much better if the openstack IRC channel manager blocked any message starting with â–„â–„â–„â–„â–„â–„10:51
*** lukebrowning has quit IRC10:56
*** udesale has quit IRC10:56
*** heyongli has quit IRC11:01
*** heyongli has joined #openstack-infra11:01
*** pgadiya has quit IRC11:01
*** lukebrowning has joined #openstack-infra11:04
*** numans has quit IRC11:06
*** lukebrowning has quit IRC11:08
*** numans has joined #openstack-infra11:09
*** heyongli has quit IRC11:11
*** heyongli has joined #openstack-infra11:11
*** pgadiya has joined #openstack-infra11:15
*** armaan_ has quit IRC11:19
*** rfolco has joined #openstack-infra11:19
*** heyongli has quit IRC11:21
*** heyongli has joined #openstack-infra11:21
openstackgerritFabien Boucher proposed openstack-infra/zuul feature/zuulv3: Make Zuul able to start with a broken config  https://review.openstack.org/52906011:29
*** heyongli has quit IRC11:31
*** heyongli has joined #openstack-infra11:32
*** markvoelker has joined #openstack-infra11:32
*** wolverineav has joined #openstack-infra11:36
*** markvoelker has quit IRC11:37
*** heyongli has quit IRC11:42
*** heyongli has joined #openstack-infra11:42
*** lukebrowning has joined #openstack-infra11:42
*** lukebrowning has quit IRC11:47
*** e0ne has quit IRC11:48
*** lukebrowning has joined #openstack-infra11:48
*** e0ne has joined #openstack-infra11:49
*** heyongli has quit IRC11:52
*** heyongli has joined #openstack-infra11:52
*** lukebrowning has quit IRC11:53
*** lukebrowning has joined #openstack-infra11:55
*** nicolasbock has joined #openstack-infra11:58
*** lukebrowning_ has joined #openstack-infra11:59
*** lukebrowning has quit IRC12:00
*** ykarel has quit IRC12:00
*** annp has quit IRC12:01
*** heyongli has quit IRC12:02
*** heyongli has joined #openstack-infra12:02
*** lukebrowning_ has quit IRC12:03
*** lukebrowning has joined #openstack-infra12:05
*** jkilpatr has joined #openstack-infra12:07
toskysambetts: it's not just the nickname; freenode admins have a bot with rules for k-lining based on various parameters12:08
sambettstosky: ah thats good then12:09
*** ijw has quit IRC12:09
*** lukebrowning has quit IRC12:10
*** heyongli has quit IRC12:12
*** cshastri has quit IRC12:12
*** heyongli has joined #openstack-infra12:13
*** links has quit IRC12:13
*** lukebrowning has joined #openstack-infra12:15
openstackgerritSam Betts proposed openstack-infra/devstack-gate master: Move location of devstack-gate unstack to post-tempest  https://review.openstack.org/53072612:17
*** lukebrowning has quit IRC12:19
*** edmondsw has joined #openstack-infra12:19
*** edmondsw has quit IRC12:20
*** ijw has joined #openstack-infra12:22
*** heyongli has quit IRC12:23
*** heyongli has joined #openstack-infra12:23
*** lukebrowning has joined #openstack-infra12:24
*** lukebrowning has quit IRC12:29
*** lucasagomes is now known as lucas-hugry12:30
*** ykarel has joined #openstack-infra12:30
*** lukebrowning has joined #openstack-infra12:31
*** heyongli has quit IRC12:33
*** heyongli has joined #openstack-infra12:33
*** markvoelker has joined #openstack-infra12:34
*** dave-mccowan has joined #openstack-infra12:35
*** lukebrowning has quit IRC12:35
*** jaypipes has joined #openstack-infra12:37
*** markvoelker has quit IRC12:38
*** lukebrowning has joined #openstack-infra12:38
*** smatzek has joined #openstack-infra12:41
*** lukebrowning has quit IRC12:43
*** heyongli has quit IRC12:43
*** heyongli has joined #openstack-infra12:43
*** lukebrowning has joined #openstack-infra12:45
*** lukebrowning_ has joined #openstack-infra12:49
*** lukebrowning has quit IRC12:50
*** heyongli has quit IRC12:53
*** lukebrowning_ has quit IRC12:53
*** heyongli has joined #openstack-infra12:54
*** lukebrowning has joined #openstack-infra12:55
*** larainema has joined #openstack-infra12:57
*** jcoufal has joined #openstack-infra12:59
*** lukebrowning has quit IRC13:00
*** lukebrowning has joined #openstack-infra13:02
*** heyongli has quit IRC13:04
*** heyongli has joined #openstack-infra13:04
*** lukebrowning has quit IRC13:07
*** rlandy has joined #openstack-infra13:07
*** lukebrowning has joined #openstack-infra13:12
*** heyongli has quit IRC13:14
*** rlandy is now known as rlandy|ruck13:14
*** heyongli has joined #openstack-infra13:14
*** pblaho has joined #openstack-infra13:14
*** coolsvap has quit IRC13:15
*** gmann has joined #openstack-infra13:18
*** dave-mccowan has quit IRC13:21
*** lukebrowning has quit IRC13:21
*** pblaho has quit IRC13:22
*** pgadiya has quit IRC13:23
*** heyongli has quit IRC13:24
*** heyongli has joined #openstack-infra13:24
fungiclarkb: i don't expect to be around for the infra meeting later today. some last-minute travel came up with has me booked through tomorrow now13:25
fungis/with.which/13:25
* fungi is teh typo king13:25
*** ramishra has quit IRC13:26
*** erlon has joined #openstack-infra13:26
*** annp has joined #openstack-infra13:28
*** markvoelker has joined #openstack-infra13:28
erlon@all, guys, anyone else experiencing the glance error on devstack? http://logs.openstack.netapp.com/logs/periodic-internal/periodic-cinder-SVM-cDOT-NFS/20ff876/logs/devstack-early.txt13:34
*** heyongli has quit IRC13:34
*** heyongli has joined #openstack-infra13:35
*** pblaho has joined #openstack-infra13:36
*** udesale has joined #openstack-infra13:36
stephenfinclarkb: When you're about, fancy taking a look at this? https://review.openstack.org/#/c/450918/513:36
fricklerinfra-root: neutron proposes a test that will check connectivity from an instance to the outside world, in that case 8.8.8.8. I think it is a very bad idea to assume third-party hosts to be reachable, but would like your feedback on that. see https://review.openstack.org/53003613:37
*** lucas-hugry is now known as lucasagomes13:38
*** edmondsw has joined #openstack-infra13:39
*** annp has quit IRC13:43
*** edmondsw has quit IRC13:44
*** heyongli has quit IRC13:45
*** heyongli has joined #openstack-infra13:45
*** eharney has joined #openstack-infra13:47
*** pgadiya has joined #openstack-infra13:47
*** pcichy_ has quit IRC13:48
*** rhallisey has joined #openstack-infra13:49
*** trown|outtypewww is now known as trown|rover13:51
*** annp has joined #openstack-infra13:53
*** heyongli has quit IRC13:55
*** heyongli has joined #openstack-infra13:55
fricklererlon: without access to the logs from glance, that will be impossible to debug.13:55
*** kgiusti has joined #openstack-infra13:56
*** mriedem has joined #openstack-infra13:56
*** bh526r has joined #openstack-infra13:56
erlonfrickler, yeap, but at this stage glance is not up yet, so the only logs we have are the devstack ones13:57
*** mihalis68 has joined #openstack-infra13:59
*** kgiusti has quit IRC14:00
*** kgiusti has joined #openstack-infra14:02
fricklererlon: glance api is installed and started before you get that error, so there would be a log from that, possibly in systemd journal14:03
erlonfrickler, hmm, well that wasnt saved in the job, Ill start on manually and hold the node for checking, just wanted to know if that was a problem that affected uptream CIs14:04
*** heyongli has quit IRC14:05
*** heyongli has joined #openstack-infra14:05
cloudnullmornings all, happy new year!14:09
*** Goneri has joined #openstack-infra14:10
*** pblaho has quit IRC14:11
*** pblaho has joined #openstack-infra14:11
*** edmondsw has joined #openstack-infra14:14
*** heyongli has quit IRC14:15
*** heyongli has joined #openstack-infra14:16
*** lukebrowning has joined #openstack-infra14:17
*** kgiusti has quit IRC14:20
*** numans has quit IRC14:22
*** lukebrowning has quit IRC14:22
*** numans has joined #openstack-infra14:23
*** pgadiya has quit IRC14:24
*** heyongli has quit IRC14:26
*** heyongli has joined #openstack-infra14:26
*** lukebrowning has joined #openstack-infra14:28
*** lukebrowning has quit IRC14:32
*** heyongli has quit IRC14:36
*** heyongli has joined #openstack-infra14:36
*** psachin has quit IRC14:37
*** esberglu has joined #openstack-infra14:37
*** mdrabe_ is now known as mdrabe14:38
*** kgiusti has joined #openstack-infra14:38
*** bobh has joined #openstack-infra14:39
*** hongbin has joined #openstack-infra14:41
*** lukebrowning has joined #openstack-infra14:42
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Reject mail to openstack-infra-owner  https://review.openstack.org/53074914:44
*** pall is now known as pabelanger14:45
pabelangermorning14:45
*** heyongli has quit IRC14:46
*** heyongli has joined #openstack-infra14:46
*** lukebrowning has quit IRC14:47
pabelangerfrickler: Yah, not sure I would depend on 8.8.8.8 for a test, but that is just me14:48
*** lukebrowning has joined #openstack-infra14:49
*** lyarwood has joined #openstack-infra14:52
corvusinfra-root: if you would please approve 530749 that would give my spam filter a respite.14:53
*** annp has quit IRC14:53
*** lukebrowning has quit IRC14:53
*** heyongli has quit IRC14:56
*** heyongli has joined #openstack-infra14:57
*** dsariel has quit IRC14:57
shardyHi all, happy new year!14:58
shardyCan anyone please help my troubleshoot http://logs.openstack.org/83/530483/1/check/requirements-tox-py27-check-uc/7bc8388/tox/py27-check-uc-1.log14:58
shardyit looks like the pypi mirror is out of date, because it can't find this:14:58
shardyhttps://pypi.python.org/pypi/tripleo-common/8.3.014:59
shardyis some sync job failing for the mirror perhaps?14:59
shardypabelanger: ^^ perhaps you can help when you get some time?14:59
*** rlandy|ruck is now known as rlandy|ruck|mtg15:00
*** dave-mccowan has joined #openstack-infra15:01
pabelangerhttp://mirror.dfw.rax.openstack.org/pypi/last-modified15:02
pabelangerseems to be a little old15:02
pabelangerso it is possible pypi is out of sync15:02
pabelangerlet me get some coffee and have a look15:02
shardypabelanger: ack thanks!15:02
*** heyongli has quit IRC15:07
*** heyongli has joined #openstack-infra15:07
pabelangermirror.pypi               1200000000 1199997050  100%<<       17%    <<WARNING15:07
pabelangerguess we are out of space15:07
lyarwoodAre there any docs covering how to run Zuulv3 jobs locally against a VM? I'm trying and failing to reproduce https://bugs.launchpad.net/tempest/+bug/1739829 that's currently blocking stable/pike for Nova15:09
openstackLaunchpad bug 1739829 in tempest "tempest-full job failing in stable/pike with 404 from keystone during tempest verify-config" [Undecided,Confirmed]15:09
*** Apoorva has joined #openstack-infra15:11
*** lukebrowning has joined #openstack-infra15:11
*** pcichy_ has joined #openstack-infra15:12
pabelangerokay, bumped quota on mirror.pypi15:15
pabelangerchecking logs now15:15
*** lukebrowning has quit IRC15:16
*** heyongli has quit IRC15:17
*** heyongli has joined #openstack-infra15:17
*** cshastri has joined #openstack-infra15:20
pabelangerokay, bandersnatch running again15:21
pabelangerhopefully it will complete properly with enough space15:22
*** dsariel has joined #openstack-infra15:23
*** cshastri has quit IRC15:26
*** heyongli has quit IRC15:27
pabelangerHmm, I think we might need to re-initialize bandersnatch, I only see a single package that was mirrored15:27
*** heyongli has joined #openstack-infra15:27
corvuspabelanger: let it run again15:28
corvuspabelanger: that may have been it finishing up the previous run (it saves state); it may pick up the new changes on the next run15:28
pabelangercorvus: okay15:28
pabelangercrontab running now15:30
*** lukebrowning has joined #openstack-infra15:32
*** sobersabre has joined #openstack-infra15:37
*** lukebrowning has quit IRC15:37
sobersabrehi, any jenkins job builder devels around ?15:37
*** heyongli has quit IRC15:37
*** gmann has quit IRC15:37
*** heyongli has joined #openstack-infra15:38
*** eharney has quit IRC15:38
*** lukebrowning has joined #openstack-infra15:39
corvussobersabre: there's an #openstack-jjb channel which may be the best place to find them15:41
sobersabrecorvus: oh, ok. I looked on https://wiki.openstack.org/wiki/IRC15:42
sobersabrethanks, corvus15:42
sobersabre:)15:42
*** sobersabre has left #openstack-infra15:42
corvusi'll add it15:43
*** bnemec-xmas is now known as bnemec15:46
*** lukebrowning has quit IRC15:46
*** david-lyle has joined #openstack-infra15:47
*** lukebrowning has joined #openstack-infra15:47
*** heyongli has quit IRC15:48
*** heyongli has joined #openstack-infra15:48
*** mlavalle has joined #openstack-infra15:49
*** baoli has joined #openstack-infra15:49
fricklerlyarwood: IMHO the tempest-full job isn't ready to be voting yet, because it is lacking lots of important log collection, making debugging like this difficult. your best bet for reproducing locally would be manually tracing the ansible job definitions which you can see in the inheritance_path here http://logs.openstack.org/84/529384/1/check/tempest-full/fbefda8/zuul-info/inventory.yaml15:49
fricklerlyarwood: also this is more an issue for #openstack-qa I'd say15:50
*** eharney has joined #openstack-infra15:51
*** baoli has quit IRC15:51
*** lukebrowning has quit IRC15:52
*** lukebrowning has joined #openstack-infra15:54
*** heyongli has quit IRC15:58
*** heyongli has joined #openstack-infra15:58
*** lukebrowning has quit IRC15:59
*** gouthamr has joined #openstack-infra16:00
*** lukebrowning has joined #openstack-infra16:00
*** armax has joined #openstack-infra16:00
*** rlandy|ruck|mtg is now known as rlandy|ruck16:01
lyarwoodfrickler: ack thanks16:04
*** xarses_ has joined #openstack-infra16:04
*** lukebrowning has quit IRC16:05
*** lukebrowning has joined #openstack-infra16:06
*** heyongli has quit IRC16:08
*** heyongli has joined #openstack-infra16:08
*** kmalloc has joined #openstack-infra16:09
*** lukebrowning has quit IRC16:11
*** lukebrowning has joined #openstack-infra16:13
smcginnispabelanger: Do we regularly clean out the bandersnatch mirror? I seem to remember needing to do that once or twice when I ran a local one.16:16
fricklerpabelanger: multiple jobs failures due to issues with mirror.lon1.citycloud.o.o, do you think that could be related to your mirror actions? http://logs.openstack.org/27/527027/3/check/legacy-tempest-dsvm-py35/ff2107e/job-output.txt.gz#_2018-01-02_14_30_11_063608 http://logs.openstack.org/27/527027/3/check/legacy-grenade-dsvm-neutron-multinode/dece1c5/logs/devstack-gate-setup-host.txt16:16
*** pcichy_ has quit IRC16:17
*** slaweq has quit IRC16:17
*** lukebrowning has quit IRC16:18
pabelangersmcginnis: I don't believe so, IIRC we keep a full mirror at all times16:18
*** heyongli has quit IRC16:18
*** heyongli has joined #openstack-infra16:19
*** lukebrowning has joined #openstack-infra16:19
pabelangerfrickler: looks like possible networking issue, we'd have to check and see how often it has been happening16:19
openstackgerritDavid Moreau Simard proposed openstack-infra/system-config master: Setup bash history to contain standard timestamps (RFC3339)  https://review.openstack.org/52942816:19
clarkbpabelanger: frickler I think we have an elastic recheck query for that issue already16:20
pabelangerhowever, bandersnatch is still running, hopefully once we do a full mirror / release we can confirm logs again16:20
clarkbit doesn't always result in a failure though as pip will retry up to 5 times by default now iirc16:20
*** iyamahat has joined #openstack-infra16:20
*** rlandy|ruck is now known as rlandy|ruck|mtg16:21
dmsimardmnaser, clarkb: I'll add a freenode spam (continued) topic to the agenda16:21
dmsimardI was 99% disconnected throughout the holiday period but there was several waves16:21
clarkbya though there isn't much we can do I expect16:21
clarkbfreenode staff seems to have been trying to kline things as they notice it16:22
dmsimardclarkb: what I did on the original spam wave was to +r all channels which enforces the requirement for users to be authenticated against nickserv before joining the channel16:23
*** lukebrowning has quit IRC16:24
dmsimardhowever, there needs to be a better way than http://paste.openstack.org/raw/629176/ (which takes ~3 minutes to run) :D16:24
clarkbpabelanger: frickler http://status.openstack.org/elastic-recheck/#1449136 is the bug tracking it and it does hvae a large spike currently16:24
mnaseri added myself to operators and i would ban things when i get notifications (i have irccloud so it would hit my phone)16:24
*** iyamahat has quit IRC16:24
*** lukebrowning has joined #openstack-infra16:25
clarkbit is happening in more than one cloud as well16:26
dmsimardjust pyyaml &?16:26
pabelangerclarkb: frickler: okay, I can dig more shortly, bandersnatch is still running but need to relocate to coffee shop16:26
clarkbdmsimard: pyyaml tripleo-common pycrypto devstack-tools I see after a quick skim16:27
clarkblikely just instals that happen early in jobs are common16:27
corvusi just got a random ssh connectivity failure on a job in rax-ord16:27
*** heyongli has quit IRC16:29
fricklerpabelanger: clarkb: o.k., looks like I just had bad luck with my patch, then. waiting for the next recheck results now16:29
*** heyongli has joined #openstack-infra16:29
corvusclarkb, pabelanger, Shrews: https://review.openstack.org/529705 and child will help us get the zuul finger gateway running16:30
clarkbfrickler: well it does look like somethin is going on, possibly related to the bandersnatch update but lso possibly something else16:30
*** lukebrowning has quit IRC16:30
*** ijw has quit IRC16:30
pabelangeryah, we have about ~10days worth of data to mirror from pypi16:31
pabelangerokay, relocating now16:31
Shrewscorvus: oh neat! i forgot that was a thing16:31
*** iyamahat has joined #openstack-infra16:31
*** lukebrowning has joined #openstack-infra16:32
corvusShrews: yeah... i can't remember who wrote that.... ;)16:32
*** dsariel has quit IRC16:32
Shrewsi was lucky to remember my laptop password this morning16:32
mordredShrews: what's a laptop?16:33
*** ijw has joined #openstack-infra16:33
Shrewsmordred: that thing i stole from you that one time16:33
Shrews(oops)16:33
mordredShrews: oh sweet. so I don't have one and can just go back to bed?16:33
*** dsariel has joined #openstack-infra16:34
dmsimardcorvus: new handle for the new year ?16:34
corvusmordred: you're not in bed?16:34
mordredcorvus: I'm not?16:34
clarkbcorvus: question on https://review.openstack.org/#/c/529704/2 but not worthy of a -116:34
corvusdmsimard: i heard not everyone was familiar with it :)16:34
dmsimardbah16:35
dmsimardso it's my fault then16:35
corvusclarkb: 529874 and 529875 lgtm16:35
dmsimard:D16:35
clarkbcorvus: thanks, I figure if I can get those changes in today I will get the existing host updated then deploy new host with less of the old config ever touching it16:35
*** lukebrowning has quit IRC16:36
fricklerclarkb: pabelanger: all the recent matches on http://status.openstack.org/elastic-recheck/#1449136 seem to be located in ovh-* now, saying "No matching distribution found for PyYAML" as opposed to the triple-o thing earlier16:37
corvusclarkb: i think the logging stuff is all copy pasta from the other zuul services.  however, does the python time logger compress?16:37
* frickler moves home now, not sure yet whether I'll make it to the meeting time later16:37
clarkbcorvus: looks like no compression, so probably worth using logrotate16:38
*** dsariel has quit IRC16:38
*** heyongli has quit IRC16:39
*** heyongli has joined #openstack-infra16:39
*** lukebrowning has joined #openstack-infra16:40
*** pcaruana has quit IRC16:41
*** ijw has quit IRC16:44
*** lukebrowning has quit IRC16:44
*** Apoorva has quit IRC16:44
*** lukebrowning has joined #openstack-infra16:46
*** bh526r has quit IRC16:46
*** heyongli has quit IRC16:49
*** heyongli has joined #openstack-infra16:49
*** lukebrowning has quit IRC16:50
*** lukebrowning has joined #openstack-infra16:52
*** mgoddard has joined #openstack-infra16:52
mgoddardHello infra team, I would like to create an IRC channel for the kayobe project (https://github.com/stackhpc/kayobe)16:54
corvusmgoddard: see info here: https://docs.openstack.org/infra/system-config/irc.html#channel-requirements16:54
*** udesale has quit IRC16:55
corvusmgoddard: also https://docs.openstack.org/infra/manual/creators.html#configure-gerritbot-to-announce-changes16:55
corvusmgoddard: also https://docs.openstack.org/infra/system-config/irc.html#accessbot16:56
corvussorry that's not all together :|16:56
mgoddardcorvus: great, thanks16:56
*** lukebrowning has quit IRC16:57
*** lukebrowning has joined #openstack-infra16:58
*** heyongli has quit IRC16:59
*** heyongli has joined #openstack-infra17:00
*** armaan has joined #openstack-infra17:00
*** gyee has joined #openstack-infra17:01
*** lukebrowning has quit IRC17:03
*** lukebrowning has joined #openstack-infra17:05
*** rlandy|ruck|mtg is now known as rlandy|ruck17:07
*** mihalis68 has quit IRC17:08
*** heyongli has quit IRC17:10
*** heyongli has joined #openstack-infra17:10
*** lukebrowning has quit IRC17:14
*** lukebrowning has joined #openstack-infra17:16
*** d0ugal has quit IRC17:19
*** d0ugal has joined #openstack-infra17:20
*** heyongli has quit IRC17:20
*** heyongli has joined #openstack-infra17:20
*** lukebrowning has quit IRC17:20
openstackgerritMark Goddard proposed openstack-infra/project-config master: Add #openstack-kayobe to accessbot channel list  https://review.openstack.org/53077817:21
*** lukebrowning has joined #openstack-infra17:22
*** jpich has quit IRC17:23
*** lucasagomes is now known as lucas-afk17:25
*** ijw has joined #openstack-infra17:26
*** lukebrowning has quit IRC17:27
*** lukebrowning has joined #openstack-infra17:28
*** heyongli has quit IRC17:30
*** heyongli has joined #openstack-infra17:30
*** iyamahat has quit IRC17:32
*** iyamahat has joined #openstack-infra17:33
*** ykarel has quit IRC17:33
*** iyamahat has quit IRC17:33
*** lukebrowning has quit IRC17:34
*** e0ne has quit IRC17:35
*** iyamahat has joined #openstack-infra17:35
*** iyamahat has quit IRC17:35
*** iyamahat has joined #openstack-infra17:36
*** jascott1 has joined #openstack-infra17:38
*** Apoorva has joined #openstack-infra17:39
*** caphrim007 has joined #openstack-infra17:39
*** heyongli has quit IRC17:40
*** heyongli has joined #openstack-infra17:41
*** armaan has quit IRC17:41
*** armaan has joined #openstack-infra17:43
*** AJaeger has joined #openstack-infra17:43
*** armaan has quit IRC17:43
AJaegerHappy new year!17:43
clarkbAJaeger: to you as well17:44
AJaegerthanks, clarkb !17:44
*** lukebrowning has joined #openstack-infra17:45
*** armaan has joined #openstack-infra17:45
AJaegerconfig-core, https://etherpad.openstack.org/p/Nvt3ovbn5x contains a list of reviews ready to merge - nothing critical, just a larger backlog. Reviews are welcome.17:46
*** trown|rover is now known as trown|lunch17:48
*** caphrim007 has quit IRC17:49
*** caphrim007 has joined #openstack-infra17:49
*** caphrim007 has quit IRC17:50
*** armaan has quit IRC17:51
*** heyongli has quit IRC17:51
*** heyongli has joined #openstack-infra17:51
*** armaan has joined #openstack-infra17:51
*** caphrim007 has joined #openstack-infra17:51
*** catintheroof has joined #openstack-infra17:51
clarkbAJaeger: thank you for putting that together, gives me an easy thing t ostart the year on now that I have caffeine :)17:51
smcginnisThere was a problem with the tempest-full job on stable branches that I think someone was working on. Anyone know if that is fixed?17:52
clarkbandreaf: know the answer to ^17:52
smcginnisI guess I can just try a recheck and find out.17:53
openstackgerritMerged openstack-infra/puppet-zuul master: Add support for zuul-fingergw  https://review.openstack.org/52970417:53
openstackgerritMerged openstack-infra/system-config master: Add finger gateway to zuulv3  https://review.openstack.org/52970517:53
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Remove ensure-reno role  https://review.openstack.org/52114317:54
*** lukebrowning has quit IRC17:54
*** armaan has quit IRC17:55
AJaegerclarkb: thanks for reviewing!17:56
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Remove legacy jobs in storlets  https://review.openstack.org/51295817:56
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Remove legacy jobs in Murano  https://review.openstack.org/51143817:59
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Remove legacy novaclient jobs  https://review.openstack.org/52210217:59
*** derekh has quit IRC17:59
*** armaan has joined #openstack-infra17:59
*** heyongli has quit IRC18:01
*** heyongli has joined #openstack-infra18:01
*** lukebrowning has joined #openstack-infra18:03
*** lamt` has joined #openstack-infra18:04
openstackgerritClark Boylan proposed openstack-infra/system-config master: Treat mail sent to superuser-owner as spam  https://review.openstack.org/53078518:04
clarkbinfra-root corvus ^ similar to an earlier change today to black hole list owner spam18:05
lamt`o/ I have a question regarding the doc publication. It looks like the latest update for some of the doc was Dec 21, even though newer doc changes were merged in after (and are not reflected): e.g. https://docs.openstack.org/openstack-helm/latest/18:07
*** armaan has quit IRC18:07
clarkblamt`: have an example change that merged more recently that should've updated the docs? we should be able to work back from that to find logs and figure out what is going on18:08
lamt`clarkb, thanks lemme find a patch set18:08
*** lukebrowning has quit IRC18:09
lamt`clarkb, https://review.openstack.org/#/c/530468/ - merged on Dec 30.18:09
*** lukebrowning has joined #openstack-infra18:10
*** heyongli has quit IRC18:11
*** heyongli has joined #openstack-infra18:11
*** harlowja has joined #openstack-infra18:12
clarkblamt`: ok logs show the job ran in what it thinks was a successful manner18:12
*** mgoddard has quit IRC18:12
clarkblamt`: http://logs.openstack.org/6c/6c27dd852cb92a4d8a67b9b8d2f356f72dbcdd9a/post/publish-openstack-sphinx-docs/5c5437e/ ( found that via the zuul status page's builds list and added openstack/openstack-helm as the project and post as the pipeline filter)18:13
lamt`the page linked from gerrit looks updated, but docs.o.o is not18:14
AJaegerclarkb: seems we have a general problem with docs.o.o - look at http://files.openstack.org/docs/?C=M;O=D18:14
AJaegerlast updated 23rd of December ;(18:15
*** lukebrowning has quit IRC18:15
clarkblamt`: the page linked from gerrit is generated by check and gate builds and is only published to the ci logs server not the actual afs production server18:15
AJaegermordred: is there a problem with the pti-updated changes? We merged some around that time ^18:15
AJaegerinteresting, this is wrong: http://files.openstack.org/docs/python-tripleoclient/html/18:17
openstackgerritClark Boylan proposed openstack-infra/system-config master: Treat mail sent to foundation-board-owner as spam  https://review.openstack.org/53078818:17
clarkbanother list owner spam blackhole request ^18:17
clarkbAJaeger: supposed to be just latest/ not html/latest?18:18
AJaegerclarkb: indeed, should not be http://files.openstack.org/docs/tempest/html/latest/ but http://files.openstack.org/docs/tempest/latest/18:18
clarkbhelm at least does not have an html/latest18:18
AJaegerbut no publishing *after* the 23rd anywhere that I checked18:18
*** sambetts is now known as sambetts|afk18:19
AJaegerclarkb: any zuul log files for that helm change?18:20
AJaegerI mean: Debug log from zuul?18:20
clarkbAJaeger: I can check in a sec18:21
*** heyongli has quit IRC18:21
*** heyongli has joined #openstack-infra18:22
*** lukebrowning has joined #openstack-infra18:22
AJaegerhttps://review.openstack.org/#/c/525734/ was the change that merged last on this.18:23
openstackgerritJames E. Blair proposed openstack-infra/nodepool feature/zuulv3: WIP: Convert from legacy to native devstack job  https://review.openstack.org/52066418:25
*** lukebrowning has quit IRC18:27
openstackgerritJames E. Blair proposed openstack-infra/nodepool feature/zuulv3: WIP: Convert from legacy to native devstack job  https://review.openstack.org/52066418:27
lamt`brb, thanks for looking into it.  From spotchecking docs from other projects last week, it looks like nothing got published past 12/22-23.18:28
clarkbAJaeger: not seeing anything weird in the executor debug log for that job18:28
*** lukebrowning has joined #openstack-infra18:29
AJaegerclarkb, mordred, so I think the extra html/ got introduced by 525734 - but I don't know why we stop publishing on the 23rd. 525734 merged on the 21st already18:29
AJaegerclarkb: is AFS ok?18:29
clarkbAJaeger: that was the next thing I was going t ocheck18:29
clarkbpabelanger: was it you that updated files01 to files02? we still have a files01 dns record fwiw18:29
pabelangerclarkb: yah, I need to check my notes, I am not sure I deleted files01 yet18:30
pabelangerhttps://review.openstack.org/527519/ and https://review.openstack.org/528135/ are related reviews18:31
*** heyongli has quit IRC18:32
*** heyongli has joined #openstack-infra18:32
*** mgoddard has joined #openstack-infra18:32
corvuspabelanger: fwiw, filesNN is intended to support HA -- we can run more than one and load-balance/failover them if we want.18:32
corvus(i'm just reacting to the commit message in 527519)18:32
AJaegerfound out the reason for extra html/ - not sure yet how to fix best ;/18:33
AJaegeras usual a missing "/" (in this case a removed one)18:33
clarkbAJaeger: http://paste.openstack.org/show/636430/18:33
*** lukebrowning has quit IRC18:33
clarkbAJaeger: so its publishing to html/ for helm too18:33
clarkbthat makes me think we hvae two problems, the first is writing to html and the second must be related to afs publishing to ro volume18:33
pabelangerclarkb: okay, cleaning up files01.o.o DNS now, I'll also remove the server. files.o.o has been pointing to files02.o.o for a few weeks now18:34
AJaegerclarkb: agreed with the two unrelated problems. I'm working on html18:34
clarkbpabelanger: well18:34
pabelangercorvus: ack18:34
clarkbpabelanger: docs publishing hasn't worked for a few days18:34
openstackgerritDavid Shrewsbury proposed openstack-infra/system-config master: Open finger port for zuulv3  https://review.openstack.org/53078918:35
*** hemna_ has joined #openstack-infra18:35
clarkbI've got to step out fo ra bit but can poke at the afs side when back if no one beats me to it18:35
pabelangerclarkb: okay, I can look. But believe I confirmed doc publishing to be working once we updated files.o.o to files02.o.o18:36
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: networking-midonet: Remove some of legacy jobs  https://review.openstack.org/52247118:37
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Remove legacy jobs in Cyborg  https://review.openstack.org/51108718:37
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Remove *-js-release-branch/-master/-draft jobs  https://review.openstack.org/52949918:37
pabelanger[Sat Dec 23 08:34:55 2017] afs: Lost contact with volume location server 104.130.136.20 in cell openstack.org (code -1)18:41
pabelangerthat is on ze0118:41
*** heyongli has quit IRC18:42
openstackgerritMerged openstack-infra/system-config master: Reject mail to openstack-infra-owner  https://review.openstack.org/53074918:42
*** heyongli has joined #openstack-infra18:42
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: Fix sphinx publishing  https://review.openstack.org/53079018:44
AJaegerclarkb, mordred, that's the first part of the change, please review and tell me whether that is ok ^18:44
pabelangerclarkb: Yah, I am seeing that message on a most zuul-executors on Dec. 23rd18:44
* AJaeger works on second part now18:44
*** jkilpatr has quit IRC18:46
pabelangerwe also getting OOM killer running on executors too, pretty sure we are aware of that18:46
AJaegerclarkb, mordred, alternative is revert the removal of  sphinx_output_src in 52573418:47
*** trown|lunch is now known as trown18:49
*** trown is now known as trown|rover18:49
*** heyongli has quit IRC18:52
*** heyongli has joined #openstack-infra18:52
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Use sphinx_output_suffix  https://review.openstack.org/53079118:53
*** wolverineav has quit IRC18:53
AJaegerclarkb, mordred, second part ^18:53
*** lukebrowning has joined #openstack-infra18:54
*** wolverineav has joined #openstack-infra18:55
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Use sphinx_output_suffix  https://review.openstack.org/53079118:55
pabelangerAJaeger: that is starting to look pretty complicated^. Why can't we just default to html/ and not create sphinx_output_suffix18:56
AJaegerpabelanger: for check and gate we need "html" to publish to the html subdir ;(18:57
AJaegerbut for post, we don't want it.18:57
pabelangerAJaeger: where is that code again for check / gate?18:58
AJaegerpabelanger: have a look at https://review.openstack.org/#/c/525734/ and the removal of sphinx_output_src18:58
AJaegerpabelanger: it's the same role in zuul-jobs18:58
AJaegerpabelanger: if you or mordred have a better idea - welcome to see that!18:59
*** tesseract has quit IRC18:59
clarkbok back18:59
*** lukebrowning has quit IRC18:59
clarkbpabelanger: while there may have been a blip on the 23rd I don't know that that is the problem since we have writes to the RW volume on the 30th18:59
clarkbpabelanger: I'm wondering if we deleted the host that does the cron to publish to RO volume or something like that as part of the trusty to xenial stuff19:00
* clarkb looks to find where we do that 5 minute cron19:00
pabelangerAJaeger: yah, I'll have to catch up on docs reorg, did know we removed sphinx_output_src19:00
*** jkilpatr has joined #openstack-infra19:00
AJaegerclarkb: note that http://files.openstack.org/docs/openstack-helm/html does not exist while /afs/.openstack.org/docs/openstack-helm/html exists19:01
clarkbAJaeger: ya that is why I think its a vos release problem19:01
AJaegerpabelanger: we can revert the part of 525734 and reintroduce sphinx_output_src for the post jobs.19:01
AJaegerpabelanger: if you prefer that solution, I can do it...19:01
*** heyongli has quit IRC19:02
*** heyongli has joined #openstack-infra19:03
pabelangerclarkb: Hmm, was it on afsdb01.o.o? If so, I am not able to SSH to that server currently19:03
openstackgerritMerged openstack-infra/project-config master: Add neutron-lib-commit group  https://review.openstack.org/52966119:04
clarkbpabelanger: ya site.pp says it is the host with afsrelease included19:04
Shrewsumm, meeting now??19:04
corvusyeah, the auto-release thing is a cron job that only runs on one host19:05
*** lukebrowning has joined #openstack-infra19:05
pabelangerwonder if host was live migrated19:05
clarkbShrews: yes sorry I had to akf then distracted by afs things19:05
*** priteau has joined #openstack-infra19:08
*** lukebrowning has quit IRC19:10
*** lukebrowning has joined #openstack-infra19:11
*** priteau has quit IRC19:11
*** priteau has joined #openstack-infra19:12
*** heyongli has quit IRC19:13
*** heyongli has joined #openstack-infra19:13
*** priteau has quit IRC19:13
*** priteau has joined #openstack-infra19:14
*** lukebrowning has quit IRC19:16
*** wolverineav has quit IRC19:17
*** lukebrowning has joined #openstack-infra19:17
*** wolverineav has joined #openstack-infra19:19
NobodyCamhappy New Year To all the awesome Infra folk!19:21
*** zerick has joined #openstack-infra19:22
*** lukebrowning has quit IRC19:22
ShrewsNobodyCam: you too!19:23
NobodyCam:)19:23
*** heyongli has quit IRC19:23
*** heyongli has joined #openstack-infra19:23
mordredhogepodge: saw a mentoin in the meeting channel - ping me if you have any issues realted to doc PTI things19:24
*** lukebrowning has joined #openstack-infra19:24
AJaegermordred: please read backscroll - and see https://review.openstack.org/530790 and https://review.openstack.org/53079119:24
mordredAJaeger: okie - will do - still catching up post-break19:25
AJaegermordred: alternative: revert part of 525734 -  reintroduce sphinx_output_src for the post jobs.19:25
AJaegerthanks, mordred19:25
*** dbecker has quit IRC19:25
AJaegermordred: and feel free to take over the changes if you have better ideas - I won't be around for much longer today...19:26
mordredAJaeger: those look good on first glance - out of curiosity, why not just remove the html part always?19:26
mordredAJaeger: (I thnk either way is fine, mostly wondering if there is a use-case or aconcern I might be missing)19:27
AJaegermordred: check/gate preview publishing19:27
AJaegermordred: http://logs.openstack.org/90/530790/1/check/build-openstack-sphinx-docs/6d2c3c0/html/ is our location there19:27
AJaegerbut we could reorganize that and publish the *target* to html perhaps?19:28
AJaegerdidn't look at that option19:28
mordredyah - that's what I was starting to think - but I haven't thought about it for long enough to know whether it's a better or worse idea ...19:28
AJaegerneither have I...19:29
*** lukebrowning has quit IRC19:29
*** lukebrowning has joined #openstack-infra19:30
*** heyongli has quit IRC19:33
*** heyongli has joined #openstack-infra19:33
*** lukebrowning has quit IRC19:35
ssbarneado we have any openstack gerrit projects where gates are triggered only after someone adds a label to gerrit? if so, which ones? i want to know the label name used for this case. note that this is not "Code-Review" one.19:37
clarkbdmsimard: talking freenode spam in #openstack-meeting now19:38
dmsimardsaw the meeting notification but got sidetracked, joining19:39
clarkbssbarnea: gate requires a +1 workflow vote19:39
ssbarneaclarkb: ok, i imagined this but failed to find documentation for "Workflow" label.19:40
*** dimak has joined #openstack-infra19:40
clarkbssbarnea: its a label we add to Gerrit. Not sure what documentation you are looking for19:40
ssbarneaclarkb: so "Workflow" meaning is: "looks safe to be tested by CI".19:40
clarkbssbarnea: no, workflow has three states: 0, default. +1 approved means this should go through gate testing and if that psses merge the change. -1 Work in progress (do not merge)19:41
ssbarneaclarkb: is there any relation between Code-Review label in Workflow one? Or they are used parallel? (merge will happen only when Code-Review + Workflow + Verified are present with +2 or +! values)19:43
openstackgerritMerged openstack-infra/system-config master: Treat mail sent to superuser-owner as spam  https://review.openstack.org/53078519:43
clarkbssbarnea: they are used in parallel, correct all of them need to have the positive votes19:43
*** heyongli has quit IRC19:43
*** heyongli has joined #openstack-infra19:44
ssbarneaclarkb: thanks!19:44
*** heyongli has quit IRC19:54
*** heyongli has joined #openstack-infra19:54
*** lukebrowning has joined #openstack-infra19:56
dmsimardcorvus: so it appears that we indeed removed the mlock settings on channels such as #shade (thus breaking the forward). Do we have a list of those channels somewhere ?19:59
corvusdmsimard: not that i'm aware19:59
*** d0ugal has quit IRC20:00
corvusdmsimard: adding mlock support to accessbot might be a way of keeping such a list for the future, however.20:00
*** lukebrowning has quit IRC20:01
corvusdmsimard: if we're lucky, there may be mentions in the infrastructure status log20:01
dmsimarddigging right now20:01
*** heyongli has quit IRC20:04
*** shardy has quit IRC20:04
*** heyongli has joined #openstack-infra20:04
corvusdmsimard: i don't see any there other than shade :(20:05
pabelangeroffline for a bit while I relocate again20:06
*** lukebrowning has joined #openstack-infra20:06
dmsimardI know that, for example, #kolla was renamed to #openstack-kolla and the redirection is still in place .. but #kolla is also no longer in accessbot's list20:07
clarkbI confirm that afsdb01 is not sshable20:07
fricklerthe channel was #openstack-shade and that redirects fine for me20:07
clarkbopenstack says its active20:07
dmsimard#shade is no longer there as well20:07
clarkbI'm goingto check its console really quickly but expect that its a victim of the live migration stuff we've seen fial periodically recently20:08
clarkbcorvus: ^ anything special I should account for if I hard reboot that instnace via the openstack api?20:08
fricklernote the redirect only works if you are not joined to #openstack-sdks already20:08
dmsimardhmmm, yeah #shade seems something entirely different and the registration doesn't show anything openstack related20:08
corvusdmsimard, frickler: oh in that case, maybe we're lucky enough that none of those are in the list now.20:08
corvusclarkb: no... theoretically it should resync with afsdb0220:09
corvusi have to run grab lunch, biab.20:09
*** mgoddard has quit IRC20:10
mordreddmsimard, corvus: perhaps the #shade channel pre-dates the registration system and we never fully migrated it over?20:10
*** larainema has quit IRC20:10
dmsimardmordred: was #shade ever a thing ? It seems to have been registered by "ffmad" more than 3 years ago20:11
*** lukebrowning has quit IRC20:11
clarkbya lots of kernel hung task timeouts on afsdb01 which is what we've seen with the failed live migrations20:11
clarkbhard rebooting it now20:11
clarkbok reboot is done20:13
clarkbit should run the vos release for docs at 2015UTC20:13
dmsimardasking #freenode about general advice how to best manage our channels "at scale", they do mention an interesting bit in that you can enable the chanserv verbose mode to have it echo changes to the operators (channel operators? chanserv operators? not sure)20:13
mordreddmsimard: oh - maybe not - maybe it was just #openstack-shade20:13
*** dbecker has joined #openstack-infra20:13
*** heyongli has quit IRC20:14
dmsimardah, but the help message only mentions it echoes access list changes20:14
*** Goneri has quit IRC20:14
*** heyongli has joined #openstack-infra20:14
mordreddmsimard: if you're starting to get sucked in to hacking on irc things - are you aware of the long-standing intent to rework the bots to be a single errbot bot? (and by single bot, I actually mean scale-out-collection of copies of a single bot, because of channel limits)20:15
dmsimardI discussed that with harlowja a while back since I've used errbot before (and we use it right now for RDO) but I'm not aware of anyone working on that right now20:16
*** slaweq has joined #openstack-infra20:16
mordreddmsimard: oh - definitely nobody is working on it20:16
dmsimardiirc harlowja ended up writing his own bot framework for his stuff20:16
harlowjaya, SpamapS can vouch for it, lol20:16
harlowjai'll prob just opensource it :-P20:16
harlowjaand then people can use it, lol20:16
mordreddmsimard: I just figured I'd let you know that it's been on the todo listfor forever in case you get ragey and wind up wanting to hack on that20:17
harlowjaits better than errbot (though its targeted at slack, not being a driver-based-all-backends crap)20:17
mordredharlowja: oh, well, targetted at slack probably won't help us too much :)20:17
harlowjadown with IRC, lol20:17
harlowjalol20:17
mordredthe driver-based-all-backends was actually one of the reasons we were looking at errbot (so that people running infra clones inside of companies where they used not-irc could re-use stuff we've got)20:18
dmsimardmordred: I'd love to tbh, errbot is great to work with -- here, let me put it on my never-ending list of things I'd like to do :p20:18
harlowjaya, driver crap is driver crap, imho20:18
mordredharlowja: tell me about it20:18
*** d0ugal has joined #openstack-infra20:18
harlowjau get what u pay for with driver LCD crap20:18
harlowjalol20:18
harlowjathat's in part why i said ummm, ya, no thx to errbot20:18
harlowjalol20:18
mordreddmsimard: main 'hurdle' with errbot is that in its current form it wants config to be done via IRC commands, which is obviously not how we want to configure it20:18
dmsimardmordred: yeah.. it's been that way before but they've moved a lot in recent months, maybe worth looking at it again. They don't even support python2 anymore20:19
mordredharlowja: yah - #1 priority on our side is handling IRC *well* - supporting non-IRC is a nice-to-have but certainly not essential or anything20:19
clarkbI mean we're talking a bot that has to write 512 byte messages to message busses20:19
clarkbits not like the LCD is lacking in features20:19
harlowjaso that starts to get into how useful u want a bot to be20:20
mordredooh20:20
mordredhttp://errbot.io/en/latest/user_guide/setup.html#id120:20
openstackgerritMerged openstack-infra/system-config master: Treat mail sent to foundation-board-owner as spam  https://review.openstack.org/53078820:20
dmsimardmordred: that's not new20:20
mordredit's not? poo20:20
dmsimardmordred: you could always use config.py and then have your plugins pull from that (that's what we do for RDO)20:21
dmsimardbut there are certain things that don't work that way20:21
mordredah. gotcha20:21
dmsimardor at least back when I last tried it20:21
dmsimardhttps://github.com/rdo-infra/ansible-role-rdo-bot/blob/master/tasks/setup.yml#L66-L6920:21
clarkbvos release for docs is still going20:21
clarkb(taking more than 5 minutes but we don't have multiple processes)20:21
*** rfolco has quit IRC20:22
pabelangerhttp://git.openstack.org/cgit/openstack-infra/err2d2/ :)20:22
dmsimardmordred: oh, this is new though and fixes exactly our problem: http://errbot.io/en/latest/user_guide/provisioning.html20:22
pabelangerbest repo name20:22
mordreddmsimard: ++20:23
*** dbecker has quit IRC20:24
mordreddmsimard: that seems fairly perfect - we could have our config in yaml and then just write a tiny thing that reads it and pipes it to errbot --storage-set20:24
dmsimardsomething like that20:24
*** heyongli has quit IRC20:24
mordreddmsimard: I mean, we could just store things in python dict format so it's just 'cat foo.py | errbot --storage-set' ... but the other thing seems pretty trivial20:24
jaypipesas long as it doesn't jaypipes it, I'm fine.20:25
*** heyongli has joined #openstack-infra20:25
dmsimarda wild jaypipes has appeared20:25
jaypipes:P20:25
mordredjaypipes: nothing is possible without jaypipesing things20:25
jaypipeslol20:25
* jaypipes goes back to reading about UEFI20:25
dmsimardmordred: you could even *gasp* make an idempotent ansible module that does a get on the config before doing a set so you'd know if it changed20:26
clarkbAJaeger: http://files.openstack.org/docs/openstack-helm/html/ exists now so I think once we fix the job side of things it will be all happy again20:26
mordreddmsimard: ++20:26
*** jkilpatr has quit IRC20:27
clarkband with that I need to figure out lunch plans20:27
mordreddmsimard: so - since that exists now - the largest amount of work, iirc, is porting our meetbot code to be an errbot plugin20:27
dmsimardmeetbot is a debian thing right ?20:27
dmsimardthere's probably no way it can be ported as-is20:28
dmsimardbut we can probably write something similar-ish20:28
mordreddmsimard: well, it started that way, but we have an openstack fork: http://git.openstack.org/cgit/openstack-infra/meetbot20:28
mordreddmsimard: but yah - it'll certainly an amount of porting effort - which may involve just rewriting from scratch based on design, or may be able to use *some* of the code20:29
clarkbmordred: pabelanger AJaeger ok I think the last thing to sort out now is the html/ stuff20:30
*** iyamahat has quit IRC20:30
clarkbthen maybe clean out all the unwatned html/ dirs and tell projects they may need to merge new changes to get updates20:30
mordredclarkb: cool - I've got AJaeger's changes open, will engage with that in just a sec20:31
pabelangerclarkb: yah, I'm going to see if I can get a console on afsdb01.o.o, still not able to SSH to it20:32
pabelangeroh20:33
pabelangerit just worked20:33
pabelanger2018-01-02 20:35:01,351 release INFO     Checking docs20:33
pabelangerjust started again20:33
pabelangerso, somebody restart it?20:33
clarkbpabelanger: ya I did20:33
*** lukebrowning has joined #openstack-infra20:33
pabelanger2017-12-23 02:50:01,753 release DEBUG      afs01.ord.openstack.org /vicepa 2017-12-23 00:25:59 docs.dev.readonly20:33
pabelangerwas last20:33
clarkbI chceked the console first it had the kernel hung task messages20:33
pabelangerkk20:33
pabelangercool20:33
clarkbwhich is what we've seen with the failed live migrations20:34
pabelangeryah, see backscoll now20:34
*** dangers_away has quit IRC20:34
*** heyongli has quit IRC20:35
*** heyongli has joined #openstack-infra20:35
*** yamahata has quit IRC20:35
*** e0ne has joined #openstack-infra20:37
*** dangers_away has joined #openstack-infra20:38
*** erlon has quit IRC20:38
*** lukebrowning has quit IRC20:38
*** lukebrowning has joined #openstack-infra20:41
harlowjamordred dmsimard u guys can steal from https://github.com/harlowja/gerritbot2/blob/master/plugins/gerritbot/gerritbot.py also if u want20:41
harlowjahttps://github.com/harlowja/gerritbot2/blob/master/plugins/gerritbot/gerritbot.py#L203 ya da ya da20:41
*** jkilpatr has joined #openstack-infra20:41
harlowjai really didn't like the whole configuration method of errbot20:41
harlowjait seemed ummmm, sorta ummmm, overdone, lol20:42
*** kgiusti has quit IRC20:42
*** heyongli has quit IRC20:45
*** heyongli has joined #openstack-infra20:45
*** lukebrowning has quit IRC20:45
*** catintheroof has quit IRC20:46
*** catintheroof has joined #openstack-infra20:46
*** lukebrowning has joined #openstack-infra20:47
*** e0ne has quit IRC20:47
*** armax has quit IRC20:49
*** e0ne has joined #openstack-infra20:51
*** catintheroof has quit IRC20:51
*** lukebrowning has quit IRC20:51
*** lukebrowning has joined #openstack-infra20:52
*** smatzek has quit IRC20:54
*** smatzek has joined #openstack-infra20:55
*** smatzek has quit IRC20:55
*** heyongli has quit IRC20:55
*** smatzek has joined #openstack-infra20:55
*** heyongli has joined #openstack-infra20:55
*** lukebrowning has quit IRC20:56
dmsimardharlowja: so it relies on firehose/mqtt then ? is that how the current gerritbot works ? /me has no idea20:57
harlowjai think the current one does now use mqtt yes20:57
dmsimardI figured the bot connected directly to the gerrit event stream20:57
harlowjait used to afaik20:57
harlowjadon't think it does anymore20:57
harlowjassh keys and blah blah were required20:57
*** kgiusti has joined #openstack-infra21:00
*** smatzek has quit IRC21:00
*** jkilpatr has quit IRC21:04
*** jlk` has joined #openstack-infra21:05
openstackgerritAlex Krzos proposed openstack-infra/grafyaml master: Add support to change the default time filter range  https://review.openstack.org/53079821:05
*** heyongli has quit IRC21:05
*** heyongli has joined #openstack-infra21:06
*** jlk has quit IRC21:06
openstackgerritAlex Krzos proposed openstack-infra/grafyaml master: Add legend for graphs  https://review.openstack.org/53079921:09
*** slaweq_ has joined #openstack-infra21:13
*** slaweq_ has quit IRC21:14
*** jlk` is now known as jlk21:15
*** heyongli has quit IRC21:16
*** jlk has quit IRC21:16
*** jlk has joined #openstack-infra21:16
*** heyongli has joined #openstack-infra21:16
*** Swanson has joined #openstack-infra21:18
*** armax has joined #openstack-infra21:19
*** Goneri has joined #openstack-infra21:22
*** markmcclain has quit IRC21:25
*** heyongli has quit IRC21:26
*** heyongli has joined #openstack-infra21:26
*** markmcclain has joined #openstack-infra21:28
*** aeng has joined #openstack-infra21:30
*** jkilpatr has joined #openstack-infra21:33
dmsimardinfra-root: an hopefully simple review to add date/time to our bash history: https://review.openstack.org/#/c/529428/21:33
corvus+2 but leaving for more infra-roots to see21:35
dmsimardyeah, that's the intent21:35
mordredsame here - +2 but leaving for more infra-roots21:36
*** heyongli has quit IRC21:36
*** heyongli has joined #openstack-infra21:36
*** kgiusti has left #openstack-infra21:37
*** eharney has quit IRC21:41
*** jcoufal has quit IRC21:42
*** rcernin has joined #openstack-infra21:45
*** priteau has quit IRC21:45
*** hongbin has quit IRC21:45
*** heyongli has quit IRC21:46
*** dave-mccowan has quit IRC21:47
*** heyongli has joined #openstack-infra21:47
*** wolverineav has quit IRC21:47
*** jaypipes has quit IRC21:51
*** heyongli has quit IRC21:57
*** heyongli has joined #openstack-infra21:57
*** slaweq has quit IRC21:59
*** trown|rover is now known as trown|outtypewww21:59
*** heyongli has quit IRC22:07
*** heyongli has joined #openstack-infra22:07
*** heyongli has quit IRC22:17
*** heyongli has joined #openstack-infra22:17
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: WIP: Support cross-source dependencies  https://review.openstack.org/53080622:21
*** jascott1 has quit IRC22:26
*** jascott1 has joined #openstack-infra22:26
openstackgerritEmilien Macchi proposed openstack-infra/project-config master: Remove legacy beaker jobs from Puppet OpenStack modules  https://review.openstack.org/53080722:26
*** heyongli has quit IRC22:27
*** jascott1 has quit IRC22:28
*** heyongli has joined #openstack-infra22:28
*** jascott1 has joined #openstack-infra22:28
*** bobh has quit IRC22:28
*** jascott1 has quit IRC22:30
*** heyongli has quit IRC22:38
*** heyongli has joined #openstack-infra22:38
*** catintheroof has joined #openstack-infra22:46
clarkblunch always takes twice as long when having it with family22:46
* clarkb pulls up docs fixs22:46
*** heyongli has quit IRC22:48
*** heyongli has joined #openstack-infra22:48
openstackgerritClark Boylan proposed openstack-infra/zuul-jobs master: Fix sphinx publishing  https://review.openstack.org/53079022:49
clarkbminor fix to the docs really quickly since it didn't have any +2s yet22:49
clarkbpabelanger: mordred corvus dmsimard ^ reviews on that and 530791 much appreciated22:49
clarkbdmsimard: corvus ara updates on executors is working now right?22:57
clarkblooks like ze04 is out to lunch. I will check in on its console and reboot it if it has the hung task messages22:58
*** heyongli has quit IRC22:58
*** heyongli has joined #openstack-infra22:58
Swansonretry_limit is one I've never seen before.22:59
clarkbSwanson: means your job failed in the pre/setup phase multiple times (we retry jobs if they fail there until the retry limit)23:00
clarkbSwanson: checking ara is typically a good way to see what is failing23:00
corvusmaybe we should rename that result to 'pre_failure'23:00
*** iyamahat has joined #openstack-infra23:00
Swansonclarkb, ara?23:01
Swansonnevermind23:02
clarkbSwanson: it is part of your job logs on failure, its a nice rendering of the ansible run23:02
clarkbmtreinish: are the subunit2sql issues at the bottom of https://etherpad.openstack.org/p/zuulv3-issues addressed at this point? the two changes listed there merged23:05
clarkbandreaf: ^ you may know23:05
*** hashar has quit IRC23:05
mordredclarkb: ok. both +A'd - I think we could improve on it, but I think unbreaking is more important and we can swing back by and improve later23:05
*** heyongli has quit IRC23:08
*** heyongli has joined #openstack-infra23:09
*** jascott1 has joined #openstack-infra23:12
*** jtomasek has quit IRC23:17
*** yamahata has joined #openstack-infra23:18
clarkbya ze04 exhibits the hung task timeout message on its console going to hard reboot it now23:18
clarkbthe other ze's and zm's seem fine though23:18
*** heyongli has quit IRC23:19
*** edmondsw has quit IRC23:19
*** heyongli has joined #openstack-infra23:19
*** edmondsw has joined #openstack-infra23:19
clarkbze04 has a bunch of leaked builds in /var/lib/zuul/builds so going to delete those before starting services23:20
*** edmondsw has quit IRC23:24
corvusclarkb: what's the gist of 'hung task timeout'?23:25
clarkbcorvus: when rax does live migrations sometimes they fail and the failures exhibit as the kernel complaining about hung tasks timing out23:26
corvusclarkb: gotcha, thx23:26
*** ijw has quit IRC23:26
clarkbcorvus: my hunch is that when the live migration does the suspend of the old host, then copy of memory delta, then start again on the other side sometimes that memory delta copy takes long enough that linux is very unhappy when it "starts" again23:26
*** ijw_ has joined #openstack-infra23:26
clarkbcorvus: any idea why lists.openstack.org is still in the emergency file for ansible puppetry?23:27
clarkbalso puppet run all log shows we have some keystone failures when building inventory. I am going to look into that next23:27
corvusclarkb: oh, i thought i removed it, let me dig23:27
corvusclarkb: we need https://review.openstack.org/52989323:28
corvusclarkb: and https://review.openstack.org/52989523:28
corvusinfra-root: ^23:28
* clarkb looks23:29
*** heyongli has quit IRC23:29
*** heyongli has joined #openstack-infra23:29
clarkboh I worte the one change so won't approve it23:29
*** pahuang has joined #openstack-infra23:32
clarkbinfra-root much appreciated if you can review https://review.openstack.org/#/c/529893/ so that puppet can puppet lists.o.o again23:35
mordredclarkb: done23:38
clarkbmordred: also don't go anywhere I'm about to try and figure out hwich cloud is 401ing keystone23:38
*** heyongli has quit IRC23:39
*** heyongli has joined #openstack-infra23:39
*** tosky has quit IRC23:39
clarkbits ovh23:40
clarkband its only to the openstackci-ovh creds not the openstackjenkins-ovh creds23:43
*** e0ne has quit IRC23:45
clarkbhrm ze04 zuul-executor service doesn't appear to be starting properly23:45
clarkbI think we need to disable ovh23:48
clarkbthe mirror nodes unping and the openstackci-ovh account unpings23:48
*** e0ne has joined #openstack-infra23:49
clarkbis someone able to do that while I debug ze04?23:49
*** heyongli has quit IRC23:49
*** heyongli has joined #openstack-infra23:50
mordredclarkb: yah - on it - all of ovh?23:51
clarkbmordred: I think so. both mirrors are gone (but you should double check that assertion)23:51
clarkbsystemd says zuul-executor exited with return code 023:52
clarkbProcess: 1507 ExecStart=/etc/init.d/zuul-executor start (code=exited, status=0/SUCCESS) so confused. I guess I try to start it in the foreground? (there are no logs that help)23:52
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Disable OVH  https://review.openstack.org/53081723:54
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Revert "Disable OVH"  https://review.openstack.org/53081823:54
mordredclarkb: ^^ there's the disable and then a patch for later to re-enable23:54
clarkbmordred: +2, thanks23:55
*** olaph has quit IRC23:55
clarkbthe executor socket is not being recreated so we aren't making it that far23:56
mordredclarkb: and yes, I verify independently that the ovh account is unhappy23:57
*** olaph has joined #openstack-infra23:57
clarkbmordred: I wonder if that account met the fate of the other account, basically running out of funds or however they comp it23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!