Tuesday, 2016-09-27

clarkbfungi: 15 still looks odd, not sure puppet ran there successfully00:01
*** salv-orlando has joined #openstack-infra00:01
fungiworker15 still seems unhappy as you noted, but the others have 4 workers running now00:01
clarkbbut if 19/20 are happy we can likely leave debugging 15 for toorrow00:01
fungii'm updating https://etherpad.openstack.org/p/logstash-debugging with current state and what we did00:02
*** adrian_otto has joined #openstack-infra00:02
clarkbkk00:02
fungijhesketh: maybe worker15 is something you want to look into00:02
fungi?00:02
clarkblooking at worker logs we have error: [Errno 111] Connection refused00:03
*** ijw has quit IRC00:04
clarkbthe connection it is complaining about tehre is to the logstash jvm daemon00:04
clarkbI think its likely that the mqtt things caused logstash daemons to go away for long enough that the workers died00:04
*** shu-mutou-AFK is now known as shu-mutou00:04
mtreinishclarkb: oh, yeah there was a downtime for a few hours between when we flipped the switch and pushed and landed the fix00:05
clarkbya I think that could do it00:05
mtreinishwith lunch and broken puppet master in the middle00:05
*** salv-orlando has quit IRC00:05
mtreinishso maybe we didn't actually need to revert the mqtt config stuff00:06
mtreinishoh well00:06
clarkbelasticsaerch head shows me an incrementing docs number at about the rate I expect00:06
clarkbso I think things are happy for the moment00:06
*** Sukhdev_ has joined #openstack-infra00:06
*** adrian_otto has quit IRC00:07
*** zhurong has quit IRC00:07
*** Sukhdev has quit IRC00:07
*** rfolco has quit IRC00:09
*** zz_dimtruck is now known as dimtruck00:09
*** tuannguyen has quit IRC00:10
*** tuannguyen has joined #openstack-infra00:10
*** adrian_otto has joined #openstack-infra00:11
openstackgerritMatthew Treinish proposed openstack-infra/system-config: Revert "Disable mqtt output plugin on logstash workers"  https://review.openstack.org/37713000:14
*** spzala has joined #openstack-infra00:15
*** tuannguyen has quit IRC00:15
*** adrian_otto has quit IRC00:16
*** kaisers has joined #openstack-infra00:16
*** kaisers2 has joined #openstack-infra00:17
*** kaisers4 has quit IRC00:17
*** pvaneck_ has quit IRC00:17
*** kaisers1 has quit IRC00:17
*** gongysh has joined #openstack-infra00:17
*** ijw has joined #openstack-infra00:18
*** adrian_otto has joined #openstack-infra00:18
*** dimtruck is now known as zz_dimtruck00:19
*** gildub has joined #openstack-infra00:19
sdakehey folks , quck q00:20
*** Swami has quit IRC00:20
sdakei noticed this page: https://www.openstack.org/software/project-navigator/00:20
*** thcipriani is now known as thcipriani|afk00:20
sdakeis there a repo where that is stored?00:21
clarkbsdake: its part of the www site so is in the openstack-org org on github00:21
sdakeclarkb cool thanks00:21
*** AlexxNica has quit IRC00:23
*** AlexxNica has joined #openstack-infra00:24
*** ijw has quit IRC00:25
sdakeclarkb is what i'm looking for "openstack-org" ?00:25
melwittcould someone please add hudson-openstack (https://launchpad.net/~hudson-openstack) back to the nova bug team (https://launchpad.net/~nova-bugs) please? I think the user has to be a member of the team in order to update launchpad bugs when the gerrit hook runs00:26
sdakethat namespace doesn't appear to exist00:26
clarkbhrm I wonder if its case sensitive /me finds url00:26
*** tuanla has joined #openstack-infra00:26
sdakeclarkb thanks for the assistance00:27
clarkbits openstack-org on launchpad00:27
clarkbsdake: https://github.com/OpenStackweb/openstack-org/ there we go00:27
sdakeclarkb00:27
sdakecool thanks pretty sure i would never havefound  that ;)00:27
melwittjeblair: when you get a chance, could you please add the hudson-openstack user back to the nova bug team? I think it has to be a member and its membership must have expired recently. it looks like you're the only admin of the nova bug team00:30
melwitt*has to be a member in order to update our bugs when the gerrit hook runs00:30
*** markvoelker has quit IRC00:30
*** caowei has joined #openstack-infra00:31
*** adrian_otto has quit IRC00:31
*** Apoorva has quit IRC00:34
*** chem has quit IRC00:35
*** zz_dimtruck is now known as dimtruck00:36
melwittjeblair: argh, I was looking at the wrong member page when I said that. I need to ask an admin of the nova bug team. sorry for the noise00:37
*** david-lyle_ has joined #openstack-infra00:38
openstackgerritMerged openstack-infra/devstack-gate: Enable tlsproxy on ocata and above  https://review.openstack.org/37577600:39
clarkbwoot, just one more devstack change to go then we can actually turn that on in jobs00:41
*** gongysh has quit IRC00:41
*** david-lyle has quit IRC00:41
*** thorst has joined #openstack-infra00:42
jeblairmelwitt: i added hudson-openstack to nova-bugs00:42
melwittjeblair: thanks, you're the best00:44
*** amitgandhinz has joined #openstack-infra00:45
jeblairmelwitt: aww shucks.  you're welcome.  i also set its membership to never expire00:45
*** dimtruck is now known as zz_dimtruck00:46
melwittwoo awesome00:47
*** csomerville has joined #openstack-infra00:47
*** tqtran has quit IRC00:48
*** cody-somerville has quit IRC00:49
*** gordc has joined #openstack-infra00:49
*** esikachev has joined #openstack-infra00:50
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Make bug name, type, and url explicit  https://review.openstack.org/37511300:53
*** mdrabe has quit IRC00:53
*** mriedem is now known as mriedem_debate00:54
*** esikachev has quit IRC00:55
*** yamamot__ has joined #openstack-infra01:01
*** salv-orlando has joined #openstack-infra01:02
*** vinaypotluri has quit IRC01:02
*** zz_dimtruck is now known as dimtruck01:03
*** Apoorva has joined #openstack-infra01:05
*** tuannguyen has joined #openstack-infra01:06
*** salv-orlando has quit IRC01:06
*** gongysh has joined #openstack-infra01:07
*** zhurong has joined #openstack-infra01:10
openstackgerritMerged openstack-infra/tripleo-ci: Remove test-environments/swap-partition.yaml  https://review.openstack.org/37350001:12
*** dimtruck is now known as zz_dimtruck01:13
*** kaisers_ has quit IRC01:14
*** amitgandhinz has quit IRC01:19
*** thorst has quit IRC01:21
*** thorst has joined #openstack-infra01:22
*** mtanin___ has joined #openstack-infra01:23
*** mtanino has quit IRC01:24
*** ijw has joined #openstack-infra01:26
*** alex_xu has quit IRC01:27
*** yanyanhu has joined #openstack-infra01:28
*** sdake has quit IRC01:28
*** zz_dimtruck is now known as dimtruck01:30
*** alex_xu has joined #openstack-infra01:31
*** thorst has quit IRC01:31
*** markvoelker has joined #openstack-infra01:31
*** ijw has quit IRC01:34
*** baoli has joined #openstack-infra01:34
*** verdurin has quit IRC01:34
*** aspiers has quit IRC01:34
*** ijw has joined #openstack-infra01:34
*** harlowja has quit IRC01:35
*** huanxie has joined #openstack-infra01:35
*** markvoelker has quit IRC01:36
*** thorst has joined #openstack-infra01:36
*** thorst has quit IRC01:37
*** verdurin has joined #openstack-infra01:39
*** ijw has quit IRC01:39
*** aspiers has joined #openstack-infra01:39
*** dimtruck is now known as zz_dimtruck01:40
*** esikachev has joined #openstack-infra01:51
*** amotoki has joined #openstack-infra01:51
*** esikachev has quit IRC01:55
*** spzala has quit IRC01:55
*** zz_dimtruck is now known as dimtruck01:57
*** rfolco has joined #openstack-infra01:59
*** salv-orlando has joined #openstack-infra02:03
*** xarses has joined #openstack-infra02:05
*** dimtruck is now known as zz_dimtruck02:07
*** salv-orlando has quit IRC02:07
*** csomerville has quit IRC02:07
*** csomerville has joined #openstack-infra02:07
*** thorst has joined #openstack-infra02:07
openstackgerritDong Ma proposed openstack-infra/jenkins-job-builder: Update email-ext plugin  https://review.openstack.org/37715202:08
*** gordc has quit IRC02:14
*** amitgandhinz has joined #openstack-infra02:15
*** thorst has quit IRC02:16
openstackgerritJohn Hua proposed openstack-infra/project-config: Shorten image list  https://review.openstack.org/37715902:20
*** zz_dimtruck is now known as dimtruck02:24
*** adrian_otto has joined #openstack-infra02:27
*** yamamot__ has quit IRC02:29
*** jianghuaw has quit IRC02:29
*** Qiming_ has joined #openstack-infra02:30
*** Odd_Bloke has quit IRC02:30
*** sfinucan has quit IRC02:31
*** sfinucan has joined #openstack-infra02:31
*** jianghuaw has joined #openstack-infra02:32
openstackgerritMonty Taylor proposed openstack-infra/shade: Normalize images  https://review.openstack.org/37716302:34
openstackgerritMonty Taylor proposed openstack-infra/shade: Make sure we're matching image status properly  https://review.openstack.org/37716402:34
*** dimtruck is now known as zz_dimtruck02:34
*** adrian_otto has quit IRC02:35
*** Qiming has quit IRC02:37
*** Qiming_ is now known as Qiming02:38
*** gordc has joined #openstack-infra02:39
*** maishsk has joined #openstack-infra02:41
*** tuannguyen has quit IRC02:42
*** EricGonczer_ has quit IRC02:44
*** hongbin has quit IRC02:44
*** hongbin has joined #openstack-infra02:45
*** yuanying has quit IRC02:47
*** david-lyle_ has quit IRC02:48
*** mtanin___ has quit IRC02:49
*** david-lyle has joined #openstack-infra02:49
stevemarfungi: if you have a minute, i'd like your take on a licensing patch: https://review.openstack.org/#/c/377170/102:50
stevemarfungi: last thing i want is thing to turn into a fix-all-the-projects type of patch02:50
*** amitgandhinz has quit IRC02:50
*** yuanying has joined #openstack-infra02:50
*** rhallisey has quit IRC02:50
*** zz_dimtruck is now known as dimtruck02:51
*** esikachev has joined #openstack-infra02:51
*** baoli has quit IRC02:53
*** sdake has joined #openstack-infra02:55
*** esikachev has quit IRC02:56
*** gildub has quit IRC02:56
*** mriedem_debate has quit IRC02:59
*** dimtruck is now known as zz_dimtruck03:01
*** david-lyle has quit IRC03:04
*** EricGonczer_ has joined #openstack-infra03:06
*** Apoorva has quit IRC03:06
*** rfolco has quit IRC03:09
*** yamamot__ has joined #openstack-infra03:10
*** moravec has joined #openstack-infra03:12
openstackgerritDong Ma proposed openstack-infra/jenkins-job-builder: Update fingerprint plugin  https://review.openstack.org/37718103:14
*** thorst has joined #openstack-infra03:14
*** sdake has quit IRC03:15
*** adrian_otto has joined #openstack-infra03:16
*** maishsk has quit IRC03:17
*** gildub has joined #openstack-infra03:17
*** zz_dimtruck is now known as dimtruck03:18
*** armax has quit IRC03:19
*** EricGonczer_ has quit IRC03:20
*** thorst has quit IRC03:21
*** sdake has joined #openstack-infra03:23
*** huanxie has quit IRC03:25
*** vikrant has joined #openstack-infra03:25
*** amotoki has quit IRC03:26
*** dimtruck is now known as zz_dimtruck03:28
*** EricGonczer_ has joined #openstack-infra03:28
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: WIP - Deploy TripleO with Puppet 4  https://review.openstack.org/37120903:32
*** markvoelker has joined #openstack-infra03:32
*** EricGonczer_ has quit IRC03:37
*** yuanying has quit IRC03:37
*** amotoki has joined #openstack-infra03:37
*** markvoelker has quit IRC03:38
*** vgadiraj has quit IRC03:39
*** yarkot has quit IRC03:39
*** sdake has quit IRC03:40
*** sdake has joined #openstack-infra03:41
*** yarkot has joined #openstack-infra03:45
*** adrian_otto has quit IRC03:45
*** zz_dimtruck is now known as dimtruck03:46
*** amitgandhinz has joined #openstack-infra03:46
*** tqtran has joined #openstack-infra03:47
*** sdake has quit IRC03:47
*** adrian_otto has joined #openstack-infra03:48
*** amotoki has quit IRC03:49
*** yuanying has joined #openstack-infra03:50
*** Sukhdev_ has quit IRC03:51
*** rkukura has joined #openstack-infra03:51
*** Sukhdev has joined #openstack-infra03:51
*** tqtran has quit IRC03:52
*** esikachev has joined #openstack-infra03:52
*** dave-mccowan has quit IRC03:53
*** gongysh has quit IRC03:54
*** amotoki has joined #openstack-infra03:55
*** dimtruck is now known as zz_dimtruck03:55
*** adrian_otto has quit IRC03:57
*** esikachev has quit IRC03:57
*** amitgandhinz has quit IRC04:01
*** gordc has quit IRC04:04
*** hongbin has quit IRC04:07
*** zz_dimtruck is now known as dimtruck04:13
*** sdake has joined #openstack-infra04:15
*** caowei has quit IRC04:17
*** amotoki has quit IRC04:18
*** links has joined #openstack-infra04:19
*** thorst has joined #openstack-infra04:21
*** sdake_ has joined #openstack-infra04:22
*** dimtruck is now known as zz_dimtruck04:22
*** senk_ has joined #openstack-infra04:23
*** sdake has quit IRC04:24
*** thorst has quit IRC04:26
*** rlandy has quit IRC04:28
*** amotoki has joined #openstack-infra04:28
*** roxanaghe has quit IRC04:29
*** roxanaghe has joined #openstack-infra04:29
*** tqtran has joined #openstack-infra04:29
*** coolsvap has quit IRC04:32
*** winggundamth has quit IRC04:33
*** roxanaghe has quit IRC04:34
*** amotoki_ has joined #openstack-infra04:39
*** sdake_ has quit IRC04:40
*** zz_dimtruck is now known as dimtruck04:40
*** amotoki has quit IRC04:42
*** sdake has joined #openstack-infra04:43
*** gouthamr has quit IRC04:44
*** psachin has joined #openstack-infra04:45
*** sdake_ has joined #openstack-infra04:46
*** sdake has quit IRC04:48
*** claudiub has joined #openstack-infra04:48
*** csomerville has quit IRC04:48
*** dimtruck is now known as zz_dimtruck04:50
*** esikachev has joined #openstack-infra04:53
*** salv-orlando has joined #openstack-infra04:54
*** esikachev has quit IRC04:58
*** amitgandhinz has joined #openstack-infra04:58
*** sdake_ has quit IRC04:58
*** amitgandhinz has quit IRC05:03
*** kaisers_ has joined #openstack-infra05:03
*** pgadiya has joined #openstack-infra05:05
*** baoli has joined #openstack-infra05:06
*** zz_dimtruck is now known as dimtruck05:07
*** jaosorior has joined #openstack-infra05:07
*** gongysh has joined #openstack-infra05:08
*** baoli has quit IRC05:10
*** Sukhdev has quit IRC05:12
*** dimtruck is now known as zz_dimtruck05:17
*** OferBarber_ has quit IRC05:17
*** jaosorior has quit IRC05:19
*** rossella_s has quit IRC05:19
*** rossella_s has joined #openstack-infra05:19
*** jaosorior has joined #openstack-infra05:20
*** tonytan4ever has quit IRC05:20
*** snarwade has quit IRC05:21
*** snarwade has joined #openstack-infra05:21
*** thorst has joined #openstack-infra05:23
*** ihrachys has joined #openstack-infra05:25
openstackgerritCaleb Boylan proposed openstack-infra/nodepool: [WIP] Fix subnode deletion  https://review.openstack.org/37045505:31
*** thorst has quit IRC05:32
*** flepied has quit IRC05:32
*** zz_dimtruck is now known as dimtruck05:34
*** senk_ has quit IRC05:34
*** isaacb has joined #openstack-infra05:36
openstackgerritCaleb Boylan proposed openstack-infra/nodepool: [WIP] Fix subnode deletion  https://review.openstack.org/37045505:38
*** cody-somerville has joined #openstack-infra05:38
*** dimtruck is now known as zz_dimtruck05:44
openstackgerritAnusha Ramineni proposed openstack-infra/project-config: enable congress multi process deployment  https://review.openstack.org/37725005:47
*** claudiub has quit IRC05:49
*** nadya has joined #openstack-infra05:53
*** esikachev has joined #openstack-infra05:54
*** rajinir has quit IRC05:55
*** martinkopec has joined #openstack-infra05:58
*** amitgandhinz has joined #openstack-infra05:58
*** esikachev has quit IRC05:59
*** otherwiseguy has quit IRC06:00
*** zz_dimtruck is now known as dimtruck06:01
*** otherwiseguy has joined #openstack-infra06:01
*** nherciu has joined #openstack-infra06:05
*** woodster_ has quit IRC06:10
*** dimtruck is now known as zz_dimtruck06:11
*** senk_ has joined #openstack-infra06:11
openstackgerritzhangyanxian proposed openstack-infra/project-config: Enable py35 voting on project vitrage   https://review.openstack.org/37726006:11
openstackgerritzhangyanxian proposed openstack-infra/project-config: Enable py35 voting on project vitrage   https://review.openstack.org/37726006:13
*** xnox has quit IRC06:14
openstackgerritSridhar Ramaswamy proposed openstack-infra/project-config: Make tacker python3 jobs voting  https://review.openstack.org/37038806:15
*** isaacb has quit IRC06:18
*** tqtran has quit IRC06:21
*** gtmanfred has quit IRC06:23
*** xnox has joined #openstack-infra06:24
*** vgadiraj has joined #openstack-infra06:26
*** esikachev has joined #openstack-infra06:27
*** zz_dimtruck is now known as dimtruck06:28
*** gtmanfred has joined #openstack-infra06:28
*** thorst has joined #openstack-infra06:30
*** nadya has quit IRC06:30
*** ccamacho has joined #openstack-infra06:30
openstackgerritZhenguo Niu proposed openstack-infra/system-config: Add #openstack-nimble to statusbot and meetbot  https://review.openstack.org/37726806:31
*** rlu has joined #openstack-infra06:31
*** esikachev has quit IRC06:31
*** rlu has quit IRC06:32
*** amitgandhinz has quit IRC06:33
openstackgerritNicko Glayre proposed openstack-infra/jenkins-job-builder: Adding nexus-artifact-uploader plugin  https://review.openstack.org/37605306:34
*** Ravikiran_K has joined #openstack-infra06:35
*** thorst has quit IRC06:37
*** dimtruck is now known as zz_dimtruck06:38
zhenguohi infras, is the gerritbot ready to accept new channel now?06:40
AJaegerzhenguo: no06:41
zhenguoAJaeger :(06:41
openstackgerritMerged openstack-infra/project-config: Enable py35 voting on project vitrage   https://review.openstack.org/37726006:42
openstackgerritMerged openstack-infra/project-config: Add 3 missing deb-python-xstatic for Horizon  https://review.openstack.org/37655106:42
AJaegerzhenguo: https://review.openstack.org/#/c/355588/ needs some love, do you want to help?06:43
zhenguoAJaeger: I can try06:44
AJaegerzhenguo: I just checked the log file - perhaps a recheck is enough - let's wait first for that one.06:44
*** tosky has joined #openstack-infra06:45
zhenguoAJaeger: ok, thanks, hope recheck can help :)06:45
*** pcaruana has joined #openstack-infra06:45
*** tuannguyen has joined #openstack-infra06:47
openstackgerritMerged openstack-infra/project-config: enable congress multi process deployment  https://review.openstack.org/37725006:47
openstackgerritMerged openstack-infra/project-config: Use python-db-jobs for vmware-nsx  https://review.openstack.org/37576606:48
AJaegerzhenguo: keep an eye on it and if not, feel free to help ;)06:49
*** isaacb has joined #openstack-infra06:49
zhenguoAJaeger: ok, I will06:49
*** andreas_s has joined #openstack-infra06:50
*** tuannguyen has quit IRC06:52
*** flepied has joined #openstack-infra06:54
*** tosky has quit IRC06:55
*** zz_dimtruck is now known as dimtruck06:55
*** rcernin has joined #openstack-infra06:59
*** marst has quit IRC07:00
*** salv-orl_ has joined #openstack-infra07:01
*** yanyanhu_ has joined #openstack-infra07:01
*** Ravikiran_K has quit IRC07:02
*** Ravikiran_K has joined #openstack-infra07:02
*** rlu has joined #openstack-infra07:02
*** ihrachys has quit IRC07:02
*** ihrachys has joined #openstack-infra07:03
*** salv-orlando has quit IRC07:04
*** yanyanhu has quit IRC07:04
*** dimtruck is now known as zz_dimtruck07:05
*** salv-orl_ has quit IRC07:05
*** xnox has quit IRC07:10
yolandagood morning07:12
AJaegermorning, yolanda !07:13
*** xnox has joined #openstack-infra07:14
openstackgerritOndÅ™ej Nový proposed openstack-infra/project-config: Debian packing upstream links cleanup  https://review.openstack.org/37585307:14
*** drifterza has joined #openstack-infra07:16
*** AlexxNica has quit IRC07:20
*** Rockyg has quit IRC07:20
*** tonytan4ever has joined #openstack-infra07:21
*** zz_dimtruck is now known as dimtruck07:22
*** esikachev has joined #openstack-infra07:25
*** yamahata has joined #openstack-infra07:25
*** tonytan4ever has quit IRC07:26
openstackgerritMerged openstack-infra/project-config: Force configdrive for ironic jobs  https://review.openstack.org/37645407:26
openstackgerritMerged openstack-infra/puppet-gerrit: Add apache2.4 acls and permission to gerrit static dir  https://review.openstack.org/37660607:27
*** claudiub has joined #openstack-infra07:28
*** marst has joined #openstack-infra07:28
*** amitgandhinz has joined #openstack-infra07:29
openstackgerritMerged openstack-infra/project-config: Skip slow ironic tests on install-guide changes  https://review.openstack.org/37660307:31
openstackgerritMerged openstack-infra/project-config: Enable py35 voting on project os-client-config  https://review.openstack.org/37659707:31
*** denisra has quit IRC07:32
*** yamahata has quit IRC07:32
*** xnox has quit IRC07:33
*** jpich has joined #openstack-infra07:34
*** dimtruck is now known as zz_dimtruck07:35
*** thorst has joined #openstack-infra07:35
*** xnox has joined #openstack-infra07:35
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config: Add neutron-lib-master periodic job for networking-midonet  https://review.openstack.org/37729207:36
*** dtardivel has joined #openstack-infra07:37
*** AlexxNica has joined #openstack-infra07:38
*** jlanoux has joined #openstack-infra07:38
*** nmagnezi has joined #openstack-infra07:38
*** aviau_ has quit IRC07:40
*** aviau has joined #openstack-infra07:40
*** tnovacik has joined #openstack-infra07:40
*** danielitit has joined #openstack-infra07:41
*** thorst has quit IRC07:41
openstackgerrityolanda.robla proposed openstack/diskimage-builder: Fix typo in extracting root partition  https://review.openstack.org/37729607:41
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config: grafana: Add periodic-networking-midonet-python27-newton  https://review.openstack.org/37729907:46
*** danielitit has quit IRC07:48
*** danielitit has joined #openstack-infra07:48
*** anteaya has quit IRC07:48
*** tuannguyen has joined #openstack-infra07:48
*** hashar has joined #openstack-infra07:48
*** anteaya has joined #openstack-infra07:49
*** jpena|off is now known as jpena07:49
*** danielitit has quit IRC07:49
*** danielitit has joined #openstack-infra07:50
*** jordanP has joined #openstack-infra07:50
*** pilgrimstack has joined #openstack-infra07:50
*** danielitit has quit IRC07:50
*** danielitit has joined #openstack-infra07:50
*** tuannguyen has quit IRC07:53
*** _degorenko|afk is now known as degorenko07:57
*** e0ne has joined #openstack-infra07:59
*** zzzeek has quit IRC08:00
*** zzzeek has joined #openstack-infra08:00
*** ihrachys has quit IRC08:00
*** e0ne has quit IRC08:01
*** yaume has joined #openstack-infra08:01
*** claudiub has quit IRC08:01
*** amoralej|off is now known as amoralej08:02
*** yanyanhu_ has quit IRC08:02
*** amitgandhinz has quit IRC08:04
*** yanyanhu_ has joined #openstack-infra08:05
*** xnox has quit IRC08:09
*** rwsu has quit IRC08:09
*** watersoul has quit IRC08:11
*** xnox has joined #openstack-infra08:14
*** AlexxNica has quit IRC08:15
*** rwsu has joined #openstack-infra08:15
openstackgerrityolanda.robla proposed openstack/diskimage-builder: Disabling all previous repos registered in the system  https://review.openstack.org/37733108:20
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/37733308:21
*** tonytan4ever has joined #openstack-infra08:22
*** tonytan4ever has quit IRC08:27
*** e0ne has joined #openstack-infra08:30
*** ihrachys has joined #openstack-infra08:32
*** derekh has joined #openstack-infra08:36
*** mhickey has joined #openstack-infra08:39
*** martinkopec has quit IRC08:45
*** panda is now known as panda|bbl08:46
*** denisra has joined #openstack-infra08:56
*** dtantsur|afk has quit IRC08:57
*** dtantsur has joined #openstack-infra08:58
openstackgerritMerged openstack-infra/system-config: Use firehose.openstack.org as the mqtt server name  https://review.openstack.org/37418308:58
*** watersoul has joined #openstack-infra09:00
*** amitgandhinz has joined #openstack-infra09:00
openstackgerrityolanda.robla proposed openstack/diskimage-builder: Disabling all previous repos registered in the system  https://review.openstack.org/37733109:05
*** sambetts|afk is now known as sambetts09:12
*** kashyap has left #openstack-infra09:13
*** mptacekx has joined #openstack-infra09:14
*** jtomasek has joined #openstack-infra09:17
*** jaosorior is now known as jaosorior_lunch09:18
*** matrohon has joined #openstack-infra09:19
*** yanyanhu_ has quit IRC09:24
*** electrofelix has joined #openstack-infra09:25
*** zz_dimtruck is now known as dimtruck09:26
*** jlanoux_ has joined #openstack-infra09:27
*** e0ne has quit IRC09:27
*** e0ne has joined #openstack-infra09:27
*** ianychoi has quit IRC09:29
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton  https://review.openstack.org/32375009:29
*** ianychoi has joined #openstack-infra09:29
*** jlanoux has quit IRC09:30
*** nadya has joined #openstack-infra09:33
*** amitgandhinz has quit IRC09:35
*** gongysh has quit IRC09:35
*** dimtruck is now known as zz_dimtruck09:35
*** dizquierdo has joined #openstack-infra09:36
*** xnox has quit IRC09:43
*** thorst has joined #openstack-infra09:44
openstackgerrityolanda.robla proposed openstack/diskimage-builder: Disabling all previous repos registered in the system  https://review.openstack.org/37733109:47
*** tuannguyen has joined #openstack-infra09:50
*** denisra_ has joined #openstack-infra09:50
*** thorst has quit IRC09:51
*** denisra has quit IRC09:52
*** zz_dimtruck is now known as dimtruck09:53
*** denisra has joined #openstack-infra09:53
*** tuannguyen has quit IRC09:55
*** denisra_ has quit IRC09:55
*** gildub has quit IRC09:56
wznoinskhi infra, is review.o.o awlright?09:56
*** denisra_ has joined #openstack-infra09:56
*** jlvillal has quit IRC09:56
*** furlongm has joined #openstack-infra09:56
*** jmccrory has quit IRC09:57
*** furlongm_ has quit IRC09:57
*** Qiming has quit IRC09:57
AJaegerwznoinsk: at least for me. What's your problem?09:57
openstackgerritOpenStack Proposal Bot proposed openstack-dev/hacking: Updated from global requirements  https://review.openstack.org/37742109:58
openstackgerritOpenStack Proposal Bot proposed openstack-dev/pbr: Updated from global requirements  https://review.openstack.org/37742209:58
*** denisra has quit IRC09:58
wznoinskAJaeger: I guess it depends on which load balancing route I am, but it throws service unavailable every now and then for me, I'll let you know if it's still happening from now09:58
*** gildub has joined #openstack-infra09:58
openstackgerritOpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements  https://review.openstack.org/37743109:59
*** Qiming has joined #openstack-infra09:59
*** jlvillal has joined #openstack-infra09:59
*** jmccrory has joined #openstack-infra10:00
*** zeus has quit IRC10:01
*** melwitt has quit IRC10:01
*** melwitt has joined #openstack-infra10:01
AJaegerwznoinsk: just mention it here - I cannot help but hope others see and can check cacti or logs.10:01
*** zhurong has quit IRC10:02
*** melwitt is now known as Guest320310:02
wznoinskAJaeger: sure, thanks10:02
*** zeus has joined #openstack-infra10:02
*** zeus is now known as Guest7154610:02
*** shu-mutou is now known as shu-mutou-AWAY10:03
*** dimtruck is now known as zz_dimtruck10:05
*** bethwhite has quit IRC10:06
*** bethwhite has joined #openstack-infra10:06
*** sdague has joined #openstack-infra10:08
*** dizquierdo has quit IRC10:08
*** gongysh has joined #openstack-infra10:10
openstackgerritAlexander Evseev proposed openstack-infra/project-config: Use global defaults for common job parameters  https://review.openstack.org/37759110:10
*** berendt has joined #openstack-infra10:10
openstackgerritMaciej Relewicz proposed openstack-infra/puppet-apps_site: Glare support for app-catalog  https://review.openstack.org/35902910:13
isaacbdiskimage-builder fails to build images due to empty repo: openstack/deb-python-xstatic-angular-schema-form.10:14
isaacbDoes anyone knows why?10:14
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Add chocolate oscc cloud to puppetmaster  https://review.openstack.org/37759410:16
*** yamamot__ has quit IRC10:20
fricklerisaacb: this "usually" happens during the interval between new project creation and import of the repo being finished. https://review.openstack.org/376551 was merged over 3 hours ago, maybe some infra-root can check the logs10:21
*** tonytan4ever has joined #openstack-infra10:23
*** sdake has joined #openstack-infra10:27
*** tonytan4ever has quit IRC10:28
*** amitgandhinz has joined #openstack-infra10:31
*** jaosorior_lunch is now known as jaosorior10:34
*** gongysh has quit IRC10:34
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: TEST: DONT RECHECK: periodic jobs  https://review.openstack.org/35921510:36
*** afonsod__ has joined #openstack-infra10:36
*** panda|bbl is now known as panda10:36
*** gordc has joined #openstack-infra10:38
*** salv-orlando has joined #openstack-infra10:39
*** scottda has quit IRC10:40
*** denisra_ has quit IRC10:40
*** masayukig has quit IRC10:45
*** masayukig has joined #openstack-infra10:45
*** jlanoux has joined #openstack-infra10:46
*** denisra has joined #openstack-infra10:48
*** jlanoux_ has quit IRC10:49
*** Ravikiran_K has quit IRC10:49
*** thorst has joined #openstack-infra10:49
*** afonsod__ has quit IRC10:50
*** gordc has quit IRC10:53
*** nmagnezi has quit IRC10:55
isaacbfrickler: Thanks10:56
*** thorst has quit IRC10:56
*** jkilpatr has quit IRC10:56
isaacbinfra cores - can anyone check why deb-python-xstatic-angular-schema-form hasn't merged yet (https://review.openstack.org/376551)10:57
*** mhickey has quit IRC10:57
*** rtheis has joined #openstack-infra10:57
isaacbThe git repository is empty ant that fails the diskimage-builder10:57
fricklerwasn't there an issue earlier with importing when the default branch isn't called master?11:00
frickleryolanda: zigo: ^^ ?11:01
*** yamamoto has joined #openstack-infra11:02
*** amitgandhinz has quit IRC11:05
*** jtomasek has quit IRC11:06
*** sdake has quit IRC11:12
*** mhickey has joined #openstack-infra11:12
*** yamamoto has quit IRC11:13
*** nherciu_ has joined #openstack-infra11:17
*** nherciu has quit IRC11:19
openstackgerritValeriy Ponomaryov proposed openstack-infra/project-config: [Manila] Add experimental tempest jobs on xenial  https://review.openstack.org/37736011:19
*** rlu has left #openstack-infra11:23
*** yamamoto has joined #openstack-infra11:24
openstackgerritMarton Kiss proposed openstack-infra/groups: Fix broken jsonpath url link  https://review.openstack.org/37761711:24
jheskethinfra-root: fyi I'm fixing up the puppet stable branches as requested (will reply to mailing list once done)11:25
openstackgerritNicko Glayre proposed openstack-infra/jenkins-job-builder: Adding nexus-artifact-uploader plugin  https://review.openstack.org/37605311:25
*** jpena is now known as jpena|lunch11:25
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Add launcher layouts for chocolate admin cloud  https://review.openstack.org/37761911:25
*** amoralej is now known as amoralej|lunch11:26
*** mhickey has quit IRC11:27
*** salv-orlando has quit IRC11:27
*** dtantsur is now known as dtantsur|mtg11:28
*** ccamacho is now known as ccamacho|lunch11:29
openstackgerritSam Betts proposed openstack-infra/irc-meetings: Add Networking Cisco IRC Meeting  https://review.openstack.org/37762211:34
openstackgerritDmitry Tantsur proposed openstack-infra/project-config: Switch ironic-inspector to tempest discovery job  https://review.openstack.org/37762311:35
*** jkilpatr has joined #openstack-infra11:36
*** jtomasek has joined #openstack-infra11:36
openstackgerritMerged openstack-infra/groups: Fix broken jsonpath url link  https://review.openstack.org/37761711:37
*** ldnunes has joined #openstack-infra11:39
*** salv-orlando has joined #openstack-infra11:40
openstackgerritMaciej Relewicz proposed openstack-infra/puppet-apps_site: Glare support for app-catalog  https://review.openstack.org/35902911:41
*** pgadiya has quit IRC11:43
*** gordc has joined #openstack-infra11:44
*** EricGonczer_ has joined #openstack-infra11:44
*** achevychalov has joined #openstack-infra11:45
*** EricGonc_ has joined #openstack-infra11:47
zigoisaacb: frickler: yolanda: As much as I know, there's no issue with debian/newton being the default.11:48
*** narayrak has joined #openstack-infra11:49
zigoisaacb: frickler: yolanda: I'm also annoyed by the fact these 3 new deb-python-xstatic-{tv4,objectpath,angular-schema-form} aren't being populated, I need them to package Horizon...11:49
*** EricGonczer_ has quit IRC11:49
*** tuannguyen has joined #openstack-infra11:51
*** thorst has joined #openstack-infra11:53
*** tuannguyen has quit IRC11:56
*** zz_dimtruck is now known as dimtruck11:56
*** pgadiya has joined #openstack-infra11:56
*** rhallisey has joined #openstack-infra11:59
*** jamesdenton has quit IRC12:01
*** amitgandhinz has joined #openstack-infra12:01
openstackgerritAndrey Nikitin proposed openstack-infra/project-config: Add one new Fuel's plugin  https://review.openstack.org/37763912:02
*** dprince has joined #openstack-infra12:03
*** sandanar has joined #openstack-infra12:03
*** ArxCruz has joined #openstack-infra12:03
*** ArxCruz has quit IRC12:04
*** sandanar_ has joined #openstack-infra12:04
*** sandanar has quit IRC12:05
*** moravec has joined #openstack-infra12:05
*** jzerebecki has quit IRC12:05
*** dimtruck is now known as zz_dimtruck12:06
*** jaosorior has quit IRC12:06
*** rcarrillocruz has quit IRC12:06
*** liusheng has quit IRC12:06
*** jaosorior has joined #openstack-infra12:06
*** rcarrillocruz has joined #openstack-infra12:07
*** liusheng has joined #openstack-infra12:07
*** jzerebecki has joined #openstack-infra12:07
*** AJaeger has quit IRC12:08
openstackgerritAndrey Nikitin proposed openstack-infra/project-config: Add one new Fuel's plugin  https://review.openstack.org/37763912:10
*** tuannguyen has joined #openstack-infra12:10
*** kgiusti has joined #openstack-infra12:11
*** rfolco has joined #openstack-infra12:12
*** Ravikiran_K has joined #openstack-infra12:12
*** rodrigods has quit IRC12:12
*** rodrigods has joined #openstack-infra12:12
*** vikrant has quit IRC12:13
*** vikrant has joined #openstack-infra12:13
*** gouthamr has joined #openstack-infra12:16
*** hrybacki is now known as hrybacki|afk12:16
*** moravec has quit IRC12:16
*** zhurong has joined #openstack-infra12:17
*** tpsilva has joined #openstack-infra12:18
*** rlandy has joined #openstack-infra12:19
*** trown|outtypewww is now known as trown12:21
*** zz_dimtruck is now known as dimtruck12:23
*** jpena|lunch is now known as jpena12:24
*** amoralej|lunch is now known as amoralej12:25
*** woodster_ has joined #openstack-infra12:25
*** psilvad has joined #openstack-infra12:27
*** sdake has joined #openstack-infra12:28
*** ccamacho|lunch is now known as ccamacho12:29
*** markvoelker has joined #openstack-infra12:29
*** matrohon has quit IRC12:30
*** edmondsw has joined #openstack-infra12:30
*** gildub has quit IRC12:31
*** gildub has joined #openstack-infra12:32
*** yamamoto has quit IRC12:33
*** burgerk has joined #openstack-infra12:33
*** gildub has quit IRC12:33
*** AJaeger has joined #openstack-infra12:34
*** amitgandhinz has quit IRC12:35
*** dimtruck is now known as zz_dimtruck12:36
*** baoli has joined #openstack-infra12:36
AJaegerclarkb: could you check a xenial change for manila, please? https://review.openstack.org/#/c/377360/12:38
*** yamamoto has joined #openstack-infra12:40
*** tuanla has quit IRC12:42
*** annegentle has joined #openstack-infra12:44
*** nadya has quit IRC12:44
openstackgerritDamian Szeluga proposed openstack-infra/project-config: Add checks to openstack/cookbook-openstack-application-catalog  https://review.openstack.org/37767112:44
*** salv-orlando has quit IRC12:45
*** nadya has joined #openstack-infra12:46
*** denisra_ has joined #openstack-infra12:47
*** denisra has quit IRC12:49
*** matrohon has joined #openstack-infra12:49
*** amitgandhinz has joined #openstack-infra12:51
*** claudiub has joined #openstack-infra12:53
*** vikrant has quit IRC12:54
*** tonytan4ever has joined #openstack-infra12:54
*** david-lyle has joined #openstack-infra12:56
*** tonytan4ever has quit IRC12:57
*** baoli_ has joined #openstack-infra12:58
*** chem has joined #openstack-infra12:58
*** tonytan4ever has joined #openstack-infra13:00
*** baoli has quit IRC13:01
rcarrillocruzpabelanger: when you get around, can you pls check https://review.openstack.org/#/c/377594/ and https://review.openstack.org/#/c/377619/ ?13:05
rcarrillocruzchocolate related changes13:05
*** amitgandhinz has quit IRC13:07
*** rlandy is now known as rlandy|mtg13:09
*** mdrabe has joined #openstack-infra13:10
*** pgadiya has quit IRC13:11
*** yamamoto has quit IRC13:12
*** salv-orlando has joined #openstack-infra13:12
*** john-davidge has joined #openstack-infra13:12
john-davidgeDoes anybody happen to know where the source for openstack.org lives?13:13
sdaguejohn-davidge: it's maintained by foundation staff in a CMS iirc13:14
*** IzikP has joined #openstack-infra13:14
john-davidgesdague: Ah, ok. Not something we can easily push fixes to then?13:14
sdaguejohn-davidge: correct13:14
john-davidgesdague: Ok thanks, I'll try to get in touch then13:15
IzikPHello, who can help me with renewing our CI gerrit account ?13:15
*** jasondotstar_ is now known as jasondotstar13:17
isaacbIzikP: send a mail in the openstack-infra mailing list13:18
*** rossella_s has quit IRC13:19
*** jaosorior has quit IRC13:19
*** mriedem has joined #openstack-infra13:19
*** rossella_s has joined #openstack-infra13:20
*** Guest71546 is now known as zeus13:20
openstackgerritMaciej Relewicz proposed openstack-infra/puppet-apps_site: Glare support for app-catalog  https://review.openstack.org/35902913:20
*** zeus has quit IRC13:20
*** zeus has joined #openstack-infra13:20
*** jaosorior has joined #openstack-infra13:20
clarkbsdague: https://review.openstack.org/#/c/375778/ should be ready now if you want to weigh in on the jobs yo use tls13:20
IzikPisaacb: thanks for responding,I sent an e mail, but although I'm on the openstack-infra mail list I got  this mail in respond "Is being held until the list moderator can review it for approval.  The reason it is being held:      Post by non-member to a members-only list"13:20
*** jheroux has joined #openstack-infra13:21
clarkbjohn-davidge: https://github.com/OpenStackweb/openstack-org is the source code, but much of the content is in the cms side of things13:21
sdagueclarkb: +A13:21
*** inc0 has joined #openstack-infra13:21
*** moravec has joined #openstack-infra13:21
sdaguewe should just keep an eye on it to make sure things are fine once it merges13:21
clarkbsdague: yup13:22
*** tongli has joined #openstack-infra13:23
*** matt-borland has joined #openstack-infra13:24
john-davidgeclarkb: Thanks!13:24
openstackgerritRobin Naundorf proposed openstack-infra/system-config: Fix small typo in OpenStack-Health doc  https://review.openstack.org/37769113:26
*** xnox has joined #openstack-infra13:26
openstackgerritMerged openstack-infra/project-config: Enable tlsproxy on "core" tempest jobs  https://review.openstack.org/37577813:29
*** cardeois has joined #openstack-infra13:30
*** esberglu has joined #openstack-infra13:33
*** fguillot has joined #openstack-infra13:33
*** yamamoto has joined #openstack-infra13:35
*** yamamoto has quit IRC13:35
*** PalTale has quit IRC13:36
*** PalTale has joined #openstack-infra13:37
*** _bmjen is now known as bmjen13:38
*** yamamoto has joined #openstack-infra13:38
*** esikachev has quit IRC13:40
*** scottda has joined #openstack-infra13:40
*** tuannguy_ has joined #openstack-infra13:41
*** annegentle has quit IRC13:42
*** salv-orl_ has joined #openstack-infra13:42
*** tuannguyen has quit IRC13:43
*** _oanson has joined #openstack-infra13:44
*** moravec has quit IRC13:44
*** mugsie__ is now known as mugsie13:45
*** jamesdenton has joined #openstack-infra13:46
*** salv-orlando has quit IRC13:46
*** oanson has quit IRC13:46
*** moravec has joined #openstack-infra13:47
*** raildo has joined #openstack-infra13:47
*** eharney has joined #openstack-infra13:48
*** denisra has joined #openstack-infra13:48
*** hferenc has quit IRC13:48
*** jpena is now known as jpena|off13:48
*** jpena|off is now known as jpena13:48
*** hichihara has joined #openstack-infra13:50
*** afonsod__ has joined #openstack-infra13:51
*** denisra_ has quit IRC13:51
*** rossella_ has joined #openstack-infra13:51
*** rossella_ has quit IRC13:51
*** denisra has quit IRC13:53
*** rlandy|mtg is now known as rlandy13:54
*** jordanP has quit IRC13:54
AJaegerclarkb: could you check a xenial change for manila, please? https://review.openstack.org/#/c/377360/13:54
clarkbAJaeger: I am really less interested in these make experimental jobs first unless we know they fail. in most cases everything has just worked13:55
clarkbmaybe we can ask manila if they have knowm failures13:55
openstackgerritMerged openstack-infra/system-config: Add chocolate oscc cloud to puppetmaster  https://review.openstack.org/37759413:56
clarkbat least in the case of neutron I dont think we have made any progress with the experimental jobs13:56
*** rajinir has joined #openstack-infra13:56
*** moravec has quit IRC13:56
*** matbu is now known as matbu|bbl13:56
clarkbthough they do actually seem to fail13:57
*** yamahata has joined #openstack-infra13:57
openstackgerritMerged openstack-infra/elastic-recheck: Add query for ceph vol snapshot bug 1627220  https://review.openstack.org/37577713:57
openstackbug 1627220 in devstack-plugin-ceph "ceph: test_volume_boot_pattern fails with "ImageNotFound: error protecting snapshot"" [Undecided,In progress] https://launchpad.net/bugs/162722013:57
*** yamahata has quit IRC13:57
AJaegervponomaryov: see clarkb's comments above13:58
*** maishsk has joined #openstack-infra13:58
*** yamahata has joined #openstack-infra13:58
AJaegerclarkb: perhaps best to discuss with vponomaryov about proper way forward?13:58
clarkbalso most of those jobs are nonvoting anyways...13:58
clarkbya13:58
vponomaryovAJaeger, clarkb: this "experimental period" is question of 1-2 days13:59
vponomaryovAJaeger, clarkb: problems I expect are devstack-plugin related13:59
clarkbvponomaryov: but they are nonvoting anyways for the most part so why dont we just switch them?13:59
*** eharney has quit IRC14:00
clarkband avoid all this churn14:00
vponomaryovclarkb: if you mean enable them in check - then ny intention was to reduce usage of CI resources14:00
*** moravec has joined #openstack-infra14:00
clarkbvponomaryov: its the same ci resources14:00
*** esikachev has joined #openstack-infra14:00
clarkbbecause trusty runs on old branches and xenial on new14:01
clarkbthe total job count per patch remaims the same14:01
vponomaryovclarkb: master will have twice bigger14:01
clarkbno master should not do that14:01
clarkbmaster will only run xenial14:01
vponomaryovit is not my point14:02
AJaegervponomaryov: we won't approve twice bigger master ;)14:02
vponomaryovit is what will be in case I do what you propose14:02
*** mtanino has joined #openstack-infra14:02
AJaegervponomaryov: what we do elsewhere is: duplicate the job in a trusty and a xenial variant.14:03
clarkbI am proposing you just run xenial on new branches now without a complocated transition because the jobs are already non voting14:03
vponomaryovAJaeger: that is why I did it as experimental14:03
AJaegerRun trusty on mitaka and older, xenial on newton and newer14:03
clarkbdo not continue to run trusty there14:03
*** isaacb has quit IRC14:03
AJaegerthat's no duplication in master14:03
vponomaryovclarkb: there is 3 voting14:03
vponomaryovall my intentions were exactly the same you propose me14:04
vponomaryovjust with intermediate step14:04
clarkbright for those just test if they work or not14:04
vponomaryovwhere I can slowly fix every problem14:04
vponomaryov))14:04
clarkbif they dont then do the experimental setup for themuntil they work14:04
clarkbif they do work just change them14:04
clarkbIf you would like help testing I can do that14:05
AJaegervponomaryov: switch over a single job this way that is non-voting, then the next one.14:05
vponomaryovclarkb: switch only non-voting or all of new?14:05
clarkbmy usual process has been to take the reproduce.sh for a job that passed on a change that merged and run that on a xenial vm14:05
rajinirThirdparty CI which was working fine is broken now. Devstack fails with Bad subnets request: allocation_pools allowed only for specific subnet requests... Any idea?14:05
*** hongbin has joined #openstack-infra14:05
pabelangergood news everybody! ubuntu-xenial works in rax-iad again, just manually tested it.14:06
pabelangeronly trusty and xenial were uploaded yesterday, the rest will happen today14:06
clarkbvponomaryov: ideally all of them but if you know the voting jobs wont pass then put them in experimental14:06
AJaegerpabelanger: great!14:06
vponomaryovclarkb: I don't know for sure14:06
clarkbrajinir: can you link to logs?14:07
vponomaryovclarkb: just don't have xenial handy14:07
AJaegervponomaryov: then let's do a first change with non-voting jobs.14:07
AJaegervponomaryov: you can build one yourself using diskimagebuilder14:07
AJaegervponomaryov: see project-config/tools/build-images.sh14:07
*** amitgandhinz has joined #openstack-infra14:08
*** senk_ has quit IRC14:08
*** moravec1 has joined #openstack-infra14:08
vponomaryovAJaeger: ok, I will upload change for non-voting only first14:08
clarkbor I can help run tests for those too14:08
rajinirclarkb: Looks like it might be broken or something new is introduced in neutron. Cisco CI also complained https://stash.opencrowbar.org/logs/71/295371/7/silent/dell-hw-tempest-dsvm-ironic-pxe_ipmitool/716d0e614:08
vponomaryovAJaeger, clarkb: thank you14:08
clarkbvponomaryov: if you link to the reproduce.sh log for voting jobs tjat passed I can wotk on running them on xenial14:08
AJaegervponomaryov: or point clarkb to a successfull run and he can test them for you ^14:08
AJaegervponomaryov: and note: We run xenial on newton and newer!14:09
vponomaryovAJaeger: it is obvious, for me14:09
vponomaryovAJaeger: but thank you for reminding ))14:09
rajinirclarkb: Logs https://stash.opencrowbar.org/logs/77/369777/16/silent/dell-hw-tempest-dsvm-ironic-pxe_ipmitool/a0b65c314:10
*** xarses has quit IRC14:10
*** moravec has quit IRC14:10
clarkbrajinir: I get a 404 on that last url14:10
*** Julien-zte has joined #openstack-infra14:11
rajinirclarkb: Logs take 30 mins to sync to the external server. Try this https://stash.opencrowbar.org/logs/77/369777/16/silent/dell-hw-tempest-dsvm-ironic-pxe_ipmitool/a0b65c314:11
*** rbrndt has joined #openstack-infra14:11
*** Goneri has joined #openstack-infra14:12
clarkbrajinir: same thing with all of them. Maybe you can paste the relevant section of the logs somewhere?14:12
AJaegerclarkb, try https://stash.opencrowbar.org/logs/77/369777/16/silent/dell-hw-tempest-dsvm-ironic-pxe_ipmitool/d01457c/14:14
AJaegerjust go up one dir and try ;)14:14
rajinirclarkb: Are you able to see this https://10.49.81.37/logs/53/377653/1/check/dell-hw-tempest-dsvm-ironic-pxe_drac/21630cc/14:16
AJaegerrajinir: but there're earlier failures as well - like "ERROR: could not access cell mapping database - has api db been created?"14:16
*** akijak has joined #openstack-infra14:17
clarkbrajinir: pretty sure the issue there is the subnetpool is created in 10/8 but then you try to make an allocation out of 172.16.99.0/2414:17
clarkbso that error is correct.14:17
*** eharney has joined #openstack-infra14:18
*** jordanP has joined #openstack-infra14:18
openstackgerritMatt Riedemann proposed openstack-infra/devstack-gate: DNM: test nova-manage db archive_deleted_rows post_test_hook  https://review.openstack.org/37711314:20
*** john-davidge has quit IRC14:20
rajinirclarkb: It used to work till yesterday14:20
akijakhi, I get 'Code Review - Error Server Error Cannot store contact information' on Gerrit. Is there a problem right now?14:20
openstackgerritAndrew Laski proposed openstack-infra/project-config: Run Nova post_test_hook on placement job  https://review.openstack.org/37653714:21
*** maishsk has quit IRC14:21
clarkbrajinir: amybe the ip ranges changed?14:23
clarkbrajinir: you'd likely have better luck talking to devstack and/or neutron14:23
rajinirclarkb: sure thanks14:23
*** burgerk has quit IRC14:23
clarkbakijak: usually that means that a step was missed during account setup. Are you following any specific documentation for account setup?14:23
*** sdake has quit IRC14:24
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Move periodic-python-jobs-with-neutron-lib-master to xenial  https://review.openstack.org/37773814:24
AJaegerclarkb, what do you think of the above change ? ^14:24
clarkbakijak: http://docs.openstack.org/infra/manual/developers.html#account-setup is the one we would like people to use14:24
akijakclarkb: no, I just did 'get review' and got 'A Contributor Agreement must be completed before uploading' which I did signed few months ago, strange14:26
clarkbakijak: ok, if you read through that account setup doc it should show you how to sign the contributor license agreement which will make that error go away14:27
*** zz_dimtruck is now known as dimtruck14:27
*** adrian_otto has joined #openstack-infra14:27
clarkbAJaeger: that change looks fine. Except I wonder if it ever ran with python3 at all since the env name isn't py34 or py3514:27
clarkbAJaeger: not sure if tox is magical that way14:27
narayrakLooks like FIXED_RANGE variable is not honoured via localrc and its defaults to 10.0.0.0/8 this has broken HPE CI as it route interfered with our Floating range14:27
*** isaacb has joined #openstack-infra14:29
narayrakseems like this is happening from yesterday14:29
*** moravec1 has quit IRC14:29
clarkbnarayrak: ya rajinir was looking at that too.14:29
clarkblooks like there was a renaming of neutron variables in devstack?14:30
akijakclarkb: I've already sign that but without 'Contact information' and now it asks be to update it but I get the error :/14:31
*** tuannguy_ has quit IRC14:31
clarkbakijak: you'll need to make sure that your foundation account primary email matches your gerrit primary email address (this should be covered in that doc but this is the primary key that is used)14:32
*** amotoki_ has quit IRC14:33
AJaegerclarkb: ah, let's check - I'll follow your suggestion for the environment...14:33
openstackgerritMerged openstack-infra/system-config: Add launcher layouts for chocolate admin cloud  https://review.openstack.org/37761914:33
*** tuannguyen has joined #openstack-infra14:33
akijakclarkb: email on ttps://review.openstack.org/#/settings/ and https://launchpad.net/~adam-kijak are the same14:33
narayrakclarkb: thanks, not sure with new variable, will explore14:34
clarkbakijak: not launchpad. your foundation account. The second paragraph of the link I provided14:34
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Move periodic-python-jobs-with-neutron-lib-master to xenial  https://review.openstack.org/37773814:34
AJaegerclarkb: updated ^14:34
*** _d34dh0r53_ is now known as d34dh0r5314:34
*** amitgandhinz has quit IRC14:35
*** amitgandhinz has joined #openstack-infra14:36
akijakclarkb: ok, got it. indeed it doesn't match with foundation account. I'll try to fix that. Thanks!14:36
*** adrian_otto has quit IRC14:36
*** senk_ has joined #openstack-infra14:37
*** sdake has joined #openstack-infra14:37
*** adrian_otto has joined #openstack-infra14:37
*** dimtruck is now known as zz_dimtruck14:37
*** marst has quit IRC14:38
*** danielitit has quit IRC14:39
openstackgerritDan Smith proposed openstack-infra/project-config: Run Nova post_test_hook on placement job  https://review.openstack.org/37653714:39
openstackgerritAndrew Laski proposed openstack-infra/project-config: Run Nova post_test_hook on placement job  https://review.openstack.org/37653714:40
openstackgerritMatt Riedemann proposed openstack-infra/devstack-gate: DNM: test nova-manage db archive_deleted_rows post_test_hook  https://review.openstack.org/37711314:41
*** amitgandhinz has quit IRC14:42
*** amitgandhinz has joined #openstack-infra14:42
zigoAJaeger: clarkb: Any idea why deb-python-xstatic-{tv4,objectpath,angular-schema-form} aren't getting filled with content?14:46
zigoIt's blocking our Horizon packaging, and we're getting really nervous about it, 3 days from the release.14:46
*** burgerk has joined #openstack-infra14:48
clarkbwhen were they added? the last logs I have for manage projects on gerrit are from the 23rd and those don't show up there (well angular and tv4)14:49
*** hichihara has quit IRC14:49
clarkbya they don't appear to be gerrit projects14:53
*** marst has joined #openstack-infra14:53
*** senk_ has quit IRC14:53
clarkbPLAY [review.openstack.org:!disabled] says skipping: no hosts matched14:53
clarkbthat explains why no manage projects/puppet is happening14:53
*** senk_ has joined #openstack-infra14:53
clarkband it is in the emergency file14:53
openstackgerritLucas Alvares Gomes proposed openstack-infra/project-config: Ironic: Replace *_ssh jobs with *_ipmitool  https://review.openstack.org/37699014:53
clarkbinfra-root anyone know why that is?14:53
*** denisra has joined #openstack-infra14:53
openstackgerritMatthew Treinish proposed openstack-infra/elastic-recheck: Handle missing ES indexes in SearchEngine.search()  https://review.openstack.org/37776214:53
*** adrian_otto has quit IRC14:53
clarkbzigo: I think its just a matter of reenabling puppet on the gerrit server but I don't want to do that without knowing why it was disabled14:53
zigoclarkb: So I should wait for fungi to get up?14:53
jeblairclarkb: oh, that was me for the launchpad thing yesterday, sorry14:53
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config: networking-midonet: Provide gate hooks for the grenade job  https://review.openstack.org/37776414:54
jeblairclarkb: i can remove now if you like14:54
*** afonsod__ has quit IRC14:54
clarkbjeblair: that would be great14:55
jeblairdone14:55
openstackgerritValeriy Ponomaryov proposed openstack-infra/project-config: [Manila] Add xenial tempest jobs  https://review.openstack.org/37736014:55
*** edtubill has joined #openstack-infra14:55
*** armax has joined #openstack-infra14:56
*** flepied has quit IRC14:56
*** senk_ has quit IRC14:57
*** roxanaghe has joined #openstack-infra14:58
openstackgerritValeriy Ponomaryov proposed openstack-infra/project-config: [Manila] Add xenial tempest jobs  https://review.openstack.org/37736014:58
*** trukise_ has joined #openstack-infra14:59
*** denisra has quit IRC14:59
*** denisra_ has joined #openstack-infra14:59
fungistevemar: on https://review.openstack.org/377170 i'm not a lawyer but advise you to read https://www.apache.org/licenses/LICENSE-2.0#apply and see what conclusions you come to14:59
*** trukise_ has left #openstack-infra15:00
clarkbhe just plays one on tv15:00
olaphor maybe he stayed at a holiday inn15:01
clarkbthats autogenerated by sphinx right?15:01
stevemarfungi: thanks for the reference fungi15:01
*** ssancheztrujillo has joined #openstack-infra15:01
stevemarclarkb: yes, but it's also in the project cookie cutter template15:01
*** persia__ has quit IRC15:01
stevemarwhich i think most people use...15:01
zigofungi: Any idea about what we just talked with clarkb? How come the new python-xstatic-{tv4,objectpath,angular-schema-form} aren't being populated? Did you disable the puppet stuff?15:01
clarkbsure but if its autogenerated then sphinx is the people who should be asserting copyright/licensing (also I am not a lawyer either)15:02
*** bin_ has joined #openstack-infra15:02
clarkbzigo: jeblair explained it above...15:02
zigoOh ok ! :)15:02
*** zhurong has quit IRC15:02
stevemarclarkb: maybe i'll send a note to legal-discuss15:02
*** mrunge_ has quit IRC15:02
*** EricGonc_ has quit IRC15:02
zigoclarkb: But now, I should wait for another patch to get in project-config?!?15:02
clarkbzigo: no, just need to wait for puppet to run again15:03
zigoOk, fair enough then.15:03
*** isaacb has quit IRC15:03
zigo:)15:03
*** r-mibu has quit IRC15:03
zigoThanks a lot.15:03
*** mrunge has joined #openstack-infra15:03
*** mhickey has joined #openstack-infra15:03
*** r-mibu has joined #openstack-infra15:03
*** spzala has joined #openstack-infra15:03
*** Ravikiran_K has quit IRC15:04
*** persia has joined #openstack-infra15:04
fungialso waiting for that upstream url fix for deb-python-jsonrpclib to end up on the server so that manage-projects doesn't hang15:04
*** mriedem is now known as mriedem_meeting15:05
zigofungi: Didn't that one land yesterday evening?15:05
openstackgerritEmilien Macchi proposed openstack-infra/project-config: Convert tripleo-container jobs into multinode  https://review.openstack.org/37700015:05
fungizigo: it did, but puppet's been disabled on review.o.o for about 24 hours15:05
zigoAh...15:06
fungifallout from dealing with the openid outage for launchpad sso yesterday15:06
*** pcaruana has quit IRC15:06
*** annegentle has joined #openstack-infra15:06
*** yamamoto has quit IRC15:06
*** rcernin has quit IRC15:07
*** esikachev has quit IRC15:10
*** pilgrimstack has quit IRC15:10
*** tuannguyen has quit IRC15:12
*** pilgrimstack has joined #openstack-infra15:12
*** tuannguyen has joined #openstack-infra15:13
*** thcipriani|afk is now known as thcipriani15:13
*** mdrabe has quit IRC15:14
*** amotoki has joined #openstack-infra15:15
*** lucasagomes is now known as lucas-hungry15:16
*** tuannguyen has quit IRC15:17
*** openstackgerrit has quit IRC15:18
*** openstackgerrit has joined #openstack-infra15:18
*** tosky has joined #openstack-infra15:19
openstackgerritRodolfo Alonso Hernandez proposed openstack-infra/project-config: Open a repository for NFV filters.  https://review.openstack.org/37779415:19
*** denisra_ has quit IRC15:20
*** mdrabe has joined #openstack-infra15:20
clarkbgreghaynes: mtreinish sdague http://logs.openstack.org/99/342299/5/check/gate-tempest-dsvm-full-ubuntu-xenial/8ba3bed/logs/tempest.txt.gz#_2016-09-27_14_48_35_155 check that out15:21
*** zz_dimtruck is now known as dimtruck15:21
mtreinishclarkb: hehe, nice15:22
mtreinishclarkb: it's the future now15:22
*** chem has quit IRC15:23
stevemarfungi: there are 1000+ repos under the openstack namespace correct?15:24
*** cardeois_ has joined #openstack-infra15:25
*** cardeois has quit IRC15:25
*** tuannguyen has joined #openstack-infra15:25
*** yamahata has quit IRC15:26
*** cardeois has joined #openstack-infra15:26
pleia2stevemar: yeah, there are15:26
stevemarpleia2: oof15:26
stevemarnot including openstack-infra or openstack-dev15:26
mat128stevemar: "public_repos": 1286 for "openstack" alone, according to github15:27
*** andreas_s has quit IRC15:27
mat128stevemar: 15 under -dev, 155 under -infra15:27
mat128stevemar: https://api.github.com/users/openstack-infra15:27
clarkbyou can ssh gerrit gerrit ls-projects | grep | wc -l15:27
pleia2mat128: github seems to be missing a few then, best to go by numbers on git.o.o15:28
pleia21289 on git.openstack.org15:28
*** esikachev has joined #openstack-infra15:28
*** ralonsoh has joined #openstack-infra15:28
pleia2or ask gerrit, as clarkb says :)15:28
*** amotoki has quit IRC15:29
mat128pleia2: that's surprising, were they just created?15:29
pleia2mat128: I don't know, just noting that your number is different than mine (I don't have a iff)15:29
pleia2diff15:29
*** rbrndt has quit IRC15:29
*** cardeois_ has quit IRC15:30
stevemarpleia2: mat128 i just wrote it as "1000+" :)15:30
*** cardeois_ has joined #openstack-infra15:31
pleia2stevemar: accurate! ;)15:31
*** dimtruck is now known as zz_dimtruck15:31
*** cardeois has quit IRC15:31
stevemarpleia2: i was actually surprised by the number, didn't realize it was over 100015:31
*** rbrndt has joined #openstack-infra15:32
*** yamamoto has joined #openstack-infra15:33
*** rohanion has joined #openstack-infra15:33
dmsimardAnyone have that Zuul console telnet script handy ? I might have an older version that support v6 and it breaks horribly.15:35
*** daniel1 has joined #openstack-infra15:36
*** daniel1 is now known as Odd_Bloke15:36
zulhi, we aprroved this yesterday but it hasnt been merged yet? anyone know whats gonig on? https://review.openstack.org/#/c/377075/15:37
clarkbzul: gerrit says "Cannot merge"15:37
*** senk_ has joined #openstack-infra15:37
clarkbzul: if you hover over that string you get more info15:37
zulclarkb: *sigh* ok15:38
*** psachin has quit IRC15:39
fungimat128: there are almost always some in the process of being created15:40
fungiwith as many repos as we have, the churn is constant15:40
anteayaperhaps our mascot should have been a butter churn15:42
*** ssancheztrujillo has quit IRC15:42
pabelangeryes!15:42
*** senk_ has quit IRC15:42
*** cardeois_ has quit IRC15:43
*** mptacekx has quit IRC15:43
jeblairfungi: meeting agenda updated15:44
*** esikachev has quit IRC15:44
jeblairclarkb, fungi: how terrible is it if we stop nodepool and lose the image builds?15:45
fungithanks for the heads up, jeblair15:45
clarkbjeblair: I think pabelanger has rackspace image updates that are semi important. They include glean fixes for ipv6 and config drive configurations15:45
fungijeblair: probably not at all terrible since the broken repos list is likely causing them all to fail when caching git repos anyway15:45
*** amotoki has joined #openstack-infra15:45
jeblairfungi, clarkb: thanks, that helps me figure out the situation, though https://review.openstack.org/377123 did not land so i guess i have a bit more work to do first :)15:46
*** adrian_otto has joined #openstack-infra15:46
* jeblair actually runs tests locally this time15:46
*** sandanar_ has quit IRC15:46
*** cardeois has joined #openstack-infra15:47
*** rbrndt has quit IRC15:47
*** xarses has joined #openstack-infra15:47
*** tuannguyen has quit IRC15:47
*** zz_dimtruck is now known as dimtruck15:48
*** panda is now known as panda|bbl15:49
*** mdrabe has quit IRC15:50
*** mdrabe has joined #openstack-infra15:50
openstackgerritDmitry Tantsur proposed openstack-infra/project-config: Switch ironic-inspector to tempest discovery job  https://review.openstack.org/37762315:52
pabelangerjeblair: clarkb: I'm fine with losing them if needed.15:53
pabelangerclarkb: jeblair: infact, image builds failed today because of git repos zigo is depending on15:55
openstackgerritGuillaume Espanel proposed openstack-infra/project-config: Create puppet-cloudkitty repository  https://review.openstack.org/36453515:57
*** mhickey has quit IRC15:58
*** tnovacik has quit IRC15:59
*** amitgandhinz has quit IRC16:00
*** dimtruck is now known as zz_dimtruck16:01
*** EricGonczer_ has joined #openstack-infra16:02
*** tosky has quit IRC16:03
*** csomerville has joined #openstack-infra16:04
*** tnovacik has joined #openstack-infra16:04
*** gyee has joined #openstack-infra16:05
*** cody-somerville has quit IRC16:07
*** esikachev has joined #openstack-infra16:07
*** jlanoux has quit IRC16:08
*** inc0 has quit IRC16:09
*** amitgandhinz has joined #openstack-infra16:10
*** salv-orl_ has quit IRC16:11
*** salv-orlando has joined #openstack-infra16:12
*** matrohon has quit IRC16:12
*** vinaypotluri has joined #openstack-infra16:14
*** dprince has quit IRC16:14
*** drifterza has quit IRC16:14
*** lucas-hungry is now known as lucasagomes16:16
*** dprince has joined #openstack-infra16:16
*** jaosorior has quit IRC16:16
dhellmannhey, folks, I'm getting errors from apt-get about an unsigned repository, so the release jobs fail and the change won't merge: http://logs.openstack.org/96/377696/4/gate/gate-releases-tox-validate-ubuntu-xenial/91db446/console.html#_2016-09-27_16_06_33_18133916:18
dhellmannlooks like maybe an osic issue?16:18
* dhellmann wonders if that's the "broken repos list" issue fungi was referring to16:19
clarkbdhellmann: that is true of our ubuntu and debian mirrors but we should configure apt to be ok with that16:19
clarkbthe errors are a little lower16:19
clarkbthe hash mismatches16:20
dhellmannah16:20
sdaguedhellmann: http://logs.openstack.org/96/377696/4/gate/gate-releases-tox-validate-ubuntu-xenial/91db446/console.html#_2016-09-27_16_06_33_181527 is actually where we start to Error "E:"16:20
zaromorning16:20
clarkbzaro: good morning16:20
dhellmannsdague , clarkb : I've never hit this error before. is this something I can deal with? should I recheck and hope for a different node?16:21
*** ptoohill has quit IRC16:21
sdagueclarkb: could we be overloading that mirror?16:22
clarkbdhellmann: different node likely won't help as its the same mirror hosted everywhere with afs16:22
sdagueosic did double it's nodes16:22
clarkbI am looking at the logs really quickly to see what they say16:22
fungidhellmann: yeah, unrelated to the "broken repos list" i referred to (gerrit/projects.yaml includes newly added projects that hadn't been created in gerrit yet, which causes our image updates to fail when attempting to cache those nonexistent repos)16:22
dhellmannfungi : ok, thanks16:22
clarkbhttp://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=302 looks fine16:23
*** jtomasek has quit IRC16:23
*** szaher_ has quit IRC16:23
pabelangerwe did pull down some new packages with reprepro last run16:23
pabelangerpossible we didn't get everything16:23
clarkbat 1600UTC we did update xenial updates16:23
pabelangerhowever, looks like changes to security repo16:24
dhellmannyes, there were a few patches in the gate queue before this one and those passed16:24
clarkbI think this is actually a race with mirror updates16:24
*** ldnunes has quit IRC16:24
clarkbit failed during the update16:24
clarkbso while its downloading Packages.gz the underlying file changes which results in more bytes and different hash sum16:24
dhellmannoh, fun16:25
pabelangerya, that might explain it16:25
dhellmannshall I recheck and see how that goes?16:25
clarkbI would expect that once apache opens Packages.gz it will get the right one16:25
*** ldnunes has joined #openstack-infra16:25
clarkbdhellmann: yes I think try a recheck16:25
pabelangerWrong checksum during receive of 'http://security.ubuntu.com/ubuntu/dists/xenial-security/main/binary-amd64/Packages.xz':16:25
pabelangerI see that in longs16:25
dhellmannok, rechecking16:26
clarkbI wonder if apache gets the file length by checking the dirent16:26
*** baoli_ has quit IRC16:26
clarkbso get direct, updates, read file would be the race16:26
clarkbs/direct/dirent/16:26
pabelangerThere have been errors!16:27
irtermitesdague: have you all confirmed it was the doubling that caused the issue? Too much for the mirrors to handle?16:27
pabelangerso, something is up with reprepro, cause that is in our mirror log16:27
clarkbirtermite: no I think the mirror is fine performance wise16:27
fungizigo: openstack/deb-python-jsonschema has the same issue you fixed for openstack/deb-python-jsonrpclib in 37711716:27
irtermiteclarkb: ack16:27
clarkbirtermite: this has to do with the actual content I think16:27
zigo:/16:27
zigofungi: Doing the patch.16:27
irtermitethank you16:27
clarkbpabelanger: where di you see that?16:27
clarkbpabelanger: I am looking at the reprepro xenial updates log and don't see that16:27
dmsimardI see you guys discussing mirror issues, is that for Temporary failure resolving 'mirror.regionone.osic-cloud1.openstack.org' ?16:27
clarkbdmsimard: no16:28
pabelangerclarkb: /var/log/reprepro/ubuntu-mirror.log16:28
pabelangermirror-update.o.o16:28
clarkbpabelanger: thanks16:28
dhellmannclarkb, pabelanger : it appears the jobs are running ok now16:28
pabelangerdmsimard: where did you see that?16:28
*** szaher has joined #openstack-infra16:28
*** berendt has quit IRC16:28
fungizigo: the good news is i see no other lines in that file matching upstream:.*\.$16:29
clarkbpabelanger: the error in that log is basically the same error we got downstream16:29
zigofungi: https://review.openstack.org/37784316:29
dmsimardpabelanger: I've seen it happen twice now, in the same job: http://logs.openstack.org/60/337860/58/check/gate-puppet-openstack-integration-4-scenario003-tempest-ubuntu-xenial/516709e/console.html#_2016-09-27_15_54_51_053878 && http://logs.openstack.org/60/337860/58/check/gate-puppet-openstack-integration-4-scenario003-tempest-ubuntu-xenial/10e45c7/console.html#_2016-09-27_15_19_06_04790716:29
pabelangerclarkb: looks like we tried to update at 2016-09-27T14:00:01,824587186+0000 and failed16:29
clarkbpabelanger: I think if dhellmanns jobs are happy then this is just a race in how Packages.gz is served16:29
pabelangerbut worked at 2016-09-27T16:00:01,570026553+000016:29
zigofungi: I got my 3 missing deb-python-xstatic gerrit repo up and running, so I can finally do Horizon ! :)16:29
clarkband we hit it with the upstream source then jobs hit it with our downstream mirror too16:29
*** ldnunes_ has joined #openstack-infra16:29
clarkbwoo apt16:29
dmsimardpabelanger: If it's not occuring anywhere else, it might be my particular review -- it's related to designate implementation so maybe there's some underlying DNS condition I'm introducing as part of that. I'll double check.16:30
*** matbu|bbl is now known as matbu16:30
pabelangerclarkb: agreed16:30
zigofungi: FYI, everything else is already packaged as RC1...16:30
zigo:)16:30
zaroclarkb: what's the possibility of running a zuul-dev to trigger on jobs from review-dev?16:30
*** ekarlso_ has quit IRC16:31
clarkbdmsimard: pabelanger we check dns resolution prior to putting nodes into action with a nodepool ready script. So dns should work when your job starts16:31
mordredjeblair: I think I'm past meetings for a bit - what, if anything, do you need from me related to nodepool?16:31
pabelangerdmsimard: I'd check to see if puppet is somehow changing /etc/reslov.conf, it would be using unbound16:31
jeblairmordred: nothing at the moment; waiting for updated https://review.openstack.org/377123 to pass/land16:32
*** annegentle has quit IRC16:32
mordredjeblair: cool16:32
dmsimardclarkb, pabelanger: yeah, like I said, if it's not a general occurence (I'll look at logstash?) .. I will investigate if I'm doing something wrong.16:32
*** jordanP has quit IRC16:32
*** ldnunes has quit IRC16:32
fungizaro: not sure if you saw yesterday, but i added the jgit upgrade discussion to our meeting agenda for today. do you expect to be around for the meeting?16:33
jeblairmordred, clarkb, fungi: jhesketh suggested a minor release of zuul to get the zuul-cloner changes... i think that's fine.  any objections?16:33
*** zz_dimtruck is now known as dimtruck16:33
fungijeblair: no, i agree it would be a good idea based on that thread16:33
*** hongbin_ has joined #openstack-infra16:34
zarofungi: cool, you beat me to it.  thanks.  i'll be around16:34
*** [HeOS] has joined #openstack-infra16:34
*** roxanagh_ has joined #openstack-infra16:34
fungizaro: excellent. i hoped that would be the case16:34
*** lezbar has joined #openstack-infra16:34
*** tonytan_brb has joined #openstack-infra16:34
*** asselin__ has joined #openstack-infra16:34
*** burgerk_ has joined #openstack-infra16:34
*** esikache1 has joined #openstack-infra16:34
*** kaisers__ has joined #openstack-infra16:35
*** david-lyle_ has joined #openstack-infra16:35
*** tongli_ has joined #openstack-infra16:35
*** rfolco_ has joined #openstack-infra16:35
*** zhenguo_ has joined #openstack-infra16:36
*** zhenguo has quit IRC16:36
*** ildikov has quit IRC16:36
*** mrunge has quit IRC16:36
*** kambiz has quit IRC16:36
*** unrahul has quit IRC16:36
*** njohnston has quit IRC16:36
*** patrickeast has quit IRC16:36
*** kong has quit IRC16:36
*** ggherdov- has quit IRC16:36
*** kuromagi has quit IRC16:36
*** makowals has quit IRC16:36
*** pots has quit IRC16:36
*** alaski has quit IRC16:36
*** fmccrthy has quit IRC16:36
*** xgerman has quit IRC16:36
*** jraim has quit IRC16:36
*** samueldmq has quit IRC16:36
*** kuromagi has joined #openstack-infra16:36
*** asingh_ has quit IRC16:36
*** mtanin___ has joined #openstack-infra16:36
*** katyafervent_awa has quit IRC16:36
*** roxanaghe has quit IRC16:36
*** Qiming has quit IRC16:36
*** Odd_Bloke has quit IRC16:36
*** edtubill has quit IRC16:36
*** puiterwijk|cld has quit IRC16:36
*** armax has quit IRC16:36
*** rtheis has quit IRC16:36
*** Shrews has quit IRC16:36
*** fabo_ has quit IRC16:36
*** lezbar__ has quit IRC16:36
*** dansmith has quit IRC16:36
dmsimardpabelanger: unbound on Ubuntu too ?16:37
*** jzerebecki has quit IRC16:37
*** thorst has quit IRC16:37
*** yee3793 has quit IRC16:37
*** andymccr has quit IRC16:37
*** EricGonc_ has joined #openstack-infra16:37
*** manfred has joined #openstack-infra16:37
*** daniel has joined #openstack-infra16:37
*** mattoliverau has quit IRC16:37
*** esikachev has quit IRC16:37
*** bin_ has quit IRC16:37
*** marst has quit IRC16:37
*** hongbin has quit IRC16:37
*** tbarron has quit IRC16:37
*** chmouel has quit IRC16:37
*** dkehn_ has quit IRC16:37
*** jlvillal has quit IRC16:37
*** derekh has quit IRC16:37
*** snarwade has quit IRC16:37
*** ajafo has quit IRC16:37
*** narayrak has quit IRC16:37
*** kun_huang has quit IRC16:37
*** warpc has quit IRC16:37
*** yee379 has joined #openstack-infra16:37
*** pilgrimstack1 has joined #openstack-infra16:37
*** doffm has joined #openstack-infra16:37
*** salv-orlando has quit IRC16:37
*** jimbaker has quit IRC16:37
*** bexelbie- has joined #openstack-infra16:37
*** gtmanfred has quit IRC16:37
*** dmanchado has joined #openstack-infra16:37
*** jlk` has joined #openstack-infra16:37
*** szaher has quit IRC16:37
*** david-lyle has quit IRC16:37
*** kragniz has quit IRC16:37
*** Douhet has quit IRC16:37
*** jimbaker` has joined #openstack-infra16:37
*** dmanchado has quit IRC16:37
*** dmanchado has joined #openstack-infra16:37
*** manfred is now known as gtmanfred16:37
*** Adri2000_ has joined #openstack-infra16:37
*** mriedem_meeting has quit IRC16:37
*** jasondotstar has quit IRC16:37
*** serverascode has quit IRC16:37
*** pkoniszewski has quit IRC16:37
*** dmanchad has quit IRC16:37
*** Guest22950 has quit IRC16:37
*** MarkAtwood has quit IRC16:37
*** stevemar has quit IRC16:37
*** andrewbogott has quit IRC16:37
*** M-docaedo_vector has quit IRC16:37
*** coreyob has quit IRC16:37
*** jlk has quit IRC16:37
*** mrhillsman has quit IRC16:37
*** mfisch has quit IRC16:37
*** Adri2000 has quit IRC16:37
*** clarkb has quit IRC16:37
*** vipul has quit IRC16:37
*** taron has joined #openstack-infra16:37
*** stevemar has joined #openstack-infra16:37
*** tbarron_ has joined #openstack-infra16:37
*** rtheis_ has joined #openstack-infra16:37
*** rohanion has quit IRC16:37
*** jkilpatr has quit IRC16:37
*** Jeffrey4l has quit IRC16:37
*** Marx314 has quit IRC16:37
*** acabot has quit IRC16:37
*** stevebak` has quit IRC16:37
dmsimardpabelanger: seeing dnsmasq messages in Ubuntu syslog16:37
*** admcleod_ has quit IRC16:37
*** med_ has quit IRC16:37
*** ajafo has joined #openstack-infra16:37
*** tdasilva has quit IRC16:37
*** sigmavirus has quit IRC16:37
*** claudiub has quit IRC16:37
*** otherwiseguy has quit IRC16:37
*** HeOS has quit IRC16:37
*** migi has quit IRC16:37
*** rods has quit IRC16:37
*** ggillies_ has quit IRC16:37
*** pleia2 has quit IRC16:37
*** darvon has quit IRC16:37
*** 14WAAHO67 has joined #openstack-infra16:37
*** alaski_ has joined #openstack-infra16:37
*** icey has quit IRC16:37
*** tdasilva- has joined #openstack-infra16:37
*** rfolco has quit IRC16:37
*** rhallisey has quit IRC16:37
*** kaisers_ has quit IRC16:37
*** krtaylor has quit IRC16:37
*** GheRivero has quit IRC16:37
*** megm has quit IRC16:37
*** roaet has joined #openstack-infra16:37
*** Qiming_ has joined #openstack-infra16:37
*** dmanchado is now known as dmanchad16:37
*** rcarrillocruz has quit IRC16:37
*** daniel is now known as Guest1118116:37
*** watersoul has quit IRC16:37
*** dosaboy has quit IRC16:37
*** trash has quit IRC16:37
*** matbu has quit IRC16:37
*** michaelxin has quit IRC16:37
*** mikal has quit IRC16:37
*** jgriffith has quit IRC16:37
*** hughhalf has quit IRC16:37
*** lennyb has quit IRC16:37
*** coreycb has quit IRC16:37
*** redrobot has quit IRC16:37
*** ldnunes_ has quit IRC16:37
*** csomerville has quit IRC16:37
*** tsufiev has quit IRC16:37
*** mnaser has quit IRC16:37
*** puiterwijk has quit IRC16:37
*** devananda has quit IRC16:37
*** BobBall_ has quit IRC16:37
*** camunoz has quit IRC16:37
*** _sigmavirus24 has joined #openstack-infra16:37
*** matt6434 has joined #openstack-infra16:37
*** fabo has joined #openstack-infra16:37
*** sflanigan_ has joined #openstack-infra16:37
*** trash has joined #openstack-infra16:37
*** Guest11181 is now known as Odd_Bloke16:37
*** tbarron_ is now known as tbarron16:37
*** pilgrimstack2 has joined #openstack-infra16:37
*** fabo has quit IRC16:37
*** fabo has joined #openstack-infra16:37
*** sflanigan_ has quit IRC16:37
*** sflanigan_ has joined #openstack-infra16:37
*** sflanigan has quit IRC16:37
*** dmellado has quit IRC16:37
*** thcipriani has quit IRC16:37
*** pabelanger_ has joined #openstack-infra16:37
*** puiterwi1 has joined #openstack-infra16:38
*** pfallenop has quit IRC16:38
*** drawsmcgraw has quit IRC16:38
*** EricGonczer_ has quit IRC16:38
*** burgerk has quit IRC16:38
*** mtanino has quit IRC16:38
*** xnox has quit IRC16:38
*** tongli has quit IRC16:38
*** anteaya has quit IRC16:38
*** asselin_ has quit IRC16:38
*** pabelanger has quit IRC16:38
*** rhallisey_ has joined #openstack-infra16:38
*** anteaya has joined #openstack-infra16:38
*** Julien-zte has quit IRC16:38
*** yolanda has quit IRC16:38
*** thingee has quit IRC16:38
*** krtaylor has joined #openstack-infra16:38
*** xnox_ has joined #openstack-infra16:38
*** alaski_ is now known as alaski16:38
*** njohnston has joined #openstack-infra16:38
*** cardeois has quit IRC16:38
*** liusheng has quit IRC16:38
*** aviau has quit IRC16:38
*** notmyname has quit IRC16:38
*** ramishra has quit IRC16:38
*** sileht has quit IRC16:38
*** aunnam has quit IRC16:38
*** clayton has quit IRC16:38
*** filler has quit IRC16:38
*** Madasi has quit IRC16:38
*** dkehn_ has joined #openstack-infra16:38
*** migi has joined #openstack-infra16:38
*** Odd_Bloke has quit IRC16:38
*** Odd_Bloke has joined #openstack-infra16:38
*** eharney has quit IRC16:38
*** matt-borland has quit IRC16:38
*** gouthamr has quit IRC16:38
*** nherciu_ has quit IRC16:38
*** Hal1 has quit IRC16:38
*** xarses has quit IRC16:38
*** gema_ has quit IRC16:38
*** pilgrimstack has quit IRC16:38
*** jmorgan1_ has joined #openstack-infra16:38
*** firenka has joined #openstack-infra16:38
*** chmouel has joined #openstack-infra16:38
*** drawsmcgraw has joined #openstack-infra16:38
*** kambiz has joined #openstack-infra16:38
*** pfallenop has joined #openstack-infra16:38
*** pfallenop has joined #openstack-infra16:38
*** megm has joined #openstack-infra16:38
*** adrian_otto1 has joined #openstack-infra16:38
*** jzerebecki has joined #openstack-infra16:38
*** kaisers1 has joined #openstack-infra16:38
*** makowals has joined #openstack-infra16:38
*** bexelbie has quit IRC16:38
*** zzzeek has quit IRC16:38
*** coreycb has joined #openstack-infra16:38
*** taron_ has quit IRC16:38
*** bexelbie- is now known as bexelbie16:38
*** tonytan4ever has quit IRC16:38
*** doffm_ has quit IRC16:38
*** puiterwi1 is now known as puiterwijk16:38
*** adrian_otto has quit IRC16:38
*** dprince has quit IRC16:38
*** kaisers has quit IRC16:38
*** tobiash_ has quit IRC16:38
*** roaet_ has quit IRC16:38
*** jmorgan1 has quit IRC16:38
*** jlvillal_ has joined #openstack-infra16:38
*** pkoniszewski has joined #openstack-infra16:38
*** firenka_ has quit IRC16:38
*** amitgandhinz has quit IRC16:38
*** lennyb has joined #openstack-infra16:38
*** Shrews has joined #openstack-infra16:38
*** dmellado has joined #openstack-infra16:38
*** kragniz1 has joined #openstack-infra16:38
*** derekh has joined #openstack-infra16:38
*** gordc has quit IRC16:38
*** links has quit IRC16:38
*** beisner has quit IRC16:38
*** dgonzalez has quit IRC16:38
*** hwoarang_ has quit IRC16:38
*** tphummel has joined #openstack-infra16:38
*** jheroux has quit IRC16:38
*** rossella_s has quit IRC16:38
*** clarkb has joined #openstack-infra16:38
*** Kiall has quit IRC16:38
*** xarses has joined #openstack-infra16:38
*** Julien-zte has joined #openstack-infra16:38
*** szaher has joined #openstack-infra16:38
*** coreyob has joined #openstack-infra16:38
*** hwoarang has joined #openstack-infra16:38
*** cardeois has joined #openstack-infra16:38
*** thcipriani has joined #openstack-infra16:38
*** pleia2 has joined #openstack-infra16:38
*** admcleod has joined #openstack-infra16:38
*** ggillies has joined #openstack-infra16:39
*** xarses has quit IRC16:39
*** tobiash has joined #openstack-infra16:39
*** darvon has joined #openstack-infra16:39
clarkbwoo is it going to be one of those days with freenode?16:39
*** jgriffith has joined #openstack-infra16:39
*** liusheng has joined #openstack-infra16:39
*** xnox_ is now known as xnox16:39
*** jlvillal_ is now known as jlvillal16:39
*** mrunge has joined #openstack-infra16:39
jeblairtagged zuul 2.5.116:39
*** rcarrillocruz has joined #openstack-infra16:39
*** nherciu_ has joined #openstack-infra16:39
*** amitgandhinz has joined #openstack-infra16:39
*** xarses has joined #openstack-infra16:39
*** hongbin_ has quit IRC16:39
*** ramishra has joined #openstack-infra16:39
*** hughhalf has joined #openstack-infra16:39
*** redrobot has joined #openstack-infra16:39
*** vipul has joined #openstack-infra16:39
*** jlvillal is now known as Guest9547716:39
*** gordc has joined #openstack-infra16:40
*** gema has joined #openstack-infra16:40
*** matt-borland has joined #openstack-infra16:40
cloudnullafternoons.16:40
*** redrobot is now known as Guest2778016:40
*** yuanying has quit IRC16:40
*** dansmith has joined #openstack-infra16:40
*** ldnunes_ has joined #openstack-infra16:40
*** hongbin has joined #openstack-infra16:40
*** jmorgan1_ has quit IRC16:40
*** pabelanger_ is now known as pabelanger16:40
*** Kiall has joined #openstack-infra16:40
*** mrhillsman has joined #openstack-infra16:40
*** snarwade has joined #openstack-infra16:40
anteayaclarkb: i'll be prepared to be alone in the meeting again today16:40
*** jak-jak has joined #openstack-infra16:40
*** zhenguo_ is now known as zhenguo16:40
anteayacloudnull: did your town get washed away?16:40
anteayaor just washed?16:40
*** _sigmavirus24 is now known as sigmavirus16:40
*** sigmavirus has joined #openstack-infra16:40
cloudnullanteaya: nope . safe and sound :)16:40
anteayaglad to hear it16:40
*** jak-jak is now known as Guest989316:40
*** petems has quit IRC16:41
*** pilgrimstack1 has quit IRC16:41
*** doffm has quit IRC16:41
*** yee379 has quit IRC16:41
*** yamamoto has quit IRC16:41
*** csmart_ has quit IRC16:41
pabelangerclarkb: greghaynes: If we can get a few more eyes on https://review.openstack.org/#/c/374366/ (runtime-ssh-host-keys element), I'm hoping we could cut a new release of diskimage-builder today16:41
*** Apoorva has joined #openstack-infra16:41
*** watersoul has joined #openstack-infra16:41
cloudnullclarkb: RE: osic repo issues. is that something busted in cloud1 ?16:41
*** electrical has quit IRC16:41
*** DuncanT has quit IRC16:41
*** gouthamr has joined #openstack-infra16:41
cloudnullsomething I need to look into ?16:41
clarkbcloudnull: no I think its more related to how apt works and how we are hosting the pcakges on the mirror16:41
*** tsufiev has joined #openstack-infra16:41
cloudnullanteaya: thanks for asking though :)16:41
anteaya:)16:41
*** aviau has joined #openstack-infra16:41
*** yuanying has joined #openstack-infra16:41
*** yamamoto has joined #openstack-infra16:42
cloudnullok.16:42
*** stevebaker has joined #openstack-infra16:42
*** mfisch has joined #openstack-infra16:42
*** mfisch has quit IRC16:42
*** mfisch has joined #openstack-infra16:42
*** mikal has joined #openstack-infra16:42
*** doffm has joined #openstack-infra16:42
*** jmorgan1 has joined #openstack-infra16:42
*** dimtruck is now known as zz_dimtruck16:43
*** rtheis has joined #openstack-infra16:44
*** trown is now known as trown|lunch16:44
*** lock_ has joined #openstack-infra16:44
*** hrybacki|afk has quit IRC16:44
*** ildikov has joined #openstack-infra16:44
*** csmart has joined #openstack-infra16:44
*** dosaboy has joined #openstack-infra16:44
*** yamahata has joined #openstack-infra16:45
*** yamamoto_ has joined #openstack-infra16:45
*** eranrom_ has joined #openstack-infra16:45
*** dgonzalez has joined #openstack-infra16:45
*** gema has quit IRC16:45
*** Apoorva_ has joined #openstack-infra16:45
*** apuimedo has quit IRC16:45
*** yuanying_ has joined #openstack-infra16:45
*** clayton has joined #openstack-infra16:46
*** yee379 has joined #openstack-infra16:46
*** katyafervent_awa has joined #openstack-infra16:46
*** toabctl_ has joined #openstack-infra16:46
*** gema has joined #openstack-infra16:46
*** hongbin_ has joined #openstack-infra16:46
*** xarses_ has joined #openstack-infra16:46
*** yamahata has quit IRC16:46
*** Guest66676 has quit IRC16:46
*** alex_xu has quit IRC16:46
*** EricGonc_ has quit IRC16:46
*** mrmartin has quit IRC16:46
*** mrunge has quit IRC16:46
*** jmorgan1_ has joined #openstack-infra16:46
*** vipul has quit IRC16:46
*** thcipriani has quit IRC16:46
*** ajafo has quit IRC16:46
*** jesusaur has quit IRC16:46
*** kaisers has joined #openstack-infra16:46
*** ajafo has joined #openstack-infra16:46
*** cburgess_ has joined #openstack-infra16:46
*** nherciu__ has joined #openstack-infra16:47
*** lennyb has quit IRC16:47
*** adrian_otto1 has quit IRC16:47
*** jmorgan1 has quit IRC16:47
*** dtardivel has quit IRC16:47
*** cburgess has quit IRC16:47
*** beisner has joined #openstack-infra16:47
*** beisner has quit IRC16:47
*** 14WAAHO67 has quit IRC16:47
*** mtanino has joined #openstack-infra16:47
*** hughhalf_ has joined #openstack-infra16:47
*** makowals has quit IRC16:47
*** skipp has quit IRC16:47
*** lindsayk_ has quit IRC16:47
*** coreycb has quit IRC16:47
*** pilgrimstack2 has quit IRC16:47
*** sfinucan has quit IRC16:47
*** slaweq has quit IRC16:47
*** liusheng has quit IRC16:47
*** zeus has quit IRC16:47
*** slaweq_ has quit IRC16:47
*** paulobanon has quit IRC16:47
*** eranrom has quit IRC16:47
*** derekh has quit IRC16:47
*** mtanino has quit IRC16:47
*** mtanino has joined #openstack-infra16:47
*** hongbin has quit IRC16:47
*** matt-borland has quit IRC16:47
*** gordc_ has joined #openstack-infra16:47
*** jlk` has quit IRC16:47
*** thcipriani has joined #openstack-infra16:47
*** mrunge_ has joined #openstack-infra16:47
*** sflanigan_ has quit IRC16:47
*** cardeois has quit IRC16:47
*** jzerebecki has quit IRC16:47
*** trash has quit IRC16:47
*** tbarron has quit IRC16:47
*** sdague has quit IRC16:47
*** rbradfor has quit IRC16:47
*** makowals_ has joined #openstack-infra16:47
*** watersoul has quit IRC16:47
*** xarses has quit IRC16:47
*** nherciu_ has quit IRC16:47
*** jgriffith has quit IRC16:47
*** darvon has quit IRC16:47
*** sigmavirus has quit IRC16:47
*** jzerebecki has joined #openstack-infra16:47
*** rbradfor has joined #openstack-infra16:47
*** tbarron has joined #openstack-infra16:47
*** Madasi has joined #openstack-infra16:47
*** mfisch` has joined #openstack-infra16:47
*** Shrews has quit IRC16:47
*** mattoliverau has joined #openstack-infra16:47
*** _oanson has quit IRC16:47
*** dmellado_ has joined #openstack-infra16:47
*** kaisers2 has quit IRC16:47
*** paul___ has quit IRC16:47
*** sslypushenko has quit IRC16:47
*** skath has quit IRC16:48
*** toabctl has quit IRC16:48
*** rtheis_ has quit IRC16:48
*** matt6434 has quit IRC16:48
*** gouthamr has quit IRC16:48
*** admcleod has quit IRC16:48
*** dham1 has quit IRC16:48
*** briancurtin has quit IRC16:48
*** Apoorva has quit IRC16:48
*** mrhillsman has quit IRC16:48
*** mtanin___ has quit IRC16:48
*** kuromagi has quit IRC16:48
*** jamielennox has quit IRC16:48
*** vkmc has quit IRC16:48
*** pahuang has quit IRC16:48
*** jasondotstar has joined #openstack-infra16:48
*** toabctl_ is now known as toabctl16:48
*** zzzeek has joined #openstack-infra16:48
*** mrmartin_ has joined #openstack-infra16:48
*** dansmith has quit IRC16:48
*** hughhalf has quit IRC16:48
*** alex_xu_ has joined #openstack-infra16:48
*** mfisch has quit IRC16:48
*** yuanying has quit IRC16:48
*** gordc has quit IRC16:48
*** kragniz1 has quit IRC16:48
*** Guest95477 has quit IRC16:48
*** dansmith has joined #openstack-infra16:48
*** gouthamr_ has joined #openstack-infra16:48
*** trash_ has joined #openstack-infra16:48
*** alex_xu_ has quit IRC16:48
*** alex_xu_ has joined #openstack-infra16:48
*** vkmc- has joined #openstack-infra16:48
*** coreyob has quit IRC16:48
*** dmellado has quit IRC16:48
*** logan- has quit IRC16:48
*** mrmartin_ is now known as mrmartin16:48
*** rcarrillocruz has quit IRC16:48
*** beisner- has joined #openstack-infra16:48
*** EricGonczer_ has joined #openstack-infra16:48
*** filler has joined #openstack-infra16:48
*** amitgandhinz has quit IRC16:48
*** anteaya has quit IRC16:48
*** yamahata has joined #openstack-infra16:48
*** pahuang_ has joined #openstack-infra16:48
*** yamamoto has quit IRC16:48
*** tobiash has quit IRC16:48
*** ianychoi has quit IRC16:48
*** matt-borland has joined #openstack-infra16:48
*** pahuang_ has quit IRC16:48
*** pahuang_ has joined #openstack-infra16:48
*** med_ has joined #openstack-infra16:48
*** darvon has joined #openstack-infra16:48
*** coreyob has joined #openstack-infra16:48
*** _sigmavirus24 has joined #openstack-infra16:48
*** kuromagi has joined #openstack-infra16:48
fungistevemar: now that i'm catching up on the ml, i see your concern about the license header addition voiced: that it's an autogenerated file. i would be inclined to consider that file not actually source code and therefore not needing a license header16:48
*** sflanigan_ has joined #openstack-infra16:48
*** sflanigan_ has joined #openstack-infra16:48
*** dansmith is now known as Guest374716:48
*** beisner- is now known as beisner16:48
*** rcarrillocruz has joined #openstack-infra16:48
*** vkmc- is now known as vkmc16:48
*** vkmc has quit IRC16:48
*** vkmc has joined #openstack-infra16:48
*** notmyname has joined #openstack-infra16:48
*** med_ is now known as Guest4406416:48
*** fmccrthy has joined #openstack-infra16:48
*** jamielennox has joined #openstack-infra16:48
*** jlk has joined #openstack-infra16:49
*** jlk has quit IRC16:49
*** jlk has joined #openstack-infra16:49
*** cardeois has joined #openstack-infra16:49
*** tobiash has joined #openstack-infra16:49
*** kragniz1 has joined #openstack-infra16:49
fungii initially thought you were just asking whether adding licence headers to apache-licensed files that lacked them was something worth reviewer time16:49
*** acabot has joined #openstack-infra16:49
stevemarfungi: glad i make more sense when i write things out, instead of just shouting random stuff in an irc room :P16:49
dmsimardpabelanger: yeah turns out the implementation of the BIND backend for designate in our tests had bind listen on port 53 which was probably breaking things. Fixed now and let's see what happens. Sorry for the noise.16:49
*** jgriffith has joined #openstack-infra16:49
*** Guest3203 is now known as melwitt16:49
*** _sigmavirus24 is now known as sigmavirus16:49
*** paul___ has joined #openstack-infra16:49
*** matbu has joined #openstack-infra16:49
*** otherwiseguy has joined #openstack-infra16:49
*** vipul has joined #openstack-infra16:49
*** adrian_otto has joined #openstack-infra16:50
*** mnaser has joined #openstack-infra16:50
*** liusheng has joined #openstack-infra16:50
*** thorst has joined #openstack-infra16:50
*** sigmavirus is now known as Guest4509616:50
*** jkilpatr has joined #openstack-infra16:50
*** lennyb has joined #openstack-infra16:50
*** camunoz has joined #openstack-infra16:50
openstackgerritKyle Haley proposed openstack-infra/project-config: Make docs gate non-voting for openstack/quark  https://review.openstack.org/37790816:50
*** sfinucan has joined #openstack-infra16:50
*** anteaya has joined #openstack-infra16:50
*** eharney has joined #openstack-infra16:50
*** aunnam has joined #openstack-infra16:51
*** simonmcc has quit IRC16:51
*** Guest45096 is now known as sigmavirus16:51
*** sigmavirus has joined #openstack-infra16:51
*** jraim has joined #openstack-infra16:51
openstackgerritMerged openstack-infra/project-config: Remove trailing dot in deb-python-jsonschema  https://review.openstack.org/37784316:51
*** jheroux has joined #openstack-infra16:51
*** sileht has joined #openstack-infra16:51
*** coreycb has joined #openstack-infra16:51
*** jesusaur has joined #openstack-infra16:51
*** jianghuaw has quit IRC16:51
*** jklare has quit IRC16:51
*** Hal1 has joined #openstack-infra16:51
*** links has joined #openstack-infra16:51
*** skipp has joined #openstack-infra16:52
*** watersoul has joined #openstack-infra16:52
*** fkautz_ has quit IRC16:52
*** Guest3747 is now known as dansmith16:52
*** rossella_s has joined #openstack-infra16:52
*** devananda has joined #openstack-infra16:53
*** Adri2000_ is now known as Adri200016:53
*** zeus has joined #openstack-infra16:53
*** yolanda has joined #openstack-infra16:53
*** Adri2000 has quit IRC16:53
*** Adri2000 has joined #openstack-infra16:53
*** zeus is now known as Guest6643016:53
*** rods has joined #openstack-infra16:53
*** jianghuaw has joined #openstack-infra16:53
*** logan- has joined #openstack-infra16:54
*** patrickeast has joined #openstack-infra16:54
*** samueldmq has joined #openstack-infra16:54
*** apuimedo has joined #openstack-infra16:54
anteayalooks like I'm getting removed a lot today16:54
*** slaweq has joined #openstack-infra16:55
*** asingh_ has joined #openstack-infra16:55
*** e0ne has quit IRC16:56
*** jlvillal has joined #openstack-infra16:56
*** admcleod has joined #openstack-infra16:56
*** admcleod has joined #openstack-infra16:56
*** jlvillal has quit IRC16:56
*** jlvillal has joined #openstack-infra16:57
openstackgerritMerged openstack-infra/nodepool: Add options to skip deleting or launching nodes  https://review.openstack.org/37712316:57
AJaegerproject-config cores, could you review https://review.openstack.org/#/c/368948/ , please?16:57
*** Guest66666 has joined #openstack-infra16:57
*** warpc has joined #openstack-infra16:58
*** kzaitsev_mb has joined #openstack-infra16:58
*** jklare has joined #openstack-infra16:58
*** sdague has joined #openstack-infra16:58
*** annegentle has joined #openstack-infra16:58
*** mriedem has joined #openstack-infra16:58
*** bin_ has joined #openstack-infra16:58
*** hongbin_ has quit IRC16:58
*** hongbin has joined #openstack-infra16:59
zaroclarkb: did you catch my question earlier, about zuul-dev?16:59
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Only query indexes that exist  https://review.openstack.org/37712916:59
*** salv-orlando has joined #openstack-infra16:59
*** BobBall has joined #openstack-infra16:59
openstackgerritClark Boylan proposed openstack-infra/devstack-gate: Copy ssl certs for multinode  https://review.openstack.org/37321916:59
clarkbzaro: looking17:00
*** BobBall is now known as Guest7562417:00
clarkbzaro: its probably doable, but we likely need to deploy a zuul launcher for it and give it a test instance17:00
*** lindsayk_ has joined #openstack-infra17:00
*** rajinir has quit IRC17:00
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use mod_autoindex on files.o.o  https://review.openstack.org/37791917:00
*** _oanson has joined #openstack-infra17:00
*** slaweq_ has joined #openstack-infra17:00
jeblairpabelanger, mordred: ^ quick afs-docs patch makes http://files.openstack.org/ work17:01
jeblair(currently manually applied)17:01
openstackgerritRodolfo Alonso Hernandez proposed openstack-infra/project-config: Add new project: devstack-plugin-libvirt-qemu  https://review.openstack.org/37792017:01
*** Apoorva has joined #openstack-infra17:01
anteayaIzikP: what do you mean renewing your gerrit CI account? http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-09-27.log.html#t2016-09-27T13:15:3417:02
*** unrahul has joined #openstack-infra17:02
*** Guest66430 is now known as zeus`17:02
anteayaIzikP: what is the name of your gerrit CI account?17:02
zaroclarkb: i think we should do that to prep for a gerrit upgrade. will allow testing from gerrit->zuul->nodepool and back.17:02
*** zeus` is now known as zeus17:02
*** zeus has quit IRC17:02
*** zeus has joined #openstack-infra17:02
clarkbzaro: ok, you'll want to look at puppeting a zuul launcher on zuul-dev17:02
*** amoralej is now known as amoralej|off17:02
clarkbzaro: and we can probably give it a static test instance for now (just sort of hand wave around that to start)17:02
pabelangerjeblair: +217:03
*** andrewbogott has joined #openstack-infra17:03
openstackgerritRichard Theis proposed openstack-infra/project-config: Enable networking-ovn plugin on subnode config  https://review.openstack.org/37792117:03
*** armax has joined #openstack-infra17:03
*** jpena is now known as jpena|off17:04
jeblairpabelanger: do you know the progress of getting the keytab installed on the launchers, and have they been restarted with the version of zuul-launcher that understands the afs publisher?17:04
*** ralonsoh has quit IRC17:04
*** Shrews has joined #openstack-infra17:04
pabelangerjeblair: we just need to +A 371818, and should be ready.  I restarted zuul-launchers last week17:04
*** Apoorva_ has quit IRC17:04
*** roxanagh_ has quit IRC17:05
pabelangerif we have a test job setup, we should be ready17:05
zaroclarkb: thanks i'll look into that.17:06
jeblairpabelanger: +317:06
jeblairpabelanger, AJaeger: i don't *think* we have added an afs publisher yet, have we?17:06
AJaegerinfra-root, https://github.com/openstack/deb-python-xstatic-angular-schema-form is still empty but https://git.openstack.org/cgit/openstack/deb-python-xstatic-angular-schema-form has content - could somebody set it up in github, please?17:06
AJaegerjeblair: no, not that I'm aware off.17:07
pabelangerjeblair: I have not17:07
anteayaAJaeger: I believe fungi is working on untangling that17:07
jeblairokay, how about i do that, maybe for an infra publish job or something?  any suggestions?17:07
anteayaunless I misread backscroll17:07
*** tphummel_ has joined #openstack-infra17:07
*** cardeois_ has joined #openstack-infra17:08
*** nherciu_ has joined #openstack-infra17:08
*** roxanaghe has joined #openstack-infra17:08
*** kzaitsev1ws is now known as kzaitsev_ws17:09
*** puiterwi1 has joined #openstack-infra17:09
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Move periodic-python-jobs-with-neutron-lib-master to xenial  https://review.openstack.org/37773817:09
*** puiterwijk has quit IRC17:09
*** jd__` has joined #openstack-infra17:09
*** puiterwi1 is now known as puiterwijk17:09
AJaegerthanks, anteaya17:09
AJaegerjeblair: just pick a small job for a start - infra-publish looks fine.17:10
*** jmorgan1 has joined #openstack-infra17:10
*** Apoorva_ has joined #openstack-infra17:10
jeblairAJaeger: yeah, looks like it will be easiest to add to all the infra publish jobs17:10
*** notmyname_ has joined #openstack-infra17:10
*** matbu_ has joined #openstack-infra17:10
*** jgriffith_ has joined #openstack-infra17:10
*** dmanchado has joined #openstack-infra17:10
*** coreyob has quit IRC17:10
*** matbu has quit IRC17:10
*** notmyname has quit IRC17:10
*** ajafo has quit IRC17:10
*** kaisers has quit IRC17:10
*** njohnston has quit IRC17:10
pabelangerI can do that now if needed17:10
*** ajafo has joined #openstack-infra17:10
*** acabot has quit IRC17:10
*** Guest44064 has quit IRC17:10
*** matt-borland has quit IRC17:10
*** jzerebecki has quit IRC17:10
*** kaisers2 has joined #openstack-infra17:10
*** notmyname_ is now known as notmyname17:10
*** zeus has quit IRC17:11
jeblairpabelanger: i'm on it17:11
*** nherciu__ has quit IRC17:11
*** jmorgan1_ has quit IRC17:11
*** megm has quit IRC17:11
*** xnox has quit IRC17:11
*** thorst_ has joined #openstack-infra17:11
*** jgriffith has quit IRC17:11
*** dmanchad has quit IRC17:11
*** adrian_otto has quit IRC17:11
pabelangerack17:11
*** dmanchado is now known as dmanchad17:11
*** acabot has joined #openstack-infra17:11
*** unrahul has quit IRC17:11
*** apuimedo has quit IRC17:11
*** logan- has quit IRC17:11
*** stevemar has quit IRC17:11
*** woodster_ has quit IRC17:11
*** peterlisak has quit IRC17:11
*** tomhambleton has quit IRC17:11
*** lindsayk_ has quit IRC17:11
*** makowals_ has quit IRC17:11
*** stevemar has joined #openstack-infra17:11
*** kzaitsev_mb has quit IRC17:11
*** ggherdov- has joined #openstack-infra17:11
*** matt-borland_ has joined #openstack-infra17:11
*** mtanino__ has joined #openstack-infra17:11
*** andrewbogott has quit IRC17:11
*** sdague has quit IRC17:11
*** asingh_ has quit IRC17:11
*** patrickeast has quit IRC17:11
*** dmellado_ has quit IRC17:11
*** alaski has quit IRC17:11
*** ggherdov- has quit IRC17:11
*** ggherdov- has joined #openstack-infra17:11
*** mtanino__ has quit IRC17:11
*** mtanino__ has joined #openstack-infra17:11
*** megm has joined #openstack-infra17:11
*** xnox has joined #openstack-infra17:11
*** EricGonc_ has joined #openstack-infra17:11
*** mnaser has quit IRC17:11
*** rbradfor has quit IRC17:11
*** mdrabe has quit IRC17:11
*** makowals has joined #openstack-infra17:11
*** jak-jak_ has joined #openstack-infra17:11
*** annegentle has quit IRC17:11
*** jheroux has quit IRC17:11
*** eharney has quit IRC17:11
*** camunoz has quit IRC17:11
*** liusheng has quit IRC17:11
*** tbarron has quit IRC17:11
*** mtanino has quit IRC17:11
*** ramishra has quit IRC17:11
*** kambiz has quit IRC17:11
*** coreyob has joined #openstack-infra17:11
openstackgerritMerged openstack-infra/project-config: Integrate neutron-fwaas with neutron-lib CI  https://review.openstack.org/36894817:11
*** sfinucan has quit IRC17:11
*** thorst has quit IRC17:11
*** trash_ has quit IRC17:11
*** dansmith has quit IRC17:11
*** mrmartin has quit IRC17:11
*** links has quit IRC17:11
*** jlk has quit IRC17:11
*** gouthamr_ has quit IRC17:11
*** ldnunes_ has quit IRC17:11
*** Guest9893 has quit IRC17:11
*** mminesh has quit IRC17:11
*** jd__ has quit IRC17:11
*** cardeois has quit IRC17:11
*** jd__` is now known as jd__17:11
*** Julien-zte has quit IRC17:12
*** jhesketh has quit IRC17:12
*** Odd_Bloke has quit IRC17:12
*** jraim has quit IRC17:12
*** tobiash has quit IRC17:12
*** jamielennox has quit IRC17:12
*** darvon has quit IRC17:12
*** pleia2 has quit IRC17:12
*** firenka has quit IRC17:12
*** Guest66666 has quit IRC17:12
*** peterlisak has joined #openstack-infra17:12
jeblairAJaeger, pabelanger: 100% of the instances of doc-publisher-site are docs.openstack.org, so i think maybe we should un-template it?17:12
*** eharney has joined #openstack-infra17:12
*** jraju has joined #openstack-infra17:12
*** jklare has quit IRC17:12
*** filler has quit IRC17:12
*** kzaitsev1mb has joined #openstack-infra17:12
*** njohnston_ has joined #openstack-infra17:12
*** rods has quit IRC17:12
*** clarkb has quit IRC17:12
*** tphummel has quit IRC17:12
*** jklare has joined #openstack-infra17:12
*** clarkb has joined #openstack-infra17:12
*** otherwiseguy has quit IRC17:12
*** mrmartin_ has joined #openstack-infra17:12
*** yolanda has quit IRC17:12
*** EricGonczer_ has quit IRC17:12
clarkbhrm if freenode is going to be flappy today do we want to have a fallback for our meeting?17:12
clarkbseems like there are some semi important items on the topic list17:12
*** darvon_ has joined #openstack-infra17:12
*** armax has quit IRC17:12
*** devananda has quit IRC17:12
*** ldnunes__ has joined #openstack-infra17:12
*** kambiz_ has joined #openstack-infra17:12
*** filler has joined #openstack-infra17:12
*** gouthamr has joined #openstack-infra17:12
*** jgriffith_ is now known as jgriffith17:12
*** tbarron has joined #openstack-infra17:12
*** jheroux has joined #openstack-infra17:12
*** _oanson has quit IRC17:12
*** yolanda_ has joined #openstack-infra17:12
*** Apoorva has quit IRC17:12
*** admcleod has quit IRC17:12
*** liusheng has joined #openstack-infra17:12
*** camunoz has joined #openstack-infra17:12
*** tphummel_ is now known as tphummel17:12
*** tonytan_brb has quit IRC17:12
*** apuimedo_ has joined #openstack-infra17:12
*** rbradfor has joined #openstack-infra17:12
*** adrian_otto has joined #openstack-infra17:12
*** tobiash has joined #openstack-infra17:12
*** mrmartin_ is now known as mrmartin17:12
*** _oanson has joined #openstack-infra17:12
*** dmellado has joined #openstack-infra17:12
*** jlk has joined #openstack-infra17:12
*** logan- has joined #openstack-infra17:12
*** kambiz_ is now known as kambiz17:12
*** jlk has quit IRC17:12
*** jlk has joined #openstack-infra17:12
*** apuimedo_ is now known as apuimedo17:12
fungii think the fallback is our mailing list17:12
*** sslypushenko has joined #openstack-infra17:12
*** Odd_Bloke has joined #openstack-infra17:13
clarkbfungi: sounds good17:13
*** Odd_Bloke has quit IRC17:13
*** Odd_Bloke has joined #openstack-infra17:13
AJaegerclarkb: the py34-with-neutron venv seems to use python3.4: http://logs.openstack.org/periodic/periodic-networking-ovn-py34-with-neutron-lib-master/37295b1/console.html#_2016-09-27_06_02_28_18615817:13
*** sfinucan has joined #openstack-infra17:13
fungii can start ml threads corresponding to each topic if the meeting ends up being untenable17:13
pabelangerjeblair: sure17:13
clarkbAJaeger: neat I guess that means that tox does do magic there17:13
openstackgerritMerged openstack-infra/project-config: Add Grafana dashboard for FWaaS checks  https://review.openstack.org/36894517:13
*** Julien-zte has joined #openstack-infra17:13
AJaeger;)17:13
openstackgerritMerged openstack-infra/project-config: Make neutron-fwaas functional job not experimental  https://review.openstack.org/35932017:13
*** dansmith has joined #openstack-infra17:13
clarkbfungi: I just know I am completely lsot on whatever is going on with afs docs17:13
*** trash has joined #openstack-infra17:13
*** sdague has joined #openstack-infra17:13
clarkbbecause freenode is flappy. (Which is fine I wasn't really working on it anyways but thinking might make meeting hard)17:13
*** jhesketh has joined #openstack-infra17:13
fungii wish the fallback were #openstack-meeting on oftc (i'm already there, fwiw), but worry that not enough people would join and also we don't have the meetbot in there17:14
*** kzaitsev1mb is now known as kzaitsev_mb17:14
*** alaski has joined #openstack-infra17:14
anteayafungi: thanks17:14
pabelangerclarkb: I think we are pretty much ready for test jobs with afsdocs, everything is in place for zuul17:14
AJaegerjeblair: see https://review.openstack.org/37759117:14
*** jamielennox has joined #openstack-infra17:14
AJaegerdinner time, bbl17:14
anteayaAJaeger: enjoy17:14
*** otherwiseguy has joined #openstack-infra17:14
*** MarkAtwood has joined #openstack-infra17:14
*** kragniz1 is now known as kragniz17:15
*** pleia2 has joined #openstack-infra17:15
anteayacould we have a meeting in an etherpad?17:15
*** darvon_ is now known as darvon17:15
openstackgerritKyle Haley proposed openstack-infra/project-config: Make docs gate non-voting for openstack/quark  https://review.openstack.org/37790817:15
*** icey has joined #openstack-infra17:15
fungimeetings are a dialogue, and etherpads are more of a collaboratively edited document (aside from the terribly hacky commenting box). i don't think we can easily hold a large group conversation in there17:16
*** rods has joined #openstack-infra17:17
*** roxanaghe has quit IRC17:17
*** zeus has joined #openstack-infra17:17
*** Guest66666 has joined #openstack-infra17:17
*** harlowja has joined #openstack-infra17:17
*** patrickeast has joined #openstack-infra17:17
anteayafair enough17:17
anteayaa bad idea that floated through my head17:17
*** makowals_ has joined #openstack-infra17:18
funginot a bad idea, just i think the ml is probably our best fallback option17:18
jeblairclarkb: yeah, meeting topic is to resync on afs-docs, since i haven't touched it for a week, i don't know where we are either.  i'm poking at where i think the edges are based on the etherpad: https://etherpad.openstack.org/p/afs-docs17:18
anteayayup17:18
*** ramishra has joined #openstack-infra17:19
*** strigazi is now known as strigazi_AFK17:19
*** kzaitsev1mb has joined #openstack-infra17:19
*** mtanino has joined #openstack-infra17:19
*** gouthamr_ has joined #openstack-infra17:20
*** nadya has quit IRC17:20
*** Shalmazar has joined #openstack-infra17:20
*** armax has joined #openstack-infra17:20
*** apuimedo_ has joined #openstack-infra17:20
openstackgerritLucas Alvares Gomes proposed openstack-infra/project-config: Ironic: Replace *_ssh jobs with *_ipmitool  https://review.openstack.org/37792817:20
*** mrunge has joined #openstack-infra17:21
*** chmouel has quit IRC17:21
*** pabelanger has quit IRC17:21
*** trash_ has joined #openstack-infra17:21
*** gordc has joined #openstack-infra17:21
*** dmellado_ has joined #openstack-infra17:21
*** borland__ has joined #openstack-infra17:21
*** alex_xu has joined #openstack-infra17:21
*** mikal_ has joined #openstack-infra17:21
*** sslypushenko has quit IRC17:21
*** dmellado has quit IRC17:21
*** makowals has quit IRC17:21
*** ggherdov- has quit IRC17:21
*** stevemar has quit IRC17:21
*** mrunge_ has quit IRC17:21
*** Qiming has joined #openstack-infra17:21
*** admcleod has joined #openstack-infra17:21
*** trash has quit IRC17:21
*** bin_ has quit IRC17:21
*** stevemar has joined #openstack-infra17:21
*** artmr has joined #openstack-infra17:21
*** mminesh has joined #openstack-infra17:21
*** chmouel_ has joined #openstack-infra17:21
*** admcleod has quit IRC17:21
*** admcleod has joined #openstack-infra17:21
*** zeus is now known as Guest4538517:21
*** rodrigod` has joined #openstack-infra17:21
*** rbradfor has quit IRC17:21
*** tbarron has quit IRC17:21
*** coreyob has quit IRC17:21
*** mtanino__ has quit IRC17:21
*** sileht has quit IRC17:21
*** hughhalf_ has quit IRC17:21
*** mjblack has quit IRC17:21
*** tuannguyen has joined #openstack-infra17:21
*** hughhalf has joined #openstack-infra17:22
*** harlowja_ has joined #openstack-infra17:22
*** lindsayk_ has joined #openstack-infra17:22
*** beekneemech has joined #openstack-infra17:22
*** Guest66676 has joined #openstack-infra17:22
*** Qiming_ has quit IRC17:22
*** alex_xu_ has quit IRC17:22
*** stevebaker has quit IRC17:22
*** zzzeek has quit IRC17:22
*** krtaylor has quit IRC17:22
*** pabelanger_ has joined #openstack-infra17:22
*** harlowja has quit IRC17:22
*** Guest45385 has quit IRC17:22
*** icey has quit IRC17:22
*** pleia2 has quit IRC17:22
*** logan- has quit IRC17:22
*** jheroux has quit IRC17:22
*** matt-borland_ has quit IRC17:22
*** EricGonc_ has quit IRC17:22
*** EricGonczer_ has joined #openstack-infra17:22
*** krtaylor has joined #openstack-infra17:22
*** apuimedo has quit IRC17:22
*** kambiz has quit IRC17:22
*** filler has quit IRC17:22
*** jklare has quit IRC17:22
*** apuimedo_ is now known as apuimedo17:22
*** tbarron has joined #openstack-infra17:22
*** dteselkin has joined #openstack-infra17:22
*** amitgandhinz has joined #openstack-infra17:22
*** kzaitsev_mb has quit IRC17:22
*** stevebak` has joined #openstack-infra17:22
*** stevebak` has quit IRC17:22
*** stevebak` has joined #openstack-infra17:22
*** clarkb has quit IRC17:22
*** mikal has quit IRC17:22
*** vgadiraj has quit IRC17:22
*** kambiz has joined #openstack-infra17:22
*** devananda has joined #openstack-infra17:22
*** Guest27780 has quit IRC17:22
*** bnemec has quit IRC17:22
*** Guest66666 has quit IRC17:22
*** gouthamr has quit IRC17:22
*** jianghuaw has quit IRC17:22
*** yamahata has quit IRC17:22
*** beisner has quit IRC17:22
*** salv-orl_ has joined #openstack-infra17:22
*** rhallisey_ has quit IRC17:22
*** pleia2 has joined #openstack-infra17:22
*** jklare has joined #openstack-infra17:22
openstackgerritJames E. Blair proposed openstack-infra/project-config: Publish infra docs to AFS  https://review.openstack.org/37793017:23
*** zeus- has joined #openstack-infra17:23
*** serverascode has joined #openstack-infra17:23
*** roxanaghe has joined #openstack-infra17:23
*** clarkb has joined #openstack-infra17:23
*** jjheroux has joined #openstack-infra17:23
*** pvaneck has joined #openstack-infra17:23
*** rbradfor has joined #openstack-infra17:23
*** electrichead has joined #openstack-infra17:23
*** jianghuaw has joined #openstack-infra17:23
*** szaher_ has joined #openstack-infra17:23
*** kzaitsev1mb is now known as kzaitsev_mb17:23
jeblairpabelanger_: ^17:23
jeblairAJaeger: ^17:23
*** icey has joined #openstack-infra17:23
*** filler has joined #openstack-infra17:23
*** zeus- is now known as zeus`17:24
*** zzzeek has joined #openstack-infra17:24
*** migi_ has joined #openstack-infra17:24
*** sileht has joined #openstack-infra17:24
*** david-lyle_ is now known as david-lyle17:24
*** beisner has joined #openstack-infra17:25
*** migi_ has quit IRC17:25
*** rhallisey_ has joined #openstack-infra17:25
*** migi_ has joined #openstack-infra17:25
*** salv-orlando has quit IRC17:25
*** simonmcc has joined #openstack-infra17:26
*** pzhurba__ has joined #openstack-infra17:26
*** lindsayk_ has quit IRC17:26
*** migi has quit IRC17:26
*** migi_ is now known as migi17:27
*** petems has joined #openstack-infra17:27
*** GheRivero has joined #openstack-infra17:27
*** rossella_ has joined #openstack-infra17:27
*** coreyob has joined #openstack-infra17:27
*** Madasi has quit IRC17:28
*** gordc_ has quit IRC17:28
*** szaher has quit IRC17:28
*** jimbaker` has quit IRC17:28
*** bexelbie has quit IRC17:28
*** raildo has quit IRC17:28
*** fguillot has quit IRC17:28
*** rodrigods has quit IRC17:28
*** rwsu has quit IRC17:28
*** dteselkin_ has quit IRC17:28
*** Shalmazar is now known as Madasi17:28
*** logan- has joined #openstack-infra17:28
cloudnullqq -- is it possible to split the infra workload into smaller subsets within noodpool?  Basically I'd like to split the max-servers in cloud1 across various host aggregates using different flavors.17:29
cloudnullI can see in nodepool/nodepool.yaml that it "should" be possible by changing the provider to something like this http://cdn.pasteraw.com/33hgt9vi4sy6nl60iaimu30h80kmvxc but I wanted to know if this is something that could be supported?17:29
cloudnullor if there was a better way ?17:29
cloudnullor if i'm just talking nonsense17:29
cloudnullall could be equally true17:29
cloudnull:)17:30
*** firenka has joined #openstack-infra17:30
*** salv-orlando has joined #openstack-infra17:31
jeblaircloudnull: i believe that will work.  nodepool will treat those as two separate clouds (which means that it won't know to share a rate-limit between them).  otherwise i think it will work as expected.17:31
mordredcloudnull: I agree with jeblair17:31
jeblaircloudnull: (we had to do something similar with hpcloud back in the day because their neutron networks couldn't handle more than X hosts)17:31
*** jzerebecki has joined #openstack-infra17:31
cloudnullah. ok.17:31
*** mnaser has joined #openstack-infra17:32
*** mminesh` has joined #openstack-infra17:32
cloudnullwe have different hardware profiles in the cloud, right now its just scheduling to all of them17:32
*** pabelanger_ has quit IRC17:32
*** rossella_s has quit IRC17:32
*** aunnam has quit IRC17:32
*** lennyb has quit IRC17:32
*** pzhurba_ has quit IRC17:32
cloudnulland I'd like to start collecting metrics on hardware specific workloads. if at all possible17:32
jeblaircloudnull: out of curiosity, what will this let you do...17:32
*** filler_ has joined #openstack-infra17:32
jeblairah, got it :)17:32
*** mminesh has quit IRC17:32
*** Qiming has quit IRC17:32
*** GheRivero has quit IRC17:32
*** petems has quit IRC17:32
*** simonmcc has quit IRC17:32
*** filler has quit IRC17:32
*** stevemar has quit IRC17:32
*** kuromagi has quit IRC17:32
*** kuromagi has joined #openstack-infra17:32
*** stevebaker has joined #openstack-infra17:32
*** admcleod has quit IRC17:32
*** mtreinish has quit IRC17:33
*** SamYaple has quit IRC17:33
*** unrahul has joined #openstack-infra17:33
*** mrmartin_ has joined #openstack-infra17:33
*** stevebaker has quit IRC17:33
*** stevebaker has joined #openstack-infra17:33
*** unrahul has quit IRC17:33
*** unrahul has joined #openstack-infra17:33
*** pabelanger has joined #openstack-infra17:33
*** mminesh` is now known as mminesh17:33
*** mjblack has joined #openstack-infra17:33
*** admcleod has joined #openstack-infra17:33
*** admcleod has joined #openstack-infra17:33
*** logan- has quit IRC17:33
*** zeus` has quit IRC17:33
*** tbarron has quit IRC17:33
*** hughhalf has quit IRC17:33
*** harlowja_ has quit IRC17:33
*** sdague has quit IRC17:33
*** yolanda_ has quit IRC17:33
*** jak-jak_ has quit IRC17:33
*** bradm has quit IRC17:33
*** pvaneck_ has joined #openstack-infra17:33
*** degorenko is now known as _degorenko17:33
cloudnullwill that cause you all any undue pain ?17:33
*** ldnunes_ has joined #openstack-infra17:33
*** filler_ is now known as filler17:33
jeblaircloudnull: so if we care about rate-limits,  we may want to slow it down a bit (increase the 'rate' value, sorry).  but maybe it's okay to just let nodepool throw api reqs at 2x the current rate.17:33
*** pzhurba has joined #openstack-infra17:33
*** yolanda_ has joined #openstack-infra17:33
jeblaircloudnull: i don't think so.17:33
cloudnullthat's not something I'd want if thats the case.17:33
*** tbarron has joined #openstack-infra17:33
*** andymccr has joined #openstack-infra17:33
*** stevebak` has quit IRC17:33
*** mrmartin has quit IRC17:33
*** mrmartin_ is now known as mrmartin17:33
*** jklare_ has joined #openstack-infra17:33
*** pzhurba__ has quit IRC17:33
*** pvaneck has quit IRC17:34
*** jklare has quit IRC17:34
*** salv-orl_ has quit IRC17:34
*** dteselkin has quit IRC17:34
*** ldnunes__ has quit IRC17:34
*** rcarrillocruz has quit IRC17:34
*** sileht has quit IRC17:34
*** devananda has quit IRC17:34
*** spzala has quit IRC17:34
*** rwsu has joined #openstack-infra17:34
*** raildo has joined #openstack-infra17:34
*** sileht has joined #openstack-infra17:34
*** stevemar has joined #openstack-infra17:34
cloudnulljeblair: i'm good with the increase in api workload. :)17:34
* cloudnull has a name to live up to17:34
*** harlowja has joined #openstack-infra17:34
clarkbI can't forsee any issues with doing that other than needing to deal with alien commands not being accurate like we had to do with hpcloud17:34
*** kun_huang has joined #openstack-infra17:34
*** rcarrillocruz has joined #openstack-infra17:35
*** sdague has joined #openstack-infra17:35
*** devananda has joined #openstack-infra17:35
jeblairclarkb: oh right.  but i bet we can fix that if it gets annoying.17:35
cloudnull"that's not something I'd want if thats the case." -- was in relation to causing pain.17:35
jeblaircloudnull: thought so, thanks for the clarification.17:35
clarkbjeblair: yup and we lived with it before so not a huge deal17:35
*** hughhalf has joined #openstack-infra17:35
jeblaircloudnull: have we dos'd one of your clouds yet?17:35
cloudnullnot yet.17:35
*** trown|lunch is now known as trown|brb17:35
*** trown|brb is now known as trown17:35
clarkbclearly we must try harder17:36
cloudnullI'm hoping soon though17:36
*** spzala has joined #openstack-infra17:36
pabelanger2.038 killo-jobs per hour @ 2016-09-27 10:00 UTC17:36
*** dteselkin has joined #openstack-infra17:36
jeblaircloudnull: we're usually pretty good at it -- i think you found our achilles heel though.  give us so much quota we dos'd our cloud-doser (nodepool).17:36
*** xgerman has joined #openstack-infra17:36
*** baoli has joined #openstack-infra17:36
cloudnullhhahahahaha17:36
cloudnulldoes that mean i win ?17:36
clarkbcloudnull: yes one cookie17:36
cloudnullw00t!17:37
* cloudnull happy dance17:37
*** Qiming has joined #openstack-infra17:37
jeblaircloudnull: you win this round!  :)17:37
jeblairpabelanger: whee!17:37
clarkbfrickler: any luck with fixing the identity v3 only job?17:37
* mordred hands cloudnull a mostly unused cookie17:37
jeblairmordred: ^ we legit broke 2kjph17:37
*** jak-jak_ has joined #openstack-infra17:37
clarkbfrickler: since its running on all the changes it would be nice to get that owrking soon17:38
mordredjeblair: \o/17:38
* cloudnull eats mostly unused cookie17:38
openstackgerritMerged openstack-infra/system-config: Add zuul-launcher keytab to zl0(1-8) nodes  https://review.openstack.org/37181817:38
*** zeus- has joined #openstack-infra17:38
*** bradm has joined #openstack-infra17:38
cloudnullwould other changes be needed in noodpool to support that config ?17:38
*** SamYaple has joined #openstack-infra17:38
cloudnullsomething I can take care of in a set of changes?17:38
*** logan- has joined #openstack-infra17:38
*** sambetts is now known as sambetts|afk17:39
clarkbcloudnull: the only other thing is we will have to reupload all the images...17:39
*** zeus- is now known as zeus`17:39
*** jraim has joined #openstack-infra17:39
cloudnullclarkb: will the images be different?17:39
*** zeus` is now known as zeus17:39
fungioh, right, we have multiple copies of the images in that case?17:39
*** zeus has quit IRC17:39
*** zeus has joined #openstack-infra17:39
cloudnulloh bummer.17:39
clarkbcloudnull: so you may want to add those and keep the old provider in place with a reduced max-servers value. Then remove the old provider once the new ones are active17:39
jeblairso probably best to add the 2 new clouds with no quota to get the images in place, then shift the quota over to them17:39
jeblairwhat clarkb said17:39
*** dtardivel has joined #openstack-infra17:40
clarkbfungi: cloudnull correct because nodepool treats it as logically separate17:40
*** mtreinish has joined #openstack-infra17:40
*** lennyb has joined #openstack-infra17:40
cloudnullah... That sucks. anyway I can point it at a consolodated glance?17:40
*** andymccr has quit IRC17:41
clarkbcloudnull: you might be able to rewrite bits in nodepool but probably not worth the effort17:41
cloudnullok.17:41
*** puiterwijk|cld has joined #openstack-infra17:41
*** Apoorva_ has quit IRC17:41
*** sslypushenko has joined #openstack-infra17:41
*** Swanson has joined #openstack-infra17:41
*** harlowja has quit IRC17:42
*** Apoorva has joined #openstack-infra17:42
cloudnullwhere is the image upload-erator code live?17:42
cloudnulljust to have a look17:42
*** rodrigod` is now known as rodrigods17:42
*** rodrigods has quit IRC17:42
*** rodrigods has joined #openstack-infra17:42
*** tomhambleton has joined #openstack-infra17:43
*** tnovacik has quit IRC17:43
clarkbcloudnull: openstack-infra/nodepool/nodepool/nodepool.py I think17:43
fungiyeah, it's not just uploads you'd have to change. you'd need some way to signal that instances for cloud B should be booted from images in cloud A17:43
mordredclarkb, cloudnull: if it's a consolidated glance ...17:43
* cloudnull looking17:43
clarkbfungi: ya that17:43
mordredthen it's entirely possible that the image duplication prevention code will take care of it17:43
cloudnullmordred:  it would all be the same control plain17:44
clarkbmordred: it won't because by default glance acl things prevent shared images17:44
jeblairclarkb: we're in the same tenant17:44
jeblairproject17:44
jeblairwhatever17:44
clarkbjeblair: oh right17:44
clarkbderp17:44
mordredyah. just different flavors17:44
mordredso I _think_ it should work17:44
clarkbso ya it may just work17:44
cloudnull-tm17:44
fungiit may deduplicate them at rest, but i doubt it would prevent us from trying to upload?17:44
mordredwe use the same image name everywhere, yeah?17:44
*** med_ has joined #openstack-infra17:44
mordredfungi: it would17:44
jeblairmordred: so shade will "upload" the image and just return the already extant image id?17:45
*** andrewbogott has joined #openstack-infra17:45
clarkbmordred: it has the timestamp attached to it and its timestamp per provider I think so maybe that wouldn't work17:45
*** med_ is now known as Guest7927817:45
fungior is there some signaling with glance where you tell it a checksum of the image before uploading and it checks to see whether it already has an image with that checksum?17:45
mordredfungi: the create_image call checks to see if the existing image in the cloud has the same name and checksum as the image you're trying to upload17:45
mordredclarkb: oh - the timestamp may screw it17:45
fungiaha17:45
cloudnullsomething we could look at the MD5sum of the existing image before uploading ?17:45
jeblairclarkb: i think the timestamp is only on the dib?17:45
mordredcloudnull: we do17:45
jeblaircloudnull: i believe that's what mordred is saying shade does17:45
clarkbjeblair: dib files are just increasing but I think we use timestamps for glance17:45
cloudnulloh. so then it "should" go ?17:45
mordredwell - unless we're doing a different image name per upload17:46
mordredin which case shade will not prevent that17:46
*** kong has joined #openstack-infra17:46
clarkbI will look if its timestamp per upload or timestamp per image17:46
fungiwell, nodepool will probably continue to think it lacks any images for cloud B in that case, right?17:46
mordred++17:46
mordredfungi: the create_image call will succeed17:46
fungioh!17:46
mordredfungi: so nodepool will think "I uploaded an image"17:46
mordredand consider its job done17:46
fungiwell, now that's right fancy if i do say so17:46
*** dprince has joined #openstack-infra17:46
mordredso it will still "upload" the image - it'll just happen REALLY quickly17:46
*** lezbar has quit IRC17:46
*** electrical has joined #openstack-infra17:47
cloudnullwell that can't happen ... back to the drawing board.17:47
mordredhehehe17:47
*** harlowja has joined #openstack-infra17:47
*** lezbar has joined #openstack-infra17:47
*** DuncanT has joined #openstack-infra17:47
*** GheRivero has joined #openstack-infra17:47
*** njohnston_ is now known as njohnston17:48
mordredfwiw - this feature exists because doing testing of "upload image and then launch vm from it" really sucks otherwise17:48
jeblairclarkb: i think what *should* happen is that the timestamp should be associated with the dib, and the name of the image should contain the timestamp of the dib, and so therefore should be the same across all uploads.  if that *isn't* happening, i think we should change it.  :)17:48
mordredjeblair: ++17:48
clarkbya the version looks different per17:49
clarkbso image name in nodepool is eg ubuntu-xenial but then we tack on extra infos when uploading to glance to make them unique17:49
jeblairclarkb, mordred: examining 'server list' output it looks like it's different per upload, so we will need a fix.17:49
*** asingh_ has joined #openstack-infra17:50
*** skath has joined #openstack-infra17:50
mordredI feel like maybe we did that on purpose17:50
clarkbhttps://git.openstack.org/cgit/openstack-infra/nodepool/tree/nodepool/nodepool.py#n176017:50
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Move py34-with-oslo-master jobs to xenial  https://review.openstack.org/37794717:50
*** bin_ has joined #openstack-infra17:51
mordredI can't come up with a plausible reason that makes sense now, so it's also possible it was a reason that used to make sense but doesn't any longer ...17:51
*** hrybacki|afk has joined #openstack-infra17:51
jeblairmordred: i suspect it has more to do with the snapshot builds17:51
clarkbthere is a handy todo there with mordreds name on it :P17:51
dimsfungi : i am tempted to add a line in requirements that we should NOT propose an entry for defusedxml :)17:51
*** ggherdov- has joined #openstack-infra17:51
mordredwhy does that function have both snapshot and upload in it? what am I misreading?17:52
*** Swami has joined #openstack-infra17:52
jeblairmordred: 'snapshot' is an overloaded term that means 'instance of a dib upload' in some places.17:52
clarkbsnapshot really means image in glance17:52
mordredoy17:52
*** rajinir has joined #openstack-infra17:52
jeblairright17:52
clarkbat least in the db in nodepool17:52
fungidims: sure, that might make it more obvious it's already been discussed17:52
mordredwe should fix that in the next version17:52
jesusaurasselin__: I've responded to your comment on 37710517:53
*** tonytan4ever has joined #openstack-infra17:53
jeblairmordred: we are17:53
mordred\o/17:53
*** simonmcc has joined #openstack-infra17:53
*** tqtran has joined #openstack-infra17:53
*** woodster_ has joined #openstack-infra17:53
fungidims: or put some documentation in the requirements repo (readme?) with a list of packages that have already been discussed and ruled out17:53
AJaegerarmax: please review https://review.openstack.org/#/c/37773817:53
dimsfungi : right17:54
armaxAJaeger: can I get a favor in return and get someone to review https://review.openstack.org/#/c/361432/?17:54
armax:)17:54
jeblairmordred, clarkb: i'm not sure that's the code that gets us template-centos-7-1474987696 as the glance image name17:54
*** dham1 has joined #openstack-infra17:54
mordreddims: can we put in a comment that just says "no new xml libs please?"17:54
*** jraju has quit IRC17:54
dimsmordred : haha ++17:54
AJaegerarmax: I'm not core on that repo - but let me review nevertheless ;)17:55
clarkbjeblair: its derived from that 1474987696 is the version and centos-7 the image name17:55
armaxAJaeger: thanks17:55
armaxAJaeger: ot17:55
clarkbjeblair: template- comes from the image name template string. Let me find that17:55
armaxit’s a simple change17:55
AJaegerclarkb, fungi https://review.openstack.org/#/c/361432 removes the hardcoded neutron names from reviewday17:56
clarkbjeblair: https://git.openstack.org/cgit/openstack-infra/nodepool/tree/nodepool/config.py#n20317:56
clarkbjeblair: its possible we could provide a different string that would still get us unique images within a cloud17:56
jeblairclarkb: yeah, i'm just saying that's not where they are being assembled together17:56
*** claudiub has joined #openstack-infra17:57
*** psilvad has quit IRC17:57
*** fkautz_ has joined #openstack-infra17:57
*** psilvad has joined #openstack-infra17:57
mordredhttps://git.openstack.org/cgit/openstack-infra/nodepool/tree/nodepool/builder.py#n34217:57
mordredthat's where the image name gets built17:57
openstackgerritK Jonathan Harker proposed openstack-infra/project-config: Re-enable the logstash filter jobs  https://review.openstack.org/36966717:57
mordredhttps://git.openstack.org/cgit/openstack-infra/nodepool/tree/nodepool/builder.py#n348 actually, that's the line17:58
jeblairoh my, we don't even use the same timestamp17:58
clarkbmordred: no thats the on disk name17:58
AJaegerjeblair: regarding your comment from earlier that docs-publisher-site is always docs.o.o - did you have a look at https://review.openstack.org/#/c/377591/ ? is that something we should do? That change moves some settings to defaults.yaml...17:58
clarkbthe dib names are not timestamps they are just an incrementing count17:58
mordredclarkb: ext_image_name is what is passed to create_image as the image name17:58
mordredfilename is the name of the dib file on disk17:58
mordreds/name of the/path to the/17:59
clarkbah ya. It uses the same template string though17:59
fungiarmax: see comment on 361432, it makes an (invalid) assumption about the projects.yaml data model17:59
AJaegerdo we have any puppet experts that can look into a failure on puppet-gerritbot, please? https://review.openstack.org/35558817:59
jeblairmordred, clarkb: we can actually change this just with configuration17:59
mordredwoot17:59
*** thorst_ has quit IRC17:59
armaxfungi: I noticed, I am personally not fond of ideas of deliverables17:59
clarkbjeblair: ya we may want to add more fields to dummy_image though18:00
*** tnovacik has joined #openstack-infra18:00
armaxfungi: I am trying to move away from it as far as Neutron goes, but I can see your point18:00
jeblairclarkb: yes, you're right18:00
clarkbjeblair: mordred I think we want template-{image.name}-{image.ondisknumber}18:00
fungiarmax: me too. i wish we could go back to only one repo per deliverable18:00
clarkbso its a small change to the builder and to our config18:00
AJaegerjeblair: " Unknown entry point or macro 'afs' for component type: 'publisher'." for your afs change ;(18:00
*** briancurtin has joined #openstack-infra18:00
fungiarmax: my main concern is for other teams reusing this script18:00
jeblairclarkb: the ondisknumber is a timestamp18:00
armaxfungi: technically that’s what we’re going for ever since I have become in charge18:00
jeblairclarkb: but otherwise i agree18:00
openstackgerritK Jonathan Harker proposed openstack-infra/puppet-logstash: Make the java heap size configurable  https://review.openstack.org/35742618:00
armaxfungi: the script is riddled for Neutron references, it’ll be hard to be reused as it18:01
armaxis18:01
jeblairclarkb: it's the "version" field in the dib table18:01
clarkbjeblair: ls /opt/nodepool_dib says its just an incrementing integer18:01
fungiarmax: fair enough. +2 from me then18:01
jeblair| 1151 | centos-7       | /opt/nodepool_dib/centos-7-1474799640       | 1474799640 | ready | 02:00:12:22 |18:01
clarkbya so 1151 is what we have on disk18:01
jeblairclarkb: 1474799640 is the number i'd like to see18:01
clarkbwe can use that or the timestamp I guess18:01
jeblairclarkb: 1151 is the mysql autoincrement18:01
armaxfungi: I can revise in another pass though18:01
fungiarmax: when i said i wish "we" could go back to one deliverable per repo, i meant all of openstack not just neutron ;)18:02
clarkbthey will both produce a unique image within the provider that is useable18:02
*** matbu_ is now known as matbu18:02
armaxfungi: ack18:02
jeblairclarkb: ya18:02
*** nadya has joined #openstack-infra18:02
*** rbrndt has joined #openstack-infra18:02
fungier, one repo per deliverable18:02
clarkbjeblair: wait so does this already work?18:02
* clarkb checks in clouds18:02
jeblairclarkb: no18:02
jeblairclarkb: that's from dib-image list18:03
jeblairclarkb: the upload is done with the timestamp of the upload time18:03
*** thorst_ has joined #openstack-infra18:03
jeblairclarkb: which isn't even the same number that's recorded as the version in the database18:03
clarkbah at the beginning of the function18:03
clarkbright so part of that is the builder has no db access iirc18:03
jeblairclarkb: you know, i think you could talk me into using the id rather than the version18:03
clarkbso easy mode is to use the db id18:04
clarkbotherwise we have to make sure that the gearman rpc call incldues the version in its blob18:04
clarkbjeblair: so I think maybe we should do the easy thing now with is use the table id, then we can worry about better communication and terminology with zk18:05
openstackgerritMerged openstack-infra/system-config: Use mod_autoindex on files.o.o  https://review.openstack.org/37791918:05
jeblairclarkb: i think the id will actually transition nicely into the zk18:05
*** yamahata has joined #openstack-infra18:05
*** tuannguyen has quit IRC18:05
clarkbok should I go ahead and push that patch here in a minute?18:05
*** kzaitsev_mb has quit IRC18:06
*** yee3794 has joined #openstack-infra18:06
*** yee379 has quit IRC18:07
jeblairclarkb: yeah18:07
jeblairclarkb: also, i'm not entirely sure we need to continue to use template-hostname for this18:07
*** kzaitsev_mb has joined #openstack-infra18:07
jeblairclarkb: i wonder if we should just hardcode this as '{image_name}-{image_id}'18:07
*** psilvad has quit IRC18:08
*** xarses_ has quit IRC18:08
jeblairhrm18:08
jeblairi guess we'd want something for the user to be able to disambiguate multiple nodepools18:08
*** tonytan_brb has joined #openstack-infra18:08
openstackgerritDong Ma proposed openstack-infra/jenkins-job-builder: Update cppcheck plugin  https://review.openstack.org/37795318:08
jeblairso nevermind, let's keep using template-hostname for now18:08
jeblairand clean it up later18:08
*** nadya_ has joined #openstack-infra18:09
EmilienMsomeone root on tarballs.o.o might want to rm -r http://tarballs.openstack.org/%7bproject%7d/18:09
*** kambiz_ has joined #openstack-infra18:09
*** petems has joined #openstack-infra18:09
*** aspiers has quit IRC18:09
openstackgerritClark Boylan proposed openstack-infra/nodepool: Add image id to DIB image upload name params  https://review.openstack.org/37795418:10
*** harlowja has quit IRC18:10
*** ajafo has quit IRC18:10
*** jpich has quit IRC18:10
*** ggherdov- has quit IRC18:10
*** sslypushenko has quit IRC18:10
*** puiterwijk|cld has quit IRC18:10
*** xgerman has quit IRC18:10
*** jklare_ has quit IRC18:10
*** crinkle has quit IRC18:10
*** ggherdov- has joined #openstack-infra18:10
*** electrical has quit IRC18:10
*** andrewbogott has quit IRC18:10
*** dtardivel has quit IRC18:10
*** kun_huang has quit IRC18:10
*** serverascode has quit IRC18:10
*** patrickeast_ has joined #openstack-infra18:10
*** mtreinish_ has joined #openstack-infra18:10
*** ajafo has joined #openstack-infra18:10
clarkbhttps://review.openstack.org/377954 and I think I am on the wrong side of a netsplit18:10
*** ggherdov- has quit IRC18:10
*** ggherdov- has joined #openstack-infra18:10
*** patrickeast_ has quit IRC18:10
*** patrickeast_ has joined #openstack-infra18:10
*** tonytan4ever has quit IRC18:10
*** Guest79278 has quit IRC18:10
*** lennyb has quit IRC18:10
*** bethwhite has quit IRC18:10
*** jak-jak_ has quit IRC18:10
*** rcarrillocruz has quit IRC18:10
*** yolanda_ has quit IRC18:10
*** pvaneck_ has quit IRC18:10
*** redrobot has joined #openstack-infra18:10
*** bradm_ has joined #openstack-infra18:10
*** tnovacik has quit IRC18:10
*** bethwhite has joined #openstack-infra18:10
anteayaclarkb: I'm on the same side as you18:11
clarkbwoo irc went wonky for me there but I think ^ will do it then we just need to update our config18:11
*** redrobot is now known as Guest4507918:11
*** nadya has quit IRC18:11
*** sdague has quit IRC18:11
jeblairclarkb: ++18:11
*** thorst_ has quit IRC18:11
*** rbrndt has quit IRC18:11
*** kambiz has quit IRC18:11
*** mtreinish has quit IRC18:11
*** rods has quit IRC18:11
*** mtreinish_ is now known as mtreinish18:11
*** njohnston has quit IRC18:11
*** electrichead has quit IRC18:11
*** bradm has quit IRC18:11
*** sileht has quit IRC18:11
*** tuannguyen has joined #openstack-infra18:11
*** pvaneck__ has joined #openstack-infra18:11
*** rhallisey_ has quit IRC18:11
*** tuannguyen is now known as 17SAAEKL818:11
*** crinkle_ has joined #openstack-infra18:11
*** jklare has joined #openstack-infra18:11
*** kambiz_ is now known as kambiz18:11
jeblairclarkb: and we can change the default to match our config later18:11
clarkbyup18:11
*** rhallisey_ has joined #openstack-infra18:11
*** njohnston has joined #openstack-infra18:11
clarkband this should be backward compatibile too18:11
*** rods has joined #openstack-infra18:11
*** psilvad has joined #openstack-infra18:12
*** crinkle_ is now known as crinkle18:12
*** eggshell has joined #openstack-infra18:12
*** serverascode has joined #openstack-infra18:12
jeblairmordred, cloudnull: https://review.openstack.org/37795418:12
*** stevemar_ has joined #openstack-infra18:12
*** rcarrillocruz has joined #openstack-infra18:13
*** yolanda_ has joined #openstack-infra18:13
*** kzaitsev_mb has quit IRC18:13
jeblairoh, looks like my --no-launches etc patch landed18:13
*** sdague has joined #openstack-infra18:13
*** electrofelix has quit IRC18:13
*** sileht has joined #openstack-infra18:13
*** lennyb has joined #openstack-infra18:13
clarkbok with that I am going to take a break to get some food before meetings18:13
clarkbif that breaks somehow feel free to psuh an update18:13
jeblairclarkb: if we can go ahead and approve your patch, we can probably restart it all after the meeting18:14
jeblairoh right, regarding https://review.openstack.org/377930 i just remembered i had a plan to make a jjb extension that understands the afs publisher and use it in the layout job18:14
jeblairbut i had not written that yet18:14
*** aspiers has joined #openstack-infra18:15
*** jak-jak has joined #openstack-infra18:15
mordredjeblair: oops :)18:16
*** jak-jak is now known as Guest8804618:16
mordredclarkb: I do not understand how that patch does what you say18:17
mordredclarkb: oh - wait - because the template comes from config18:17
*** stevemar_ is now known as stevemar__18:17
*** tuannguyen has joined #openstack-infra18:18
*** stevemar__ has quit IRC18:18
mordredclarkb: +A18:18
*** e0ne has joined #openstack-infra18:18
*** sslypushenko has joined #openstack-infra18:19
*** andrewbogott has joined #openstack-infra18:20
*** stevemar____ has joined #openstack-infra18:20
*** electrical has joined #openstack-infra18:20
*** dtardivel has joined #openstack-infra18:20
*** patrickeast_ has quit IRC18:20
*** rbrndt has joined #openstack-infra18:20
*** Guest45079 has quit IRC18:21
*** darvon_ has joined #openstack-infra18:21
*** artmr has quit IRC18:21
*** morgan has joined #openstack-infra18:21
*** electrichead has joined #openstack-infra18:21
*** 17SAAEKL8 has quit IRC18:21
*** darvon has quit IRC18:21
*** sileht has quit IRC18:21
*** pvaneck__ has quit IRC18:21
*** kun_huang has joined #openstack-infra18:22
*** yolanda_ is now known as yolanda18:22
*** electrichead is now known as Guest7809118:22
*** dstufft has quit IRC18:22
* morgan looks longingly at oftc again.18:23
*** dstufft has joined #openstack-infra18:23
*** tuannguyen has quit IRC18:23
clarkbmordred yup so we also need a config change that uses image.id18:23
*** artmr has joined #openstack-infra18:23
*** zara_the_lemur__ has joined #openstack-infra18:24
*** harlowja has joined #openstack-infra18:24
*** sileht has joined #openstack-infra18:24
*** psilvad has quit IRC18:24
AJaegerproject-config cores, here're two changes to move to xenial that I'd like to see in soonish: https://review.openstack.org/377738 and https://review.openstack.org/377947 . Please review18:25
*** thorst_ has joined #openstack-infra18:25
*** psilvad has joined #openstack-infra18:25
mordredclarkb: ++18:25
*** [HeOS] has quit IRC18:26
*** tnovacik has joined #openstack-infra18:26
*** xarses_ has joined #openstack-infra18:26
openstackgerritMerged openstack-infra/elastic-recheck: Only query indexes that exist  https://review.openstack.org/37712918:27
*** panda|bbl is now known as panda18:27
*** patrickeast_ has joined #openstack-infra18:28
*** taron has quit IRC18:28
*** taron has joined #openstack-infra18:28
*** ihrachys has quit IRC18:28
*** yaume has quit IRC18:29
*** mjblack has quit IRC18:29
*** pilgrimstack has joined #openstack-infra18:30
*** artmr_ has joined #openstack-infra18:30
*** darvon_ is now known as darvon18:30
*** artmr has quit IRC18:30
*** mjblack has joined #openstack-infra18:31
*** nadya_ has quit IRC18:31
*** xgerman has joined #openstack-infra18:32
*** puiterwijk|cld has joined #openstack-infra18:32
*** jimbaker` has joined #openstack-infra18:32
*** bexelbie has joined #openstack-infra18:32
*** M-docaedo_vector has joined #openstack-infra18:33
*** pvaneck has joined #openstack-infra18:34
*** yamahata has quit IRC18:34
*** andymccr has joined #openstack-infra18:34
*** salv-orlando has quit IRC18:35
*** thorst_ has quit IRC18:35
*** Rajini_Ram_ has joined #openstack-infra18:35
*** quade1 has joined #openstack-infra18:35
*** sdake has quit IRC18:35
*** thorst_ has joined #openstack-infra18:36
*** thorst_ has quit IRC18:36
*** thorst_ has joined #openstack-infra18:36
anteayaif someone can, both -meeting and -meeting-alt don't have ongoing meetings at the moment and could use a topic refresh18:37
anteayakeystone is still using -meeting but meetbot isn't taking minutes18:37
Rajini_Ram_Can someome provide an example to use the Cleanup trap in devstack gate please?18:37
*** dtardivel has quit IRC18:37
*** tuannguyen has joined #openstack-infra18:37
morgananteaya: tyvm18:38
*** tnovacik has quit IRC18:39
anteayamorgan: I do like your stevemar nick suggestion18:39
anteayafunny he didn't take it18:39
*** logan- has quit IRC18:39
morgantoo many characters18:39
anteayamorgan: and you're welcome :)18:39
anteayahe has too much character already18:39
anteayasorry, you set me up18:39
fungianteaya: done18:39
anteayaI couldn't walk away18:39
anteayafungi: thank you18:39
*** M-docaedo_vector has quit IRC18:40
openstackgerritKyle Haley proposed openstack-infra/project-config: Make docs gate non-voting for openstack/quark  https://review.openstack.org/37790818:40
*** andymccr has quit IRC18:40
*** andymccr has joined #openstack-infra18:40
*** pvaneck has quit IRC18:40
*** jjheroux has quit IRC18:40
*** pvaneck has joined #openstack-infra18:41
*** jjheroux has joined #openstack-infra18:41
*** thingee has joined #openstack-infra18:41
openstackgerritKyle Haley proposed openstack-infra/project-config: Make docs gate non-voting for openstack/quark  https://review.openstack.org/37790818:42
*** roxanaghe has quit IRC18:42
*** logan- has joined #openstack-infra18:42
*** med_ has joined #openstack-infra18:43
*** med_ is now known as Guest4087618:44
*** jtomasek has joined #openstack-infra18:46
openstackgerritKevin Carter (cloudnull) proposed openstack-infra/project-config: Updated OSIC project for HW specific flavors  https://review.openstack.org/37796518:47
*** lindsayk_ has joined #openstack-infra18:47
cloudnullso i lowered the rate and gave each entry a low 24 node quota, which was subtracted out of the general cloud1 config .18:47
cloudnullI hope that's right.18:48
cloudnullI also lowered the rate. I assume that once we know all is well with the configs we can up that too.18:48
*** andymccr has quit IRC18:48
*** sdake has joined #openstack-infra18:49
*** tnovacik has joined #openstack-infra18:49
*** thingee has quit IRC18:49
anteayaby rate do you mean quota?18:49
pabelangerthat also means we need new AFS mirrors too18:49
*** slaweq_ has quit IRC18:50
*** mtanino_ has joined #openstack-infra18:50
pabelangeror setup DNS to just point to the existing18:50
*** thorst_ has quit IRC18:50
cloudnullanteaya: I believe thats the api rate https://review.openstack.org/#/c/377965/1/nodepool/nodepool.yaml@75618:50
clarkbpabelanger: oh ya I think we can just dns hack18:50
openstackgerritMerged openstack/os-testr: Updated from global requirements  https://review.openstack.org/37752618:51
pabelangerYa, we are still doing that for cloud118:51
cloudnullpabelanger: does the DNS key off of the name or the cloud entry ?18:51
pabelangershould loop back to fixing that18:51
clarkbcloudnull: its the provider name iirc18:51
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove publishing jobs from salt  https://review.openstack.org/37796818:51
cloudnullah . ok .18:51
*** Apoorva has quit IRC18:52
*** mtani____ has joined #openstack-infra18:52
*** mtanino has quit IRC18:52
anteayacloudnull: ah thank you18:52
pabelangerclarkb: nodepool_region / nodepool_cloud18:52
cloudnullpabelanger: is there something I should go PR against to update that?18:52
*** Apoorva has joined #openstack-infra18:52
anteayacloudnull: so calling them s3500, s3700 and disk, yes?18:52
cloudnullyes.18:52
pabelangercloudnull: I think it is actually setup properly, we just need to try launching it again18:52
cloudnullok.18:52
clarkbpabelanger: if it doesnt use the name at all then we should be fine18:52
cloudnullanteaya:  we have a couple hardware profiles which are named based on the disks they have within them .18:53
openstackgerritMerged openstack/os-testr: Update home-page url  https://review.openstack.org/35617918:53
openstackgerritMerged openstack/os-testr: Make subprocess outputs to text for Python 3  https://review.openstack.org/30321718:53
cloudnullin this case we're running s35/700 SSD s18:53
cloudnullwe have been running that gear for a while but simply scheduling to all the things everywhere.18:54
cloudnullnow we want to test on specific harware profiles and record metrics from that.18:54
*** annegentle has joined #openstack-infra18:54
pabelangerclarkb: it will use provider.name18:54
anteayacloudnull: okay thank you18:55
pabelangerclarkb: actually...18:55
*** mtanino_ has quit IRC18:55
pabelangerclarkb: you are correct, so we should be fine18:55
cloudnullw00t!18:55
*** adrian_otto has quit IRC18:55
*** thorst_ has joined #openstack-infra18:56
openstackgerritClark Boylan proposed openstack-infra/devstack-gate: Copy ssl certs for multinode  https://review.openstack.org/37321918:57
*** jimbaker` has quit IRC18:57
*** bexelbie has quit IRC18:57
*** DuncanT has quit IRC18:58
*** puiterwijk|cld_ has joined #openstack-infra18:59
*** DuncanT has joined #openstack-infra19:00
*** Rockyg has joined #openstack-infra19:00
fungiit's that (weekly infra team meeting) time again! find us in #openstack-meeting for the next hour (freenode ddos permitting)19:00
*** lindsayk_ has quit IRC19:00
*** puiterwijk|cld has quit IRC19:00
*** annegentle has quit IRC19:00
*** puiterwijk|cld_ is now known as puiterwijk|cld19:01
*** stevemar____ has quit IRC19:02
*** simonmcc has quit IRC19:02
*** simonmcc has joined #openstack-infra19:03
*** gyee has quit IRC19:08
*** xarses_ has quit IRC19:08
*** tuannguyen has quit IRC19:10
*** flaviof has quit IRC19:10
*** wfoster has quit IRC19:10
*** sdake has quit IRC19:10
*** chmouel_ has quit IRC19:11
*** taron has quit IRC19:11
*** wfoster has joined #openstack-infra19:11
*** wcriswell has quit IRC19:12
*** dstufft has quit IRC19:12
*** dstufft has joined #openstack-infra19:12
*** chmouel has joined #openstack-infra19:12
*** zzzeek has quit IRC19:12
*** zzzeek has joined #openstack-infra19:12
*** taron has joined #openstack-infra19:12
*** salv-orlando has joined #openstack-infra19:13
*** flaviof has joined #openstack-infra19:14
*** patrickeast has quit IRC19:14
*** patrickeast_ is now known as patrickeast19:14
*** wcriswell has joined #openstack-infra19:14
*** tuannguyen has joined #openstack-infra19:15
*** annegentle has joined #openstack-infra19:17
*** eggshell has quit IRC19:18
*** salv-orlando has quit IRC19:18
*** xyang1 has joined #openstack-infra19:18
*** xarses_ has joined #openstack-infra19:19
*** esikache1 has quit IRC19:20
*** Rajini_Ram_ has left #openstack-infra19:21
*** Rajini_Ram_ has quit IRC19:21
*** beisner- has joined #openstack-infra19:23
*** andreww has joined #openstack-infra19:23
*** burgerk__ has joined #openstack-infra19:23
*** ihrachys has joined #openstack-infra19:24
jeblairfungi, clarkb, mordred: does http://xkcd.com/1737/ remind you of our HA strategy for the ci system?19:24
openstackgerritMerged openstack-infra/nodepool: Add image id to DIB image upload name params  https://review.openstack.org/37795419:24
*** Rocky_g has joined #openstack-infra19:24
mordredjeblair: yup19:24
*** tuannguy_ has joined #openstack-infra19:25
*** crinkle_ has joined #openstack-infra19:26
*** xarses_ has quit IRC19:26
*** burgerk_ has quit IRC19:26
*** kun_huang_ has joined #openstack-infra19:26
*** logan- has quit IRC19:26
*** psilvad has quit IRC19:26
*** kun_huang has quit IRC19:26
*** GheRivero has quit IRC19:26
*** tomhambleton has quit IRC19:26
*** peterlisak has quit IRC19:26
*** serverascode has quit IRC19:27
*** ggherdov- has quit IRC19:27
*** briancurtin has quit IRC19:27
*** fkautz_ has quit IRC19:27
*** stevemar has quit IRC19:27
*** MarkAtwood has quit IRC19:27
*** kuromagi has quit IRC19:27
*** kuromagi has joined #openstack-infra19:27
*** sileht has quit IRC19:27
*** Guest88046 has quit IRC19:27
*** unrahul has quit IRC19:27
*** clarkb has quit IRC19:27
*** GheRivero has joined #openstack-infra19:27
rajinirCan someone point out an example of cleanup hook/trap for the devstack-gate please?19:27
*** beisner has quit IRC19:27
*** jlvillal has quit IRC19:27
*** GheRivero has quit IRC19:27
*** GheRivero has joined #openstack-infra19:27
*** cardeois_ has quit IRC19:27
*** beisner- is now known as beisner19:27
*** Guest40876 has quit IRC19:27
*** aspiers has quit IRC19:27
*** yolanda has quit IRC19:27
*** SamYaple has quit IRC19:27
*** tbarron has quit IRC19:27
*** mrmartin has quit IRC19:27
*** mtani____ has quit IRC19:27
*** tnovacik has quit IRC19:27
*** pvaneck has quit IRC19:27
*** xgerman has quit IRC19:27
*** cardeois has joined #openstack-infra19:27
*** peterlisak has joined #openstack-infra19:27
*** SamYaple has joined #openstack-infra19:27
*** mrmartin has joined #openstack-infra19:27
*** Rockyg has quit IRC19:27
*** serverascode has joined #openstack-infra19:27
*** slaweq_ has joined #openstack-infra19:27
*** xyang1 has quit IRC19:27
*** yolanda has joined #openstack-infra19:27
*** tuannguyen has quit IRC19:28
*** crinkle has quit IRC19:28
*** tbarron has joined #openstack-infra19:28
*** lucasagomes is now known as lucas-afk19:28
*** clarkb has joined #openstack-infra19:28
*** michaelxin has joined #openstack-infra19:28
*** kun_huang_ is now known as kun_huang19:28
*** rkukura has quit IRC19:29
erlonguys, is there any acceptable limit to the duration of a job in the gate?19:29
*** aspiers has joined #openstack-infra19:29
clarkberlon: we like to aim for an hour19:29
anteayaI'm not a guy19:29
anteayaand yeah we like an hour19:29
*** patrickeast_ has joined #openstack-infra19:30
openstackgerritMatthew Treinish proposed openstack/os-testr: Handle overlapping black regexes  https://review.openstack.org/37799919:30
*** stevemar has joined #openstack-infra19:30
*** jak-jak has joined #openstack-infra19:31
*** jroll has quit IRC19:31
*** jlvillal has joined #openstack-infra19:32
*** patrickeast__ has joined #openstack-infra19:32
*** PalTale has quit IRC19:33
*** eliqiao has quit IRC19:33
*** srobert has joined #openstack-infra19:33
*** sambetts|afk has quit IRC19:34
*** ajo has quit IRC19:34
*** eset has quit IRC19:34
*** hrubi_ has quit IRC19:34
*** DinaBelova has quit IRC19:34
*** logan- has joined #openstack-infra19:34
*** trown has quit IRC19:34
*** _ody has quit IRC19:34
*** SotK has quit IRC19:34
*** eantyshev has quit IRC19:34
*** Vivek___ has quit IRC19:34
*** htruta has quit IRC19:34
*** mdbooth has quit IRC19:34
*** dtantsur|mtg has quit IRC19:34
*** SlickNik has quit IRC19:34
*** timothyb89 has quit IRC19:34
*** chmouel_ has joined #openstack-infra19:35
*** lbeliveau has quit IRC19:35
*** ianw has quit IRC19:35
*** freerunner has quit IRC19:35
*** zeroDivisible has quit IRC19:35
*** raissa has quit IRC19:35
*** mkoderer__ has quit IRC19:35
*** tonyb has quit IRC19:35
*** adreznec has quit IRC19:35
*** rook has quit IRC19:35
*** jcook has quit IRC19:35
*** kencjohnston has quit IRC19:35
*** arif-ali has quit IRC19:35
*** Krenair has quit IRC19:35
*** akscram1 has quit IRC19:35
*** panda has quit IRC19:35
*** akijak has quit IRC19:35
*** flaviof has quit IRC19:35
*** chmouel has quit IRC19:35
*** patrickeast has quit IRC19:35
*** bin_ has quit IRC19:35
*** peterlisak has quit IRC19:35
*** GheRivero has quit IRC19:35
*** stevebaker has quit IRC19:35
*** kuromagi has quit IRC19:35
*** patrickeast__ is now known as patrickeast19:35
*** aspiers has quit IRC19:35
*** serverascode has quit IRC19:35
*** DuncanT has quit IRC19:35
*** rajinir has quit IRC19:35
*** pleia2 has quit IRC19:35
*** dkehn_ has quit IRC19:35
*** prometheanfire has quit IRC19:35
*** zul has quit IRC19:35
*** jak-jak is now known as Guest8421819:35
*** timothyb89 has joined #openstack-infra19:36
*** mdbooth_ has joined #openstack-infra19:36
*** Guest84218 has quit IRC19:36
*** tbarron has quit IRC19:36
*** simonmcc has quit IRC19:36
*** puiterwijk|cld has quit IRC19:36
*** rbrndt has quit IRC19:36
*** mtreinish has quit IRC19:36
*** hrybacki|afk has quit IRC19:36
*** krtaylor has quit IRC19:36
*** vkmc has quit IRC19:36
*** mattoliverau has quit IRC19:36
*** roaet has quit IRC19:36
*** peterlisak has joined #openstack-infra19:36
*** _ody has joined #openstack-infra19:36
*** akscram1 has joined #openstack-infra19:36
*** PalTale has joined #openstack-infra19:36
*** rhallisey__ has joined #openstack-infra19:36
*** stevemar has quit IRC19:36
*** patrickeast_ has quit IRC19:36
*** zzzeek has quit IRC19:36
*** rhallisey_ has quit IRC19:36
*** aeng_ has quit IRC19:36
*** loquacities has quit IRC19:36
*** tonyb_ has joined #openstack-infra19:36
*** matt6434 has joined #openstack-infra19:36
*** flepied has joined #openstack-infra19:36
*** kuromagi has joined #openstack-infra19:36
*** tbarron has joined #openstack-infra19:36
*** arif-ali has joined #openstack-infra19:36
*** rhallisey__ is now known as rhallisey19:36
*** eliqiao has joined #openstack-infra19:36
*** pleia2 has joined #openstack-infra19:36
*** mtreinish has joined #openstack-infra19:36
*** raissa has joined #openstack-infra19:36
*** adreznec has joined #openstack-infra19:36
*** stevemar has joined #openstack-infra19:36
*** akijak has joined #openstack-infra19:36
*** rook has joined #openstack-infra19:36
*** jcook- has joined #openstack-infra19:36
*** loquacity has joined #openstack-infra19:36
*** hrubi has joined #openstack-infra19:36
*** sileht has joined #openstack-infra19:36
*** ajo has joined #openstack-infra19:36
*** ianw has joined #openstack-infra19:36
*** trown has joined #openstack-infra19:37
*** SlickNik has joined #openstack-infra19:37
*** MarkAtwood has joined #openstack-infra19:37
*** aspiers has joined #openstack-infra19:37
*** panda has joined #openstack-infra19:37
*** rook is now known as Guest4447919:37
*** freerunner has joined #openstack-infra19:37
*** jcook- is now known as jcook19:37
*** vkmc has joined #openstack-infra19:37
*** sambetts|afk has joined #openstack-infra19:37
*** dkehn_ has joined #openstack-infra19:37
*** SotK has joined #openstack-infra19:37
*** krtaylor has joined #openstack-infra19:37
*** jroll has joined #openstack-infra19:37
*** jroll has quit IRC19:37
*** jroll has joined #openstack-infra19:37
*** dtantsur has joined #openstack-infra19:37
*** SotK has quit IRC19:37
*** aeng_ has joined #openstack-infra19:37
*** SotK has joined #openstack-infra19:38
*** zeroDivisible has joined #openstack-infra19:38
*** rbrndt has joined #openstack-infra19:38
*** eset has joined #openstack-infra19:38
*** eset has quit IRC19:38
*** eset has joined #openstack-infra19:38
*** htruta has joined #openstack-infra19:38
*** kencjohnston has joined #openstack-infra19:38
*** DinaBelova has joined #openstack-infra19:39
*** petems has quit IRC19:39
*** stevebaker has joined #openstack-infra19:39
*** mkoderer__ has joined #openstack-infra19:39
*** roaet has joined #openstack-infra19:39
*** lezbar has quit IRC19:40
*** lbeliveau has joined #openstack-infra19:40
*** lezbar has joined #openstack-infra19:40
*** pvaneck has joined #openstack-infra19:40
*** pvaneck has quit IRC19:41
*** prometheanfire has joined #openstack-infra19:42
*** med_ has joined #openstack-infra19:43
*** bknudson has quit IRC19:43
*** med_ is now known as Guest4578919:43
*** zerick has joined #openstack-infra19:43
*** timrc has quit IRC19:43
*** PalTale has quit IRC19:44
*** ruhe has quit IRC19:44
*** rvba` has quit IRC19:44
*** mandre has quit IRC19:44
*** kfox1111 has quit IRC19:44
*** gbraad_ has quit IRC19:44
*** Zara has quit IRC19:44
*** peterlisak has quit IRC19:44
*** kuromagi has quit IRC19:44
*** krtaylor has quit IRC19:44
*** kuromagi has joined #openstack-infra19:44
*** stevebaker has quit IRC19:44
*** eset has quit IRC19:44
*** jroll has quit IRC19:44
*** stevemar has quit IRC19:44
*** adreznec has quit IRC19:44
*** nadya has joined #openstack-infra19:44
*** DinaBelova has quit IRC19:44
*** krtaylor has joined #openstack-infra19:44
*** trown has quit IRC19:45
*** ajo has quit IRC19:45
*** matt6434 has quit IRC19:45
*** aspiers has quit IRC19:45
*** loquacity has quit IRC19:45
*** DinaBelova has joined #openstack-infra19:45
*** bknudson_ has joined #openstack-infra19:45
*** pahuang__ has joined #openstack-infra19:45
*** mdbooth has joined #openstack-infra19:45
*** mdbooth_ has quit IRC19:45
*** panda_ has joined #openstack-infra19:45
*** matt6434 has joined #openstack-infra19:45
*** salv-orlando has joined #openstack-infra19:45
*** eantyshev has joined #openstack-infra19:45
*** tonytan_brb has quit IRC19:45
*** vkmc has quit IRC19:45
*** jheroux has joined #openstack-infra19:45
*** Zara has joined #openstack-infra19:45
*** mkoderer__ has quit IRC19:45
*** panda has quit IRC19:45
*** jjheroux has quit IRC19:45
*** pahuang_ has quit IRC19:45
*** mkoderer__ has joined #openstack-infra19:45
*** adreznec_ has joined #openstack-infra19:46
*** adreznec_ is now known as adreznec19:46
*** eset has joined #openstack-infra19:46
*** eset has quit IRC19:46
*** eset has joined #openstack-infra19:46
*** loquacities has joined #openstack-infra19:46
*** PalTale has joined #openstack-infra19:46
*** timrc has joined #openstack-infra19:46
*** stevebaker has joined #openstack-infra19:46
*** vkmc has joined #openstack-infra19:46
*** jroll has joined #openstack-infra19:46
*** jroll has quit IRC19:46
*** jroll has joined #openstack-infra19:46
*** trown has joined #openstack-infra19:46
*** aspiers has joined #openstack-infra19:47
*** ajo has joined #openstack-infra19:47
*** rvba has joined #openstack-infra19:47
*** rvba has quit IRC19:47
*** rvba has joined #openstack-infra19:47
*** peterlisak has joined #openstack-infra19:47
*** patrickeast has quit IRC19:48
*** ruhe has joined #openstack-infra19:48
*** Vivek has joined #openstack-infra19:48
*** mandre has joined #openstack-infra19:48
*** rkukura has joined #openstack-infra19:48
*** Vivek is now known as Guest9629019:48
*** kfox1111 has joined #openstack-infra19:49
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Check changed track-upstreams in gerrit/projects.yaml  https://review.openstack.org/37800319:49
*** zul has joined #openstack-infra19:49
*** patrickeast has joined #openstack-infra19:49
*** Krenair has joined #openstack-infra19:50
*** amitgandhinz has quit IRC19:51
*** GheRivero has joined #openstack-infra19:51
*** flaviof has joined #openstack-infra19:51
*** amitgandhinz has joined #openstack-infra19:51
*** zzzeek has joined #openstack-infra19:52
*** stevemar has joined #openstack-infra19:52
*** gbraad has joined #openstack-infra19:53
*** tuannguy_ has quit IRC19:53
*** artmr_ has quit IRC19:53
*** tuannguyen has joined #openstack-infra19:53
*** DuncanT has joined #openstack-infra19:54
*** Apoorva has quit IRC19:54
*** unrahul has joined #openstack-infra19:54
*** lindsayk_ has joined #openstack-infra19:54
*** simonmcc has joined #openstack-infra19:54
*** adrian_otto has joined #openstack-infra19:54
*** psilvad has joined #openstack-infra19:55
*** rajinir has joined #openstack-infra19:56
*** tomhambleton has joined #openstack-infra19:56
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Check changed track-upstreams in gerrit/projects.yaml  https://review.openstack.org/37800319:57
*** panda_ is now known as panda19:57
*** briancurtin has joined #openstack-infra19:58
*** ggherdov- has joined #openstack-infra19:58
*** hrybacki|afk has joined #openstack-infra19:58
openstackgerritAndy Smith proposed openstack-infra/project-config: Update the gate-tempest-neutron-dsvm-src-oslo.messaging test  https://review.openstack.org/37150219:59
fricklerclarkb: https://review.openstack.org/369675 seems ready from my pov, if you can help nagging devstack cores to get it merged, that would be great19:59
clarkbmtreinish: sdague ^ consider yourselves nagged :)19:59
*** bin_ has joined #openstack-infra20:00
sdaguefrickler: +2, thanks for the respin20:00
clarkbanteaya: russellb the idea being have a section of the devs see how real world ops debug real problems20:00
russellbsounds cool20:00
clarkbanteaya: russellb beacuse we had a lot of "fun" with it in walldorf20:00
anteayaclarkb: well sounds like at least two tc members would like to see it20:01
russellbi'll help if you want to use OVN  ;-)20:01
anteayaclarkb: so I hope it gets accepted for a cross project session20:01
jeblairttx, anteaya, russelb: the infra-cloud debugging we did in germany was amazing, if we can do it with more project reps, that would be swell20:01
*** xgerman has joined #openstack-infra20:01
anteayasounds like folks want it20:01
pabelanger++20:01
jeblair(we weren't expecting to do that, it just happened, and we learned a ton and said "hey, i bet folks from the projects would love to see this")20:02
fungijeblair: we have indeed roped off entire service providers after they burned down, or maybe flooded20:02
*** pilgrimstack has quit IRC20:02
jeblairfungi: the ones that burn down are never near the ones flooding :(20:02
*** serverascode has joined #openstack-infra20:02
erlonclarkb: hmm, thanks!20:03
fungijeblair: sometimes they do one and then the other though20:03
funginot all fire suppression systems are oxygen evacuation20:03
*** thingee has joined #openstack-infra20:03
AJaegerfungi, is https://github.com/openstack/deb-python-xstatic-angular-schema-form on your radar as anteaya mentioned earlier? The syncing does not work yet...20:04
*** zara_the_lemur__ has left #openstack-infra20:05
*** puiterwijk|cld has joined #openstack-infra20:05
*** petems has joined #openstack-infra20:05
fungiAJaeger: fixed now20:05
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Check changed track-upstreams in gerrit/projects.yaml  https://review.openstack.org/37800320:06
AJaegerthanks, funig20:06
AJaegerfungi, here's a test to check *changed* upstream entreis ^20:06
fungithanks for mentioning anteaya, AJaeger20:06
anteayathanks for fixing20:06
AJaegerthat should avoid problems like the ones that zigo introduced ;(20:06
fungiAJaeger: awesome, that would have caught a couple of those incorrect tracked upstreams for deb repos20:06
AJaegerfungi: exactly20:06
*** dprince has quit IRC20:06
AJaegerme uses the python script to double check *all* 580+ entries20:07
AJaegerfungi: So, reviews for script are welcome - best before we import another bunch of broken track-upstreams ;(20:07
clarkbdid we have an invalid upstream url/20:07
AJaegerclarkb: those marked as track-upstream were not tested at all - and we had at least two broken ones20:08
*** kaisers1 has quit IRC20:08
clarkbah20:08
AJaegerclarkb: zigo trusted our testing - but we only tested those upstream URLs that had no track-upstream set20:08
*** fkautz_ has joined #openstack-infra20:08
AJaegerclarkb: so, now I check for *changes* in 378003 - instead of testing 580+ repos...20:09
clarkbya looks like you check the diff20:09
*** kaisers has joined #openstack-infra20:09
*** adrian_otto has quit IRC20:09
*** mhickey has joined #openstack-infra20:10
*** fguillot has joined #openstack-infra20:10
*** senk_ has joined #openstack-infra20:11
*** patrickeast_ has joined #openstack-infra20:13
*** sdake has joined #openstack-infra20:13
*** adrian_otto has joined #openstack-infra20:13
*** claudiub has quit IRC20:14
*** claudiub has joined #openstack-infra20:14
*** mrunge has quit IRC20:15
*** cardeois has quit IRC20:15
*** julim has joined #openstack-infra20:15
*** julim has quit IRC20:16
AJaegerfungi, clarkb : Found two more broken track-upstream URLs ;(20:16
*** cardeois has joined #openstack-infra20:16
AJaegermore than two ;(20:17
fungiouch20:17
*** mat128 is now known as mat128|mtg20:17
clarkbI will admit I still don't understand this setup in those repos20:17
clarkband think we should do the other thing that is apparently possible20:17
*** e0ne has quit IRC20:18
*** e0ne has joined #openstack-infra20:18
*** cardeois has quit IRC20:18
AJaegerclarkb: I'm not happy about this either. This is far too ad-hoc IMHO20:19
fungiclarkb: yeah, the problem is their workflow before moving into gerrit involved working from forks of the repos being packaged, and we didn't realize that we were importing forks of tons of our repos (plus some forks of repos that aren't ours)20:20
*** mrunge has joined #openstack-infra20:20
*** nadya_ has joined #openstack-infra20:20
*** cardeois has joined #openstack-infra20:20
fungiand their workflow also involves pulling into and merging tags from the upstreams into their forked packaging branches20:20
*** tuannguy_ has joined #openstack-infra20:21
*** patrickeast__ has joined #openstack-infra20:21
*** ggherdov- has quit IRC20:21
*** tomhambleton has quit IRC20:21
*** tonytan4ever has joined #openstack-infra20:21
*** logan- has quit IRC20:21
fungiso at this point it was easier to add track-upstream on them, until the workflow can be redesigned20:21
*** puiterwijk|cld has quit IRC20:21
*** mminesh has quit IRC20:21
*** pkoniszewski has quit IRC20:21
*** weshay_ has quit IRC20:21
*** jschwarz has quit IRC20:21
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Add support for testing predictable placement  https://review.openstack.org/37801420:21
*** PalTale has quit IRC20:21
*** e0ne_ has joined #openstack-infra20:22
*** unrahul has quit IRC20:22
*** e0ne_ has quit IRC20:22
*** dteselkin has quit IRC20:22
*** flip214 has quit IRC20:22
*** j_king has quit IRC20:22
clarkbit just seems like a massive amount of technical debt that is fostering more as we go20:22
fungibasically, most of the damage was already done at import20:22
*** PalTale has joined #openstack-infra20:22
*** rvba` has joined #openstack-infra20:22
*** frickler has quit IRC20:22
*** asselin has quit IRC20:22
*** _ari_ has quit IRC20:22
*** sai has quit IRC20:22
*** vtapia_ has quit IRC20:22
*** skraynev has quit IRC20:22
*** leifmadsen has quit IRC20:22
*** bknudson has joined #openstack-infra20:22
fungithough i agree, some of it feels like throwing good money after bad, so to speak20:22
*** mkoderer___ has joined #openstack-infra20:22
openstackgerritSarafraj Singh proposed openstack-infra/devstack-gate: Do not merge: Patch to test LM in nova  https://review.openstack.org/37801520:22
*** fkautz_ has quit IRC20:22
*** xgerman has quit IRC20:22
*** hrybacki|afk has quit IRC20:22
*** rajinir has quit IRC20:22
*** DuncanT has quit IRC20:22
*** flaviof has quit IRC20:22
*** GheRivero has quit IRC20:22
*** timrc has quit IRC20:22
*** bknudson_ has quit IRC20:22
*** hrubi has quit IRC20:22
*** MarkAtwood has quit IRC20:23
*** simonmcc has quit IRC20:23
*** rvba has quit IRC20:23
*** sambetts|afk has quit IRC20:23
*** kuromagi has quit IRC20:23
*** kuromagi^ has joined #openstack-infra20:23
*** senk_ has quit IRC20:23
*** thingee has quit IRC20:23
*** bin_ has quit IRC20:23
*** Guest96290 has quit IRC20:23
*** ajo has quit IRC20:23
*** pahuang__ has quit IRC20:23
*** DinaBelova has quit IRC20:23
*** nadya has quit IRC20:23
*** dkehn_ has quit IRC20:23
*** Guest44479 has quit IRC20:23
*** sdake_ has joined #openstack-infra20:23
*** timrc_ has joined #openstack-infra20:23
*** matt6434 has quit IRC20:23
*** pahuang__ has joined #openstack-infra20:23
*** dkehn_ has joined #openstack-infra20:23
*** thingee has joined #openstack-infra20:23
*** zzzeek has quit IRC20:23
*** jroll has quit IRC20:23
*** vkmc has quit IRC20:23
*** mdbooth has quit IRC20:23
*** rhallisey has quit IRC20:23
*** asingh_ has quit IRC20:23
*** j_king has joined #openstack-infra20:23
*** Vivek__ has joined #openstack-infra20:23
*** ajo has joined #openstack-infra20:23
*** jroll has joined #openstack-infra20:23
*** jroll has quit IRC20:23
*** jroll has joined #openstack-infra20:23
*** patrickeast has quit IRC20:23
*** adreznec has quit IRC20:24
*** mkoderer__ has quit IRC20:24
*** e0ne has quit IRC20:24
*** asselin_ has joined #openstack-infra20:24
*** mminesh has joined #openstack-infra20:24
*** tuannguyen has quit IRC20:24
*** stevebaker has quit IRC20:24
*** eset has quit IRC20:24
*** michaelxin has quit IRC20:24
*** sdake has quit IRC20:24
*** zul has quit IRC20:24
*** aspiers has quit IRC20:24
*** rhallisey_ has joined #openstack-infra20:24
*** matt6434 has joined #openstack-infra20:24
*** patrickeast__ is now known as patrickeast20:24
openstackgerritMerged openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/37733320:24
*** stevebak` has joined #openstack-infra20:24
*** weshay has joined #openstack-infra20:24
*** _ari_ has joined #openstack-infra20:24
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Fix upstream tracking repos  https://review.openstack.org/37801620:24
*** vkmc has joined #openstack-infra20:24
*** sambetts|afk has joined #openstack-infra20:24
*** gouthamr_ has quit IRC20:24
*** pkoniszewski has joined #openstack-infra20:24
*** gyee has joined #openstack-infra20:24
*** sai has joined #openstack-infra20:24
AJaegerfungi, clarkb,zigo : 5 broken repos found and fixed ^20:24
*** adreznec has joined #openstack-infra20:24
*** gouthamr has joined #openstack-infra20:25
*** Apoorva has joined #openstack-infra20:25
*** aspiers has joined #openstack-infra20:25
*** DinaBelova has joined #openstack-infra20:25
*** skraynev has joined #openstack-infra20:25
*** srobert has quit IRC20:25
*** leifmadsen has joined #openstack-infra20:25
*** mrunge has quit IRC20:25
*** hrubi has joined #openstack-infra20:25
*** mrunge has joined #openstack-infra20:26
* AJaeger refuses to review any changes that change track-upstream repos until https://review.openstack.org/#/c/378016 and https://review.openstack.org/#/c/378003 are merged20:26
*** logan- has joined #openstack-infra20:26
*** jschwarz has joined #openstack-infra20:26
*** wfoster has quit IRC20:27
*** dteselkin has joined #openstack-infra20:27
AJaegerargh, forgot to save the 5th entry ^ ;(20:27
*** trown has quit IRC20:27
*** flip214 has joined #openstack-infra20:27
*** flip214 has quit IRC20:27
*** flip214 has joined #openstack-infra20:27
AJaegerclarkb: this is on top of my other change - so this time, the updates get checked ;)20:27
*** ansiwen_ has quit IRC20:27
clarkbah20:27
clarkbperfect20:27
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Fix upstream tracking repos  https://review.openstack.org/37801620:27
AJaegerhere comes the update ^20:27
*** zzzeek has joined #openstack-infra20:27
*** trown has joined #openstack-infra20:28
*** burgerk_ has joined #openstack-infra20:28
*** lindsayk_ has quit IRC20:28
AJaegerclarkb: could you +2 again, please? Feel free to wait for the tests to complete ;)20:28
*** ansiwen has joined #openstack-infra20:29
clarkbya I need to do a few things before a phome call but will definitely rereview20:29
*** eset has joined #openstack-infra20:29
*** eset has quit IRC20:29
*** eset has joined #openstack-infra20:29
*** mdbooth has joined #openstack-infra20:29
*** Apoorva_ has joined #openstack-infra20:30
*** burgerk has joined #openstack-infra20:31
*** burgerk__ has quit IRC20:31
openstackgerritMatt Riedemann proposed openstack-infra/devstack-gate: DNM: test nova-manage db archive_deleted_rows post_test_hook  https://review.openstack.org/37711320:31
*** mminesh` has joined #openstack-infra20:31
*** PalTale has quit IRC20:31
* AJaeger waves good night20:32
*** mminesh has quit IRC20:32
*** petems has quit IRC20:32
*** jschwarz has quit IRC20:32
*** sambetts|afk has quit IRC20:32
*** weshay has quit IRC20:32
*** ajo has quit IRC20:32
*** serverascode has quit IRC20:32
*** kuromagi^ has quit IRC20:32
*** kuromagi has joined #openstack-infra20:32
*** xgerman has joined #openstack-infra20:32
*** matt6434 has quit IRC20:32
*** rook has joined #openstack-infra20:32
*** adreznec has quit IRC20:32
*** sai has quit IRC20:32
*** aspiers has quit IRC20:32
*** briancurtin has quit IRC20:32
*** peterlisak has quit IRC20:32
*** roaet has quit IRC20:32
*** DinaBelova has quit IRC20:32
*** mminesh` is now known as mminesh20:32
*** hrybacki|afk has joined #openstack-infra20:32
*** xgerman has quit IRC20:32
*** xgerman has joined #openstack-infra20:32
*** rook has quit IRC20:32
*** rook has joined #openstack-infra20:32
*** hrybacki|afk has quit IRC20:32
*** hrybacki|afk has joined #openstack-infra20:32
*** wfoster has joined #openstack-infra20:32
*** sambetts_ has joined #openstack-infra20:32
*** DinaBelova2 has joined #openstack-infra20:33
*** leifmadsen has quit IRC20:33
*** rhallisey_ has quit IRC20:33
*** burgerk_ has quit IRC20:33
*** hrubi has quit IRC20:33
*** Apoorva has quit IRC20:33
*** vkmc has quit IRC20:33
*** jroll has quit IRC20:33
*** thingee has quit IRC20:33
*** pahuang__ has quit IRC20:33
*** tbarron has quit IRC20:33
*** rook is now known as Guest9480420:33
*** jschwarz has joined #openstack-infra20:33
*** matt6434 has joined #openstack-infra20:33
*** PalTale has joined #openstack-infra20:33
*** leifmadsen_ has joined #openstack-infra20:33
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Test hostname map  https://review.openstack.org/37801720:33
*** rhallisey has joined #openstack-infra20:33
*** peterlisak has joined #openstack-infra20:33
*** ajo has joined #openstack-infra20:33
*** tbarron has joined #openstack-infra20:33
*** flip214 has quit IRC20:33
*** pahuang__ has joined #openstack-infra20:33
*** rakhmerov has quit IRC20:33
*** flip214 has joined #openstack-infra20:33
*** flip214 has quit IRC20:33
*** flip214 has joined #openstack-infra20:33
*** DinaBelova2 is now known as DinaBelova20:33
*** adreznec has joined #openstack-infra20:33
*** sai has joined #openstack-infra20:33
*** jroll has joined #openstack-infra20:34
*** jroll has quit IRC20:34
*** jroll has joined #openstack-infra20:34
*** weshay has joined #openstack-infra20:34
*** roaet has joined #openstack-infra20:34
anteayaAJaeger: good night20:34
*** tuannguy_ has quit IRC20:34
*** vkmc has joined #openstack-infra20:34
*** tuannguyen has joined #openstack-infra20:35
*** thingee has joined #openstack-infra20:35
*** simonmcc has joined #openstack-infra20:35
*** sc68cal_ is now known as sc68cal20:35
*** hrubi has joined #openstack-infra20:36
*** aspiers has joined #openstack-infra20:36
*** rakhmerov has joined #openstack-infra20:36
*** tuannguyen has quit IRC20:36
*** zul has joined #openstack-infra20:36
*** tuannguyen has joined #openstack-infra20:37
*** burgerk has quit IRC20:37
*** frickler has joined #openstack-infra20:37
*** senk_ has joined #openstack-infra20:38
*** erikmwilson has quit IRC20:38
*** jklare has quit IRC20:38
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Make bug name, type, and url explicit  https://review.openstack.org/37511320:39
*** asingh_ has joined #openstack-infra20:39
*** DuncanT has joined #openstack-infra20:39
*** puiterwijk|cld has joined #openstack-infra20:39
*** Kiall has quit IRC20:40
*** quade1 has quit IRC20:40
*** unrahul has joined #openstack-infra20:41
*** tuannguy_ has joined #openstack-infra20:42
*** kgiusti has left #openstack-infra20:42
*** jklare has joined #openstack-infra20:42
*** GheRivero has joined #openstack-infra20:43
*** asilenkov has quit IRC20:43
*** senk_ has quit IRC20:43
*** erikmwilson has joined #openstack-infra20:43
*** rajinir has joined #openstack-infra20:43
*** eharney has quit IRC20:43
*** serverascode has joined #openstack-infra20:43
*** rakhmerov__ has joined #openstack-infra20:43
*** rakhmerov has quit IRC20:43
*** MarkAtwood has joined #openstack-infra20:43
*** adreznec_ has joined #openstack-infra20:44
*** dteselkin_ has joined #openstack-infra20:44
*** patrickeast_ has quit IRC20:44
*** asselin has joined #openstack-infra20:44
*** PalTale has quit IRC20:44
*** Anticimex has quit IRC20:44
*** ajo_ has joined #openstack-infra20:44
*** michaelxin has joined #openstack-infra20:44
*** roaet_ has joined #openstack-infra20:44
*** dtantsur has quit IRC20:44
*** PalTale has joined #openstack-infra20:45
*** ajo has quit IRC20:45
*** dteselkin has quit IRC20:45
*** tuannguyen has quit IRC20:45
*** jcook has quit IRC20:45
*** DinaBelova has quit IRC20:45
*** asselin_ has quit IRC20:45
*** hughhalf has quit IRC20:45
*** dtantsur has joined #openstack-infra20:45
*** DinaBelova2 has joined #openstack-infra20:45
*** kuromagi^ has joined #openstack-infra20:45
*** leifmadsen has joined #openstack-infra20:45
*** psilvad has quit IRC20:46
*** DinaBelova2 is now known as DinaBelova20:46
*** hughhalf has joined #openstack-infra20:46
*** DuncanT has quit IRC20:46
*** roaet has quit IRC20:46
*** adreznec has quit IRC20:46
*** xgerman has quit IRC20:46
*** dkehn_ has quit IRC20:46
*** sdake_ has quit IRC20:46
*** jamielennox has quit IRC20:46
*** puiterwijk|cld has quit IRC20:46
*** vkmc has quit IRC20:46
*** jroll has quit IRC20:46
*** tbarron has quit IRC20:46
*** matt6434 has quit IRC20:46
*** leifmadsen_ has quit IRC20:46
*** hrybacki|afk has quit IRC20:46
*** Guest94804 has quit IRC20:46
*** kuromagi has quit IRC20:46
*** krtaylor has quit IRC20:46
*** bradm_ has quit IRC20:46
*** Anticime1 has joined #openstack-infra20:46
*** tbarron has joined #openstack-infra20:46
*** jroll has joined #openstack-infra20:46
*** bradm has joined #openstack-infra20:46
*** dkehn_ has joined #openstack-infra20:46
*** rook_ has joined #openstack-infra20:46
*** Kiall has joined #openstack-infra20:46
*** Kiall has quit IRC20:46
*** Kiall has joined #openstack-infra20:46
*** matt6434 has joined #openstack-infra20:46
*** tuannguy_ has quit IRC20:46
*** krtaylor has joined #openstack-infra20:46
*** adreznec_ is now known as adreznec20:46
*** asilenkov has joined #openstack-infra20:46
*** jcook has joined #openstack-infra20:46
*** jroll is now known as Guest4994520:46
*** andymccr has joined #openstack-infra20:47
*** vtapia_ has joined #openstack-infra20:47
*** sdake has joined #openstack-infra20:47
*** briancurtin has joined #openstack-infra20:47
*** Guest49945 is now known as jroll|dupe20:47
*** jroll|dupe has quit IRC20:47
*** jroll|dupe has joined #openstack-infra20:47
*** quade1 has joined #openstack-infra20:48
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Make bug name, type, and url explicit  https://review.openstack.org/37511320:49
*** tuannguyen has joined #openstack-infra20:49
*** psilvad has joined #openstack-infra20:50
*** vkmc has joined #openstack-infra20:50
*** jimbaker` has joined #openstack-infra20:50
*** bexelbie has joined #openstack-infra20:50
*** bin_ has joined #openstack-infra20:51
*** psilvad has quit IRC20:51
*** psilvad has joined #openstack-infra20:51
*** jamielennox has joined #openstack-infra20:51
*** beekneemech is now known as bnemec20:51
*** trown is now known as trown|outtypewww20:54
*** cardeois has quit IRC20:54
*** jroll|dupe is now known as jroll20:54
*** flaviof has joined #openstack-infra20:56
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Filter bugs shown in graphs based on regex  https://review.openstack.org/25763320:56
*** raildo has quit IRC20:56
*** cardeois has joined #openstack-infra20:56
*** puiterwijk|cld has joined #openstack-infra20:57
*** jkilpatr has quit IRC20:57
*** DuncanT has joined #openstack-infra20:57
*** ggherdov- has joined #openstack-infra20:58
*** psilvad has quit IRC20:58
*** fkautz_ has joined #openstack-infra20:58
*** psilvad has joined #openstack-infra20:58
*** andreww is now known as xarses20:59
*** patrickeast_ has joined #openstack-infra20:59
*** petems has joined #openstack-infra21:00
*** tuannguyen has quit IRC21:00
*** adrian_otto has quit IRC21:00
*** hrybacki|afk has joined #openstack-infra21:00
*** xgerman has joined #openstack-infra21:00
openstackgerritRichard Theis proposed openstack-infra/project-config: Move networking-ovn multinode job to check queue  https://review.openstack.org/37803221:00
*** tuannguyen has joined #openstack-infra21:01
*** slaweq_ has quit IRC21:01
*** nadya_ has quit IRC21:01
*** tuannguyen has joined #openstack-infra21:01
*** tomhambleton has joined #openstack-infra21:01
*** Apoorva has joined #openstack-infra21:02
*** ssancheztrujillo has joined #openstack-infra21:03
*** lindsayk_ has joined #openstack-infra21:04
*** tuannguyen has quit IRC21:04
*** adrian_otto has joined #openstack-infra21:04
*** eharney has joined #openstack-infra21:04
pabelangerclarkb: going to see if I can get a release of diskimage-builder, our new runtime-ssh-host-keys element was merged21:04
*** tuannguyen has joined #openstack-infra21:04
pabelangeror, about to merge21:04
pabelangereep, gate failure21:04
*** ijw has joined #openstack-infra21:04
*** Apoorva_ has quit IRC21:05
*** ddieterly has joined #openstack-infra21:05
*** mhickey has quit IRC21:09
*** tuannguyen has quit IRC21:09
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Include query results in graph  https://review.openstack.org/26018821:13
*** patrickeast has quit IRC21:15
sdagueasselin: looks like a bad merge in that last patch set21:16
openstackgerritKyle Haley proposed openstack-infra/project-config: Make docs gate non-voting for openstack/quark  https://review.openstack.org/37790821:16
*** patrickeast_ is now known as patrickeast21:16
asselin__sdague, thanks...will fix that up21:17
cloudnullanyone around that might want to give this a little push21:18
cloudnullhttps://review.openstack.org/#/c/377965/21:18
cloudnullsorry to ask late in the evenings.21:18
*** mat128|mtg is now known as mat12821:18
*** chmouel_ is now known as chmouel21:18
*** ddieterly is now known as ddieterly[away]21:18
cloudnullalso thanks pabelanger for the review already :)21:19
openstackgerritNate Johnston proposed openstack-infra/project-config: Bifurcate neutron-fwaas tempest gate for v2  https://review.openstack.org/37174921:19
*** kzaitsev_mb has joined #openstack-infra21:20
openstackgerritKyle Haley proposed openstack-infra/project-config: Make docs gate non-voting for openstack/quark  https://review.openstack.org/37790821:21
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Include query results in graph  https://review.openstack.org/26018821:21
*** ddieterly[away] is now known as ddieterly21:21
*** jkilpatr has joined #openstack-infra21:21
*** sdake has quit IRC21:22
*** kmartin has joined #openstack-infra21:22
*** thorst_ has quit IRC21:23
*** thorst_ has joined #openstack-infra21:24
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Include query results in graph  https://review.openstack.org/26018821:25
*** aviau has quit IRC21:26
*** aviau has joined #openstack-infra21:26
openstackgerritPaul Belanger proposed openstack/diskimage-builder: Default to http://ftp.us.debian.org/debian for debian-minimal  https://review.openstack.org/37803821:28
*** thorst_ has quit IRC21:29
clarkbcloudnull: I think jeblair wanted to get my nodepool patch in as well as his split the streams patch in then restart and apply your thing21:29
clarkbcloudnull: let me see where my patch ended up21:29
*** mikal_ is now known as mikal21:29
clarkboh cool it merged21:29
cloudnullclarkb: sounds good. no pressure regardless.21:29
clarkbso now I need to update our nodepool configs21:30
clarkbthen we can rebase your thing on that21:30
cloudnullI appreciate it though21:30
clarkbthis is the trick to make image shuffling less bad21:30
cloudnullnice!21:30
cloudnullthat's awesome21:30
*** hashar has quit IRC21:32
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Test hostname map  https://review.openstack.org/37801721:32
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Test with scheduler hints  https://review.openstack.org/37804021:32
*** gouthamr has quit IRC21:33
anteayafungi: so talking with dhellmann, we chatted about my patch to rename the file https://review.openstack.org/#/c/376828/ and his concern is that he doesn't want to rename it twice if a name is selected that is not good for election tools usage21:35
*** flepied has quit IRC21:35
anteayaso he and I have agreed to calling it legacy.yaml21:35
openstackgerritKyle Haley proposed openstack-infra/project-config: Make docs gate non-voting for openstack/quark  https://review.openstack.org/37790821:35
*** matt6434 is now known as mattoliverau21:35
anteayafungi: would that be a name that would work for eleciton tools, do you foresee?21:35
*** jheroux has quit IRC21:36
openstackgerritClark Boylan proposed openstack-infra/project-config: Use the DIB image id in glance uploads  https://review.openstack.org/37804321:36
*** rtheis has quit IRC21:36
openstackgerritDamian Szeluga proposed openstack-infra/project-config: Add checks to openstack/cookbook-openstack-application-catalog  https://review.openstack.org/37767121:36
clarkbcloudnull: pabelanger ^ that is what we need then we have to restart the nodepool builder21:36
clarkbjeblair: you around to do the other thing?21:36
*** ldnunes_ has quit IRC21:36
*** esberglu has quit IRC21:37
pabelangerclarkb: +221:38
clarkbjeblair: I am here to help21:38
*** flepied has joined #openstack-infra21:38
openstackgerritJames E. Blair proposed openstack-infra/project-config: Publish infra docs to AFS  https://review.openstack.org/37793021:38
openstackgerritJames E. Blair proposed openstack-infra/project-config: Add JJB AFS module  https://review.openstack.org/37804521:38
openstackgerritJames E. Blair proposed openstack-infra/project-config: Stop running compare-xml jobs  https://review.openstack.org/37804621:38
cloudnull+1 :)21:38
jeblairclarkb: yes!21:38
clarkbcool. I think we want to merge my project-config change first, then restart the builder21:39
jeblairmordred: when you have a sec, can you give those changes a once over?  there's some setup.py funness in them.21:39
fungianteaya: i don't see anything in either the old or new name that would preclude me integrating it in the tools/owners.py script in openstack-infra/system-config21:39
clarkband maybe while the builder is off we also do that other thing of yours? also is your thing going to submit twice the image builds?21:39
*** annegentle has quit IRC21:39
jeblairclarkb: i added a --no-images option, so we can run nodepoolds that do *nothing* for images, and then also one that does *only* images if we want21:39
clarkbjeblair: perfect21:40
jeblair(or we can combine that with either of the other functions)21:40
jeblairclarkb: your change looks good, though i wonder, should we take the opportunity to drop 'template-' from the name?21:40
jeblairclarkb: (that's a holdover from when we wanted to be able to distinguish a snapshot image build *host* from other hosts)21:41
clarkbjeblair: I thought about that too but I kind of like having a string to grep when listing images21:41
clarkbjeblair: maybe we change it from template to something else.21:41
jeblairclarkb: 'nodepool' then?21:41
clarkbit was handy to have that when cleaning up leaked images21:41
clarkbsure I can make it 'nodepool'21:41
jeblairclarkb: ++21:41
anteayafungi: thank you21:41
*** amitgandhinz has quit IRC21:41
openstackgerritsebastian marcet proposed openstack-infra/openstackid: Upgrade to LV 5.2  https://review.openstack.org/30552121:42
anteayafungi: once I offer the new name i'll bug you for a review since seeing your +1 was what dhellmann agreed to to support it21:42
openstackgerritClark Boylan proposed openstack-infra/project-config: Use the DIB image id in glance uploads  https://review.openstack.org/37804321:42
jeblairclarkb: +321:44
mtreinishfungi: so I was just thinking I wonder if I can reproduce the mosquitto crash if I hit it with a bunch of websocket connections21:44
* clarkb realizes we need to go through an image upload cycle before it actually helps21:44
clarkbso maybe cloudnull's change shouldn't really need to wait on this but too late :P21:44
mtreinishbecause it's been up for > 24hours21:44
fungianteaya: probably what i would do would be to add a new _retired meta-project to lump all the removed teams into, and aggregate its contributors into the tc electorate. then for teams which are listed in both the normal projects.yaml and that file, just add the contributions from retired repos back into those official teams as well as the tc electorate21:44
*** Apoorva has quit IRC21:44
fungimtreinish: give it a shot. i can always restart it again21:44
*** Apoorva has joined #openstack-infra21:45
jeblairzaro, fungi: who are current jjb cores who should review https://review.openstack.org/378046 ?21:45
jeblair(also, clearly not quite there technically, but we can certainly review it now based on intent :)21:47
fungijeblair: electrofelix and zxiiro are the two most active cores on jjb at the moment21:47
mtreinishfungi: hmm well nothing so far, established 200 websocket connections from my desktop and it's still going21:47
mtreinishI doubt we had that many when it was crashing21:47
*** spzala has quit IRC21:47
mtreinishoh, I spoke too soon21:47
mtreinishit's dead21:47
*** spzala has joined #openstack-infra21:48
fungianteaya: though this brings up another concern, we probably need a datestamp in there recording (roughly) when they were removed or retired so that contributions merged after that won't be counted toward official electorate21:48
fungianteaya: oh! it already has it21:49
fungigood enough21:49
anteayafungi: this is the rename bit if you can see yourself clear to agreeing with the name: https://review.openstack.org/#/c/376828/221:49
openstackgerritJames E. Blair proposed openstack-infra/project-config: Stop running compare-xml jobs  https://review.openstack.org/37804621:49
openstackgerritJames E. Blair proposed openstack-infra/project-config: Add JJB AFS module  https://review.openstack.org/37804521:49
openstackgerritJames E. Blair proposed openstack-infra/project-config: Publish infra docs to AFS  https://review.openstack.org/37793021:49
anteayaI can do other fixes in a follow on patch so as to not get things bogged down in the tc review21:49
anteayafungi: if that is okay with you21:49
*** adrian_otto has quit IRC21:49
mordredjeblair: I look at changes21:49
*** ijw has quit IRC21:50
anteayasounds like a song lyric21:50
mtreinishfungi: I wonder if we should disable websockets support?21:50
mordredjeblair: we have setup.py files in subdirs of jjb?21:50
anteayaI look at changes21:50
anteayachanges everyday21:50
anteayachanges change before my eyes21:50
mtreinishI'm not really a fan of that though because it gives us the firewall advantage21:51
anteayado-wah do-wah21:51
*** GheRivero has left #openstack-infra21:51
clarkbcloudnull: would you like me to rebase your change for you?21:51
jeblairmordred: no, the intent is to have a jjb module that exists only in the context of the project-config repo21:51
clarkbcloudnull: not sure how late in the day it is21:51
cloudnulli can make it21:51
cloudnullgo.21:51
fungimtreinish: well, again there's no certainty that the gpf in the websockets lib is the cause of teh crashes, rather than just a result of them21:52
anteayafungi: thank you21:52
clarkbjeblair: thinking we should just go ahead and get cloudnull's change in too since it will require a builder restart too21:52
fungimtreinish: though if you want to temporarily disable that to help rule it out, i'm fine with it21:52
jeblairmordred: we could root it higher in project-config (we could even make project-config a python project itself), but this seemed like a good place considering how... ephemeral? it is...21:52
mordredjeblair: AHHHHH21:52
mordredyah - makes sense21:52
jeblairmordred: but i'm totally open to suggestions :)21:52
openstackgerritKevin Carter (cloudnull) proposed openstack-infra/project-config: Updated OSIC project for HW specific flavors  https://review.openstack.org/37796521:52
cloudnullit just needed to be rebased on master right?21:52
*** spzala has quit IRC21:52
jeblairclarkb: that one^  ?21:52
mtreinishfungi: well it was websocket connections that crashed it. Like it ran fine with regular mqtt connections21:53
cloudnullor should it be rebased on a different commit21:53
clarkbjeblair: yup21:53
clarkbcloudnull: it needs to be rebased on https://review.openstack.org/378043 I think21:53
clarkbit may not actually conflict in which case win I guess21:53
cloudnullok. one sec21:53
*** ihrachys has quit IRC21:54
mordredjeblair: stack looks good to me21:54
clarkbcloudnull: but you need to add in the template-hostname thing21:54
openstackgerritKevin Carter (cloudnull) proposed openstack-infra/project-config: Updated OSIC project for HW specific flavors  https://review.openstack.org/37796521:54
jeblairclarkb: i think the sequence should be: land your change, stop nodepool, start launcher+deleter, land cloudnull's change, start imager21:54
clarkbcloudnull: which you can see in 37804321:54
cloudnullahahah. more noise inbound21:54
clarkbjeblair: I think that will work21:54
jeblairclarkb: i think if we land cloudnull's change early, we accidentally upload images with the old names21:54
clarkbjeblair: oh maybe21:55
*** mrmartin has quit IRC21:55
*** tonyb_ is now known as tonyb21:56
jeblairclarkb: i'm going to take a quick break while your change lands, then i'll be back to execute that with you21:57
openstackgerritKevin Carter (cloudnull) proposed openstack-infra/project-config: Updated OSIC project for HW specific flavors  https://review.openstack.org/37796521:57
cloudnullok so that should be it21:57
clarkbjeblair: ok I will grab caffeine21:57
*** mrmartin has joined #openstack-infra21:57
*** jamesdenton has quit IRC21:58
*** igormarnat has quit IRC21:58
*** igormarnat has joined #openstack-infra21:58
*** nherciu_ has quit IRC21:58
clarkbcloudnull: ty that looks right21:59
mtreinishfungi: when you get a sec can you restart mosquitto :)21:59
*** kaisers2 has quit IRC21:59
clarkbcloudnull: and those flavors are ready on your end ? I can check using a flavor list but that would bump into get caffeine time21:59
*** kaisers1 has joined #openstack-infra21:59
*** annegentle has joined #openstack-infra22:00
*** sdague has quit IRC22:02
fungimtreinish: oh, you managed to crash it?22:02
fungidone22:03
mtreinishfungi: thanks22:03
mtreinishfungi: yep, I needed to wait another 30 secs after I initially said it didn't break anything22:03
fungihah22:04
fungimtreinish: i saw you had a change up for using a newer source of mosquitto packages. do you think they've improved the websockets implementation in there?22:04
mtreinishfungi: TBH, I don't know. I saw a few websockety things in the release notes for a newer version, but I'm not sure22:05
mtreinishfungi: I'm spinning things up on a local vm right now to see if I can reproduce it22:05
fungialso, i guess if you can crash the one for firehose.o.o. then you should be able to reproduce locally22:05
mtreinishand get more details22:05
fungiindeed!22:05
fungiand if you can't reproduce locally, then that may give us clues what to check next as well22:06
*** njohnston_ has joined #openstack-infra22:06
openstackgerritMatthew Treinish proposed openstack-infra/system-config: Temporarily block port 80 and port 8080 on firehose  https://review.openstack.org/37806022:06
mtreinishfungi: ^^^ is probably a good idea for the time being though22:07
mtreinishfungi: hahaha, oops I forgot to kill all my backgrounded websocket clients and it crashed again22:07
openstackgerritMerged openstack-infra/project-config: Use the DIB image id in glance uploads  https://review.openstack.org/37804322:08
fungirestarted again22:08
mtreinishfungi: ok thanks22:08
anteayafungi: is anyone working on a replacement cacti server do you know?22:09
fungianteaya: i think the idea was to get it reporting more detailed stats on itself for a few hours to confirm 4gb will still give us headroom, then build the replacement and copy the data over to it22:10
*** ssancheztrujillo has quit IRC22:10
*** ijw has joined #openstack-infra22:10
fungiand update firewall rules everywhere for the new ip address22:11
anteayaI'm curious to know what swap is set to on it22:11
*** mriedem has quit IRC22:12
anteayait appears to just use the max amount of swap allocated, going with 4M is allocated: http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=1&rra_id=all22:12
anteayait used all of 1M in the prior server22:12
anteayawhen swap was allocated22:12
*** EricGonczer_ has quit IRC22:15
*** salv-orlando has quit IRC22:15
fungithose graphs are misleading since they're "free ram" and "free swap"22:15
*** borland__ has quit IRC22:15
fungiit's currently using about half a gigabyte out of the 4gb of swap we allocated22:15
anteayaah22:16
anteayaI was misled22:16
*** ijw has quit IRC22:16
jeblairhttp://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=28986&rra_id=all and http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=28988&rra_id=all are far better22:16
anteayathank you22:16
clarkbfungi: pabelanger mordred jeblair https://review.openstack.org/373219 is the last piece we need before turning on tls in multinode jobs22:16
mtreinishfungi: hmm, doesn't seem to be crashing so far on me locally22:16
mtreinishfungi: I've got 500 websocket connections up to the vm22:16
clarkbif you have spare review cycles to look at that change it would be excellent22:17
*** sshnaidm is now known as sshnaidm|afk22:17
clarkband now I need to look at AJaeger's fixes for tracking upstream22:17
jeblairclarkb: are you ready for me to stop nodepool?22:17
clarkbjeblair: yup22:17
jeblairokay, i will do that, and then start up a deleter process and then a launcher22:17
jeblairi will probably just do that in screen manually22:17
jeblairthen work on init scripts, etc, if we like it22:18
clarkbok22:18
* clarkb checks that puppet won't make that unhappy for us by starting the service22:18
anteayafungi: there was a time frame when it used all of the swap22:18
anteayaaccording to the second link jeblair shared22:18
clarkblooks like puppet should do what we want here22:18
openstackgerritAkihiro Motoki proposed openstack-infra/project-config: Show more graphs in a single row  https://review.openstack.org/37806222:18
*** ssancheztrujillo has joined #openstack-infra22:19
fungiso those new gaps in the graphs even though we have available swap and are no longer seeing insufficient memory conditions, indicate the gaps are caused by something else22:19
fungiseeing this in dmesg: [Tue Sep 27 22:17:35 2016] nf_conntrack: table full, dropping packet22:19
fungithere's a lot of that22:19
clarkbhuh22:20
fungilike basically, the dmesg ring buffer is only those and lines like: [Tue Sep 27 20:53:13 2016] net_ratelimit: 32 callbacks suppressed22:20
clarkbany idea if conntrack lets us tune the table size(s)?22:20
fungiprobably. i expect this is just not tuned to handle the recent increase in udp traffic from adding infra-cloud22:20
*** ijw has joined #openstack-infra22:21
fungiwe have a /proc/sys/net/netfilter/nf_conntrack_max of 3162822:21
mtreinishfungi: hmm still not crashing. The 1 interactive websock client I'm running doesn't seem to be keeping up with the stream as well. But I'm not running with qos so there aren't any guarantees that it should22:22
fungiand /proc/sys/net/netfilter/nf_conntrack_count has the same value22:22
fungii'm going to try temporarily doubling /proc/sys/net/netfilter/nf_conntrack_max and seeing what happens22:22
clarkbjeblair: looks like you have one of two daemons running now?22:22
*** Guest45789 is now known as med_22:22
jeblairclarkb: yep, the deleter only22:22
*** Julien-zte has quit IRC22:22
*** med_ has quit IRC22:23
*** med_ has joined #openstack-infra22:23
jeblairi'm going to restart it with a different logging config22:23
clarkbah right logging22:23
mtreinishfungi: I'm gonna wait for test lpmqtt to catch up (I expect it has to go through a lot of emails since I haven't processed the throwaway gmail in a month or so) and spam it all at once22:23
jeblairbecause i just realized that interleaved logs will be weird22:23
clarkbya22:23
*** tdasilva- has quit IRC22:23
*** salv-orlando has joined #openstack-infra22:23
mtreinishfungi: if it still doesn't crash after that I'm gonna have to conclude it's not soemthing I can locally reproduce22:23
*** annegentle has quit IRC22:23
fungiraised nf_conntrack_max to 63256 (twice its earlier value) and watching nf_conntrack_count balloon now22:24
cloudnullclarkb: sorry was afk for a sec22:24
clarkbcloudnull: its ifne I think everything is ready for going in22:24
cloudnullyes we're good to go on this end with the flavors and such22:24
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Test with scheduler hints  https://review.openstack.org/37804022:24
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Add support for testing predictable placement  https://review.openstack.org/37801422:24
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Test hostname map  https://review.openstack.org/37801722:25
*** njohnston_ has quit IRC22:25
clarkbwe are letting jeblair do his thing first then once he is done we will come around and update the builder configs22:25
*** njohnston_ has joined #openstack-infra22:26
cloudnullsweet22:26
cloudnullif there's something I can do to help, let me know.22:26
clarkbjeblair: looks like you have both running now but both are with --no-images. shouldn't one be set to build iamges? or are you thinking of running a third?22:26
anteayafungi: so far the cacti graphs I'm looking at are continuing to render data, instead of blank areas in the graph22:26
*** ijw has quit IRC22:27
fungianteaya: you don't see gaps at http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=28988&rra_id=all ?22:27
anteayafungi: yes I see gaps, but old gaps, new info appears to be displayed22:28
anteayaoh spoke too soon22:28
*** mtanino has joined #openstack-infra22:29
*** fguillot has quit IRC22:29
jeblairclarkb: i'm going to run a third22:29
jeblairclarkb: but i'm unable to run the second because of the webapp22:30
clarkboh22:30
clarkbconflict over the port?22:30
clarkbSO_REUSEADDR might help?22:30
clarkbI think it should be fine for any of these daemons to process the request if our kernel supports it /me checks22:30
clarkber not that flag22:31
fungithat only lets you reuse an address+port combination that was recently closed without waiting to make sure all old connections are closed out22:31
*** baoli has quit IRC22:31
fungiif that's the one i think it is22:31
clarkbSO_REUSEPORT22:31
clarkbya its port not addr that we want22:32
clarkbnew in linux 3.922:32
clarkbso we can use it if we want22:32
*** njohnston_ is now known as njohnston__22:33
clarkbthough it looks like you have to set the option on all of the sockets before calling bind which might be tricky to organize without more marshalling22:33
mtreinishfungi: yeah, no luck crashing it with a ton of websocket connections to mosquitto running on my vm22:33
jeblairclarkb: i need another fix anyway, i'll just add another flag22:33
*** spzala has joined #openstack-infra22:33
clarkbjeblair: ok22:33
fungiclarkb: oh, wow, that sockopt does actually allow multiple listeners on one port?22:33
fungii never knew22:34
clarkbyup22:34
clarkbits shiny22:34
openstackgerritJames E. Blair proposed openstack-infra/nodepool: Add --no-webapp option  https://review.openstack.org/37806422:34
fungimanpages to the rescue22:34
mtreinishfungi: so that probably means it's something isolated to however firehose01.o.o is setup differently22:34
jeblairclarkb:  https://review.openstack.org/37806422:34
fungimtreinish: ubuntu 14.04 lts vm?22:34
jeblairclarkb: i'm going to restart it with a combined launcher/deleter for a few mins until that lands22:34
clarkbjeblair: ok22:34
jeblairclarkb: i'll still start it with --no-images, so we can safely land cloudnull's change22:35
mtreinishfungi: 16.04.1 LTS, firehose should also be a xenial node22:35
clarkbjeblair: I think as long as we don't restart the builder we are safe to do that? the builder won't reread the config off disk unless we restart it22:35
fungimtreinish: oh, yep, you're right22:36
jeblairclarkb: i think the main daemon is responsible for sending the job to the builder22:36
jeblairclarkb: it's the thing that will notice the missing images and start to upload them22:36
fungimtreinish: http://docs.openstack.org/infra/system-config/sysadmin.html#making-a-change-in-puppet has information on reusing our puppet class/modules which should get you pretty close to an identical system22:36
jeblairclarkb: (maybe it will fail because the builder hasn't reloaded the config though?)22:36
clarkbjeblair: yup but the builder won't know anything about the new cloud providers so it should fail cleanly22:36
*** ijw has joined #openstack-infra22:36
clarkbjeblair: yup that22:36
mtreinishfungi: I used the puppet mosquitto, lpmqtt, and germqtt modules to setup the vm22:36
fungimtreinish: oh, then it should be virtually identical22:37
mtreinishfungi: sigh, I have to stop saying things work, it looks like it just crashed while we were talking22:38
fungiunless the issue has to do with it running on a xen domu or something else rackspaceish22:38
jeblairclarkb, cloudnull: +3 37796522:38
mtreinishfungi: traps: mosquitto[4788] general protection ip:7fc372ca1a0c sp:7ffc172b3d20 error:0 in libwebsockets.so.7[7fc372c99000+1f000]22:38
mtreinishdon't know why it was so much harder to reproduce on my vm22:39
clarkbjeblair: commented22:39
clarkb+2'd but with things to consider22:39
*** cardeois has quit IRC22:39
anteayaclarkb: did you propose the cross project interactive debugging session yet? https://etherpad.openstack.org/p/ocata-cross-project-sessions22:40
clarkbanteaya: no22:41
anteayaokay thanks22:41
jeblairclarkb: agreed, though backwards compat with "no flags means run all the things" is a consideration22:42
cloudnulljeblair: tyvm22:42
*** amitgandhinz has joined #openstack-infra22:42
fungimtreinish: maybe because there are other more diverse users hitting firehose22:42
fungimtreinish: but that does indeed look like what we're hitting in production22:43
openstackgerritMonty Taylor proposed openstack-infra/shade: Add caching decorators to zones and record_sets  https://review.openstack.org/37806922:43
*** amotoki has quit IRC22:43
*** roxanaghe has joined #openstack-infra22:44
clarkbactually looks like you may just have to set the option before binding on each socket22:44
clarkbnot that all sockets must have it before binding any one socket22:44
*** ddieterly is now known as ddieterly[away]22:44
* clarkb preps a patch because whynot22:45
*** roxanaghe has quit IRC22:45
*** salv-orlando has quit IRC22:46
clarkboh its paste hrm22:46
openstackgerritMerged openstack-infra/project-config: Updated OSIC project for HW specific flavors  https://review.openstack.org/37796522:46
*** roxanaghe has joined #openstack-infra22:46
mtreinishfungi: well, I'm gonna try building the latest 1.7.x series of libwebsockets on the vm and restarting the test again see if they've fixed it22:47
mtreinishthere have been a bunch of bugfix releases since 1.7.1, which is what xenial packages22:48
clarkbso ya nevermind22:48
mtreinishfungi: https://github.com/warmcat/libwebsockets/blob/v1.7-stable/changelog22:48
*** rbrndt has quit IRC22:48
*** taron has quit IRC22:49
*** taron has joined #openstack-infra22:50
*** alex_xu has quit IRC22:50
*** mrodden has quit IRC22:51
*** alex_xu has joined #openstack-infra22:52
*** jamesdenton has joined #openstack-infra22:53
*** jamesden_ has joined #openstack-infra22:53
*** mrodden has joined #openstack-infra22:54
*** sdake has joined #openstack-infra22:56
*** jamesdenton has quit IRC22:57
*** ijw has quit IRC22:57
*** sdake_ has joined #openstack-infra22:58
*** ddieterly[away] is now known as ddieterly22:58
*** ddieterly is now known as ddieterly[away]22:59
*** ddieterly[away] has quit IRC22:59
*** ssancheztrujillo has quit IRC22:59
*** claudiub|2 has joined #openstack-infra23:00
clarkbjeblair: looks like the config changes have applied on disk23:00
clarkbjeblair: I will wait for you to finish up your daemon split before restarting the builder23:00
*** sdake has quit IRC23:01
clarkboh hrm I think I may have realized tehre will be an issue with this setup with the image names :( when we go to rebuild the next days we can possibly race the detection of duplicates and upload duplicates anyways. Thats no worse than the old situation so not really a regression just not as awesome as I thought this might end up being23:02
*** claudiub has quit IRC23:03
jeblairclarkb: hrm. this is true.23:04
clarkbjeblair: I think that generally since we upload provider by provider we will see it helps in practice23:04
clarkbbtu there will still be some racing23:04
anteayafungi: I haven't gotten cacti data for the past 30-40 minutes: http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=28988&rra_id=all23:05
jeblairclarkb: ya23:06
mtreinishfungi: hah, the mosquitto logging is at least consistent. I got: "Error: Unable to create websockets listener on port 80." with libwebsocket built from source on the latest version on the 1.7 branch23:07
fungianteaya: yeah, i think something may be odd with it trying to collect data from itself... though i see no new "nf_conntrack: table full, dropping packet" since upping nf_conntrack_max so that may help the less frequent remaining gaps in some of the other hosts' graphs23:07
*** gordc has quit IRC23:08
*** hongbin has quit IRC23:08
jeblairclarkb: well, 378064 is still waiting for a +3.23:09
mtreinishactually I guess consistent would be nothing, so that's an improvement I guess23:09
fungianteaya: and now that pressure on memory and the connection tracking table have been released, i see system load climbing up around 2-3 (with a single cpu on this flavor) so maybe we have enough data to inform our choice of replacement flavor23:09
jeblairclarkb: i need to start thinking about eod23:09
clarkbjeblair: ok23:09
jeblairclarkb: so if we're not going to land it now, i guess i should restart nodepool normally23:09
jeblairclarkb: do you want me to restart the builder too?23:09
clarkbjeblair: sure23:09
jeblairand we'll end up with dual images23:10
openstackgerritMerged openstack/diskimage-builder: Create runtime-ssh-host-keys element  https://review.openstack.org/37436623:10
clarkbjeblair: ya which I think is fine then we can see in the future how often we avoid duplicates23:10
anteayafungi: yay23:10
anteayaclarkb: so logstash using all of its space is a known thing, yes? http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=422&rra_id=all23:10
openstackgerritMonty Taylor proposed openstack-infra/shade: Add caching decorators to zones and record_sets  https://review.openstack.org/37806923:10
fungii think logstash has been regularly filling up its rootfs with logs23:11
clarkbanteaya: I think someone mentioned we weren't rotating logs properly?23:11
mtreinishclarkb, fungi, jeblair: if you get a sec for: https://review.openstack.org/#/c/378060/1 to stop the potential DOS on firehose23:11
clarkbwhich started happening in july?23:11
fungiwell, if the logs get too large, logrotate can also just fail to rotate them23:11
mtreinishI'll probably keep playing with debugging it later tonight or tomorrow23:11
pabelangergreghaynes: do you have bandwidth to tag 1.20.0 (I31ad6) today?23:12
anteayaclarkb: the graph confirms July23:12
pabelangergaelL: sorry, b0d72a23:12
clarkbfungi: ya we may have to reduce the total retention23:12
pabelangergreghaynes: err: b0d72a3123:12
anteayabit of a dip on the 29th but then right back up to full23:12
pabelangergaelL: wrong nick23:12
fungiyeah, right now there's a 9.9G /var/log/logprocessor/log-client-debug.log.1, and a 27G /var/log/logprocessor/log-client-debug.log23:13
clarkboh wow23:13
clarkbwhat changed in july23:14
jeblairclarkb, cloudnull: nodepool is restarted with the new config23:14
jeblairclarkb, cloudnull: apparently we didn't notice that we're missing some things though23:14
fungiclarkb: is it any loss if we drop the current logs, restart the process and see what if logrotate does? we have /var/log/logprocessor/log-client-debug.log { compress copytruncate missingok rotate 7 daily notifempty } configured for it23:14
jeblairclarkb, cloudnull: the image entries for fedora-24 look like they are truncated23:14
clarkbjeblair: I see it with a bunch of images in a building state for the osic providers23:14
fungiso at least it should be trying to rotate these23:14
clarkboh ya fedora 24 not happy there23:15
greghaynespabelanger: I should, one sec23:15
jeblairclarkb, cloudnull: and centos-7 seems to be missing23:15
clarkboh and we haev to update the labels anyways23:15
jeblairshould just be able to go ahead and land a change to fix that23:15
* clarkb edits nodepool.yaml23:16
cloudnullwas that something i missed in my PR ?23:16
cloudnullor something I should update?23:16
cloudnullnvmd. clarkb you doing a thing?23:16
*** gildub has joined #openstack-infra23:17
clarkbcloudnull: ya let me quickly update it23:17
*** amitgandhinz has quit IRC23:17
cloudnullkk23:17
*** adrian_otto has joined #openstack-infra23:18
*** ijw has joined #openstack-infra23:18
*** tphummel has quit IRC23:18
*** ijw has quit IRC23:19
*** ijw has joined #openstack-infra23:19
*** markvoelker has quit IRC23:19
openstackgerritClark Boylan proposed openstack-infra/project-config: Add missing osic images and add to labels  https://review.openstack.org/37807723:19
clarkbcloudnull: jeblair ^ that should do it I think23:19
*** tphummel has joined #openstack-infra23:20
pabelangergreghaynes: great, thanks23:20
cloudnullopps. my bad, sorry i missed those23:20
clarkbfungi: pabelanger can you also review 378077?23:22
clarkbI can restart the builder once that gets in place23:22
pabelangerclarkb: looking23:22
mordredclarkb: lgtm23:23
clarkbalso jeblair's nodepool change 37806423:23
jeblaircloudnull: what's the '-disk' provider?23:24
jeblairi got the others are different machines, i missed the memo on disk23:24
clarkbjeblair: I think the differences are s3* are ssds and disk is spinning rust23:25
cloudnullwe have spinning disk machines which we simply call the disk hw profile.23:25
jeblairare they 3500 or 3700s?23:25
jeblairoh23:26
jeblairthe 3500 or 3700 are ssd model numbers23:26
clarkbya23:26
jeblairso it's all the same chassis, but 3 different kinds of storage23:26
cloudnullyes.23:26
jeblairgotcha, thanks :)23:26
clarkbso we are about to become a case study in VM IO performance23:26
jeblairneat!23:26
cloudnull^ :)23:26
pabelangerexciting23:27
mordredthat is exciting23:27
jeblairand i am happy to know that it's one variable with 3 values :)23:27
cloudnullit is. this will be like real world rally.23:27
mordred\o/23:27
* cloudnull so excite :D23:28
*** ddieterly has joined #openstack-infra23:29
*** ddieterly is now known as ddieterly[away]23:31
*** lezbar has quit IRC23:31
*** ijw has quit IRC23:31
*** lezbar has joined #openstack-infra23:31
*** rcarrillocruz has quit IRC23:33
*** ijw has joined #openstack-infra23:33
*** ijw has quit IRC23:34
*** rcarrillocruz has joined #openstack-infra23:34
*** ijw has joined #openstack-infra23:34
*** ijw has quit IRC23:36
*** baoli has joined #openstack-infra23:37
mtreinishfungi: finally, got a stack trace: http://paste.openstack.org/show/583184/23:37
*** njohnston has quit IRC23:37
mtreinishnow what to do with it :)23:37
*** rhallisey has quit IRC23:38
anteayamtreinish: those are timestamps?23:39
*** tdasilva has joined #openstack-infra23:39
openstackgerritMerged openstack-infra/project-config: Add missing osic images and add to labels  https://review.openstack.org/37807723:40
*** vinaypotluri has quit IRC23:42
*** ijw has joined #openstack-infra23:42
*** jkilpatr has quit IRC23:42
fungimtreinish: quick batman, to the upstream bug tracker!23:43
mtreinishanteaya: those are memory addresses23:43
anteayaah23:43
fungianteaya: those aren't timestamps, they're stack call addresses23:43
anteayathank you23:43
*** lock_ is now known as ianychoi23:44
fungi(that's no moon...!!!)23:44
mtreinishfungi: which one is the question?23:44
anteaya:)23:44
mtreinishI guess mosquitto23:44
openstackgerritMerged openstack-infra/nodepool: Add --no-webapp option  https://review.openstack.org/37806423:44
clarkbnow I need to dig out my blurays23:44
mtreinishit's probably passing in a bad pointer to libwebsockets or something23:44
anteayayou have star wars on bluray?23:45
fungimtreinish: yeah, i'd start in mosquitto and they can forward you as needed23:45
mordredI like it when we start doing C23:45
clarkbdid someone say C?23:45
fungimtreinish: were you running it under gdb or did you get a core dump and map it?23:45
clarkbthe last time I did C I debugged the segfault in the python garbage collector23:45
mtreinishfungi: I just ran it under gdb23:45
fungiat least we have source line numbers23:46
funginice it was compiled with debugging23:46
*** Swami has quit IRC23:46
mtreinishI had to install the debug symbol package too :)23:46
fungior did you compile it yourself?23:46
fungiahh, yep23:46
clarkblooks like we need symbols for the websocket lib?23:46
fungiso this was the packaged version in xenial then?23:46
fungiclarkb: yeah, i guess we only have them for mosquitto23:47
mtreinishclarkb: I couldn't find a debug symbol package for libwebsockets23:47
clarkbyou know what I think of when you saw mosquitto is the rayman mosquitto levels23:47
greghaynespabelanger: http://git.openstack.org/cgit/openstack/diskimage-builder/tag/?h=1.20.023:47
fungii never did play rayman23:47
fungibut it looked fun23:47
mtreinishI was having trouble getting mosquitto to start building libwebsockets from source23:48
*** zhurong has joined #openstack-infra23:48
*** mtanino has quit IRC23:49
fungimtreinish: you can use debuild to build the libwebsockets source package in ubuntu. might not be hard to forward-port the package. and then rebuild mosquitto against the headers for that23:49
*** ddieterly[away] is now known as ddieterly23:49
fungior you might want to take an icepick to your eyes, one at a time. i don't quite know what sort of masochist you are23:49
* mordred is expecting mtreinish to pick icepick to the eyes23:50
mtreinishfungi: heh, yeah I would probably just abandon using anything ubuntu if I were gonna take it that far23:50
mtreinishand just build both from source23:50
mtreinishstraight from git23:50
openstackgerritKevin Carter (cloudnull) proposed openstack-infra/project-config: Update server distribution to HW specific entries  https://review.openstack.org/37810023:51
mtreinishI probably should just do that after I file the bug to see if I can reproduce it with the latest version of mosquitto23:51
cloudnullidk if we're ready for that just yet , but that rebalances the vms across the new entries.23:51
fungimtreinish: good call23:51
fungimtreinish: upstream may know the problem straight away or something, and save you the extra pain23:51
clarkbcloudnull: not quite, once we get my last change in place and then the remaining images upload then we can rebalance23:51
cloudnullfor sure, i figured i'd just queue up the PR23:52
clarkbyup no problem getting it lined up23:52
fungiclarkb: not sure if you saw my question about logstash.o.o earlier... is it any loss if we drop the current logs, restart the process and see what if logrotate does? we have /var/log/logprocessor/log-client-debug.log { compress copytruncate missingok rotate 7 daily notifempty } configured for it23:52
fungijust no idea if you want the old logs for anything, or are okay losing them so we can start collecting new logs23:53
clarkbfungi: I think its ok to move them aside and move forward23:53
*** salv-orlando has joined #openstack-infra23:53
clarkb(I didn't see the question earlier sorry)23:53
fungiwell, there's no aside here unless you want them copied into /opt (and even it doesn't have enough remaining room)23:54
clarkber s/aside/delete/23:54
*** tonytan4ever has quit IRC23:55
fungidone23:55
clarkbthere are no known issues iwth it and its a secondary service so worst case we can just shove things back through again to get new logs23:55
*** spzala has quit IRC23:55
fungiit's the jenkins-log-client service which writes those? just making sure i know what needs to be restarted23:56
fungithough after a full rootfs, it's usually good hygiene to reboot23:57
clarkbfungi: yes23:57
fungiany concerns with me rebooting logstash.o.o23:57
fungi?23:57
clarkbshouldn't be. its just apache to proxy ES, kibana, and that gearman service23:57
clarkbwell zmq to gearman layer23:57
fungiokay, rebooting then23:57
fungii'm just not usually comfortable that all services on a system are sure to be running properly if they ran into a failure to open a file handle at some point because of enospc23:58
*** salv-orlando has quit IRC23:58
clarkbcloudnull: down to 9 image uplaods and that includes the ones as part of my update change23:59
clarkbcloudnull: so we should be good to do the rebalancing tomorrow morning23:59
cloudnullsweet!23:59
cloudnullno rush23:59
cloudnullthanks though23:59
*** spzala_ has joined #openstack-infra23:59
* fungi imagines taking nodepool into the mechanic to get its oil changed and providers rebalanced23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!