Monday, 2015-05-04

*** mikeit_ has quit IRC00:00
*** Alexandra_ has joined #openstack-infra00:01
*** mikeit_ has joined #openstack-infra00:02
*** sdake_ has joined #openstack-infra00:07
*** woodster_ has quit IRC00:10
*** sdake has quit IRC00:11
*** akshai has joined #openstack-infra00:18
*** mikeit_ has quit IRC00:18
*** ashleighfarnham has joined #openstack-infra00:19
*** markvoelker has joined #openstack-infra00:21
*** akshai has quit IRC00:21
*** mikeitt has joined #openstack-infra00:21
*** armax has joined #openstack-infra00:22
*** markvoelker has quit IRC00:25
*** mikeitt has quit IRC00:31
*** mikeit has joined #openstack-infra00:32
*** mpaolino has joined #openstack-infra00:34
*** camunoz has quit IRC00:34
*** mpaolino has quit IRC00:36
*** zhiwei has joined #openstack-infra00:37
*** otter768 has joined #openstack-infra00:46
*** camunoz has joined #openstack-infra00:47
*** salv-orlando has joined #openstack-infra00:47
*** salv-orlando has quit IRC00:50
*** kiseok7_ has quit IRC00:53
*** ashleighfarnham has quit IRC00:53
*** cody-somerville has quit IRC00:54
*** marcusvrn has joined #openstack-infra00:57
*** salv-orlando has joined #openstack-infra01:05
*** armax has quit IRC01:08
*** salv-orlando has quit IRC01:12
*** marcusvrn has quit IRC01:19
*** markvoelker has joined #openstack-infra01:21
*** sdake has joined #openstack-infra01:24
*** sdake_ has quit IRC01:26
*** markvoelker has quit IRC01:26
*** sdake_ has joined #openstack-infra01:27
openstackgerritMonty Taylor proposed openstack-infra/shade: Enhance the OperatorCloud constructor  https://review.openstack.org/17964701:29
mordredShrews: ^^ there ya go01:30
*** sdake has quit IRC01:30
openstackgerritJeremy Stanley proposed openstack-infra/system-config: Split diskimage-builder built images out  https://review.openstack.org/16783701:40
*** Alexandra_ has quit IRC01:41
*** sputnik13 has joined #openstack-infra01:41
*** sputnik13 has quit IRC01:43
*** sputnik13 has joined #openstack-infra01:43
*** camunoz has quit IRC01:44
*** kiseok7_ has joined #openstack-infra01:44
*** sputnik13 has quit IRC01:45
*** sputnik13 has joined #openstack-infra01:49
*** patrickeast has joined #openstack-infra01:49
*** zz_dimtruck is now known as dimtruck01:54
*** rkukura_ has quit IRC01:55
*** rkukura has joined #openstack-infra01:56
*** sarob has quit IRC01:57
*** camunoz has joined #openstack-infra01:57
*** gokrokve has joined #openstack-infra01:57
*** btully has quit IRC01:58
*** patrickeast has quit IRC02:00
*** Kennan has quit IRC02:01
*** dimsum__ has quit IRC02:04
*** Kennan has joined #openstack-infra02:06
openstackgerritMerged openstack-infra/system-config: Split diskimage-builder built images out  https://review.openstack.org/16783702:06
*** Kennan has quit IRC02:08
*** emagana has joined #openstack-infra02:11
*** Kennan has joined #openstack-infra02:11
*** dimtruck is now known as zz_dimtruck02:15
*** sabeen1 has quit IRC02:17
*** Somay has joined #openstack-infra02:19
*** Somay has quit IRC02:22
*** markvoelker has joined #openstack-infra02:22
*** achanda has quit IRC02:25
*** markvoelker has quit IRC02:27
*** sputnik13 has quit IRC02:28
*** mikeit has quit IRC02:38
*** patrickeast has joined #openstack-infra02:41
*** patrickeast has quit IRC02:42
openstackgerritMerged openstack-infra/bindep: Allow mixed-case package names  https://review.openstack.org/17911102:44
*** sdake_ is now known as sdake02:44
openstackgerritMerged openstack-infra/bindep: Add positive/negative tests exercising the parser  https://review.openstack.org/17837802:46
openstackgerritJamie Lennox proposed openstack-infra/project-config: Add django-openstack-auth-kerberos project  https://review.openstack.org/17280302:51
*** salv-orlando has joined #openstack-infra02:56
*** sergsh has quit IRC02:56
*** sergsh has joined #openstack-infra02:56
*** salv-orlando has quit IRC03:01
*** dimsum__ has joined #openstack-infra03:05
openstackgerritMerged openstack-infra/bindep: Allow hyphens in profile strings  https://review.openstack.org/17837903:09
*** dimsum__ has quit IRC03:10
*** salv-orlando has joined #openstack-infra03:14
*** sputnik13 has joined #openstack-infra03:16
*** yfried|afk is now known as yfried03:19
*** salv-orlando has quit IRC03:19
*** coolsvapl is now known as coolsvap03:22
*** SumitNaiksatam has joined #openstack-infra03:22
openstackgerritJeremy Stanley proposed openstack-infra/project-config: Look for yum instead of rpm when caching devstack  https://review.openstack.org/17968703:22
*** markvoelker has joined #openstack-infra03:23
*** achanda has joined #openstack-infra03:26
openstackgerritJeremy Stanley proposed openstack-infra/project-config: Import subprocess in cache devstack element  https://review.openstack.org/17968803:27
*** markvoelker has quit IRC03:27
*** unicell has joined #openstack-infra03:30
*** achanda has quit IRC03:31
*** unicell1 has quit IRC03:32
openstackgerritIan Wienand proposed openstack-infra/devstack-gate: [WIP] run centos7 with neutron  https://review.openstack.org/17968903:34
*** reed_ has joined #openstack-infra03:35
*** reed_ has quit IRC03:37
*** jyuso1 has joined #openstack-infra03:42
*** Somay has joined #openstack-infra03:43
openstackgerritMerged openstack-infra/project-config: Import subprocess in cache devstack element  https://review.openstack.org/17968803:48
*** bswartz has quit IRC03:50
*** bswartz has joined #openstack-infra03:52
lifelesstchaypo: I've closed off the pr's you had open on my github tripleo-end-to-end repo: I'm not hacking on that, nor maintaining it atm04:01
*** ivar-laz_ has joined #openstack-infra04:02
*** ivar-laz_ has quit IRC04:03
*** otter768 has quit IRC04:04
*** fedexo has joined #openstack-infra04:04
*** ivar-laz_ has joined #openstack-infra04:04
*** ivar-lazzaro has quit IRC04:04
tchaypothanks04:07
*** sarob has joined #openstack-infra04:11
*** sputnik13 has quit IRC04:12
*** Hal has joined #openstack-infra04:14
*** Hal is now known as Guest1759804:14
*** AlecTaylor has joined #openstack-infra04:18
AlecTaylorhi04:18
*** jyuso has joined #openstack-infra04:20
*** Guest17598 has quit IRC04:21
*** sputnik13 has joined #openstack-infra04:23
*** jyuso1 has quit IRC04:23
*** markvoelker has joined #openstack-infra04:24
openstackgerritJeremy Stanley proposed openstack-infra/project-config: Do a better job of identifying CentOS 6  https://review.openstack.org/17969404:25
*** markvoelker has quit IRC04:28
*** heyongli has quit IRC04:30
*** heyongli has joined #openstack-infra04:31
*** tjones1 has left #openstack-infra04:31
*** sks has joined #openstack-infra04:31
*** cryptarium has quit IRC04:31
*** yfried has quit IRC04:39
*** unicell has quit IRC04:41
*** unicell has joined #openstack-infra04:43
*** tjones1 has joined #openstack-infra04:51
*** abregman has joined #openstack-infra04:54
AlecTaylorPaaS cloud, considering: libcloud+consul+etcd+mesos+deis+drone, thoughts?04:56
*** woodster_ has joined #openstack-infra04:57
*** achanda has joined #openstack-infra05:02
*** chhavi has joined #openstack-infra05:04
*** btully has joined #openstack-infra05:04
chhaviHi all05:04
chhaviI am facing issues while commit the changes to openstack05:04
chhavifatal: The remote end hung up unexpectedly05:05
chhavichhavi@chhavi-KVM:~/chhavi/nova$ git push gerrit HEAD:refs/for/master/fix-bug-145103005:05
chhavifatal: ICLA contributor agreement requires current contact information.05:05
chhaviPlease review your contact information:05:05
chhavi  https://review.openstack.org/#/settings/contact05:05
chhavifatal: The remote end hung up unexpectedly05:05
jheskethchhavi: have you updated your contact information as per the instructions?05:21
chhaviyes, the issue just got resolved. there was issue while running git remote05:22
chhaviI am able to commit the change now05:22
jheskethexcellent05:24
*** markvoelker has joined #openstack-infra05:24
*** ivar-laz_ has quit IRC05:27
*** markvoelker has quit IRC05:29
*** ashleighfarnham has joined #openstack-infra05:34
tchaypodhellmann: .05:36
*** gokrokve has quit IRC05:37
*** chhavi has left #openstack-infra05:37
*** gokrokve has joined #openstack-infra05:37
*** ashleighfarnham has quit IRC05:39
*** gokrokve has quit IRC05:42
*** hdd has joined #openstack-infra05:42
*** BharatK has joined #openstack-infra05:43
*** sarob has quit IRC05:43
tchaypowell done me.05:49
*** ibiris_away is now known as ibiris05:50
*** salv-orlando has joined #openstack-infra05:53
*** Somay has quit IRC05:55
*** Somay has joined #openstack-infra05:56
*** salv-orlando has quit IRC05:56
*** Darkwan has quit IRC05:58
*** salv-orlando has joined #openstack-infra05:59
*** salv-orlando has quit IRC05:59
*** tnovacik has joined #openstack-infra06:03
*** otter768 has joined #openstack-infra06:05
*** AlecTaylor has quit IRC06:05
*** gokrokve has joined #openstack-infra06:08
*** gokrokve has quit IRC06:09
*** gokrokve has joined #openstack-infra06:09
*** otter768 has quit IRC06:10
*** salv-orlando has joined #openstack-infra06:11
*** deepakcs has joined #openstack-infra06:14
*** salv-orlando has quit IRC06:15
*** mrunge has joined #openstack-infra06:16
*** vlaza has joined #openstack-infra06:18
*** sdake_ has joined #openstack-infra06:22
*** fedexo has quit IRC06:22
openstackgerritAndreas Jaeger proposed openstack/requirements: Rename keystoneclient-federation to saml2  https://review.openstack.org/17970606:23
*** sergsh has quit IRC06:25
*** sergsh_ has quit IRC06:25
*** yfried has joined #openstack-infra06:25
*** sdake has quit IRC06:25
*** sdake_ is now known as sdake06:35
*** yfried is now known as yfried|afk06:36
*** yfried|afk is now known as yfried06:37
*** dimsum__ has joined #openstack-infra06:42
*** ibiris is now known as ibiris_away06:43
*** xylan_kong has left #openstack-infra06:44
*** salv-orl_ has joined #openstack-infra06:45
*** ibiris_away is now known as ibiris06:47
*** dimsum__ has quit IRC06:47
*** jcoufal has joined #openstack-infra06:55
*** sushilkm has left #openstack-infra06:55
ttxlifeless: doing it now06:58
*** hdd has quit IRC06:59
*** hdd has joined #openstack-infra07:00
ttxmtreinish, morganfainberg: stable-maint-core owns $PROJECT-stable-maint because we are supposed to check that proposed members know and will follow the stable policy07:02
ttxYou don't want each $PROJECT to come up with their own culture of what is an acceptable stable branch backport -- consistency is needed there07:03
*** Ala has joined #openstack-infra07:04
ttxSo project teams propose members and stable-maint-core lecture the proposed member with Stable Branch Policy, then we add them07:05
ttxReference: https://wiki.openstack.org/wiki/StableBranch#Project-specific_teams07:06
*** scheuran has joined #openstack-infra07:09
lifelessttx: thanks07:12
*** mrmartin has joined #openstack-infra07:12
*** yamahata has quit IRC07:14
*** sdake has quit IRC07:21
*** doude has joined #openstack-infra07:21
openstackgerritSteve Kowalik proposed openstack-infra/infra-specs: Add a specification about building packages  https://review.openstack.org/17971307:22
*** achanda has quit IRC07:23
*** ashleighfarnham has joined #openstack-infra07:24
openstackgerritSebastian Kalinowski proposed openstack-infra/project-config: Remove old Fuel plugins  https://review.openstack.org/17971407:24
*** btully has quit IRC07:26
*** markvoelker has joined #openstack-infra07:26
*** achanda has joined #openstack-infra07:26
*** ashleighfarnham has quit IRC07:28
*** ildikov has joined #openstack-infra07:29
*** markvoelker has quit IRC07:30
*** Kennan has quit IRC07:33
*** Kennan has joined #openstack-infra07:34
*** markus_z has joined #openstack-infra07:36
*** dizquierdo has joined #openstack-infra07:36
*** achanda has quit IRC07:37
*** chlong has quit IRC07:37
*** mpavone has joined #openstack-infra07:40
*** woodster_ has quit IRC07:40
*** Hal has joined #openstack-infra07:43
*** Hal is now known as Guest3699807:44
*** swat30 has quit IRC07:49
*** begal-sop_ has quit IRC07:50
*** funzo_ has joined #openstack-infra07:50
*** aswadr has joined #openstack-infra07:50
*** arxcruz has joined #openstack-infra07:50
*** funzo has quit IRC07:50
*** swat30_ has joined #openstack-infra07:50
*** swat30_ is now known as swat3007:51
*** begal-sop__ has joined #openstack-infra07:51
*** tnovacik has quit IRC07:52
openstackgerritJan Klare proposed openstack-infra/project-config: added chef-rake-jobs for openstack-chef-repo  https://review.openstack.org/17972107:56
openstackgerritFabien Boucher proposed openstack-infra/infra-specs: Specification proposal about system-config testing using containers  https://review.openstack.org/17283307:56
*** hashar has joined #openstack-infra07:56
*** hashar_ has joined #openstack-infra07:57
*** jistr has joined #openstack-infra07:59
*** hashar has quit IRC08:02
*** AJaeger has joined #openstack-infra08:04
*** otter768 has joined #openstack-infra08:05
GheRiver1hi all08:08
*** GheRiver1 is now known as GheRivero08:08
*** pblaho has joined #openstack-infra08:08
*** ildikov has quit IRC08:10
*** otter768 has quit IRC08:10
*** fhubik has joined #openstack-infra08:13
*** sushilkm has joined #openstack-infra08:14
*** sushilkm has left #openstack-infra08:14
*** vlaza has quit IRC08:17
*** vlaza1 has joined #openstack-infra08:17
*** ajo has joined #openstack-infra08:26
*** ildikov has joined #openstack-infra08:26
*** markvoelker has joined #openstack-infra08:27
*** sergsh_ has joined #openstack-infra08:27
*** vlaza has joined #openstack-infra08:29
*** vlaza1 has quit IRC08:29
*** Hal has joined #openstack-infra08:30
*** SumitNaiksatam has quit IRC08:30
*** Hal is now known as Guest6497108:31
*** markvoelker has quit IRC08:31
*** fhubik is now known as fhubik_afk08:31
*** sergsh has joined #openstack-infra08:31
openstackgerritHuang Rui proposed openstack-infra/project-config: enable python-jobs for networking-zvm  https://review.openstack.org/17973008:32
*** Guest36998 has quit IRC08:33
*** tnovacik has joined #openstack-infra08:33
*** SumitNaiksatam has joined #openstack-infra08:34
*** Guest64971 has quit IRC08:35
*** mrunge has quit IRC08:36
*** vlaza has quit IRC08:38
*** vlaza has joined #openstack-infra08:38
openstackgerritMerged openstack-infra/project-config: Puppet/Beaker: drop puppet-{gnocchi,tuskar}  https://review.openstack.org/17960108:39
*** Somay has quit IRC08:41
*** Somay has joined #openstack-infra08:42
*** alexpilotti has quit IRC08:45
*** ociuhandu has joined #openstack-infra08:48
*** vlaza has quit IRC08:50
*** vlaza has joined #openstack-infra08:50
*** alexpilotti has joined #openstack-infra08:54
openstackgerritSirushti Murugesan proposed openstack-infra/project-config: Add grenade jobs for Heat  https://review.openstack.org/17835208:56
openstackgerritHuang Rui proposed openstack-infra/project-config: Move zvm-driver to stackforge-attic  https://review.openstack.org/17973809:02
*** e0ne has joined #openstack-infra09:04
*** hashar_ is now known as hasharOut09:06
*** yfried is now known as yfried|afk09:10
*** e0ne has quit IRC09:10
*** asrangne has joined #openstack-infra09:11
*** salv-orlando has joined #openstack-infra09:13
*** salv-orl_ has quit IRC09:13
*** ashleighfarnham has joined #openstack-infra09:13
*** aswadr has quit IRC09:13
*** hdd has quit IRC09:16
*** _dguerri is now known as dguerri09:17
*** ashleighfarnham has quit IRC09:17
*** hdd has joined #openstack-infra09:18
*** sergsh has quit IRC09:22
*** sergsh_ has quit IRC09:22
*** fhubik_afk is now known as fhubik09:23
*** markvoelker has joined #openstack-infra09:27
*** jyuso has quit IRC09:27
*** dguerri is now known as _dguerri09:28
*** mpaolino has joined #openstack-infra09:29
*** vlaza has left #openstack-infra09:30
*** markvoelker has quit IRC09:32
openstackgerritGiulio Fidente proposed openstack-infra/tripleo-ci: Bump up delorean pinning  https://review.openstack.org/17717609:32
openstackgerritJulian Taylor proposed openstack-infra/jenkins-job-builder: add support for measurement plots plugin to junit publisher  https://review.openstack.org/17974309:33
*** Somay_ has joined #openstack-infra09:34
*** Somay has quit IRC09:35
*** sergsh has joined #openstack-infra09:35
*** sergsh_ has joined #openstack-infra09:35
*** mpaolino has quit IRC09:35
*** abregman is now known as abregmansd09:38
*** abregmansd is now known as abregman|food09:38
*** abregman|food is now known as abregman|food-ti09:39
*** abregman|food-ti is now known as abregman|launch09:39
*** abregman|launch is now known as abregman|lunch09:39
*** _dguerri is now known as dguerri09:42
*** salv-orlando has quit IRC09:43
*** ociuhandu has quit IRC09:44
*** salv-orl_ has joined #openstack-infra09:46
*** mrunge has joined #openstack-infra09:46
*** fhubik is now known as fhubik_afk09:47
openstackgerritGiulio Fidente proposed openstack-infra/tripleo-ci: Bump up delorean pinning  https://review.openstack.org/17717609:49
*** luqas has joined #openstack-infra09:49
*** abregman|lunch is now known as abregman09:49
*** dguerri is now known as _dguerri09:50
*** fhubik_afk is now known as fhubik09:50
*** salv-orlando has joined #openstack-infra09:50
openstackgerritVictor Stinner proposed openstack/requirements: Add mysqlclient dependency  https://review.openstack.org/17974509:52
*** salv-orl_ has quit IRC09:53
*** dimsum__ has joined #openstack-infra09:53
openstackgerritSirushti Murugesan proposed openstack-infra/project-config: Add grenade jobs for Heat  https://review.openstack.org/17835209:55
*** devvesa has joined #openstack-infra09:55
*** Hal has joined #openstack-infra09:55
*** Hal is now known as Guest8488309:56
*** zhiwei has quit IRC10:00
*** rlandy has joined #openstack-infra10:04
*** dguitarbite has quit IRC10:05
*** otter768 has joined #openstack-infra10:06
*** abregman is now known as abregman|lunch10:08
*** vlaza has joined #openstack-infra10:09
*** otter768 has quit IRC10:11
lifelessa few hours late, but just FYI, new releases of testscenarios and fixtures were cut earlier today10:14
lifelessnot expecting any fallout,but just in case....10:14
*** armax has joined #openstack-infra10:15
*** yfried|afk is now known as yfried10:16
*** pc_m has joined #openstack-infra10:17
*** dguitarbite has joined #openstack-infra10:17
*** Somay_ has quit IRC10:22
*** macjack has joined #openstack-infra10:23
*** fhubik is now known as fhubik_afk10:26
*** _dguerri is now known as dguerri10:27
*** tnovacik has quit IRC10:28
*** sushilkm has joined #openstack-infra10:28
*** sushilkm has left #openstack-infra10:28
*** fhubik_afk is now known as fhubik10:28
*** salv-orl_ has joined #openstack-infra10:31
*** e0ne has joined #openstack-infra10:31
*** salv-orlando has quit IRC10:34
*** weshay has joined #openstack-infra10:36
*** mrmartin has quit IRC10:40
*** samueldmq has joined #openstack-infra10:40
samueldmqmorning :)10:40
*** asrangne has quit IRC10:43
*** dguerri is now known as _dguerri10:43
*** armax has quit IRC10:44
*** salv-orl_ has quit IRC10:46
*** salv-orlando has joined #openstack-infra10:47
openstackgerritJulian Taylor proposed openstack-infra/jenkins-job-builder: add support for measurement plots plugin to junit publisher  https://review.openstack.org/17974310:49
*** ociuhandu has joined #openstack-infra10:49
*** fhubik is now known as fhubik_afk10:50
*** fhubik_afk is now known as fhubik10:52
*** tjones2 has joined #openstack-infra10:54
*** tjones1 has quit IRC10:57
*** luqas has quit IRC10:57
*** mrmartin has joined #openstack-infra10:57
*** tnovacik has joined #openstack-infra10:59
*** ashleighfarnham has joined #openstack-infra11:01
openstackgerritMichal Kovarik proposed openstack-infra/jenkins-job-builder: Adding support for Beaker Builder Plugin  https://review.openstack.org/17975611:02
*** claudiub has joined #openstack-infra11:03
*** baoli has joined #openstack-infra11:04
*** baoli has quit IRC11:05
*** abregman|lunch is now known as abregman11:06
*** ashleighfarnham has quit IRC11:06
sdaguels11:07
sdaguedoh11:07
openstackgerritSirushti Murugesan proposed openstack-infra/project-config: Add grenade jobs for Heat  https://review.openstack.org/17835211:09
*** vlaza has left #openstack-infra11:09
samueldmqsdague, echo "hehe"; ./good-morning.sh11:11
AJaegermorning, sdague! The requirements post job propose-requirements-update fails due to a project rename - https://review.openstack.org/#/c/179706/ should fix it...11:11
samueldmqAJaeger, and this also should be renamed in the  openstack/requirements .. which was failing to find python-keystoneclient-federation11:14
samueldmqright ?11:14
AJaegersamueldmq, yep11:14
samueldmqAJaeger, nice11:15
*** Longgeek has joined #openstack-infra11:15
AJaegersamueldmq, https://jenkins.openstack.org/job/propose-requirements-updates/539/console11:15
samueldmqAJaeger, cool .. I just added a reference to your patch on the previous one (which renamed the project)11:17
AJaegerthanks, samueldmq.11:19
sdagueAJaeger: could you make propose-requirements-update more robust as well?11:19
sdagueit seems like 1 project not existing shouldn't make it go fatal11:19
*** luqas has joined #openstack-infra11:19
AJaegersdague, would be a nice idea. I doubt I have time quickly for this, so if anybody else wants to take it...11:20
*** salv-orlando has quit IRC11:23
openstackgerritGal Sagie proposed openstack-infra/project-config: Enable a check job for stackforge/dragonflow  https://review.openstack.org/17515211:23
samueldmqsdague, AJaeger how would be that change ? I'd probably be able to help, but I am not familiar with this code11:26
samueldmqany entry point to the code ?11:26
AJaegersamueldmq, jenkins/scripts/propose_update.sh in project-config11:26
*** ildikov has quit IRC11:27
samueldmqAJaeger, sdague what should be the expected behavior when it cannot fetch a project ?11:27
AJaegerhttp://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/propose_update.sh#n43 is the line that currently breaks it11:28
*** eharney has quit IRC11:28
AJaegersamueldmq, setting ALL_SUCCESS to 1 and moving on to the next project11:28
AJaegerTogether with some error message to look out for11:29
*** luqas has quit IRC11:29
sdaguesamueldmq: yeh, so it should accumulate errors somewhere to report at the end, but definitely keep looping through all projects11:30
*** wznoinsk has quit IRC11:30
*** marcusvrn has joined #openstack-infra11:30
pc_mHi, I'm getting a failure in gate-neutron-vpnaas-python27 that is quite odd. It passes all the tests and then fails in the script after. Anyone hear of this?11:30
pc_mRef:  http://logs.openstack.org/15/168115/32/check/gate-neutron-vpnaas-python27/5d9ecf1/console.html#_2015-05-02_16_28_39_90611:30
AJaegersdague, there are more projects in requirements that were renamed...11:30
* AJaeger does a second patch...11:31
sdagueAJaeger: cool, thanks11:31
openstackgerritMichal Kovarik proposed openstack-infra/jenkins-job-builder: Adding support for Beaker Builder Plugin  https://review.openstack.org/17975611:31
*** luqas has joined #openstack-infra11:31
samueldmqAJaeger, sdague should be something similar to what we do when a project fails to sync .. http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/propose_update.sh#n11411:32
samueldmqright ?11:33
AJaegeryes11:33
samueldmqk got it11:33
*** deepakcs has quit IRC11:36
openstackgerritAndreas Jaeger proposed openstack/requirements: Follow project renames  https://review.openstack.org/17970611:38
AJaegersdague, samueldmq : Updated my patch ^11:38
openstackgerritAndreas Jaeger proposed openstack/requirements: Follow project renames  https://review.openstack.org/17970611:39
AJaegerand sorted projects again ^11:39
*** mase_x200 has joined #openstack-infra11:40
*** dimsum__ is now known as dims11:41
samueldmqAJaeger, and congress/rally changed from stackforge to opesntack, right?11:41
AJaegersamueldmq, they did11:41
samueldmqAJaeger, k looks good, thanks11:42
AJaegersamueldmq, let me enhance the commit message...11:42
*** ildikov has joined #openstack-infra11:43
samueldmqAJaeger, oh, already +1'ed, the commit message already says rally and congress were renamed11:43
samueldmqAJaeger, but feel free to enhance it as you want :)11:43
openstackgerritAndreas Jaeger proposed openstack/requirements: Follow project renames  https://review.openstack.org/17970611:43
AJaegersamueldmq, if you ask, somebody else wonders as well - so let's be clear here...11:43
AJaegersamueldmq, thanks for the reviews11:43
samueldmqAJaeger, you added links to the commits on each project, nice11:44
*** claudiub_ has joined #openstack-infra11:44
samueldmqAJaeger, np, just trying to get things moving, my reviews aren't still that valuable in #infra though11:45
AJaegersamueldmq, every review is important - and you learn as well with them ;)11:45
*** claudiub has quit IRC11:45
samueldmqAJaeger, :)11:46
*** ldnunes has joined #openstack-infra11:47
*** esker has quit IRC11:47
*** markvoelker has joined #openstack-infra11:50
*** alexpilotti has quit IRC11:51
*** luqas has quit IRC11:51
*** dkranz has joined #openstack-infra11:52
openstackgerritSamuel de Medeiros Queiroz proposed openstack-infra/project-config: Don't short circuit when projects fails to clone  https://review.openstack.org/17976011:53
samueldmqAJaeger, sdague ^11:53
openstackgerritSamuel de Medeiros Queiroz proposed openstack-infra/project-config: Don't short circuit when projects fail to clone  https://review.openstack.org/17976011:53
samueldmqwell, just fixed the commit message11:53
fungipc_m: i've commented on your https://review.openstack.org/168115 change pointing out where you're introducing that issue11:54
pc_mfungi: Thanks. I was wondering if it was the copy of rootwrap.conf. will look at your comment.11:55
pc_mfungi: Ah that is what you saw. Thanks11:56
AJaegerthanks, samueldmq!11:56
pc_mfungi: I'm not too familiar with tox.ini. Can I have a command for a target, depend on another target's command?11:57
claudiub_sdague: hello. Could I have your +2 on a 1 line commit? It already has a +2. https://review.openstack.org/#/c/178993/ Thanks!11:57
samueldmqAJaeger, np, anytime :)11:57
*** rfolco has joined #openstack-infra11:58
pc_mfungi: IOW, have the command on that and the previous two lines in a "functional" target, and then have the dsvm-functional and dsvm-functional-sswan targets use that command?12:00
openstackgerritSean Dague proposed openstack-infra/system-config: use optimized url for zuul status  https://review.openstack.org/17093512:00
openstackgerritSean Dague proposed openstack-infra/system-config: add dbg function to hideci  https://review.openstack.org/17957312:00
openstackgerritSean Dague proposed openstack-infra/system-config: add jquery visibility  https://review.openstack.org/17936212:00
openstackgerritSean Dague proposed openstack-infra/system-config: create proxy for testing hideci  https://review.openstack.org/17936012:00
openstackgerritSean Dague proposed openstack-infra/system-config: Revert "Revert zuul status in gerrit"  https://review.openstack.org/17936112:00
pc_mfungi: like... command = {|functional|command}?12:00
fungipc_m: i don't think there's a way to inherit and then extend those variables from one env to the next, though you might check the documentation for tox. you'll probably just need to have that command appear explicitly in each env where it's needed12:03
pc_mfungi: ok. will do. thanks12:04
fungipc_m: oh, though it is possible to add an env to the commands list of another env12:05
fungifor example nova (and a lot of other projects) have a pep8 env that calls the flake8 env12:05
openstackgerritMonty Taylor proposed openstack-infra/shade: Convert node_set_provision_state to task  https://review.openstack.org/17798712:05
openstackgerritMonty Taylor proposed openstack-infra/shade: Update recent Ironic exceptions  https://review.openstack.org/17798612:05
openstackgerritMonty Taylor proposed openstack-infra/shade: Enhance error message in update_machine  https://review.openstack.org/17798512:05
openstackgerritMonty Taylor proposed openstack-infra/shade: Enhance the OperatorCloud constructor  https://review.openstack.org/17964712:05
AJaegerfungi, we have an idle CI system with a IMHO very slow throughput. project-config changes need over an hour to get through, waiting very long for gate-infra-puppet-apply-precise - if I interpret status.openstack.org/zuul correctly12:05
fungipc_m: wait, i just brained wrong. the pep8 env calls the flake8 command line utility. flake8 itself magically reads tox.ini too and looks for a "flake8" env containing its configuration12:06
fungipc_m: so forget what i said about calling an env from another env12:06
*** luqas has joined #openstack-infra12:07
fungiAJaeger: yes, i think that jenkins-gearman bug clarkb and zaro were looking at last week is still an issue. i'll clear out more stale ready nodes for now and that should speed things back up12:07
AJaegerfungi, thanks! That jenkins-gearman bug is becoming famous ;/12:08
jheskethsdague: oh cool, nice stuff (re zuul status patches)12:08
sdaguejhesketh: yeh, so the jquery visibility bit still needs that file to actually get deployed, I don't know where best to do that12:10
sdaguejhesketh: do you?12:10
*** mrmartin has quit IRC12:10
openstackgerritSamuel de Medeiros Queiroz proposed openstack-infra/project-config: Don't short circuit when projects fails to clone  https://review.openstack.org/17976012:10
sdaguealso, the targettted status calls aren't working still12:11
jheskethsdague: I just started reviewing at the top (the web server), so not sure which file that is sorry?12:11
samueldmqfungi, ^ thanks for your review12:11
sdaguehttps://review.openstack.org/17936212:11
jheskethsdague: oh yeah, I know where that goes... gimme a sec12:12
*** hdd has quit IRC12:12
openstackgerritMerged openstack-infra/project-config: Adds pypi-jobs to networking-hyperv  https://review.openstack.org/17899312:12
jheskethsdague: it goes in system-config/modules/openstack_project/files/gerrit/GerritSiteHeader.html12:14
sdaguejhesketh: ok, we also deploy it for zuul, so I wasn't sure if there was a common deploy point that we wanted12:14
jheskethsdague: do it similar to the hideci.js in that file, the important part is the notify => Exec['reload_gerrit_header'] because of the way gerrit signs static files12:14
sdagueright, my only concern was copying the file a second time into a tree12:15
sdagueand if that's what people want or not12:15
*** luqas has quit IRC12:15
jheskethsdague: oh right, I see.. Well we could link to the file on zuul.o.o but that's not great12:15
fungisdague: sdague what do you mean by the targeted status calls aren't working still? http://zuul.openstack.org/status/change/179754,2 returns json for me right now12:15
sdaguefungi: oh... but not over https12:16
fungisdague: oh! right, we probably need to duplicate that block for https as well. i completely forgot :/12:16
sdagueok, no prob, let me go look for that12:17
jheskethsdague: but the actual file isn't copied into the tree, so something like this: http://git.openstack.org/cgit/openstack-infra/puppet-zuul/tree/manifests/init.pp#n24912:17
*** doug-fish has left #openstack-infra12:17
fungisdague: it's later in the same file you modified previously12:17
fungisdague: one vhost template providing http and optionally https12:18
jheskethsdague: but the notify part I mentioned is still important and the gerrit.pp is the place12:18
sdaguejhesketh: any chance you want to write that puppet? I need to get my presentations for summit sorted early this week, so I'm going to loose this thread for a bit12:18
*** shakamunyi has quit IRC12:18
jheskethsdague: also, looks like I had a paste failure... I meant to say it'd go into system-config/modules/openstack_project/manifests/gerrit.pp12:18
jheskethsdague: heh, I was just about to offer12:18
jheskethhappy to12:18
*** bswartz has quit IRC12:19
*** aysyd has joined #openstack-infra12:19
*** ashleighfarnham has joined #openstack-infra12:19
openstackgerritSean Dague proposed openstack-infra/puppet-zuul: also add targeted status redirects to https  https://review.openstack.org/17976412:20
sdaguejhesketh: thanks!12:20
sdaguefungi: ok, that's hopefully the other fix12:20
*** gordc has joined #openstack-infra12:22
mordredjhesketh: ooh - you're still up12:24
*** salv-orlando has joined #openstack-infra12:24
jheskethmordred: corright12:24
*** ashleighfarnham has quit IRC12:24
mordredjhesketh: I think I took care of your notes on https://review.openstack.org/17798512:24
jheskethmordred: looking12:24
mordredjhesketh: and good points, btw, thanks12:24
openstackgerritJoshua Hesketh proposed openstack-infra/system-config: Serve up jquery-visibility.min.js for gerrit  https://review.openstack.org/17976512:26
jheskethsdague: ^12:27
jheskethmordred: cool, lgtm12:27
mordredjhesketh: yay!12:27
*** fhubik has quit IRC12:28
mordredjhesketh: wow, we weren't jquerying already?12:28
jheskethmordred: we weren't jquery-visibilitying12:28
yolandamordred, i'm having issues with https://git.openstack.org/cgit/openstack-infra/nodepool/commit/?id=8b87890b978a54be10eff272f7bb6d9955e2d99912:29
*** fhubik has joined #openstack-infra12:29
yolandathe change for nodepool and proxy12:29
yolandayou are not using that upstream, right?12:29
*** luqas has joined #openstack-infra12:29
mordredyolanda: we are not12:30
yolandai'd say that this change doesn't work12:30
yolandalet me paste you the error12:30
yolandamordred, http://paste.openstack.org/show/214843/12:30
mordredyolanda: oh! we are12:30
yolandathat code is not raising a ProxyError, but a Timeout12:30
yolandait happens when i do a manual image-update12:31
mordredyolanda: but we're not behind a proxy - so we've never seen that condition ourselves12:31
yolandaalso, according to the logs, it even doesn't reach the provider_manager.py runTask method12:31
mordredhrm12:31
*** hdd has joined #openstack-infra12:31
*** dizquierdo has quit IRC12:34
*** luqas has quit IRC12:34
*** alexpilotti has joined #openstack-infra12:35
*** woodster_ has joined #openstack-infra12:36
*** salv-orlando has quit IRC12:37
yolandamordred, so i guess the intention of that change is to override runtask, and add the proxy settings there, and call provider_manager.runTask instead of task_manage.runTask ?12:38
*** _dguerri is now known as dguerri12:38
openstackgerritMerged openstack-infra/project-config: Don't short circuit when projects fails to clone  https://review.openstack.org/17976012:38
*** dprince has joined #openstack-infra12:39
mordredyolanda: yah - I think so12:40
yolandalooking at the code i'd say this runtask is not even reached, as i cannot see traces of errors there12:40
yolandai even amended the code to accept Timeout exception, but same problem12:41
mordredyolanda: weird12:43
yolandawe have all of our images older than a day due to that error, and i'm unable to update manually, i'm testing more12:43
*** radez_g0n3 is now known as radez12:44
*** dguerri is now known as _dguerri12:45
*** kgiusti has joined #openstack-infra12:46
openstackgerritMonty Taylor proposed openstack-infra/shade: Convert node_set_provision_state to task  https://review.openstack.org/17798712:48
openstackgerritMonty Taylor proposed openstack-infra/shade: Update recent Ironic exceptions  https://review.openstack.org/17798612:48
openstackgerritMonty Taylor proposed openstack-infra/shade: Enhance error message in update_machine  https://review.openstack.org/17798512:48
openstackgerritMonty Taylor proposed openstack-infra/shade: Enhance the OperatorCloud constructor  https://review.openstack.org/17964712:48
*** dizquierdo has joined #openstack-infra12:48
*** dprince has quit IRC12:48
yolandamm, mordred, i think i got it12:51
yolandacommand lines aren't using the thread interface , right?12:51
*** mikeit has joined #openstack-infra12:53
mordredyolanda: oh! you are right12:55
*** links has joined #openstack-infra12:56
*** fhubik has quit IRC12:56
*** fhubik has joined #openstack-infra12:57
*** pc_m has quit IRC12:57
*** sdake has joined #openstack-infra12:57
anteayareed: I'm glad you are having a faster experience with git, I find it odd that ipv6 would be playing a role12:57
*** macjack has left #openstack-infra12:57
yolandamordred, actually where it fails is in createservertask, and that's embedded inside a submittask method12:58
*** jaypipes has joined #openstack-infra12:58
mordredanteaya: that's a fascinating first sentence of the day12:58
anteayaI aim to facinate12:59
anteayamy job for the day is complete12:59
anteayathank you12:59
* mordred hands anteaya a japanese eggplant and a bowl of salsa12:59
anteayawell thank you12:59
fungimmmeggplantsalsa13:01
*** bknudson has quit IRC13:01
*** BobH has joined #openstack-infra13:02
*** yfried is now known as yfried|afk13:02
anteayaI never knew it could be good for breakfast13:03
yolandamordred, ok, i think i got it, needed to add timeout exception as well to task_manager code. Will test it and send a patch upstream13:06
*** _elmiko is now known as elmiko13:07
*** signed8bit has joined #openstack-infra13:09
*** signed8bit has quit IRC13:09
*** miqui has joined #openstack-infra13:09
jheskethsdague: I wonder with the live zuul status stuff if we should have a toggle to turn it on/off. If we merge this and find the optimisations aren't enough, turning it off again will require a number of reverts13:10
*** jpeeler has joined #openstack-infra13:10
jheskethif we have a toggle switch it'll make it easier13:10
*** mrmartin has joined #openstack-infra13:10
jheskethand also mean that if it works for the next 5 months but then zuul is under stupid amounts of load, we can change it back then too while we look at further optimisations13:11
sdaguejhesketh: yes, I was thinking putting it behind an experimental flag would be good13:11
sdaguewhich we could default off, then turn on inline, like the debug13:11
sdagueso it would be easy to turn off13:11
jheskethsdague: good idea13:11
sdagueI think the only issue is we might need to change the caching model on review.openstack.org13:13
*** hasharOut is now known as hashar13:13
sdagueso that it expires hideci more often13:13
sdagueso that a change like that would deploy to people more quickly13:13
jheskethsdague: what is it currently?13:13
jheskethdeployment is always going to be slow if we have to wait for jenkins+puppet etc13:14
*** BobH has quit IRC13:14
sdagueyeh, that's not the concern, but js resources from the browser seem to have long lifetimes13:14
*** dims has quit IRC13:14
sdagueso we might need an explicit expiration time to make browsers come back13:14
*** dustins has joined #openstack-infra13:14
*** dims has joined #openstack-infra13:15
*** erlon_awaY has joined #openstack-infra13:15
jheskethsure, that's probably a good idea13:15
mordredjeblair_: when you return -I'm having an issue with gertty where it seems to have finished all of its syncing, and then I go to a change, and the change is not present, so it goes and figures out that it needs to sync new things13:15
mordredjeblair_: it's happening frequently - I would have expected all of the things to sync at some point, otherwise prep-for-being-offline becomes unhappy13:16
fungimordred: is it a change for a subscribed project?13:16
mordredfungi: yes13:16
fungiodd13:16
*** alexpilotti has quit IRC13:16
*** bcourt has joined #openstack-infra13:16
mordredfungi, jeblair_: and in fact, like, every change I go to review in devstack-gate right now is doing this.13:17
*** hdd has quit IRC13:17
*** yfried|afk is now known as yfried13:18
*** tiswanso has joined #openstack-infra13:18
*** hdd has joined #openstack-infra13:18
mrmartinmorning13:19
fungimordred: hrm, running tip of master i am not having that problem on devstack-gate changes13:20
*** jtriley has joined #openstack-infra13:20
* mordred is the SpamapS of gertty13:21
*** baoli has joined #openstack-infra13:21
mordredwow. I JUST DID a ctrl-r on the devstack-gate screen, it finished, and tried to go to a review and I got the commits-not-present message13:22
Shrewsmordred: give up. monday has won13:22
fungiproblem with your local git cache?13:22
mordredfungi: my local git cache is my source tree13:22
fungiyeah, mine too. just didn't know if there might be something funky with your devtsack-gate clone in there13:23
*** bswartz has joined #openstack-infra13:23
mordredfungi: and I'm running tip of master too13:23
mordredfungi: well, it was doing it on other repos too13:23
mordredfungi: this is just the one I'm on in this morning's reviews13:23
*** dizquierdo has quit IRC13:24
*** jcoufal_ has joined #openstack-infra13:24
* mordred gives Shrews a case of the mondays13:24
openstackgerritMerged openstack-infra/release-tools: Add a --stable-series argument to release_notes.py  https://review.openstack.org/17819413:24
*** mestery_ is now known as mestery13:25
openstackgerritMerged openstack-infra/release-tools: Add option to format release notes for email  https://review.openstack.org/17819513:26
*** amitgandhinz has joined #openstack-infra13:26
*** jcoufal has quit IRC13:27
*** erlon_awaY is now known as erlon13:27
mordredfungi, sdague: 167412 - looks safe to merge now (this is temporarily enabling trove plugin stuff in d-g) - any gotchas you're aware of?13:28
*** peristeri has joined #openstack-infra13:28
openstackgerritMerged openstack-infra/release-tools: Update rccut and rcdelivery to match Kilo process  https://review.openstack.org/17720213:28
openstackgerritJeremy Stanley proposed stackforge/gertty: Be more verbose on non-tagged versions  https://review.openstack.org/17977613:28
sdaguemordred: yeh, it should be13:29
fungilgtm, but didn't approve in case you were still reviewing13:29
sdagueit's basically a hack to deal with the fact that project-config can't be pre gated13:29
fungioh, though its dependency chain is still in need of work13:30
fungibut yeah, that one should be able to merge, it's just changes depending on it which need some updating13:30
mordredsdague: what does ;& mean in a base case statement?13:30
*** sdake_ has joined #openstack-infra13:31
sdaguemordred: fall through13:31
mordreddoesn't ;; also mean that?13:31
sdagueno, ;; is break13:31
mordredwell then13:31
sdagueI thought at least13:31
sdagueI had some sample code to ensure this did the right thing13:32
*** dprince has joined #openstack-infra13:32
*** pabelanger has joined #openstack-infra13:32
*** pabelanger has joined #openstack-infra13:32
mordredit's fine - I just didn't know that syntax13:32
*** craigbr has joined #openstack-infra13:32
sdagueyeh, is a bash 4 thing apparently13:33
sdaguebecause default shell case doesn't have fall through13:33
sdaguewhich... is weird13:33
sdaguebecause that's the point of case in my mind13:33
*** bknudson has joined #openstack-infra13:33
*** sdake has quit IRC13:34
openstackgerritSirushti Murugesan proposed openstack-infra/project-config: Add grenade jobs for Heat  https://review.openstack.org/17835213:35
*** coolsvap is now known as coolsvap|afk13:37
fungidims: you're abandoning wip changes?13:38
mordredin which project(s)?13:39
mordredI would not like my WIP changes abandoned13:39
fungioslo.*13:39
mordredk13:39
mordredas long as it's not infra13:39
fungii'm restoring one that's still waiting on an ubuntu bug fix13:39
dimsfungi: which one? (still tuning the script)13:40
*** luqas has joined #openstack-infra13:40
fungidims: 12022813:41
fungiunfortunately bugs in ubuntu lts take a while to get fixed13:42
dimsthanks fungi!13:42
openstackgerritMerged openstack-infra/shade: Enhance error message in update_machine  https://review.openstack.org/17798513:42
openstackgerritMerged openstack-infra/shade: Update recent Ironic exceptions  https://review.openstack.org/17798613:42
fungidims: you're welcome13:43
mordredsdague: could we do something clever to make a passthrough config to devstack? every time I approve a patch like 179662 it makes me think we're doing something wrong13:43
sdaguemordred: 179662 ?13:43
*** prad has joined #openstack-infra13:43
sdagueoh, yes, we already have that13:43
openstackgerritMerged openstack-dev/pbr: Allow overwriting sphinx builder from command line  https://review.openstack.org/11747813:43
*** jeblair_ is now known as jeblair13:44
*** superdan is now known as dansmith13:44
mordredsdague: yeah? what is it?13:44
sdaguegrep for enable_plugin in project-config13:44
sdagueDEVSTACK_LOCAL_CONFIG13:44
sdaguethat's it13:44
mrmartinfungi: the refactoring for puppet-askbot staging is almost ready, I just found some issues with askbot-devel. so it have issues with ubuntu precise cloudimg, so I made tests with trusty, and it works well13:45
openstackgerritYanis Guenane proposed openstack-infra/project-config: Add support for backport-potential commit flag  https://review.openstack.org/17584913:45
mrmartinso, what is the proper way of ask.o.o upgrade from precise to trusty?13:45
mordredsdague: col13:45
mordredcool13:45
mordredwe shoudl start using that more13:45
sdaguemordred: yes13:46
*** mriedem has joined #openstack-infra13:46
*** dboik has joined #openstack-infra13:46
sdagueit gets a little janky with multi lines in the project config files, we probably want to come up with a better way to make that work13:46
fungimrmartin: we build a new server running trusty and migrate from the current one to the new one (similar to how we migrated from the non-managed server to the current managed one)13:47
mrmartinok great13:47
sdaguebut the intent was about 1/2 of d-g vars should be able to go away13:47
sdaguejust never got there13:47
mordredsdague: ++13:47
mordredsdague: well, I'm going to start -1ing new variables13:47
mrmartincan we disable the puppet on the precise ask.o.o?13:47
*** lascii is now known as alaski13:47
*** luqas has quit IRC13:48
*** julim has joined #openstack-infra13:48
mrmartinThis new patch contains some breaking changes, removes the slotX directory structure, and consume askbot-devel directly from github instead of pip13:48
mordredfungi: speaking of that ^^ https://review.openstack.org/#/c/179581/13:48
mordredfungi: I'd argue we should stop spinning up new precise servers unless we really mean it13:49
mrmartinyeah, that's the case we like to upgrade the precise nodes to trusty13:50
mordredfungi: also, while I'm bothering you with things that do not matter (good morning): https://review.openstack.org/#/q/status:open+branch:master+topic:hound,n,z - and there is a server running at hound.openstack.org with those changes if you want to check it out13:50
fungimordred: yep, saw that. +2 but we should get a little more consensus. worth mentioning briefly in the meeting tomorrow perhaps13:50
fungimordred: on the default-to-trusty change i mean13:51
*** luqas has joined #openstack-infra13:51
mordredfungi: ++13:51
fungimrmartin: well, if we merge changes that won't work for the current production server, we're basically setting ourselves up to be unable to update that server further until we migrate... so we should be really sure we can arrange the migration maintenance in a short timeframe13:51
mrmartinok.13:52
samueldmqmordred, ping - just saw your review on 'Pass the flag for Identity v3 API only jobs'13:52
mordredsamueldmq: yah13:52
mrmartinbut what you think about temporarily commenting out the askbot related manifests there?13:52
samueldmqmordred, so I just need to put what I need in DEVSTACK_LOCAL_CONFIG in my project-config change13:52
samueldmqmordred, right ?13:52
mordredsamueldmq: yup!13:52
samueldmqmordred, that's happening in a lot of places13:53
mordredsamueldmq: no need to teach d-g about it at all13:53
*** emagana has quit IRC13:53
sdaguemordred: +113:53
samueldmqmordred, do we need to refactor all the other places we are already doing this ?13:53
fungimrmartin: if it's not too hard to adjust it so that it works on trusty for the ask-dev server but continues working on precise for the production ask server, that would make things a little easier to test and coordinate, but if that's a lot of extra work then i think just updating the module and migrating the server within a maintenance is probably okay13:53
samueldmqmordred, I can do this refactor if you want me too13:53
mordredsamueldmq: it would be nice - although as sdague mentions, if there are jobs taht need to set more than one thing, it's a little odd syntax-wise, so we might want to sort that13:54
mrmartinok.13:54
mordredsamueldmq: but probably all of the jobs that just set one thing that's just a passthrough and does not trigger additional logic in d-g are good candidates13:54
mrmartinwe have some open issues with askbot-devel which must be closed first, so when it works well, I'll come back with some plan13:54
samueldmqmordred, great I will do it later today13:54
samueldmqmordred, will ping you once I submit the patch, thanks13:54
openstackgerritClaudiu Belu proposed openstack-infra/project-config: Adds pypi-jobs to compute-hyperv  https://review.openstack.org/17978213:55
samueldmqmordred, yeah I will get those good candidates first, it should already be a nice cleanup13:55
fungimrmartin: great. i'll try to make sure i keep reviewing those a priority, but ping me when you have related changes i haven't noticed/reviewed just in case13:55
*** luqas has quit IRC13:55
mrmartinfungi, ok nothing urgent here13:55
mrmartinwe have only one large issue with google-auth but it still requires some work from askbot devs.13:56
*** chlong has joined #openstack-infra13:58
fungiokay, thanks for the update!14:01
*** btully has joined #openstack-infra14:02
*** yfried is now known as yfried|afk14:02
*** btully has quit IRC14:03
*** fhubik has quit IRC14:03
*** zz_naotok has quit IRC14:04
*** zz_naotok has joined #openstack-infra14:06
*** eharney has joined #openstack-infra14:08
*** ashleighfarnham has joined #openstack-infra14:08
*** otter768 has joined #openstack-infra14:08
*** chlong has quit IRC14:08
*** ayoung has joined #openstack-infra14:09
*** dguitarbite has quit IRC14:09
*** _dguerri is now known as dguerri14:12
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add query for nova unit test bug 1451389  https://review.openstack.org/17978814:13
openstackbug 1451389 in OpenStack Compute (nova) "Nova gate broke due to failed unit test" [Critical,In progress] https://launchpad.net/bugs/1451389 - Assigned to Sean Dague (sdague)14:13
*** otter768 has quit IRC14:13
*** dguerri is now known as _dguerri14:13
*** _dguerri is now known as dguerri14:13
*** ashleighfarnham has quit IRC14:13
*** e0ne has quit IRC14:14
*** sigmavirus24_awa is now known as sigmavirus2414:14
*** salv-orlando has joined #openstack-infra14:16
*** rossella_s has quit IRC14:16
*** rossella_s has joined #openstack-infra14:16
*** mrunge has quit IRC14:17
*** pc_m has joined #openstack-infra14:17
*** signed8bit has joined #openstack-infra14:19
*** yfried|afk is now known as yfried14:22
*** BharatK has quit IRC14:23
*** stevemar has joined #openstack-infra14:23
*** gsagie has joined #openstack-infra14:24
*** chlong has joined #openstack-infra14:26
openstackgerritMerged openstack-infra/elastic-recheck: Add query for nova unit test bug 1451389  https://review.openstack.org/17978814:27
openstackbug 1451389 in OpenStack Compute (nova) "Nova gate broke due to failed unit test" [Critical,In progress] https://launchpad.net/bugs/1451389 - Assigned to Sean Dague (sdague)14:27
*** dboik_ has joined #openstack-infra14:30
*** salv-orlando has quit IRC14:32
*** armax has joined #openstack-infra14:33
*** gsagie has left #openstack-infra14:33
*** gokrokve_ has joined #openstack-infra14:34
*** dboik has quit IRC14:34
openstackgerritMerged openstack-infra/devstack-gate: Temporarily add trove plugin configuration  https://review.openstack.org/16741214:34
openstackgerritPaul Belanger proposed openstack-infra/system-config: WIP: Initial commit for grafana  https://review.openstack.org/17920814:34
*** zz_jgrimm is now known as jgrimm14:35
*** ajmiller has joined #openstack-infra14:35
openstackgerritGal Sagie proposed openstack-infra/project-config: Enable a check job for stackforge/dragonflow  https://review.openstack.org/17515214:36
*** gokrokve has quit IRC14:37
*** jgrimm is now known as zz_jgrimm14:38
openstackgerritMonty Taylor proposed openstack-infra/project-config: Run shade functional tests on neutron and nova-net  https://review.openstack.org/17979714:40
mordredsdague: is that correct? ^^14:40
*** wenlock has joined #openstack-infra14:40
*** mtanino has joined #openstack-infra14:40
*** unicell1 has joined #openstack-infra14:40
*** wenlock1 has joined #openstack-infra14:40
sdagueI don't think you need L54 in - https://review.openstack.org/#/c/179797/1/jenkins/jobs/shade.yaml,cm14:41
*** packet has joined #openstack-infra14:41
sdagueotherwise, I think this is right14:42
*** unicell has quit IRC14:42
*** emagana has joined #openstack-infra14:42
mordredsdague: what if we change the default in the future?14:44
*** claudiub_ has quit IRC14:44
mtreinishmordred: heh, you really think that'll happen? :)14:46
*** Somay has joined #openstack-infra14:46
*** mase_x200 has quit IRC14:46
mordredmtreinish: *sad trombone*14:47
openstackgerritEmilien Macchi proposed openstack-infra/system-config: Create rubygems mirror from rubygems.org  https://review.openstack.org/17802614:47
*** tonytan4ever has joined #openstack-infra14:52
sdaguemordred: so... if the default flips we're going to have to update it a bunch of places in project config14:52
mordredsdague: k.14:53
sdagueI'm +2 either way14:53
sdaguejust saying it's not strictly needed14:53
*** alexpilotti has joined #openstack-infra14:53
*** vhoward has quit IRC14:54
*** vhoward has joined #openstack-infra14:55
mordrednod14:55
*** gokrokve_ has quit IRC14:55
AJaegermordred, just gave a +2 - feel free to self-approve once you finished the discussion here ;)14:55
*** gokrokve has joined #openstack-infra14:55
*** links has quit IRC14:56
*** gokrokve has quit IRC14:57
openstackgerritPeter Hamilton proposed openstack/requirements: Update PyKMIP version in requirements  https://review.openstack.org/16444914:58
*** gokrokve has joined #openstack-infra14:58
*** sushilkm has joined #openstack-infra14:58
*** sushilkm has left #openstack-infra14:58
openstackgerritVictor Stinner proposed openstack/requirements: Add mysqlclient dependency  https://review.openstack.org/17974514:58
*** craigbr has quit IRC15:00
*** sdake has joined #openstack-infra15:01
*** sarob has joined #openstack-infra15:02
*** sdake__ has joined #openstack-infra15:03
*** sdake_ has quit IRC15:03
openstackgerritMerged openstack-infra/puppet-jeepyb: Modernize template syntax  https://review.openstack.org/16814715:03
openstackgerritMerged openstack-infra/puppet-jeepyb: Parameterize git revision of jeepyb  https://review.openstack.org/17096015:03
openstackgerritMerged openstack-infra/puppet-jeepyb: Add helpers to execute manage_projects  https://review.openstack.org/17153815:03
*** sarob has quit IRC15:03
*** sarob has joined #openstack-infra15:03
*** sdake has quit IRC15:07
mordredjeblair: so - basically every time I go to a new change now, I get the no-revs error, and then gertty adds about 100 things to the queue, processes them, then I have the change15:07
mordredjeblair: the 100-ish number is consistent15:07
*** asselin has joined #openstack-infra15:07
mordredjeblair: also, I have nothing in my logs to indicate that gertty thinks anything is broken15:07
*** ociuhandu_ has joined #openstack-infra15:07
openstackgerritEmilien Macchi proposed openstack-infra/project-config: Disable voting for puppet-syntax-future on havana/icehouse  https://review.openstack.org/17980915:09
*** arxcruz has quit IRC15:10
pabelangerLooking for some feedback for provisioning grafana.  Right now, there is a manual process for connecting data sources.  If you read here: http://docs-draft.openstack.org/08/179208/5/check/gate-system-config-docs/e8333d4//doc/build/html/grafana.html15:11
pabelangerhowever, need to setup default admin credentials and looking for which openstack-infra group to be using15:12
pabelangerand even a default location for password15:12
openstackgerritMerged openstack-infra/puppet-jenkins: Allow for plugin pinning  https://review.openstack.org/16054915:12
pabelangerBasically, once I am done with that info, it would be ready for review15:12
openstackgerritMerged openstack-infra/tripleo-ci: Bump up delorean pinning  https://review.openstack.org/17717615:14
*** claudiub has joined #openstack-infra15:14
*** spzala has joined #openstack-infra15:14
mordredpabelanger: I don't suppose it's possible to set those things directly in the grafana db is it?15:15
pabelangermordred, checking15:16
mordredbecause, I mean, I know that we don't want to create graphs through the web15:16
jeblairpabelanger: i was talking with timrc recently about grafana and he suggested that it would be possible to write a short python script to translate a yaml file and write it into the db15:16
openstackgerritMerged openstack-infra/puppet-jenkins: Use jenkins group instead of nogroup  https://review.openstack.org/17536815:17
jeblairpabelanger: iirc, he said the db format was basically just a json blob describing each dashboard15:17
pabelangerjeblair, Ya, that was mentioned last week. Seems like the right approach15:17
jeblairtimrc meet pabelanger15:17
jeblairpabelanger meet timrc15:17
pabelangertimrc, ohai15:18
*** jcoufal_ has quit IRC15:18
jeblairpabelanger: so maybe check with timrc and see if he's started anything, and if not, fight it out between the two of you? :)15:18
pabelangerjeblair, timrc Sure. I'll dive into DB right now to see what it looks like15:20
openstackgerritMerged openstack-infra/project-config: Run shade functional tests on neutron and nova-net  https://review.openstack.org/17979715:20
*** craigbr has joined #openstack-infra15:22
jeblairmordred: i'm getting asymtoptically closer to post-travel usefulness, i'll get with you on gertty soon -- though in the mean time, if you have time to capture a 'refresh, missing git refs' cycle with debug logging enabled ("-d" on command line); and probably quit shortly after to keep log size small, that will probably help.15:22
mordredjeblair: on it15:23
*** dannywilson has joined #openstack-infra15:23
openstackgerritMerged openstack-infra/puppet-log_processor: Setting dependencies in metadata.json  https://review.openstack.org/15382115:24
mordredjeblair: are gertty logs safe to pastebin?15:24
*** craigbr has quit IRC15:25
*** sarob_ has joined #openstack-infra15:26
jeblairmordred: i believe so15:26
*** hdd has quit IRC15:27
*** dannywilson has quit IRC15:27
mordredjeblair: http://paste.openstack.org/show/21486115:28
*** sarob has quit IRC15:28
*** e0ne has joined #openstack-infra15:28
*** sks has quit IRC15:28
*** sabeen has joined #openstack-infra15:29
*** dannywilson has joined #openstack-infra15:29
*** ashleighfarnham has joined #openstack-infra15:29
openstackgerritMerged openstack-infra/jenkins-job-builder: Added parallelization options  https://review.openstack.org/7551415:30
*** Somay has quit IRC15:30
*** ociuhandu_ has quit IRC15:31
openstackgerritMerged openstack-infra/jenkins-job-builder: Added gerrit trigger missing options  https://review.openstack.org/16717515:32
*** maishsk has joined #openstack-infra15:32
*** mtanino has quit IRC15:33
*** harlowja_at_home has joined #openstack-infra15:35
*** sarob has joined #openstack-infra15:35
openstackgerritMerged openstack-infra/askbot-theme: updated theme to match version 0.7.53  https://review.openstack.org/17748515:35
*** yamahata has joined #openstack-infra15:36
fungijeblair: also when you're settled back in, i've noticed that gertty doesn't deal well with reviewing multi-parent changes like a merge commit (it dies in flames, actually). i haven't had time to dig into possible fixes for it, but will write a story with the trace in a bit15:36
openstackgerritMerged openstack-infra/askbot-theme: deleted unused file  https://review.openstack.org/17748615:36
jeblairfungi: yeah, there's a "TODO: support multiple parents" in the database schema :)15:36
openstackgerritMerged openstack-infra/askbot-theme: css changes for language nav and dropdown menu  https://review.openstack.org/17947215:36
jeblairfungi: so it's a substantial project15:37
fungijeblair: i figured i wasn't the only one to spot that. good to know15:37
jeblairfungi: though, i mean, hopefully "not crashing" might be a less substantial bugfix15:37
fungii thought a stopgap might be to just treat a multi-parent change as a non-parented change and move on15:38
clarkbjeblair: question about gearman, if you send a RESET_ABILITIES does that clear out a pending GET JOB?15:38
clarkbzaro: I think ^ may be the problem and it is much less complicated than I thought15:38
jeblairclarkb: hehe, i do not immediately know the answer to your pop quiz!  i'll look15:38
*** sputnik13 has quit IRC15:38
clarkbjeblair: we are seeing nodepool nodes remain "ready" for long periods of time and reading jenkins logs inidicates they go in that state if they send a GET JOB then immediately register functions15:39
*** sarob_ has quit IRC15:39
clarkbjeblair: I had thought the run loop was getting live locked but in this case I think it may be the server state is changing around the GET JOB because we RESET ABILITIES15:39
jeblairclarkb: we send 'get job' before registering functions?  that sounds buggy, right?15:40
clarkbjeblair: yes, and yes I think that is a bug15:40
clarkbjeblair: however we do receive the JOB_UNIQ response we just never seem to handle it15:40
*** mfisch has quit IRC15:40
jeblairclarkb: JOB_ASSIGN_UNIQ?15:40
*** Longgeek_ has joined #openstack-infra15:40
*** ildikov has quit IRC15:41
clarkbjeblair: I have a change up with logs inthe ocmmit message https://review.openstack.org/#/c/178929/15:41
*** salv-orlando has joined #openstack-infra15:41
clarkbjeblair: you should probably ignore the reset of the commit message though as zaro indicates my change does not work beyond the mvn run tests (which I Had run locally)15:41
*** Guest65197 has joined #openstack-infra15:41
jeblairclarkb: that makes it sound like geard is doing what we want in this situation (which is behavior i would expect -- i would expect geard to react to adding functions and then wake the worker as appropriate)15:42
*** Longgeek has quit IRC15:42
jeblairclarkb: oh, but maybe your question is -- does geard think the job it assigned is no longer running?15:42
clarkbjeblair: correct15:42
jeblairclarkb: got it15:42
clarkbjeblair: because the worker ends up not running any job at that point and does not go offline or emit events saying it did anything15:43
clarkbjeblair: so we essentially have a worker that can run a job but never does15:43
clarkb(and this disrupts nodepool's deficit calculations)15:43
jeblairclarkb: i do not think geard would take any action there; i would expect it to think the worker is running the job but slowly.  however, that would suggest that zuul would see stuck jobs15:43
timrcpabelanger, Hi, reading up.15:45
timrcjeblair, pabelanger I actually think it would be best if we used the grafana rest API to update / overwrite dashboards.15:45
*** Guest84883 has quit IRC15:46
timrcI have something somewhere on my laptop which does that.  The import script would then be a simple matter of converting yaml to json and then using the rest API to post new dashboards / update existing ones.15:46
jeblairtimrc: ooh there's one of those?  that sounds even better15:46
timrcjeblair, Yep.15:47
clarkbjeblair: I am not sure that we have seen that, I think I have seen jobs earlier in the pipeline that are queued but not running even though jobs that run on the same node type are running later in the pipeline15:47
clarkbI will keep an eye out for ^ today15:47
*** mwagner_lap has quit IRC15:47
timrcjeblair, At the time we talked I didn't realize there was a rest API but discovered there was on the long flight back from Ireland.15:47
pabelangertimrc, Oh, nice. Didn't know there was restapi15:47
clarkbfungi: fyi https://github.com/openstack-infra/git-review/pull/815:49
jeblairclarkb: i'm guessing we don't have any info on the job that was assigned (and apparently ignored)?15:49
clarkbjeblair: not that I could fine15:49
clarkbs/fine/find/15:49
fungiclarkb: conveniently, git-review does not enforce a signed icla, so i'll be happy to git-am that pr into gerrit on the author's behalf, unless you wanted dibs15:51
clarkbfungi: well I am wondering if we should close won't fix because it runs on windows natively last I checked15:52
*** mattfarina has joined #openstack-infra15:52
clarkbbut I don't know enough about windows to know why one would try to use a cygwin utility without cygwin paths15:52
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Ensure tests run with ConfigParser object  https://review.openstack.org/17982215:53
*** sigmavirus24 is now known as Apsu_15:53
pabelangertimrc, jeblair: okay, sweet.  API is there, simple json blob returned. Going to now hack up a dashboard now15:53
pabelangertimrc, do you have any code started?15:53
*** Apsu_ is now known as sigmavirus2415:53
*** mtanino has joined #openstack-infra15:54
timrcpabelanger, Not really, I just validated I could create and overwrite JSON dashboards15:54
timrcvia the rest api15:54
pabelangerroger15:54
pabelangergoing to go get some food, but will start work on something after lunch15:55
timrcpabelanger, Do you think it would be best if the import code ran on the grafana host itself and posted against the localhost:3000? I didn't know how that would actually work.15:55
*** hdd has joined #openstack-infra15:55
pabelangertimrc, My thoughts are to have it work like JJB does.  At least thats my preference15:56
timrcThat way we could use http basic auth and store those creds in hiera.  Maybe a post pipeline job to ssh and curl?15:56
pabelangerI've done both JJB local and remote15:56
timrcpabelanger, Ah okay.15:56
fungiclarkb: unless maybe people trapped on 'doze prefer a cygwin-provided shell/environment (i haven't tried it in nearly two decades at this point, honestly)15:56
*** devvesa has quit IRC15:56
*** armax has quit IRC15:56
clarkbjeblair: I wonder if the sequence of events is, worker send GET JOB, server receives GET JOB and sends JOB ASSIGN UNIQ, worker sends RESET ABILITIES, worker gets JOB ASSIGN UNIQ, server gets RESET ABILITIES, "kills" previously assigned UNIQ15:56
*** hashar is now known as hasharAway15:57
clarkbfungi: ya in any case its probably worht a follow up with the author to understand the usecase then they/you/we can push the change to gerrit if needed15:57
*** craigbr has joined #openstack-infra15:57
*** maishsk has quit IRC15:58
*** sabeen2 has joined #openstack-infra15:58
jeblairclarkb: i don't believe geard has the assumptiong that a worker can only have one job, so i don't think it would alter the previously assigned job15:58
fungiclarkb: i am asking it on the pr now15:58
*** whayutin_ has joined #openstack-infra15:59
jeblairclarkb: my current understanding is: zuul submits job to geard, geard assigns it to worker, worker receives assignment and ignores job, geard assumes job is running as long as that worker is connected, and so does zuul15:59
*** ashleighfarnham has quit IRC15:59
clarkbjeblair: I am almost sure that isn't what happens because we will have a lot of leaked nodes but no leaked jobs15:59
jeblairclarkb: so i would expect to see stuck jobs in zuul unless we were "lucky" enough for them to be canceled gate jobs or something.15:59
jeblairclarkb: i agree, but i can't falsify that yet :/15:59
clarkbok15:59
*** asselin has quit IRC16:00
jeblairclarkb: even zuul's "look for missing jobs" thing, which is mostly a holdover from jenkins days, just checks that gearman knows about the job16:00
jeblairgeard that is16:00
openstackgerritMerged openstack-infra/jenkins-job-builder: (hipchat) Accept list of hipchat rooms.  https://review.openstack.org/15383116:00
*** maishsk has joined #openstack-infra16:01
*** gokrokve_ has joined #openstack-infra16:01
*** gokrokve_ has quit IRC16:01
*** sabeen has quit IRC16:01
fungiclarkb: actually, the pr mentions http://babun.github.io/ so maybe that's the key16:01
*** whayutin_ has quit IRC16:02
*** SumitNaiksatam has quit IRC16:02
*** whayutin_ has joined #openstack-infra16:02
openstackgerritKen Dreyer proposed stackforge/python-jenkins: support six 1.3  https://review.openstack.org/17982616:02
*** weshay has quit IRC16:02
*** Guest65197 has quit IRC16:02
clarkbfungi: maybe, I am just confused becuase I would expect the cygwin path to work in cygwin and the non cygwin path to work in normal windows, we shouldn't be crossing path streams16:03
*** emagana has quit IRC16:03
fungialso, my wife and i just had an offer accepted on a house, so i'm likely not around much later today as we're scrambling to get mortgage paperwork finalized16:03
clarkbbut that likelyreflects my poor understanding of windows16:03
clarkbfungi: gl16:03
*** SumitNaiksatam has joined #openstack-infra16:03
fungiclarkb: i consider that i have basically _no_ understanding of windows at this point, except maybe still how to spell it16:04
*** gokrokve has quit IRC16:04
*** emagana has joined #openstack-infra16:04
openstackgerritMerged openstack-infra/jenkins-job-builder: Add support for 'raw' to inject direct xml to job configs.  https://review.openstack.org/16015816:05
*** dguerri is now known as _dguerri16:06
jeblairfungi: congrats!16:06
*** mikeit has quit IRC16:06
*** unicell1 has quit IRC16:06
*** gokrokve has joined #openstack-infra16:06
*** gokrokve has quit IRC16:06
fungithanks--hopefully no expensive surprises on the inspection16:07
zaromorning16:08
*** mpavone has quit IRC16:08
*** yamahata has quit IRC16:09
*** otter768 has joined #openstack-infra16:09
clarkbzaro: good morning, jeblair and I were discussing possibility that RESET ABILITIES being sent is potentially related to the gearman plugin trouble. jeblair was looking at the geard side. Might be worth reading some of the recent scrollback for that and let us know what you think16:09
*** sergsh_ has quit IRC16:10
*** sergsh has quit IRC16:10
*** markus_z has quit IRC16:11
*** salv-orlando has quit IRC16:12
*** maishsk has quit IRC16:13
*** otter768 has quit IRC16:14
*** yfried has quit IRC16:14
openstackgerritEmilien Macchi proposed openstack-infra/project-config: Disable voting for puppet-syntax-future on havana/icehouse  https://review.openstack.org/17980916:14
EmilienMAJaeger: thx for your review ^16:14
*** ashleighfarnham has joined #openstack-infra16:14
*** scheuran has quit IRC16:15
*** hdd has quit IRC16:17
*** alexsyip has joined #openstack-infra16:17
*** zz_jgrimm is now known as jgrimm16:18
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Disable voting for puppet-syntax-future on havana/icehouse  https://review.openstack.org/17980916:20
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Disable voting for puppet-syntax-future on havana/icehouse  https://review.openstack.org/17980916:20
*** _dguerri is now known as dguerri16:21
mordredfungi: woot!16:24
openstackgerritSean Dague proposed openstack-infra/system-config: use optimized url for zuul status  https://review.openstack.org/17093516:24
openstackgerritSean Dague proposed openstack-infra/system-config: add dbg function to hideci  https://review.openstack.org/17957316:24
openstackgerritSean Dague proposed openstack-infra/system-config: add jquery visibility  https://review.openstack.org/17936216:24
openstackgerritSean Dague proposed openstack-infra/system-config: create proxy for testing hideci  https://review.openstack.org/17936016:24
openstackgerritSean Dague proposed openstack-infra/system-config: Revert "Revert zuul status in gerrit"  https://review.openstack.org/17936116:24
openstackgerritMerged openstack-infra/jenkins-job-builder: Support the use of lists in trigger-build builder.  https://review.openstack.org/15059716:25
*** maishsk has joined #openstack-infra16:27
*** whoops has joined #openstack-infra16:33
*** yamahata has joined #openstack-infra16:34
*** Guest19502 has joined #openstack-infra16:34
*** unicell has joined #openstack-infra16:37
*** baoli has quit IRC16:38
*** maishsk has quit IRC16:40
*** flashgordon is now known as jogo16:40
*** MarkAtwood has joined #openstack-infra16:40
*** emagana has quit IRC16:41
openstackgerritSean Dague proposed openstack-infra/system-config: use optimized url for zuul status  https://review.openstack.org/17093516:42
openstackgerritSean Dague proposed openstack-infra/system-config: add dbg function to hideci  https://review.openstack.org/17957316:42
openstackgerritSean Dague proposed openstack-infra/system-config: add jquery visibility  https://review.openstack.org/17936216:42
openstackgerritSean Dague proposed openstack-infra/system-config: create proxy for testing hideci  https://review.openstack.org/17936016:42
openstackgerritSean Dague proposed openstack-infra/system-config: Revert "Revert zuul status in gerrit"  https://review.openstack.org/17936116:42
*** emagana has joined #openstack-infra16:42
sdaguegreghaynes: doh, there was a pep8 at the bottom of the last patch16:42
greghaynessdague: fun16:43
*** harlowja_at_home has quit IRC16:43
sdaguegreghaynes: so the issue with nginx is that it's not just a proxy, it's also rewriting urls inside of content on the fly to get sub resources to go back to the proxy16:44
greghaynessdague: yea, you can do all that fun stuff in nginx, but it would probably end up being about as much config as you had code16:44
sdagueand if anyone figures out how to do that for the openid request from gerrit so that you can actually log in, that would be super awesome16:44
sdaguegreghaynes: ok, well that would be beyond my level of experience :)16:44
*** hdd has joined #openstack-infra16:44
sdagueand this requires nothing beyond python stdlib except requests16:45
*** cody-somerville has joined #openstack-infra16:45
*** sarob_ has joined #openstack-infra16:46
greghaynesyep, and its readable, which is a big win over $webserver_config16:47
sdagueI wrote a version to test ui in elastic recheck, then thought "I wonder if this would work in gerrit"16:48
sdaguewhich led to this tweet - https://twitter.com/sdague/status/59410224987545190416:48
*** sarob_ has quit IRC16:48
sdaguebecause, gerrit's post back channel as a single page app, is kind of a pain16:48
mordredreed: ping - question about sending  out a survey to people in #openstack-horizon if yo uhave a minute16:49
reedmordred, sure16:49
*** gokrokve has joined #openstack-infra16:50
fungiclarkb: revisiting that git-review patch briefly, it looks like the use case is git running in cygwin but python running outside cygwin, so git-review is getting conflicting file paths between what its callouts to git return and what it tries to access via python16:50
* fungi has no idea if this is a sane/common configuration16:50
*** bcornec has joined #openstack-infra16:52
bcornecHello everybody. I was directed to this IRC chat room by Devananda while reporting an issue I had on the ironic channel. Hope that's fine.16:53
*** hdd has quit IRC16:53
mordredbcornec: hey! welcome to the madhouse16:53
bcornecThe issue I had is with i18n with the git review command. I put info on the wiki so that others can work around it, but I think it should behave differently (meaning without the need to abandon my LANGUAGE setup). Cf: https://wiki.openstack.org/wiki/Documentation/HowTo/FirstTimers#i18n16:54
bcornecThx mordred !16:54
*** maishsk has joined #openstack-infra16:54
openstackgerritTerry Wilson proposed openstack/requirements: global-requirements: Add ovs  https://review.openstack.org/17789116:54
*** dizquierdo has joined #openstack-infra16:55
clarkbbcornec: what version of git-review do you have? I believe that the latest version should override the envs LANGUAGE and set it to C for each git command16:56
clarkbbcornec: that way you don't have to set it yourself16:56
openstackgerritClaudiu Belu proposed openstack-infra/project-config: Adds pypi-jobs to compute-hyperv  https://review.openstack.org/17978216:56
bcornec$ rpm -q git-review16:57
bcornecgit-review-1.23-3.mga416:57
sdagueor and mordred while you are active - https://review.openstack.org/#/c/179764/ - we need the redirect on https in zuul as well16:57
*** blahdeblah has joined #openstack-infra16:57
mordredsdague: on it16:57
clarkbbcornec: can you test 1.24 and see if the problem persists? You can install it to a virtualenv and run it from there16:58
*** sdake__ is now known as sdake16:58
mordredbcornec: virtualenv foo  ; foo/bin/pip install git-review ; foo/bin/git-review  ... in case "install to a virtualenv" is new for you16:58
clarkbbcornec: a quick check of the git history shows we added the fix in 1.2416:59
clarkbbut if it isn't fixed we should know that too16:59
*** bswartz has quit IRC17:00
bcornecThanks testing now...17:01
fungiokay, heading out to run some errands, bbiaw17:01
*** ivar-lazzaro has joined #openstack-infra17:01
bcornecmordred: everything pythong is new for me ;-)17:01
bcornecs/pythong/python/17:01
*** abregman has quit IRC17:02
clarkbthough I note it sets LANG=C but does not set LANGUAGE, possible it needs an update17:02
bcornecLANGUAGE was needed for sure on my side.17:03
*** baoli has joined #openstack-infra17:03
mordredcool. good to know17:04
openstackgerritClark Boylan proposed openstack-infra/nodepool: Document running Nodepool for testing  https://review.openstack.org/17984017:04
clarkbnibalizer: ^17:04
*** dboik_ has quit IRC17:04
*** dboik has joined #openstack-infra17:05
mordredclarkb: I only see us doing it on the git branch command17:05
mordredclarkb: perhaps we should do it on every command?17:05
*** dguerri is now known as _dguerri17:06
*** baoli has quit IRC17:06
clarkboh I thought it was in the subprocess wrapper17:06
*** zul has quit IRC17:06
mordredclarkb: it's not17:06
clarkbif not, then ya it should be pushed down to there17:06
openstackgerritAndreas Jaeger proposed openstack-infra/release-tools: Add script to cleanup of translations  https://review.openstack.org/17282717:06
*** baoli has joined #openstack-infra17:06
openstackgerritMerged openstack-infra/puppet-zuul: also add targeted status redirects to https  https://review.openstack.org/17976417:07
*** zul has joined #openstack-infra17:07
pabelangerjeblair, timrc: suggestions on naming for grafana yaml app?17:08
openstackgerritMonty Taylor proposed openstack-infra/git-review: Push language override down into the cmd wrapper  https://review.openstack.org/17984117:08
timrcpabelanger, grafyaml jk...17:08
pabelangergrafana dashboard builder was first choice, but gdb seems a bad naming ;)17:08
openstackgerritMonty Taylor proposed openstack-infra/git-review: Push language override down into the cmd wrapper  https://review.openstack.org/17984117:08
pabelangertimrc, Oooh17:08
mordredclarkb, bcornec: ^^ I think that should be the thing we're talking about?17:08
pabelangertimrc, I'm not even mad17:08
clarkbmordred: ya that looks right17:09
*** Longgeek_ has quit IRC17:10
mordreduhm17:11
*** achanda has joined #openstack-infra17:11
bcornecmordred: for what I can understand yes it seems that is at least needed.17:11
clarkbmordred: yes?17:11
mordrednm17:12
mordredI'm stoopid17:12
anteayanever17:12
*** _nadya_ has joined #openstack-infra17:13
*** jistr has quit IRC17:15
openstackgerritClaudiu Belu proposed openstack-infra/project-config: Adds publish-to-pypi to networking-hyperv  https://review.openstack.org/17984417:15
*** Guest19502 has quit IRC17:15
*** Guest19502 has joined #openstack-infra17:15
*** tnovacik has quit IRC17:15
*** bswartz has joined #openstack-infra17:16
*** sputnik13 has joined #openstack-infra17:16
*** Krinkle|detached is now known as Krinkle17:16
*** dizquierdo has quit IRC17:17
pabelangertimrc, https://github.com/pabelanger/grafyaml now lives :)17:17
mordredthis is exciting17:17
*** dizquierdo has joined #openstack-infra17:17
*** patrickeast has joined #openstack-infra17:18
timrcpabelanger, Sweet.17:19
cody-somerville w00t17:21
anteayapabelanger: congratulations17:21
*** Sukhdev has joined #openstack-infra17:22
*** luqas has joined #openstack-infra17:22
*** armax has joined #openstack-infra17:23
*** ociuhandu has quit IRC17:24
mordredsdague, dhellmann, clarkb: where did we ever get with markerlib support for things in requirements/17:24
mordred?17:24
mordredis that real? or just a vision of the future still?17:24
openstackgerritEmilien Macchi proposed openstack-infra/system-config: Create rubygems mirror from rubygems.org  https://review.openstack.org/17802617:24
*** mrmartin has quit IRC17:25
*** _nadya_ has quit IRC17:25
*** Guest19502 is now known as mfisch17:26
*** gokrokve has quit IRC17:27
clarkbmordred: vision of the future iirc, lifeless had a ml response explaining why17:27
mordredk17:27
mordredso a requirements change to add a requirement that is really only intended for python3 is not happyface17:27
*** mrmartin has joined #openstack-infra17:28
timrcpabelanger, Luckily converting yaml to json relatively simple.  I don't think there's a python grafana cli that implements the rest interface.  Are you just going to put a thing wrapper around requests to post dashboards?17:28
*** mrmartin has quit IRC17:28
timrcis relatively*17:28
*** mrmartin has joined #openstack-infra17:29
timrcMan thing wrapper? _thin_ wrapper.17:29
timrcIt's going to be one of those Mondays.17:29
*** dboik_ has joined #openstack-infra17:29
pabelangertimrc, Ya, I don't see any python libs for grafana / or CLI. So, ya for now just build out some simple CRUD for it.  It likely can be moved out at some future point.17:30
*** erikmwilson is now known as Guest1480717:30
*** erikmwil_ has joined #openstack-infra17:30
*** Guest14807 has quit IRC17:30
*** erikmwil_ is now known as erikmwilson17:30
*** salv-orlando has joined #openstack-infra17:30
*** erikmwilson_ has joined #openstack-infra17:30
*** armax has quit IRC17:31
*** ociuhandu has joined #openstack-infra17:31
*** tiswanso has quit IRC17:32
*** tiswanso_ has joined #openstack-infra17:33
*** luqas has quit IRC17:33
*** dboik has quit IRC17:34
*** harlowja has joined #openstack-infra17:34
anteayatimrc: I was hoping for thing wrapper17:34
*** baoli has quit IRC17:36
openstackgerritMerged openstack-infra/shade: Rename get_endpoint() to get_session_endpoint()  https://review.openstack.org/17885917:36
*** baoli has joined #openstack-infra17:37
clarkbnibalizer: so looking up the puppet beaker stuff. Have questions about https://review.openstack.org/#/c/179249/8/spec/acceptance/basic_swift_spec.rb comments says run puppet twice but it appears to run puppet 3 times, what is going on there?17:37
clarkbnibalizer: and why would you ever want t catch failures? shouldn't a failure alwys be a failure and raised as such?17:37
openstackgerritMerged openstack-infra/shade: Convert node_set_provision_state to task  https://review.openstack.org/17798717:38
clarkbEmilienM: ^ I guess same questions for you17:38
*** baoli has quit IRC17:38
*** salv-orlando has quit IRC17:39
clarkbalso not a fan of the inlined heredoc, the test file should be a fixture so that it is reuseable as a "known working"17:39
*** baoli has joined #openstack-infra17:39
*** gokrokve has joined #openstack-infra17:41
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add query for libvirt socket hangup bug 1451506  https://review.openstack.org/17985117:42
openstackbug 1451506 in OpenStack Compute (nova) "spawn failed with "libvirtError: internal error: received hangup / error event on socket" in the gate" [Undecided,New] https://launchpad.net/bugs/145150617:42
crinkleclarkb: i think that's an error17:42
nibalizerclarkb: probably thats because that particular case isn't idempotent17:42
*** russellb has quit IRC17:42
crinkleclarkb: convention with beaker-rspec is to use the heredoc17:42
*** jogo has quit IRC17:42
timrcanteaya, :)17:42
nibalizerand doesn't swift use exported resources? so it'd have to run twice ya?17:42
nibalizercrinkle: ^?17:42
crinklemaybe17:43
crinkleif so the comment should be updated17:43
clarkbnibalizer: crinkle I see so its "run twice, then run a third time" not "run twice"17:43
*** russellb has joined #openstack-infra17:43
crinkleyeah the comment may have been cargoculted17:43
*** jogo has joined #openstack-infra17:43
nibalizerclarkb: so this is more the pattern: http://git.openstack.org/cgit/stackforge/puppet-keystone/tree/spec/acceptance/basic_keystone_spec.rb#n6117:43
clarkbfor the heredoc is there an advantage to inlining that file?17:44
clarkbits clearly a test fixture and should be treated as one...17:44
mordredclarkb, nibalizer, crinkle: the more I look at things, the more I think we should do beaker-rspec so that we can do things the same way as the puppet-openstack stuff and enhance each other's happy17:44
clarkbmordred: thats funny, the more I look at it the less I like it :)17:44
mordredI was able to read and understand and reason about writing new tests looking at the stuff in puppet-openstack17:44
clarkbmordred: it comes off as very hacky17:44
crinkleclarkb: the advantage is we can see it immediately without looking at a fixture17:45
crinkleand it's conventional17:45
*** gokrokve has quit IRC17:45
*** gokrokve has joined #openstack-infra17:45
mordredclarkb: that may be - but if it's the recommended puppet way, and it's what EmilienM and crinkle and folks are doing in the other puppet things, I think we'd be better off working together than doing it all a bit differently ourselves17:45
*** mwagner_lap has joined #openstack-infra17:45
* EmilienM reads backlog17:46
mordredbasically, I'm not commenting on whether it's a good tool - more whether we have another option that is better _enough_ to warrant divergence17:46
clarkbmordred: you have to do a fair bit of setup t make it work in nodepool nodes for example, and it completely bypasses the zuul-cloner goodness we have (though shouldn't be required, just means an extra step)17:46
mordredclarkb: well, we should figure out how to fix it to not bypass zuul cloner so that EmilienM is also not unhappy17:46
mordredwhich might mean making beaker itself less bong17:46
mordredor - we should convince EmilienM to do something different with us17:47
crinkleI think we're pretty set on using beaker-rspec for those modules17:47
crinklebut i have no opinion on whether infra decides to follow suit17:47
nibalizerclarkb: i really really want to get zuul cloner and beaker playing nice with each other17:47
mordredI spend enough time yelling at openstack devs about divergence, that I believe diverging myself would be quite hypocritical17:47
mordrednibalizer: ++17:47
nibalizerbecause zuulcloner + a gate for the puppet openstack modules would be sooo cool17:48
* nibalizer gets chills17:48
clarkbmordred: ya its definitely nice to converge, I am just pointing out the first impression problems I see with it17:48
mordredclarkb: sure17:48
nibalizerclarkb: also beaker is opensource, we can take feedback upstream and patches too if we want17:48
mordredclarkb: also- I mean, I'm sstill not really sure if I should write thigns in rspec or serverspec or both or somethign else17:48
mordredI will say I think that serverspec is very nicely readable17:49
clarkbnibalizer: well these issues appaer to be conventions17:49
clarkbnibalizer: for example rspec expects to install the module for you17:49
mordredbut beaker-rspec seems to be different?17:49
clarkber sorry beaker17:49
nibalizerclarkb: conventions evolve17:49
*** luqas has joined #openstack-infra17:49
clarkbnibalizer: and heredocing fixtures inline in code17:49
nibalizerbeaker-rspec sits above beaker because largely of conventions disagreements (as i understand it)17:49
clarkbmordred: beaker-rspec is just run rspec tests after beakering17:49
clarkbmordred: the openstack puppet modules seem to use beaker-rspec not beaker alone17:50
mordredyah17:50
mordredbut something in the beaker-rspec docs mentions serverspec17:50
mordredalthough it does not seem like that's actually used/true17:51
*** luqas has quit IRC17:51
mordredhttps://github.com/puppetlabs/beaker-rspec/blob/master/lib/beaker-rspec/helpers/serverspec.rb17:51
clarkbfwiw https://review.openstack.org/#/c/179249/8/spec/spec_helper_acceptance.rb is the thing that would need to be made to grok zuul-cloner aiui17:51
clarkbright now, it appears to want to install all the things for you17:51
*** gulic has joined #openstack-infra17:52
clarkbexcept it doesn't really understand the gate and testing things together, which is fine until you want to gate things together (unsure if that is necessary)17:52
*** gulic has left #openstack-infra17:52
mordredyah - we'd need to be able to have that find zuul cloner repos to install from17:52
mordredclarkb: I think we'll want to gate things together for sure17:52
crinkleclarkb: yeah, actually gating things together would be nice, right now it just git clones from master17:52
mordredclarkb: we ahve enough thigns with depends-on relationships17:52
*** ZZelle has quit IRC17:53
*** ZZelle has joined #openstack-infra17:53
nibalizerya so my plan to hack that was to shell out to zuul cloner from inside spec_helper_acceptance, but thats kinda gross17:53
mordredso maybe we'll need to write a ruby library called beaker-rspec-zuul or something, and have that have a function that knows how to clone things, and will ook for zuul-cloner prepared repos and clone from them, otherwise clone from source17:53
clarkbmordred: or maybe beaker should gets its hands out of everything...17:54
mordredthat way anyone doing beaker-rspec things in a zuul context can just use beaker-rspec-zuul17:54
clarkbmordred: it doesn't need to install the modules, and it doesn't need to setup test nodes17:54
jeblairmordred, nibalizer: joining late, sorry, but can you say in 20 words why this should happen inside of a tool and not as a preparatory step before running it?17:54
clarkbit just needs to assert some state17:54
mordredclarkb: sure - but let's one step at a time - I think we are the exceptional case17:54
jeblair(i think clarkb might also be asking my question)17:54
clarkbjeblair: I think so :)17:55
mordredjeblair: because not everyone in the world has an infra-quality repo prep system at their disposal, so "do the thigns for me" is a feature fora  lot of people17:55
crinklejeblair: it's meant to be able to run from a user's laptop, so having it self contained is preferred17:55
*** tnovacik has joined #openstack-infra17:55
mordredjeblair: what we need is a way to easily say "please don't do that, kthxbai" in a way that doesn't break people trying to do local testing with, say, vagrant17:55
claudiubHello, anyone can give me a helping hand regarding stackforge pushed versions through gerrit? It seems I have a little bit of a problem and I did not find a proper solution yet and I was wondering if you folks would know.17:55
nibalizerjeblair: what crinkle and mordred are saying is valid, I'm not 100% satisfied that what we have now (not includin zuul cloner) is enabling local development to match ci testing17:56
nibalizerso adding zuul cloner functionality as a hidden side effect, to me, sounds like a step in the wrong direction17:56
jeblairmordred: yeah; i feel like that's the approach to start with; as there are many testing tools out there that do not manipulate git repos, so i don't actually think we're weird.  i appreciate the extra helpfulness, but it should be extra.  if every computing tool had to understand an scm, the world would be a bad place.17:56
jeblairnibalizer: by 'what we have now' do you mean the proposal in 179249?17:57
nibalizerI worry that we're pushing vagrant files out to local developers and no one is validating that they work17:57
jeblairnibalizer: or something else?17:57
EmilienMclarkb: swift was a  bad example17:57
EmilienMall other modules needs 1 puppet run17:57
dhellmannmordred, jd__: regarding lockfile, it's in murano's requirements list. murano is not in the requirements/projects.txt list though so that won't break anything, but now that murano is an official project maybe it would want to be in the projects list?17:57
clarkbEmilienM: thanks, I will have to look at the others too, I choose swift somewhat randomly17:58
mordredjeblair: yah - and it soudns like nibalizer is going to write us some upstream ruby to help with that17:58
nibalizerjeblair: 179249 is just the swift version of over a dozen patches that EmilienM has spun up, some are landed17:58
*** sabeen2 has quit IRC17:58
jeblairnibalizer: i agree -- i believe there is no warranty of functionality or fitness of purpose for anything in infra repos if we don't gate-test it.  :)17:58
mordreddhellmann: yes. in fact, I think it should be in the projects list17:58
jeblair(and maybe not even then ;)17:58
nibalizerjeblair: this is the file im worried about: https://review.openstack.org/#/c/179249/8/spec/spec_helper_acceptance.rb17:59
*** harlowja has quit IRC17:59
*** harlowja_ has joined #openstack-infra17:59
clarkbclaudiub: what is the problem?17:59
*** tonytan4ever has quit IRC17:59
claudiubclarkb: hello. So, I've pushed 1.0.3 to https://github.com/stackforge/networking-hyperv and if I download the zip / tar.gz, unzip it and python setup.py install it, this occurs: Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. Are you sure that git is installed?17:59
nibalizerin that I think there are assumptions about running in infra, that wouldn't be there if a developer was using vagrant17:59
mordredclaudiub: yes, that's totally not supported17:59
*** luqas has joined #openstack-infra17:59
clarkbclaudiub: ya, you either need a git repository or an sdist17:59
mordredclaudiub: github zip/tarball files are not and never will be a format that you can use17:59
dhellmannmordred: so do we remove lockfile and force murano to update before they can be added, or do we do it in the other order?17:59
mordredclaudiub: what you want is http://tarballs.openstack.org/networking-hyperv/18:00
nibalizerso I think we're in a grey area now, and we need to be careful about how we add zuul-cloner so that we enable both local development and CI18:00
mordredclaudiub: http://tarballs.openstack.org/networking-hyperv/networking-hyperv-master.tar.gz if you want a thing that tracks master18:00
clarkbnibalizer: to some degree, but say I am making a change to one module that requires an update to a dep too, I need to be able to express that even on my laptop18:00
*** erikmwil_ has joined #openstack-infra18:00
*** erikmwilson is now known as Guest3813118:00
*** erikmwil_ is now known as erikmwilson18:00
mordredclarkb: ++18:00
*** erikmwilson has quit IRC18:00
*** armax has joined #openstack-infra18:00
clarkbnibalizer: I think vagrant areas are likely where we need to more directly work around the laptop use case. "Install the correct stuff" is something that should be the same whereever18:00
jeblairEmilienM, nibalizer: i see -- i think since puppet modules are joining the big tent we should probably work together on a solution that works for puppet-openstack and infra18:01
mordredin fact, I know the tchaypo frequently talks about wanting to locally test complex interaction chains18:01
clarkbnibalizer: and many tools solve that by making you do the install18:01
*** erikmwilson_ is now known as erikmwilson18:01
*** luqas has quit IRC18:01
claudiubah, I see. cool. So, another question.. in order to have certain versions for the tarballs, there has to be a specific job in project-config, right?18:01
mordredjeblair: ++18:01
clarkbnibalizer: which I think is a very valid way to do it18:01
dhellmannmordred: actually, it looks like they may not be using that dependency, let me try a patch18:01
claudiubalso, if the pypi job is defined on the project-config, pip install should not have any problems, right?18:01
mordredclaudiub: that's right - although the pypi jobs will only release something to pypi when you push a signed tag18:02
*** emagana has quit IRC18:02
*** chlong has quit IRC18:02
nibalizerclarkb: so you think we should set up the right repos in the right places before the test runs, and not do those things in spec_helper_acceptance ?18:03
claudiubmordred: I assume - name: openstack-server-publish-jobs should be added to layout/zuul.yaml, right?18:03
jeblairnibalizer, mordred, clarkb: is this similar to devstack vagrant testing?  where you want to use vagrant to run devstack in a local vm based on git repos on your host?18:03
clarkbnibalizer: correct, because I think its better to have the user think about what they should do and do it correctly instead of doing it poorly and most likely wrong for them18:04
openstackgerritMerged openstack-infra/elastic-recheck: Add query for libvirt socket hangup bug 1451506  https://review.openstack.org/17985118:04
claudiubmordred: and - openstack-publish-jobs to project.yaml18:04
openstackbug 1451506 in OpenStack Compute (nova) "spawn failed with "libvirtError: internal error: received hangup / error event on socket" in the gate" [Undecided,Confirmed] https://launchpad.net/bugs/145150618:04
openstackgerritDoug Hellmann proposed openstack/requirements: Remove lockfile from requirements  https://review.openstack.org/15122418:04
*** SumitNaiksatam has quit IRC18:04
jeblairclarkb, nibalizer: ++18:04
nibalizerjeblair: i have no information about that workflow, but sounds related18:04
*** Guest38131 has quit IRC18:04
dhellmannmordred, jd__ : The global requirements patch now depends on the patch to remove it from murano.18:05
*** EmilienM is now known as EmilienM|mtg18:05
*** sandywalsh_ has joined #openstack-infra18:05
*** SumitNaiksatam has joined #openstack-infra18:05
dmsimardWould love a core review on https://review.openstack.org/#/c/179809/ when anyone gets a chance, thanks :)18:05
*** sandywalsh has quit IRC18:06
nibalizerclarkb: okay, and we could write a little 'grab the tip of master for everything' script for the people who aren't doing anything fancy?18:06
clarkbnibalizer: ya18:06
clarkbnibalizer: similar to how devstack works18:06
clarkbdevstack will use what you give it or if lazy give you $BRANCH18:07
nibalizerclarkb: okay i think we can do this, note that we'll have to put /some/ logic in spec_helper_acceptance since thats our entry point into running stuff on the vagrant vm before the actuall tests start18:07
greghaynessdague: Commented on https://review.openstack.org/#/c/179573/4/modules/openstack_project/files/gerrit/hideci.js18:07
*** jcoufal has joined #openstack-infra18:08
*** maishsk has quit IRC18:08
*** packet has quit IRC18:09
*** Sukhdev has quit IRC18:09
sdaguegreghaynes: *shrug*? that was how I figured out not to crush all the important object info :)18:09
sdagueif you have a better way, I'm all for it18:09
* sdague doesn't even pretty to play a js expert on tv18:09
*** SumitNaiksatam has quit IRC18:09
sdagues/pretty/pretend/18:10
*** otter768 has joined #openstack-infra18:10
greghaynesheh, I probably shouldnt pretent18:10
greghaynespretend18:10
claudiubclarkb, mordred: anyways, thanks a lot for the help! :D18:10
anteayagreghaynes: but can you pretty18:10
*** maishsk has joined #openstack-infra18:10
greghaynes:)18:11
greghaynessdague: The thing I pasted should work I think, ill test it out using your fancy proxy patch though18:11
*** emagana has joined #openstack-infra18:11
sdagueok, I did it that way so that the objects came through natively in the chrome console, which lets you click through them18:12
greghaynesyes, IIRC that is a matter of the type of the first argument you pass in18:13
greghaynesif your first arg is not a string then itll output objects18:13
*** rbradfor has quit IRC18:14
nibalizerclarkb: so I think we have s_p_a do some inspection to see if it is in a zuul environment (maybe check $ZUUL_REF or something), then if not do the git clone from tip of master, probably need s_p_a to clean /etc/puppet/modules and copy the zuul-cloner'd repos into /etc/puppet/modules though. Also need it to do the pure dependency installs such as stahnma-epel etc18:14
*** otter768 has quit IRC18:15
*** chlong has joined #openstack-infra18:15
clarkbnibalizer: can we check for the existence of the modules? or is it possible these nodes are reused even with vagrant?18:15
*** Krinkle is now known as Krinkle|detached18:16
nibalizerclarkb: beaker works by starting up a vm and then doing things to it, then turning it off18:17
nibalizerso making that work with side effects is gonna be a bit tricky18:17
*** sabeen has joined #openstack-infra18:18
nibalizerand yes sometimes people run beaker with BEAKER_destroy=no or BEAKER_provision=no which allows for reusing the vm18:18
openstackgerritMonty Taylor proposed openstack-infra/shade: Enhance the OperatorCloud constructor  https://review.openstack.org/17964718:18
clarkbnibalizer: hrm, right because something in the new VM has to figure it out18:18
nibalizermostly when debugging the testing harness in my experience18:18
nibalizerbut I can't speak for all developers18:18
nibalizerit does make it a ton faster if you know what you're doing isn't corrupting the tests you are doing18:19
mordredso we'd want normal beaker to do the thing people expect from a normal beaker18:19
mordredand we'd want to be able to run it in a way that says not only BEAKER_provision=no - but also "hi, we've done some prep for you"18:19
clarkbmordred: sort of, normal beaker is wrong from a install the list of correct things perspective18:19
mordredwhat do you mean?18:20
clarkbmordred: normal operation appears to be install master or latest release18:20
clarkbmordred: it does not grok "install this development commit so I can iterate on my dev workflow"18:20
mordredyah18:20
openstackgerritMerged openstack-infra/project-config: Disable voting for puppet-syntax-future on havana/icehouse  https://review.openstack.org/17980918:20
mordredwell, that CLEARLY needs to be fixed18:20
nibalizermordred: so we don't set BEAKER_provision=no, we tell it to use a very dumb hypervisor called 'just ssh into localhost, itll be ok'18:20
clarkbnibalizer: its super close to metal18:21
clarkbnibalizer: not even a container in the way18:21
nibalizerclarkb: right well so... beacuse the beaker consumers are also using the github workflow, they get around that :P18:21
nibalizerso they have a stack of commits checked out locally18:21
*** EmilienM|mtg is now known as EmilienM18:21
nibalizeron a brnach when they(we) run beaker locally18:21
mordredclarkb: we need to make beaker work for the "clone from master" case like normal puppet people expect - we need to improve their experience by adding a thign that knows how to run beaker using the git commits they have locally and not pushed, and we need to support zuul-cloner envs18:22
mordredclarkb: once that's done, it'll be good :)18:22
clarkbnibalizer: how does that help if beaker is installing via puppet module?18:22
nibalizeras for depending on an unlanded commit in another module, idunno how pepole do that other than derping with spec_helper_acceptance18:22
clarkbnibalizer: (I want to know because that is essentially how zuul-cloner works)18:22
nibalizerclarkb: only applies to the repo you're working in18:22
clarkbnibalizer: ah, ok18:22
*** zaneb has joined #openstack-infra18:22
nibalizerso for some extra diversity, here is another s_p_a https://github.com/elastic/puppet-elasticsearch/blob/master/spec/spec_helper_acceptance.rb18:23
nibalizerin my experience, these things vary kinda wildly18:23
jeblairnibalizer: s_p_a means what?18:24
nibalizeralso https://github.com/puppetlabs/puppetlabs-postgresql/blob/master/spec/spec_helper_acceptance.rb18:24
nibalizerjeblair: spec_helper_acceptance.rb, it is the script that runs before the testing in beaker18:24
jeblairwhere's the p come from?18:24
nibalizerso this is where you install puppet dependencies, and even puppet itsefl sometimes18:25
jeblairto avoid collision with sha? (ha!)18:25
nibalizerjeblair: i dont know, i think i meant s_h_a but ended up saying s_p_a (my bad!)18:25
nibalizerhehe18:25
clarkbright its basically our run install_puppet.sh script step18:25
jeblairnibalizer: ok, np.  maybe you just invented a thing.  :)18:25
anteayanibalizer: I like s_p_a as acronyms go18:26
clarkbexcept our install_puppet.sh understands these things18:26
*** dizquierdo has quit IRC18:27
*** mattfarina has quit IRC18:28
*** Sukhdev has joined #openstack-infra18:29
*** nikita09 has joined #openstack-infra18:30
*** sdake_ has joined #openstack-infra18:30
EmilienMnibalizer: I think fbo found out why we can't use beaker5 on OS infra18:31
EmilienMhttps://github.com/puppetlabs/beaker/blob/master/lib/beaker/host_prebuilt_steps.rb#L52818:31
EmilienMand http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/puppet-module-jobs.yaml#n3918:31
*** dboik_ has quit IRC18:31
asselin__nibalizer, can you reevaluate your -1 on this? https://review.openstack.org/#/c/167425/18:32
*** tonytan4ever has joined #openstack-infra18:33
clarkbmordred: hahaha see EmilienM's links. We very recently had a discussion about this18:33
nibalizerasselin__: sure18:33
openstackgerritKen Dreyer proposed stackforge/python-jenkins: support six 1.3  https://review.openstack.org/17982618:33
*** tiswanso_ has quit IRC18:33
EmilienMclarkb: it's time to collaborate seriously, isn't ? :)18:33
nibalizerEmilienM: i dont understand?18:33
EmilienMnibalizer: when I started to play with beaker18:34
EmilienMI tried beaker 518:34
clarkbEmilienM: it was a week or two ago that mordred and I were arguing about how ansible should know how to pass envs through to remote end18:34
EmilienMand I failed and did not investigate because I wanted to move forward with your work18:34
clarkbEmilienM: and it appears that that is one thing beaker depends on18:34
*** sdake has quit IRC18:34
EmilienMnibalizer: fbo found our the same issue and it seems it comes from the links I shared here18:34
*** sdake_ is now known as sdake18:34
nibalizerEmilienM: yea so the issue is that infra and beaker fight over the ssh config file?18:34
clarkb(I am still not sure I know the answer around that, but having the option to with a tool like ansible seems useful)18:34
EmilienMnibalizer++18:35
mordredclarkb: for the record, I've not been arguing that passing env vars is not something it shoudl do - I've been arguing that the framing of the way that you want to do it is not the way they do it18:35
waynrdoes anyone know what a gerrit patchSet "number" is?18:36
*** samueldmq_ has joined #openstack-infra18:37
*** boris-42 has joined #openstack-infra18:37
clarkbwaynr: yes, its the number assigned to the patchset. It is an incrementing counter for each patchset starting at 118:37
waynroh right, that makes sense18:38
clarkbwaynr: in gerrit a change has a series of patchsets. Unlike github you don't continuously append to your pull request, instead you push a new version of a chnage. Each version of a change is a patchset and is assigned a unique number18:38
nibalizerasselin__: you have feedback :)18:38
waynryeah i was just confused, the number makes sense to me now18:38
nikita09hi, does Jenkins job builder support passing list parameters to macros ???18:40
nikita09i want to pass list of branches to my git macro, that my job will be building, im not sure if its allowed to pass a list of items as parameter to a macro..18:41
waynrI find the implementation of JJB macros confusing and inconsistent (can't reference otherwise globally-defined JJB variables, can't pass yaml "objects") so I tend to stay away from them18:41
clarkbnikita09: I think so, you have to annotate the type otherwise you get the macro expanded n time where n = length of list18:41
waynroh maybe you cant pass yaml objects18:41
clarkbwaynr: you should be able to pass yaml objects18:41
waynrs/cant/can/18:41
openstackgerritMerged openstack/requirements: also install a new pbr into the venvs  https://review.openstack.org/15541018:42
clarkbhttp://ci.openstack.org/jenkins-job-builder/definition.html#variable-references18:42
clarkbwaynr: nikita09 see ^18:42
waynrclarkb: i seem to recall not being able to simply use '{obj:key}' format for passing objects like that, but haven't tried recently enough--maybe it's fixed now?18:43
waynri also definitely didn't try with the type annotation18:43
clarkbI have no idea, we don't use the feature anywhere, I have yet to find a case where it is necessary18:43
clarkbbut you can in theory use it18:44
clarkband it is in theory tested by JJBs tests18:44
*** gokrokve has quit IRC18:44
waynryeah i started using it when I first tried JJB but quickly stopped when i realized I couldn't reference JJB variables defined elsewhere inside a macro18:44
clarkbwaynr: you have to pass variables into macros same as any job template18:44
clarkbso you can use them they just have to be passed through18:45
nikita09clarkb: thanks...) how do i annotate the type ? im not sure what that means ..:/18:45
waynrclarkb: yeah but if i define a variable in a project, then list job-group in that project's "jobs" the variable defined at the project-level will be available at the job-template level for job templates defined in the job-group18:45
nikita09like '{obj:list"'???18:45
clarkbnikita09: the documentation there should show you, !!python/tuple for a tuple looks like18:45
waynrbut for macros if I remember correctly i always have to explicitly pass variables when calling the macro, otherwise variables defined at other levels in the jjb object hierarchy aren't available18:46
clarkbwaynr: you can think of putting the job template in the job list of a project as an explicit pass18:46
clarkbwaynr: there is nothing global about that var, you are explicitly passing it through to every entry in the jobs list18:47
nikita09clarkb: got it / thanks alot :)18:47
waynrbut i'm saying that the variables from the project get passed to the job-template even if that job-template is only transitively referenced in the list of jobs for that project which is very different from the macro behavior18:47
waynrwhen I say "global" what I really mean is "gets passed in from outer scopes"18:47
*** gokrokve has joined #openstack-infra18:48
*** subscope_ has joined #openstack-infra18:49
*** shivharis has joined #openstack-infra18:50
*** tsg has joined #openstack-infra18:50
*** SumitNaiksatam has joined #openstack-infra18:51
anteayamtreinish: thanks for the post to that email, I was wondering what the issue was18:51
mtreinishanteaya: oh, the depends-on thing? Sure, np18:52
*** ildikov has joined #openstack-infra18:52
mtreinishyeah it took me a sec to figure out what he was asking18:52
anteayamtreinish: I looked at it too, had no idea, and marked it as something to track for the solution18:52
anteayamtreinish: I didn't know tempest-lib check jobs installed from last release not source18:53
lifelessmordred: what requirements change?18:53
mtreinishanteaya: all the openstack libs and clients install from releases (unless it's a source job)18:53
anteayamtreinish: well shows how much I know18:54
mordredlifeless: ECONTEXT18:54
anteayamtreinish: thank you18:54
openstackgerritWalter A. Boring IV (hemna) proposed openstack/requirements: Adding os-brick for managing initiators  https://review.openstack.org/17737218:54
lifeless05:27 < clarkb> mordred: vision of the future iirc, lifeless had a ml response explaining why18:54
lifeless05:27 < mordred> k18:54
lifeless05:27 < mordred> so a requirements change to add a requirement that is really only intended for python3 is not18:54
lifeless                 happyface18:54
mordredlifeless: gotcha18:55
mtreinishanteaya: heh, a few people have been tripped on it before, especially because we used to install everything from git not that long ago18:55
anteayamtreinish: not that long ago -- last week maybe?18:55
mordredlifeless: was about a proposed patch to requirements adding new mysqlclient library - except the intent was to only use it for python 3 (at least that's what the mailing list came down to)18:55
mordredlifeless: so I was poking at whether I shold -1 broadly - or -1 saying "add markerlib thing"18:56
*** shivharis has quit IRC18:56
lifelessyeah18:56
lifelessso today18:56
mtreinishanteaya: I think it was an early kilo thing, but it might have been earlier, my sense of time gets skewed18:56
lifelessadd to requirements-py3.txt18:56
lifelessbut18:56
*** tsg has quit IRC18:56
anteayamtreinish: ah18:56
lifelesspoint out reallllly clearly that that means that we can't build wheels18:56
lifelessfor that project18:56
anteayamtreinish: yes time moves in strange ways18:56
lifeless[that get published to PyPI]18:56
lifelesswithout infra changes - and it would be better to do the environment marker work than change infra18:57
*** sslypush1nko is now known as sslypushenko18:57
zaroclarkb: hey sorry for late reply.  read your conversation about gearman plugin.18:58
zaroi'm looking at it and trying to debug some more to see if i can validate your theories.18:59
anteayazaro: I've lost track, what needs to happen before Saturday?18:59
*** rockyg has joined #openstack-infra18:59
anteayazaro: is there an etherpad started yet?18:59
openstackgerritEmilien Macchi proposed openstack-infra/system-config: Create rubygems mirror from rubygems.org  https://review.openstack.org/17802619:00
zaroanteaya: https://etherpad.openstack.org/p/gerrit-2.10-upgrade19:00
*** radez is now known as radez_g0n319:01
*** julim has quit IRC19:02
anteayazaro: great thank you19:02
*** whayutin_ is now known as weshay19:03
anteayazaro: what a stunning lavender19:03
pleia2hehe19:03
*** julim has joined #openstack-infra19:04
*** craigbr has quit IRC19:04
*** rbradfor has joined #openstack-infra19:04
anteayaisn't it a great colour?19:04
*** hdd has joined #openstack-infra19:05
pleia2it is19:05
*** yfried has joined #openstack-infra19:07
*** chlong has quit IRC19:07
nikita09clarkb: well...didnt go as expected :(  im getting error about 'obj:' ... full paste is here :  http://fpaste.org/218313/43076639/19:07
*** dboik has joined #openstack-infra19:09
openstackgerritMatthew Treinish proposed openstack/requirements: Remove tempest from projects.txt on stable  https://review.openstack.org/17988719:09
*** sdake_ has joined #openstack-infra19:09
lifelessmtreinish: wat?19:09
clarkbnikita09: I don't think youare passing those vars across19:09
*** sandywalsh_ has quit IRC19:09
clarkbnikita09: I don't know why that is the case but looking at the error that seems to be what is happening?19:10
openstackgerritMatthew Treinish proposed openstack/requirements: Remove tempest from projects.txt on stable  https://review.openstack.org/17988819:10
clarkbnikita09: its saying it doesn't know what obj is but it should realize thats special before that and do the replacement19:10
openstackgerritMatthew Treinish proposed openstack/requirements: Remove tempest from projects.txt on stable  https://review.openstack.org/17988919:10
nikita09clarkb: just to be clear...variable refereces can be made to macros...not just job-templates as the docs say..19:10
nikita09rite?19:10
clarkbnikita09: orrect19:11
mtreinishlifeless: grenade fun, it is g-r syncing stable on tempest master19:11
clarkbnikita09: what version of jjb do you have?19:11
lifelessmtreinish: ah, so -19:11
mtreinishwhich is causing the wrong version of tempest-lib from being installed for the grenade job19:11
nikita09clarkb: 1.1.019:11
lifelessmtreinish: I'm worried about uninstallability happening19:11
lifelessmtreinish: can you you think about someway to prevent that?19:11
openstackgerritMerged openstack-infra/shade: Enhance the OperatorCloud constructor  https://review.openstack.org/17964719:12
mtreinishlifeless: so tempest should be venv isolated in the gate cases I'm thinking of because it should always be a clone followed by tox19:12
clarkbnikita09: ok that version should be new enough according to git logs19:12
mtreinishlifeless: I'm not sure I understand what you mean by uninstallability19:13
mordredclarkb, jeblair, Shrews: https://review.openstack.org/#/c/178886/ - I'd  like to land that shade patch and then cut a release19:13
*** sdake has quit IRC19:13
*** dprince has quit IRC19:13
nikita09clarkb: yes..i have almost latest version... i just want to pass a list to my git scm macro.... i tried !!python/tuple and it also gave same "keyerror obj"   error..:/19:14
*** tiswanso has joined #openstack-infra19:14
*** samueldmq_ has quit IRC19:14
mtreinishlifeless: we wouldn't be able to use branchless tempest on a stable devstack if it wasn't venv isolated19:14
clarkbnikita09: zaro may know more19:15
*** sandywalsh has joined #openstack-infra19:15
mordredthe isolation makes sense to me - tempest is not part of an openstack install - it's a thing taht sits outside of the install and judges it19:16
fungiokay, sort of back now19:16
mordredfungi: https://review.openstack.org/#/c/178886/ if you have a sec - I'd like to get that landed and cut a release19:16
jeblairmordred: ack; i hope to be reviewing after lunch19:17
mordredjeblair: thanks19:17
mtreinishmordred: yeah, although I don't think we want the isolation on master (we do use it for gate runs though) But, having tempest installed systemwide on a devstack install is a better dev story for most19:18
*** mjturek1 has joined #openstack-infra19:18
zaronikita09: are you doing multiple scm or just one?19:18
mordredjeblair, clarkb, fungi, Shrews: also https://review.openstack.org/#/c/175891/ ... those are the last two that need to go in this release19:19
lifelessmtreinish: co-installability, not uninstallability19:19
lifelessmtreinish: with master19:19
nikita09zaro: i have multiple scm macros...but im in my im using just one scm19:19
*** ildikov has quit IRC19:19
lifelessmtreinish: I have a chain of assumptions; let me spell it out19:19
*** mjturek1 has quit IRC19:19
lifelessmtreinish: ops value us having everything co-installable; I assume that that includes tempest19:20
zaroclarkb: are we using jjb from master or the latest release (v 0.5.5)?19:20
*** mjturek1 has joined #openstack-infra19:20
lifelessmtreinish: global-requirements inclusion in projects.txt is how we ensure that co-installability is maintained for a given branch, including master19:20
clarkbzaro: we use master, latest release is 1.2019:20
zaroclarkb: jjb/geard19:20
clarkber 1.2.019:20
*** yfried is now known as yfried|afk19:20
clarkbzaro: do you mean gearman-plugin?19:20
zaronikita09: multiple scm requires a special plugin19:21
lifelessmtreinish: ... and I just noticed your patch is to s/kilo. :)19:21
zaronikita09: do you have that?19:21
zaroclarkb: no python gear19:21
clarkbzaro: nikita09's error appears to be in jjb not jenkins19:21
zaroclarkb: the server19:21
mtreinishlifeless: oh, heh. Yeah it;s just for stable :)19:21
clarkbzaro: python gear we install from latest release19:21
*** emagana has quit IRC19:21
nikita09zaro: WHICH plugin?? jenkins multiple SCM plugin??19:21
nikita09zaro: i have that jenkins plugin...19:22
zaroclarkb: ok. i was testing with v 5.519:22
*** ZZelle_ has joined #openstack-infra19:22
*** salv-orlando has joined #openstack-infra19:22
nikita09zaro: i also tested passing multiple branches to git scm macro...got same 'keyerror obj' ... :/    i -passed like this  >>  branch: !!python/list ['branch1',branch2','branch3']   and it git scm >>> branches : '{obj:branch}'19:23
openstackgerritMerged openstack/requirements: Bump requests version support  https://review.openstack.org/17329019:24
openstackgerritMerged openstack/requirements: update openstackclient requirement to 1.0.3  https://review.openstack.org/17733719:25
mtreinishjeblair: if you get a sec: https://review.openstack.org/175008 should be super simple :)19:26
*** e0ne has quit IRC19:26
*** claudiub has quit IRC19:26
clarkbmordred: it seems really weird that the flag to use tasks for image uploads is a cloud flag19:26
clarkbmordred: but thats because rax glance v2 is not glance v219:27
clarkbmordred: right?19:27
Shrewsmordred: oh, this can merge now   https://review.openstack.org/16867319:27
Shrewserr19:27
Shrewshttps://review.openstack.org/168672, rather19:28
nikita09zaro: the thing is, JJB git SCM doesnt understand what 'obj' is ...i see thats a feature described for project templates, but not sure if its also supposed to work for git scm too....19:28
Shrewsmordred: though, hrm... guess we need swiftclient to make another release, so maybe not19:29
clarkbnikita09: I am pretty sure that we use the same substitution method on all the things, but it is possible there is a big19:29
lifelessclarkb: no, its because each v2 cloud can choose to run the tasks stuff or not19:29
clarkber s/big/bug/19:29
*** _nadya_ has joined #openstack-infra19:29
clarkblifeless: then wtf are api versions for?19:29
*** alexpilotti has quit IRC19:29
lifelessclarkb: [also rax isn't quite standard, but thats actually a distraction here]19:29
mordredclarkb: well, yes and no19:29
lifelessclarkb: confusing users.19:29
clarkblifeless: awesome19:29
zaronikita09: git001.yaml test here: http://git.openstack.org/cgit/openstack-infra/jenkins-job-builder/tree/tests/scm/fixtures has config for multiple branches.19:29
mordredclarkb: it's a feature that takes a priori knowledge that cannot be inferred from the keystone catalog19:29
zaronikita09: does the corresponding xml look correct to you?19:30
clarkbmordred: because it is not reflected in either the catalog of glance api version?19:30
mordredclarkb: so, the API for tasks that rackspace uses is the v2 tasks api19:30
mordredclarkb: that's right19:30
clarkbmordred: can we go back and -2 this glance stuff and make them delete v2 and tasks?19:30
mordredclarkb: so - there are two axes19:30
mordredclarkb: well, v2 isn't a problem19:30
*** e0ne has joined #openstack-infra19:30
mordredclarkb: PUT works in v219:30
clarkbmordred: then have them make a v3 that works?19:30
mordredthere is "is this v1 or v2"19:30
clarkbmordred: sure except that v2 doesn't have a way of telling you to use tasks or not19:30
mordredand there is "must you use tasks"19:30
lifelessclarkb: I feel your pain, but thats just going to be worse still19:30
clarkbmordred: and if the are optional but sometimes required the api should tell you19:30
mordredclarkb: well, the "must use tasks" is a deployment policy decision at rax19:30
zaronikita09: does that work in a macro?19:30
clarkbmordred: either via a capabilities requests or via making it required19:31
mordredclarkb: AIUI, what they did is reduce the available quota for PUT calls to 019:31
mordredor something19:31
* mordred waves hands19:31
nikita09clarkb: can you please link me to an example job config where you guys use same list parameters passed ? using obj:key??19:31
clarkbmordred: sure all I am suggesting is that it is glance's job to reflect this to the user19:31
mordredclarkb: 100% agree19:31
clarkbnikita09: we have none, we never use them19:31
mordredclarkb: I jjust think it's an even bigger problem19:31
mordredclarkb: because the same thign is potentially the case with every single thing that an operator can set in policy.json19:31
clarkbmordred: fun19:32
mordredclarkb: this just happens to be one that one of our deployers did set and we hit it and learned about it19:32
mordredclarkb: so, I'd argue that policy.json itself should be shot in the face19:32
clarkbso policy.json can affect the functionality of the api wihtout having any way to reflect such things to the user before they use the api19:32
*** ildikov has joined #openstack-infra19:32
mordredyes19:32
clarkbor at least before they use the api to interrogate functionality19:32
clarkbthats insane19:32
mordredand that exists as an idea across all of openstack19:32
mordred"deployer freedom" man19:32
zaronikita09: http://git.openstack.org/cgit/openstack-infra/jenkins-job-builder/tree/tests/yamlparser/fixtures/custom_distri.yaml19:32
*** salv-orlando has quit IRC19:33
zaronikita09: and http://git.openstack.org/cgit/openstack-infra/jenkins-job-builder/tree/tests/yamlparser/fixtures/expand-yaml-for-template-job/dimensionality-test001.yaml19:33
nikita09zaro: thanks..yes i see same example in docs..but thats referring to project template...i was wondering if there is similar for git scm..19:33
*** mjturek2 has joined #openstack-infra19:34
clarkbmordred: my only suggestion for shade then is to somehow distinguish between glance tasks and shade tasks19:34
*** e0ne has quit IRC19:35
zaronikita09: no, unfortunately no test for that.  so probably not supported in scm19:35
clarkbmordred: the docstring is ambiguous as to which task is in use here19:35
nikita09zaro: i have tried passing list of branches in both ways... using []  and using @branch19:35
nikita09- branch 1   - branch 2 - branch319:35
clarkbmordred: left a comment19:36
nikita09zaro: both times it says 'keyerror: obj' so it means this feature is not enabled for git scm ..right?19:36
zaronikita09: doesn't the git001.yaml do that?19:36
*** mjturek1 has quit IRC19:36
zaronikita09: i see a list of branches getting passed to git scm19:37
*** e0ne has joined #openstack-infra19:37
*** jgrimm is now known as zz_jgrimm19:37
nikita09zaro: git001.yaml is not a macro...19:38
*** mjturek2 has quit IRC19:38
*** e0ne has quit IRC19:39
*** e0ne has joined #openstack-infra19:39
nikita09zaro: i have macro of git scm just like git001.yaml but in that, under `branches` , i passed a list of branches from my jobs... so branches : '{obj:mybranchlist}'  and in my job ,  mybranchlist: !!python/list [a,b,c ,]  etc etc19:39
anteayamtreinish: tempest is being invoked here: https://etherpad.openstack.org/p/YVR-ops-user-committee19:40
*** luqas has joined #openstack-infra19:40
anteayamtreinish: you may want to climb in19:40
nikita09but it doesnt get what obj is .....19:40
*** yfried|afk is now known as yfried19:40
mtreinishanteaya: I support both of those bullet points19:40
anteayamtreinish: tim bell may appreciate your support19:41
*** e0ne has quit IRC19:41
mtreinishthose are things I've been pushing for some time, but never got volunteers before19:41
zaronikita09: ahh, sorry i misunderstood your use case.19:41
anteayanext time he reads that etherpad19:41
anteayamtreinish: good to know, I didn't know that19:41
nikita09zaro: np :)  is it fine if i report a bug on storyboard for it ??19:42
anteayamtreinish: I wonder how many user committee folks know your thoughts19:42
mtreinishanteaya: should I just +1 it in the etherpad? Sorry I don't know the contxt of it19:42
*** e0ne has joined #openstack-infra19:42
*** mjturek1 has joined #openstack-infra19:42
anteayathis is tim bells etherpad19:42
anteayasorry let me get context for you19:42
zaronikita09: yes, please do.19:42
*** e0ne has quit IRC19:42
pabelangerAnybody know of a yaml schema validator?19:43
pabelanger... for python19:43
anteayamtreinish: context: http://lists.openstack.org/pipermail/openstack-dev/2015-May/063102.html19:43
*** e0ne has joined #openstack-infra19:43
clarkbpabelanger: voluptuous, zuul uses it19:43
*** e0ne has quit IRC19:43
anteayamtreinish: this is the who can vote for tc thread, and this is tim saying he is trying to organize the user committee better via this etherpad19:43
pabelangerclarkb, Ah, yes. I knew that19:44
pabelangerthanks19:44
mtreinishanteaya: oh heh, no wonder I missed it, it was part of that thread I was keeping away from19:44
anteayamtreinish: via, using in support of19:44
anteayamtreinish: ah sorry to drag you in19:44
anteayamtreinish: but tempest19:44
anteayamtreinish: and things you want to support19:44
mtreinishhehe, no worries, I don't need to chime in on the thread, just comment on the etherpad :)19:44
*** e0ne has joined #openstack-infra19:44
*** luqas has quit IRC19:45
*** sushilkm has joined #openstack-infra19:45
*** e0ne has quit IRC19:45
*** sushilkm has left #openstack-infra19:45
anteayamtreinish: yes I concur19:47
mtreinishanteaya: ok I left some comments19:47
nikita09well...this also seems like a valid bug in jjb.. using awk statements that have {} , jjb thinks this this mapping to something.. liike awk '{print $1}' , jjb thinks 'print $1' is key in our macro :) lol  https://storyboard.openstack.org/#!/story/200018019:48
*** emagana has joined #openstack-infra19:48
*** tiswanso has quit IRC19:48
mtreinishif the blinding red didn't give that away :)19:48
anteayamtreinish: thank you19:48
anteayamtreinish: ha ha ha19:48
*** unicell has quit IRC19:49
anteayaI always have to look for my sunglasses when you speak up19:49
anteaya:)19:49
anteayanot sure what they would want in terms of a ci here19:49
jeblairnikita09: should work if you double them "{{foo}}"19:50
*** yfried is now known as yfried|afk19:50
mtreinishanteaya: honestly we probably could leverage infra to just have a job which installs tempest somewhere and configures it to talk to public facing cloud X19:50
mtreinishI know that's something that came up in a summit session at some point in the past19:51
*** emagana_ has joined #openstack-infra19:51
mtreinishI think fungi or clarkb proposed just running a periodic job to have tempest run against infra's rax account19:51
mtreinish(which it turns out would fail miserably :) )19:51
anteayaawesome19:52
*** emagana has quit IRC19:52
anteayawell I'm more at the let's keep discussing phase than let's decide on the solution, but that might be an interesting solution19:52
anteayamtreinish: any thoughts on where I might dig up the etherpad that references said idea?19:53
mtreinishI'm not sure it would be in etherpad notes, iirc it was just a random comment at a session in HK (but super hazy)19:54
fungimtreinish: yeah, you'd want a dedicated tenant for that, and make sure the job was nonconcurrent with other jobs using the same tenant probably19:54
anteayamtreinish: can you be a bit more vague please19:54
mtreinishit might have been portland too19:54
anteayamtreinish: way too much detail there19:54
mtreinishanteaya: ^^^ :)19:54
anteayamtreinish: there we go, thanks19:54
anteayaspread it a little thinner19:54
openstackgerritJoshua Harlow proposed openstack-infra/release-tools: Include 'source_url' in notes and use readme parser function  https://review.openstack.org/17652819:54
anteayafungi: have we the ability to dedicate a tenant now, if we should want to?19:55
mtreinishfungi: sure, we probably would actually need multiple separate tenants to do it right (so we could run in parallel)19:55
fungianteaya: we'd have to request one19:56
* mtreinish remembers he needs to kill his rax node running his blog and close his account as it's no longer free19:57
anteayafungi: okay thanks, not jumping the gun, just trying to understand what is required should this route be selected19:57
fungianteaya: not looking at the etherpad, i would infer this is for the benefit of confirming that tempest testing some real cloud returns sane results (to validate tempest's operation? to validate the cloud it's hitting? both?)19:57
mtreinishmikal: ^^^ unless you want to fix that for me :)19:57
anteayafungi: both19:57
*** e0ne has joined #openstack-infra19:57
mtreinishfungi: yeah, both19:57
*** _nadya_ has quit IRC19:57
anteayafungi: the way it is worded to verify the cloud code does as expected19:57
fungianteaya: also once we have an infra-cloud, this might be waaaaay easier to organize19:58
anteayabut tempest code would be evaluated too as a result19:58
anteayafungi: ohhhh19:58
clarkbmtreinish: oh ya I need to do that too19:58
anteayafungi: might we be open to trying to recruite some ops folks to help with infra-cloud?19:58
fungibut yeah, validating our infra-cloud, while something we probably should do, doesn't necessarily help rackspace. that's where stuff like refstack is aiming to fill the gap i think19:58
anteayashould I seed the etherpad with that idea?19:58
*** wenlock has quit IRC19:59
anteayafungi: funny so far none of the ops folks have mentioned refstack19:59
*** dboik has quit IRC19:59
fungii am unsure they're even aware19:59
*** mriedem has quit IRC19:59
fungithat's probably one of those places where we're getting too siloed and people are working in parallel toward similar goals without even knowing about each other19:59
*** dboik has joined #openstack-infra19:59
* mtreinish probably should try to be nice and not say anything19:59
openstackgerritMerged openstack-dev/pbr: Stop re-invoking pip  https://review.openstack.org/17536920:00
*** jcoufal has quit IRC20:00
anteayamtreinish: good strategy20:00
anteayafungi: possibly20:00
fungimtreinish: all i'll say is that at least one if not both groups might benefit from some additional help from outside their silo20:00
jeblairfungi: ++20:00
anteayafungi: that is a diplomatic thing to say20:00
anteayaand I can agree with it20:01
mtreinishfungi: heh, I wasn't taking issue with that. That's a good statement20:01
*** dboik_ has joined #openstack-infra20:02
*** claudiub has joined #openstack-infra20:03
*** wenlock1 has quit IRC20:04
*** dboik has quit IRC20:05
*** SumitNaiksatam has quit IRC20:07
* tchaypo joins the conversation hours late20:07
tchayponibalizer: you mentioned “adding zuul-cloner” but I couldn’t figure out what you were talking about adding it to20:08
*** subscope_ has quit IRC20:08
clarkbtchaypo: the openstack puppet module beaker tests20:08
clarkbtchaypo: right now they only test the current change, there is no integration testing or cogating going on20:08
tchaypoand the concern is that adding it to the tests makes it hard to run outside infra?20:09
*** Somay has joined #openstack-infra20:10
nibalizertchaypo: basically20:10
nibalizertchaypo: tests on a local workstation should do the right thing20:10
*** doude has quit IRC20:10
*** doude has joined #openstack-infra20:10
tchaypowhere “right thing” means “same tests as CI will do”?20:10
greghaynesfungi: Hey, any idea where the change to add https://zuul.openstack.org/status/change/ is?20:11
*** otter768 has joined #openstack-infra20:11
clarkbtchaypo: yes, but using vagrant20:11
fungigreghaynes: i +2'd it, i think it needs a second set of eyes20:11
clarkbtchaypo: since locally the expectation from beaker is to spin up VMs20:11
*** SumitNaiksatam has joined #openstack-infra20:12
clarkbmordred: reading https://review.openstack.org/#/c/175891/24/shade/cmd/inventory.py shade inventory is an ini file, why would we output yaml and json but not ini?20:12
clarkbShrews: ^ maybe you know?20:12
greghaynesclarkb: inventory plugin20:12
mordredclarkb: 2 things20:12
mordredclarkb: that's not an ansible inventory plugin20:12
mordredclarkb: that's a script that a user can use to get a printout of what's in their clouds without necessarily seeing the extra ansible things like group info20:13
fungigreghaynes: oh, it's https://review.openstack.org/179764 and merged several hours ago20:13
mordredclarkb: I wind up looking at printouts of my ansible inventory a lot of times to just see somethinigs about the data20:13
greghaynesfungi: ah! that would explain why my digging through open reviews isnt finding it20:13
mordredclarkb: second - a dynamic ansible inventory is not ini - it is a program that outputs json20:13
*** radez_g0n3 is now known as radez20:14
mordredclarkb: so a program that outputs inventory in an ini format is not a thing that anything would consume20:14
clarkbmordred: can I take that json and pass it to ansible -i ?20:14
mordredclarkb: that's not how the mechanism works, no20:14
fungigreghaynes: working example https://zuul.openstack.org/status/change/179706,420:14
mordredclarkb: what you do is you pass the location of teh script to ansible -i20:14
clarkbmordred: then I think you need to also output ini20:14
mordredclarkb: no20:15
clarkbmordred: hold on20:15
*** cloudnull has quit IRC20:15
fungigreghaynes: that's a requirements change in the gate right now20:15
mordredclarkb: you tell ansible what the script is, and ansible runs it for you20:15
greghaynesfungi: ahoy20:15
mordredclarkb: and the script outputs json20:15
clarkbmordred: yes I get that20:15
clarkbmordred: BUT20:15
clarkbmordred: what if I want to edit the file first? I don't want to write a new script, just take output, make some changes then pass it back in20:15
clarkbmordred: its just really weird to me that the interface here is inconsistent20:15
mordredclarkb: this script is not an ansible inventory20:15
*** otter768 has quit IRC20:16
Shrewsthis is *dynamic* inventory20:16
clarkbmordred: except you just said I could pass it to ansible for invetory purposes20:16
mordredthis is not a tool for generating static inventories20:16
mordredclarkb: sorry - what I meant was20:16
mordredyou can pass an ansible dynamic inventory script to ansible20:16
mordredthis is not an ansible inventory script20:16
clarkbok, then what is it?20:16
mordredthis is a program to produce some information about your clouds for whatever purpose you might find useful20:16
*** rockyg has quit IRC20:17
mordredI personally use it to debug looking at all of my cloud info20:17
clarkband one of those useful things is not giving the data back to ansible?20:17
Shrewsclarkb: http://docs.ansible.com/intro_dynamic_inventory.html20:17
mordredno20:17
clarkbok20:17
mordredbeause I havea n ansible dynamic inventory plugin that uses the same underlying code as this, but does othe rthings too20:17
mordredlike add group information and ansible_ssh_hostname varaibles20:17
rcarrill`i use the ansible inventory to target plays against servers that have a certain nova metadata20:17
tchaypoI think http://docs.ansible.com/developing_inventory.html is the better link for explaining what a dynamic inventory script looks like20:17
rcarrill`it's pretty cool20:17
*** rcarrill` is now known as rcarrillocruz20:17
mordredyup20:17
clarkbso I understand whay a dynamic inventory script is20:18
*** cloudkiller has joined #openstack-infra20:18
clarkbI understand it outputs jsob20:18
clarkbI also understand that ansible -i can take an ini input file20:18
clarkbso an inventory script should support that too imo20:18
clarkbbut if this isn't an inventory script then whatever20:18
lifelessjsob?20:18
clarkbsorry json20:18
lifelessdo you men json or bson...20:18
lifelessthanks20:18
mordredlifeless: json20:19
greghaynesjsob is what you do when you pass json via command line arguments20:19
fungiboson20:19
fungiawesome20:19
clarkbgreghaynes: ++20:19
mordredclarkb: I could see the thing you're saying - but it's not really a thing with ansible inventories so far20:19
mordredclarkb: I'd be happy though to add that as a feature to the openstack one20:19
mordredclarkb: if you thnk it's a thing you'd find useful20:19
clarkbmordred: sure it is, I run command get ini file. Now I can do what I want with that and pass it to -i20:19
mordredclarkb: since it would not be ahrd20:19
mordredclarkb: no, what I'm saing is that that's not how people in ansible world do this20:19
mordredand none of the current ansible tools do it20:19
mordredhowever, it's an interesting idea20:20
greghaynesclarkb: im curious why you would want to do that?20:20
* fungi is going to create a bunch of json-like serialization languages named after subatomic particles20:20
mordredand easy to add20:20
clarkbmordred: no, instead they reinvent the world and write a NEW script to do dynamic inventory20:20
*** doug-fish has joined #openstack-infra20:20
mordredwell, no - there's no need to do that20:20
mordredyou can have multiple inventoies20:20
sdaguefungi: hmmm... so when I try to urlopen from python https://zuul.openstack.org/status/change/179706,420:20
rcarrillocruzyeah20:20
clarkbgreghaynes: first thing that comes to mind is to be extra sure that any restricted roll outs are restricted20:20
mordredso if you want to suppliment the openstack dynamic inventory20:20
rcarrillocruzyou point to a folder containing several inventories20:20
clarkbgreghaynes: ansible can't touch a host if it doesn't know about it20:20
mordredyou just put some more static inventory next to it20:20
sdagueI get an SSL error20:20
sdagueURLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>20:20
clarkbgreghaynes: then I don't have to worry about bugs in my inventory20:20
mordredclarkb: see what I just said ^^20:20
mordredclarkb: you make your inventory point to a dir - one of the things in the dir is a script, the other things are static files20:21
clarkbmordred: thats additive not restricting nodes20:21
greghaynesclarkb: you can also do -l to limit your inventory ontop20:21
greghayneser, limit ontop your inventory20:21
clarkbmordred: so I am not sure how I would say "make sure this node never shows up when I run the new scary thing"20:21
greghaynes-l !host or somesuch20:21
rcarrillocruzyou can exclude in the 'hosts' section20:21
mordredclarkb: so - I understand your general usecase, but I don't have a concrete example of it in mind, so it's hard to conjecture about how it might be solved for real... that said, I think adding a thing that outputs some ini would be pretty easy to do20:22
clarkbgreghaynes: rcarrillocruz ok did not know bout -l20:22
mordredand I'm happy to do it20:23
greghaynes:) its nifty20:23
openstackgerritDoug Hellmann proposed openstack/requirements: Remove lockfile from requirements  https://review.openstack.org/15122420:23
mordredif anything - it could be used for debugging purposes20:23
mordred"give me a static ini-style output I can futz with"20:23
fungisdague: i'll take a look at why my browser is cool with it but other tools might not be20:23
fungisdague: it's possible the chain cert there is wrongish or something20:23
*** cloudkiller has quit IRC20:23
*** sigmavirus24 has quit IRC20:23
*** d34dh0r53 has quit IRC20:23
mordredclarkb: also - fwiw, you can do static file ansible inventories in json and yaml too20:24
openstackgerritMichael Krotscheck proposed openstack/requirements: Updated oslo_middleware to 1.2.0  https://review.openstack.org/17992620:24
mordredclarkb: they do not have to be ini20:24
*** hockeynut has quit IRC20:24
clarkbmordred: so I have to say this is where the docs break down20:24
clarkbmordred: everything I have read is ini20:24
mordredI believe they make all the docs ini because it's the easiest thing for people to do20:24
*** d34dh0r53 has joined #openstack-infra20:24
*** hockeynut has joined #openstack-infra20:24
clarkband honestly yaml is probably easier to read and I would likely just use that if I knew I could20:24
mordredI'm 99% sure you can do that20:24
mordredI _know_ you can for variables files20:24
*** cloudkiller has joined #openstack-infra20:25
mordredand for local facts20:25
tchayponibalizer: do you have specific concerns about zuul_cloner, or is it just the complexity?20:25
mordredclarkb: I'll verify and get back to you20:25
sdaguefungi: so I can force this back to http for my testing proxy, I'm just surprised20:25
clarkbtchaypo: I think its "how do we make beaker and zuul_cloner play nice"20:25
*** sigmavirus24 has joined #openstack-infra20:25
mordredclarkb: and yes- much docs hacking is no my TDL20:25
clarkbtchaypo: importantly beaker should not run zuul-cloner imo20:25
clarkbtchaypo: because that adds a weird dependency, Instead beaker should say "give me the thngs to use else use sane default" basically what devstack does20:26
*** tiswanso has joined #openstack-infra20:26
mordred++20:26
fungisdague: yeah, i believe crd doesn't kick in when you're supplying local javascript in the debugger console or something20:26
mordredI totally agree with clarkb there ...20:26
tchaypomaybe I should step back and ask what problem zuul_cloner could be used to solve20:26
mordred"if I made one for you, use it, otherwise, do this"20:26
clarkbtchaypo: it solves the problem of what should be tested20:26
*** e0ne has quit IRC20:26
clarkbtchaypo: currently beaker doesn't really solve that at all and also doesn't give you a good interface to communicate your decisions to it20:27
mordredtchaypo: I think clarkb is agreeing with you - that we shoudl not teach beaker about zuul cloner20:27
sdaguefungi: crd?20:27
mordredbut instead, should teach beaker how to deal with repos that have already been managed by zuul cloner (or anything else)20:27
mordredyeah?20:27
clarkbmordred: ya20:27
mordredand I agree with that20:27
clarkband from what I am reading that is basically going to look like a hook to beaker the s_h_a thing that would do the correct thing for us and not beaker itself20:27
mordredthat way the scripts run by beaker can look like things that want to clone and work with local things like people expect them to be20:27
mordredwhen they're run by $person on $laptop20:28
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: Do not automatically target bugs for stable releases  https://review.openstack.org/17944720:28
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: Add list_oslo_unreleased_changes.sh  https://review.openstack.org/17939620:28
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: Print release notes as part of releasing the library  https://review.openstack.org/17944520:28
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: Move get_last_tag to functions  https://review.openstack.org/17944420:28
*** dustins has quit IRC20:29
tchaypoas in, use these three local repos for these modules, but get HEAD for everything else?20:29
*** ociuhandu has quit IRC20:29
*** dustins has joined #openstack-infra20:29
*** Sukhdev has quit IRC20:29
sdaguefungi: is there anything special about zuul's cert?20:30
*** e0ne has joined #openstack-infra20:30
clarkbtchaypo: that would be one way, I think its probably easier to do all local repos or use HEAD for everything though20:30
clarkbtchaypo: that way you get clear failure modes rather than seemingly random workingness20:30
*** jraim has quit IRC20:30
*** boris-42 has quit IRC20:30
*** kun_huang has quit IRC20:30
clarkb"it works for module X but not Y, what is going on?:20:30
*** cyeoh has quit IRC20:31
*** apmelton has quit IRC20:31
*** Trozz has quit IRC20:31
*** dboik_ has quit IRC20:31
*** gondoi has quit IRC20:31
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: update make_library_stable_branch.sh to set defaultbranch  https://review.openstack.org/17391920:31
*** tiswanso has quit IRC20:31
*** Trozz has joined #openstack-infra20:31
*** Nakato has quit IRC20:32
*** tiswanso has joined #openstack-infra20:32
clarkbmordred: last inventory question, shouldn't the --list and --host options imply no print(output_format_dict)?20:32
*** persia has quit IRC20:32
*** cloudkiller is now known as cloudnull20:32
clarkboh wait I see, thoseare passed to output_format_dict and it renders them and inventory depending, nvm20:32
openstackgerritMonty Taylor proposed openstack-infra/shade: Switch tasks vs put on a boolean config flag  https://review.openstack.org/17888620:32
*** Nakato has joined #openstack-infra20:33
mordredclarkb: ^^ fixed your comment - good call on the confusing20:33
clarkbmordred: I just read it and its "use task manager, use tasks for uploads"20:33
*** hockeynut has quit IRC20:33
*** jraim has joined #openstack-infra20:33
clarkband went wait a minute20:33
*** fungi has quit IRC20:33
*** persia has joined #openstack-infra20:34
*** persia has quit IRC20:34
*** persia has joined #openstack-infra20:34
*** jvrbanac has quit IRC20:34
*** apmelton has joined #openstack-infra20:34
*** hockeynut has joined #openstack-infra20:34
*** unicell has joined #openstack-infra20:34
*** julim has quit IRC20:34
*** tchaypo has quit IRC20:34
*** cyeoh has joined #openstack-infra20:34
*** d34dh0r53 has quit IRC20:34
*** sigmavirus24 has quit IRC20:34
*** gondoi has joined #openstack-infra20:34
*** dboik has joined #openstack-infra20:34
*** kun_huang has joined #openstack-infra20:34
*** jvrbanac has joined #openstack-infra20:35
*** d34dh0r53 has joined #openstack-infra20:35
clarkbmordred: actually one more question :) why are you using flamel? yaml.dump(thing, default_flow_style=False) should work too right?20:35
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: Add script to update .gitreview files  https://review.openstack.org/17459420:35
*** fungi has joined #openstack-infra20:35
*** sigmavirus24 has joined #openstack-infra20:35
*** boris-42 has joined #openstack-infra20:36
mordredclarkb: flamel does a few other things that are important that I did not want to cargo cult in - most importantly it manages unicode representaiton and also knows how to deal with ordereddicts20:36
*** tchaypo has joined #openstack-infra20:37
clarkbok20:37
mordredclarkb: granted, it's only 14 lines of python - but it's in a library, so why not :)20:37
tchaypowoo, that was fun20:37
clarkbout of curiousity what unicode things need to be done? it should all be utf820:37
mordred    def unicode_representer(dumper, uni):20:37
clarkband pyyaml will handle ^ for you20:37
mordred        node = yaml.ScalarNode(tag=u'tag:yaml.org,2002:str', value=uni)20:37
mordred        return node20:37
tchaypoI love it when I randomly miss 10 minutes or irc in the middle of conversation20:37
clarkbmordred: I am not sure that is necessary20:38
clarkbmordred: but ok20:38
mordredclarkb: I believe I hit it - it's possible it isnt' - I'll go back and figure it out20:38
clarkbyaml strings are always utf820:38
clarkbunless you do other crazness which we arne't20:38
mtreinishlifeless: fwiw, I use arch linux on most of my machines and pbr works fine :)20:39
mordredclarkb: these strings don't necessarily always come from a sane place20:39
fungisdague: yep, wrong/incorrect chain... fixing now20:39
clarkbmordred: hrm how do you handle ordered dict? either you serialize to regular dict and lose ordering or you serialize a python object which yaml should already know how to do20:40
*** fawadkhaliq has joined #openstack-infra20:40
*** dizquierdo has joined #openstack-infra20:41
clarkb(or I suppose some third representation that only a special yaml loader can load)20:41
openstackgerritRamy Asselin proposed openstack-infra/puppet-openstackci: Create a log server like http://logs.openstack.org  https://review.openstack.org/16742520:41
mordredclarkb: the first - serialize to a regular dict and lose ordering20:41
mordredclarkb: if you serilize to a representaiton, then you could only load back into python with pyyaml20:41
clarkbmordred: correct, but the alternative is to lose ordering which is supposedly important20:42
mordredwell, you don't actually lose ordering it turns out20:42
clarkbboth seem bad20:42
mordredin the flamel case20:42
clarkbmordred: you should, python hashes starting with python3 (and any use of tox) have random seeds20:42
mordredright. so - this is mainly to keep things human readable - not to keep actual important data an ordered fashion20:43
*** zz_jgrimm has quit IRC20:43
mordredso far it's working to output the stuff in a mostly sane order -  i"m not 100% how that's working20:43
openstackgerritRamy Asselin proposed openstack-infra/puppet-openstackci: Simple script to setup and test log server  https://review.openstack.org/16911720:44
mordredbut in flamel's case, it means when you read the json in and then write it back out as yaml, the keys are in pretty much teh same arragement20:44
mordredand vice versa20:44
openstackgerritSean Dague proposed openstack-infra/system-config: use optimized url for zuul status  https://review.openstack.org/17093520:44
openstackgerritSean Dague proposed openstack-infra/system-config: add dbg function to hideci  https://review.openstack.org/17957320:44
openstackgerritSean Dague proposed openstack-infra/system-config: add jquery visibility  https://review.openstack.org/17936220:44
openstackgerritSean Dague proposed openstack-infra/system-config: create proxy for testing hideci  https://review.openstack.org/17936020:44
openstackgerritSean Dague proposed openstack-infra/system-config: Revert "Revert zuul status in gerrit"  https://review.openstack.org/17936120:44
*** maurosr has quit IRC20:44
sdaguefungi: cool, thank you20:44
sdaguegreghaynes: so... I think there isn't a sufficiently better interface for dbg atm given the contraints of js20:44
sdagueso how do you feel about just living with the slightly wonky interface for now, which is at least useful20:45
clarkbmordred: it looks like flamel also ships a loader that preserves order, but it does so by assuming all dicts are ordered from what I can tell20:45
clarkbmordred: either way we lose some data20:45
*** e0ne has quit IRC20:45
*** sdake has joined #openstack-infra20:45
sdaguefungi: also, in this go around of all these patches I put all the zuul embedded logic behind a kill switch - https://review.openstack.org/#/c/179361/6/modules/openstack_project/files/gerrit/hideci.js,cm20:46
*** maurosr has joined #openstack-infra20:46
sdagueso flipping it off is a single global20:46
greghaynessdague: oh, my specific suggestion was to use console.log(arguments[0]); and start the for loop at 1 and do console.dir(arguments[i]);20:46
sdagueif it's wrong20:46
*** markvan__ is now known as markvan20:46
sdaguegreghaynes: ok, what does that buy us?20:46
*** zz_jgrimm has joined #openstack-infra20:46
mordredclarkb: yah. in this case I mainly wanted to make sure that we weren't outputting unreadable pyyaml unicode tags - I don't think the ordereing really comes in to play20:46
mordredat least not for the inventory output thing20:46
clarkbmordred: ok20:46
nibalizertchaypo: I think we should use zuul_cloner20:46
greghaynessdague: youll always get a nagivable object in the case youre using - using console.log doesnt necessarially give you that20:46
sdaguebecause console.log(obj) does a useful thing, at least in chrome20:47
nibalizerbut I want to advocate early for ensuring a happy path forward for local development20:47
sdagueoh, it's always doing it here20:47
mordredclarkb: but you're right - I'll go verify a few of those things because I'm curious now20:47
*** hasharAway is now known as hashar20:47
greghaynessdague: yes, apparently that depends on argument type and browser20:47
sdaguewell, all of the interface does :)20:47
sdagueconsole is kind of a ghetto interface that no one standardized20:47
greghaynessdague: yes, supposedly .dir is consistent about giving you an object though. Im not hugely bent on it I just know ive run into a lot of cases of "huh, why am I all of a sudden not getting an object"20:48
*** samueldmq_ has joined #openstack-infra20:48
*** amitgandhinz has quit IRC20:48
sdagueok20:49
greghaynessdague: not sure if you saw https://review.openstack.org/#/c/170935/ ye20:49
*** pabelanger has quit IRC20:49
*** sdake_ has quit IRC20:49
greghaynessdague: my brain hurt reasoning that one out, so let me know if it makes sense20:49
*** mrmartin has quit IRC20:50
jogogreghaynes: FYI one of your wheel patches needs a rebase https://review.openstack.org/#/c/164927/20:50
sdagueso... yeh, the challenge is the following....20:50
tchayponibalizer: why do you think we should use it?20:51
greghaynesjogo: oy20:51
greghaynesjogo: again :(20:51
sdaguewe trigger a timer loop for zuul polling based on the current url20:51
jogogreghaynes: yeah :(20:51
sdaguewhich self propogates, because javascript doesn't have threads20:51
sdagueif you navigate to a different url20:51
*** sdake_ has joined #openstack-infra20:52
sdaguewhen the current request cycles back around it needs to realize "oh, we're not there any more, stop"20:52
sdagueif you have a better suggestion on how to structure that code, please pastebin or post a solution20:52
sdaguebecause... thems as good as I got20:52
greghaynessdague: ah. So that part is desired. What about the visibility interaction though?20:52
greghaynesI didnt realize that was a feature though ;)20:52
sdagueyou are right, we want to restart that on a return to visible20:53
sdagueoh, we do that20:53
sdague#51620:53
sdaguewe just start the top loop again20:54
asselin__nibalizer, I made the change to ::jenkins but I think it will fail b/c it's not merged yet. Let's do these one at a time20:54
zaroclarkb: i'm seeing something odd with gearman.20:54
asselin__nibalizer, so I plan to revert back...let me know your opinion though...20:54
nibalizerasselin__: okay let me review20:54
greghaynessdague: ah! ok. I think that has some race issues then20:54
sdagueyeh, for visibility that was a jeblair requested feature that if the tab goes hidden in the browser that we stop polling20:54
zaroclarkb: i send a job to the server and keep the connection until job ends and that works fine.20:54
sdagueso we aren't adding load for hidden tabs20:54
sdaguegreghaynes: probably :)20:54
greghaynessdague: I think if you flip between tabs before the timeout occurs you end up with two loops20:55
*** fawadkhaliq has quit IRC20:55
openstackgerritJeremy Stanley proposed openstack-infra/puppet-zuul: Correct the variable for the chain cert  https://review.openstack.org/17993520:55
fungisdague: ^ i finally tracked the typo back through all my puppeghetti20:55
sdaguegreghaynes: yeh20:55
*** miqui has quit IRC20:55
zaroclarkb: when i send a job to the server without keeping a connection then the server reponds with NO_JOB20:55
*** salv-orlando has joined #openstack-infra20:55
sdagueit's not the end of the world though20:55
nibalizerasselin__: you mean in line 30 of https://review.openstack.org/#/c/167425/10/manifests/logserver.pp ?20:55
clarkbzaro: ya, if you disconnect from the server then the server assumes you are no longer valid20:55
zaroclarkb: but clearly the job is registered.20:55
*** sdake has quit IRC20:56
asselin__nibalizer, yes20:56
clarkbzaro: and es it doesn't unregister the job20:56
fungisdague: i'm _shockec_ (okay not really) that my browser didn't just asplode all over that error20:56
zaroclarkb: the job doesn't get executed20:56
sdaguefungi: go go SSL?20:56
zaroclarkb: in the past the job would get executed.20:56
nibalizerasselin__: why would that fail? there is a jenkins class in the jenkins module now right?20:56
sdagueyeh, that's not making me feel super excited about security of browsers :)20:56
clarkbzaro: I think the job should stop running if it started20:57
waynrclarkb greghaynes (or anyone) interested in pairing on github/zuul integration work? I am at the point where zuul is sending jobs to gearman based on updates to PRs but i'm not sure why the zuul log doesn't seem to reference job names...maybe it's not supposed to20:57
sdaguegreghaynes: so, do you have a suggestion on how to block that double update?20:57
zaroclarkb: huh?  i expect the job to execute to completion.20:57
zaroclarkb: does zuul keep a connection open for every job it sends to the gearman server?20:58
asselin__nibalizer, there's a jenkinsuser.pp20:58
asselin__which is what I had before...right?20:58
greghaynessdague: let me poke at it a bit, it takes me a little while to reason about that bit of code ;)20:59
clarkbzaro: no, zuul has one connection to the server iirc then the server has one connection per job20:59
*** bswartz has quit IRC20:59
clarkbzaro: if a worker connection goes away then server tells client that job failed20:59
*** spzala has quit IRC20:59
zaroclarkb: ohh yeah.  that makes sense.  I'm referring to connection from client to server.21:00
*** kgiusti has left #openstack-infra21:00
waynrclarkb: do you have any advice for monitoring/inspecting the gearman queue?21:00
mordredwaynr: neat!21:00
asselin__nibalizer, back in a bit....21:00
*** Sukhdev has joined #openstack-infra21:00
* mordred isn't going to be helpful - it purely just waiting for the day that work magically makes his life better21:00
zaroclarkb: client sends job to zuul.  worker sends GRAB_JOB server reponds with NO_JOB21:01
clarkbwaynr: connect to gearman via telent/nc and run status against it to get a full listing21:01
mordredand with that - I'm going to the airport - I'll be back online in a bit21:01
clarkbwaynr: you can also configure it to emit graphite events and track things with graphite21:01
waynrokay i'll give that a shot21:01
zaroclarkb: here's my simple client: https://github.com/zaro0508/gearman-plugin-client/blob/master/gear_client.py21:01
*** luqas has joined #openstack-infra21:02
*** hdd has quit IRC21:02
openstackgerritMerged openstack/requirements: Follow project renames  https://review.openstack.org/17970621:02
waynrclarkb: just FYI, looking at my 'lsof' output it seems like gearman keeps one connection open per worker (rather than one per job)21:02
zaroclarkb: here's the log http://paste.openstack.org/show/214898/21:03
zaroclarkb: this definately used to work. so i'm guessing there is some regression going on.21:03
clarkbwaynr: sorry, ya that is correct21:03
zaroclarkb: i'm not sure how any jobs run because i'm pretty sure zuul sends jobs to gearman server just like this.21:04
fungisdague: looks like the version of firefox i'm running has the ca that signed our zuul cert in its trust ring, and since apache was serving two copies of the server cert (rather than the server cert plus the chain cert) the browser probably just ignored one of them21:04
clarkbzaro: zuul alwys has a persistent connection21:05
*** tonytan4ever has quit IRC21:05
sdaguefungi: gotcha21:05
sdaguehey, slightly off topic, but does anyone have a link to where the ops tracks are listed with time slots?21:05
lifelesssdague: https://libertydesignsummit.sched.org/ ?21:06
lifelesssdague: https://docs.google.com/spreadsheets/d/1VsFdRYGbX5eCde81XDV7TrPBfEC7cgtOFikruYmqbPY/edit#gid=569963128 has rooms but not topics21:06
sdaguelifeless: yep, thank you21:06
*** luqas has quit IRC21:07
clarkbok comments on https://review.openstack.org/#/c/175891/ not sure if any of them are worth a -121:07
sdaguethe sched link seems useful enough21:07
waynrhmm it's not obvious to me how projects specified in layout.yaml get mapped to gearman functions and how gearman knows how many of its workers are capable of running each job21:07
clarkbShrews: mordred SpamapS ^ feedback appreciated21:07
clarkbwaynr: the names must match21:07
clarkbwaynr: basically the name specific in layout.yaml is the gearman function name21:07
zaroclarkb: hmm, ok. more debugging..21:07
lifelesssdague: http://libertydesignsummit.sched.org/type/design+summit/ops#.VUffnN_va00 may be better21:07
*** ldnunes has quit IRC21:08
SpamapSclarkb: I was just reviewing that right now as well.21:08
openstackgerritEmilien Macchi proposed openstack-infra/system-config: Create rubygems mirror from rubygems.org  https://review.openstack.org/17802621:09
waynrclarkb: i see functions that match the names i have specified in my layout.yaml, but I also see functions that exist nowhere in my project.yaml such as "merger:update", and I'm not sure which part of that job name actually needs to exist on my jenkins masters21:09
openstackgerritDarragh Bailey proposed openstack-infra/git-review: Use plumbing rev-parse to get the branch name  https://review.openstack.org/17993721:09
mordredclarkb: I think they're worth fixing. new patch coming soon21:09
EmilienMclarkb, nibalizer: does it make sense if I file a bug for latest beaker not working? I would like to investigate more21:10
openstackgerritSamuel de Medeiros Queiroz proposed openstack-infra/project-config: Experimental Devstack job for Identity v3 API only  https://review.openstack.org/17966121:10
clarkbwaynr: merger:update is a merger function registered by the merger worker21:10
sdaguelifeless: they say Ops: so I was able to get what I wanted out of it21:10
SpamapSmordred: I'm also looking at why it takes 10s... ;)21:10
sdaguethanks again21:10
EmilienMnot working (in OS infra)21:10
clarkbwaynr: you get that regardless of your layout.yaml21:10
SpamapSmordred: should take like 1s at the worst.21:10
lifelesssdague: cool21:10
waynrclarkb: do i need to configure a specific worker as the "merger" worker?21:10
clarkbwaynr: layout.yaml does not register any functions, layout.yaml describes to the gearman client what functions to run on which events21:10
clarkbwaynr: your workers all register the functions for you. In this case likel jenkins and the merger worker21:11
waynroh...so the workers register the functions?21:11
clarkbwaynr: its a separate service you run to merge things. Zuul may start one automagically for you like it starts the geard server though, check ps21:11
waynrokay21:11
clarkbwaynr: yes workers advertise what work can be done, clients select from those lists21:11
waynrso there is a gearman function showing up in my gearmand status that doesn't appear on any of my workers21:12
greghaynessdague: How about just a todo comment about theres a known race here :)21:12
waynrbut it is a name that exists as a result of the interpolation of variables into my zuul project template21:12
greghaynessdague: then I can poke at it in a followup21:13
sdaguegreghaynes: I'm good with that21:13
waynroh wait i'm wrong, there is no way that function comes from any of my layout.yaml sorry21:13
openstackgerritSamuel de Medeiros Queiroz proposed openstack-infra/project-config: Experimental Devstack job for Identity v3 API only  https://review.openstack.org/17966121:14
samueldmqmordred, ^ project config patch is now updated to use DEVSTACK_LOCAL_CONFIG, thanks21:14
*** dustins has quit IRC21:15
*** amitgandhinz has joined #openstack-infra21:15
fungiclarkb: waynr: technicality, but zuul (the scheduled daemon) doesn't start the merger daemon, that's generally a second initscript21:15
waynrclarkb: so zuul will register gearman workers for stuff like "merger:update"21:15
waynroh21:15
fungis/scheduled/scheduler/21:15
mordredsamueldmq: woot!21:16
waynri'm confused right now about where some of these gearman functions are coming from and why zuul doesn't seem to try to run jobs named in my layout.yaml for the particular project I am updating21:16
fungiwaynr: those merger:.* functions are registered by the separate zuul-merger daemon which is running21:16
waynrooohhh21:17
waynrwell i didn't start any zuul-merger daemon just FYI21:17
clarkbright which is why I think zuul may fork one off for you21:18
*** mmedvede has quit IRC21:18
clarkbor run it internally21:18
fungiif it does, that's a new change21:18
fungiwe still start ours from a separate zuul-merger initscript21:18
openstackgerritSean Dague proposed openstack-infra/system-config: use optimized url for zuul status  https://review.openstack.org/17093521:18
sdaguethere you go greghaynes21:19
sdagueI hope you appreciate the comment :)21:19
fungilast i looked the only separate daemon process forked by the zuul scheduler daemon was the internalized geard21:19
clarkbquick grep says we don't fork a different process21:19
*** hdd has joined #openstack-infra21:19
greghayneshaha21:19
*** hdd has quit IRC21:20
waynrclarkb fungi that makes sense, because the merger:update function has 2 queued jobs that don't seem to be leaving the queue21:20
waynrso there is no worker actually accepting those jobs21:21
fungiwaynr: and no workers? you probably need to start the zuul-merger daemon21:21
waynrokay, but i'm not sure i want anything merged by zuul...just want to run tests on jenkins based on PR updates21:21
*** eharney has quit IRC21:22
fungiwaynr: you do want things merged by zuul. the zuul-merger is what serves the merged testing refs to your workers21:22
fungiso they have something to test21:22
fungithe zuul-merger creates your zuul refs21:22
clarkbya github doesnt solve this problem for you21:22
waynroh, i think i see what you're saying21:22
waynrbecause there may be multiple open PRs that could be merged21:23
fungifor example you proposed patch x to branch y, the zuul merger attempts to merge x to a local clone of y and then that gets served as ref z21:23
*** blahdeblah has left #openstack-infra21:23
fungiand the jenkins workers check out that ref to run jobs21:24
nibalizergithub does expose those refs for you, per PR21:24
nibalizerits right there in the api "merge commit ref" or something21:24
*** wenlock has joined #openstack-infra21:24
clarkbnibalizer and those update as masyer moves?21:24
nibalizerdoesn't help at gate time though, I think21:24
nibalizerclarkb: i think so, but :shrug:21:24
nibalizeri know i've never pressed the green button and then github has said 'oops sorry I mean it can't merge'21:25
waynryeah github has refs of the form "refs/pulls/<PR#>"21:25
nibalizerbut none of the repos I merge on are very high velocity21:25
greghaynesnibalizer: but theres also the case of it can merge but tests fail because master moved21:25
*** radez is now known as radez_g0n321:25
clarkbthat doesnt address deps though which is what you mean by gate I think21:25
greghaynesand you just werent testing that rebase21:25
waynrnibalizer: github usually tells you before you try to merge whether or not a change can be merged21:25
waynrsince the merge target is usually a known ref, whenever that ref is updated github will check to see if open PRs can be merged and provide an indication of whether or not it they can on the PR page21:26
waynrs/usually/always/21:27
funginibalizer: are you saying there's a way to have github serve you the result which would be obtained from merging pr x to branch y as ref z without actually also merging x to y in the public repo and serving that to everyone as the new tip of y?21:27
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Split security group list operations  https://review.openstack.org/17994521:27
clarkbfungi ya but it wont do x depends on a because gate21:27
waynrfungi: I think we're saying that github does expose the result of merging x to y in the form of refs/pulls/<PR#>21:27
fungiwell, it might if github prs can have dependencies specified in github, but as i avoid it for the most part i really don't know much about github's more esoteric workflow options21:28
waynri don't think github has the concept of inter-project dependencies21:28
greghaynesI dont think theres a way21:28
fungioh, i see, a way to request an arbitrary series of prs merged in order even when there's no dependency specified in github21:29
*** Krinkle|detached is now known as Krinkle21:29
greghaynesevery time thats come up in a project ive just seen it done manually21:29
clarkbyou cpuld do it with zuul21:29
nibalizerfungi: I use this: http://paste.ubuntu.com/10986825/21:29
clarkband thats why you need mergers so gate works21:30
waynralso I think I have been talking about a simplified single-PR case...in the case of ordering multiple PRs and checked that they are merge/test compatible, that is a different story21:30
fungiyeah, so anyway, those are the things the zuul-merger process does, and i don't think we can replace any of that when using teh github trigger21:30
nibalizerso ill cd into puppet-postgresql, then pr 284 and it will checkout the code of that pull request21:30
waynrso yeah i agree, zuul would need to handle detection of dependencies between PRs21:30
waynrgithubs's notion of PR allows one PR to be a subset of another PR since there can be multiple patches in a given PR, i'm not sure how the commit tree changes once the subset PR is merged into master21:31
fungiwaynr: well, and ordering multiple prs is basically necessary in the gate pipeline because you may approve several at once and want to know that they don't break each other21:31
fungiwaynr: so your simplified solution would be more like the check pipeline and then a no-op gate21:32
fungii.e. independently test the pull requests, hope they don't have negative interactions when you approve them21:32
waynrso it seems like the reason my intended build jobs aren't running then is that the zuul-merger hasn't done its thing...time to play with that now i guess21:34
fungiwaynr: and i doubt that's sufficient, since zuul lets you have more than one trigger for a pipeline, so you might be comingling prs from github with changes in gerrit on a single project21:34
fungiand would need to have them both support a common set of jobs in the same pipelines together21:35
*** jtriley has quit IRC21:35
*** mwagner_lap has quit IRC21:35
waynri think my goal for now is to implement the simplified "either or" case where a project either has "github" trigger or "gerrit" trigger21:36
waynri'm sure it should be possible to specify that kind of mutual exclusion in the layout schema21:36
waynr(to provide users immediate feedback if they try to use both styles of trigger on given pipeline)21:36
waynrunless i am misunderstanding zuul configuration21:37
waynrwhich i probably am21:37
waynrdang it's not even 3 yet21:38
clarkbwaynr: a single project will only ever come from one most likely, but it may be nice to mix projects on github and projects on gerrit together in the same pipeline for cogating. That said I don't think you need to support that off the bat if its more complicated21:38
fungii think your options there are to document the exclusivity of those triggers in a pipeline, have zuul log an error if someone mistakenly adds both, and optionally refuse to load the config21:39
*** dkranz has quit IRC21:39
waynryeah21:39
fungienforcing that exclusivity in the voluptuous validation would probably be possible21:39
waynrthat's what i'm hoping21:40
fungiwhich also makes it easier for someone to run a gate job validating their zuul layout to catch it21:40
*** samueldmq has quit IRC21:40
*** samueldmq_ is now known as samueldmq21:40
*** peristeri has quit IRC21:40
*** pc_m has quit IRC21:41
waynri really prefer gerrit's notion of changes and patchsets over github pr's21:42
clarkb++21:43
pleia2++21:44
*** dizquierdo has quit IRC21:45
mtreinishclarkb, fungi: do you have any thoughts on: https://review.openstack.org/176564 I'm debating doing something like that vs spinning up a separate liberty db21:49
*** rfolco has quit IRC21:51
clarkbmtreinish: is that going to take locks out on the db and prevent us from writing to it for long periods of time?21:51
*** Somay has quit IRC21:52
mtreinishclarkb: honestly dunno, maybe, the sqlalchemy calls are here: http://git.openstack.org/cgit/openstack-infra/subunit2sql/tree/subunit2sql/db/api.py#n64421:53
mtreinishgiven our past experience everything gets to be kinda slow with the db at this size, I forget how long that last migration took21:54
*** _dguerri has quit IRC21:54
fungihours. i too forget exactly21:54
*** zz_jgrimm has quit IRC21:54
*** zz_zz_ja has quit IRC21:55
clarkbmtreinish: http://docs.sqlalchemy.org/en/improve_toc/orm/query.html check out the warning for snychronize_session there21:56
*** zz_jgrimm has joined #openstack-infra21:56
clarkbmtreinish: I think its bascally gonna do a select for all the things that match the filter, then delete them21:57
clarkbso at the very least it won't be super efficient21:57
*** samueldmq has quit IRC21:57
*** zz_ja has joined #openstack-infra21:57
clarkbbut I am noexpert here21:58
clarkbjust worried that it will be expensive. Worth noting that we drop an entire index per day with ES because thats cheap21:58
mtreinishclarkb: yeah I remember reading that when I wrote the api call. I figured evaluate was the best tradeoff but I really didn't have enough background to weigh things well21:59
openstackgerritDarragh Bailey proposed openstack-infra/git-review: Isolate tests from user/system git configuration  https://review.openstack.org/17995421:59
mtreinishclarkb: yeah, I'm no expert either. I was just trying to come up with a solution so we don't run out of space on the trove node and this was my first thought22:00
*** dguerri has joined #openstack-infra22:00
*** dboik has quit IRC22:02
*** rbradfor has quit IRC22:03
*** rbradfor has joined #openstack-infra22:06
fungimtreinish: i would say before going forward with something like 176564 we should probably do a couple manual runs, spaced as far apart as we're considering setting the cron job, and then time the second one22:06
*** harlowja_ has quit IRC22:06
*** harlowja has joined #openstack-infra22:07
fungiand also try to pick a trough time in the historical activity graphs to minimize impact22:07
*** zz_ja has quit IRC22:07
*** gordc has quit IRC22:07
mtreinishfungi: sure, that sounds like a good idea22:08
mtreinishif I've learned anything around this db it's that the automation never works the first time anyway :)22:09
*** zz_jgrimm has quit IRC22:09
*** mmedvede has joined #openstack-infra22:09
mtreinishfungi: but you think the db pruning approach is the way to go over spinning up a second db or something else?22:10
fungi(and often not the second or third either)22:10
mtreinishfungi: heh, yep22:10
fungii hadn't previously considered adding more databases, but i suppose that could be engineered22:10
*** maurosr has quit IRC22:11
waynrwooo with zuul-merger running jobs are correctly triggered22:11
clarkbwaynr: woot22:11
*** otter768 has joined #openstack-infra22:12
*** zz_jgrimm has joined #openstack-infra22:12
waynrso how does zuul get the merged branch to the jenkins jobs?22:13
fungimtreinish: unfortunately any automation around creating and removing the trove instances themselves, if it were only exercised every 6 months, would probably bitrot and break between every invocation22:13
fungiso per-cycle databases would likely be a manual process to add and remove22:13
clarkbwaynr: the zuul merger has created a zuul ref for you in the repo with that change something like refs/zuul/Zsomesha1, you then have to serve this to your workers in a manner that makes sense for you22:14
*** zz_ja has joined #openstack-infra22:14
*** maurosr has joined #openstack-infra22:14
fungimtreinish: how about renaming tables in the existing database? start writing new data to empty tables and then prune down the old tables which shouldn't block on other writes at that point?22:15
waynrokay that part might not be functional at the moment22:15
clarkbwaynr: zuul also drops info on the location of where to get the refs in the zuul's job params, and its configurable as far as what base url (including protocol) to use22:15
clarkbwaynr: we serve via http with apache sitting in front of our zuul mergers but ssh:// should work as well as git:// iirc22:15
fungimtreinish: i have no idea if that's any more or less optimized than trying to prune the tables to which you're appending22:15
mtreinishfungi: heh, yeah that's a fair point I don't want to introduce another manual process if I can help it. I mean it was just an alternate idea I was floating around22:16
*** otter768 has quit IRC22:16
mtreinishfungi: oh, that's an interesting idea. Yeah I don't know if that would be more efficient or not22:16
*** zz_jgrimm has quit IRC22:16
fungimtreinish: also splitting your data across multiple databases means you stop being able to use one sql query to access all your data and have to perform potentially heavy (depending on your use case) post-query correlation work22:17
mtreinishit would also make dumping the delete stuff into a subunit2sql command a bit more complicated since it wouldn't know about the renamed table22:17
*** ayoung is now known as ayoung-dadmode22:17
mtreinishunless I added a migration to create that table22:17
fungiat least in the trove server-per-database model22:17
mtreinishfungi: yeah, that's what I was thinking. I mean the only reason to do the split would be because of resource limitations on a server. (I can live with some slow queries because of the table size)22:19
mtreinishmy assumption was that it would make correlation difficult, but each db would be an isolated data set for a release22:19
*** maurosr has quit IRC22:19
*** zz_jgrimm has joined #openstack-infra22:19
mtreinishso you probably wouldn't need to do direct mapping between dbs22:20
jeblairfungi: do you know if trove supports the federated engine?22:20
*** amitgandhinz has quit IRC22:20
*** ildikov has quit IRC22:20
*** maurosr has joined #openstack-infra22:20
fungijeblair: absolutely no idea whatsoever22:20
mtreinishjust thinking out loud about this, not actually sure about anything22:21
jeblairmtreinish, fungi: subunit2sql uses innodb, right?22:21
fungijeblair: i didn't even realize that was a thing until you just mentioned it22:21
mtreinishjeblair: yeah22:21
fungi"storage engine that accesses data in tables of remote databases rather than in local tables"22:21
fungineat idea22:21
jeblairif we used myisam, we could use the merge engine.  that might be okay with this dataset.  but if we want to stick with innodb, i think merge is not available, but federated would work22:22
*** mwagner_lap has joined #openstack-infra22:22
mtreinishjeblair: well iirc I have a migration test which makes sure all the created tables get set to innodb, which is only there because I copy and pasted most of the test from nova22:22
mtreinishI remember mordred telling me myisam might be a better fit for some of the tables22:22
jeblairand can be used to set up a sharding system where we put, say, one month worth of data in a database at a time, but can query the whole set transparently, and then drop those databases as we want22:22
jeblairmordred: ^ may have more input22:23
waynrclarkb: so when i cd to the directory where zuul-merger cloned my project, I don't see any reffs named 'refs/zuul/*'22:29
mordredaroo22:30
mordredreading22:30
mordredOK. yes, it's likely a great choice for subunit2sql, don't use federated22:31
mordredmyisam and merge, likely great choice22:31
*** spzala has joined #openstack-infra22:31
clarkbwaynr: nothing in .git/refs/zuul?22:32
mordredon phone, will write more in a bit22:32
mtreinishmordred: ok, I can push up a patch to add a migration to convert the tables from innodb to myisam22:32
* mtreinish can't wait to see how long that'll take22:32
*** mmedvede has quit IRC22:32
mordredrbradford: around? wanna chat with mtreinish here?22:32
fungimtreinish: assuming there's even sufficient space on the instance?22:33
mordredmtreinish: we'll get fancy22:33
*** aysyd has quit IRC22:33
mordredthese are all append only, right?22:33
mordredand you want to roll old rows off eventually?22:33
*** ildikov has joined #openstack-infra22:33
mtreinishmordred: a few tables are append only, one isn't22:33
mordredkk. let me look more deeply on laptop in 2022:34
waynrclarkb: nada22:34
clarkbwaynr: hrm, you should check your zuul-merger logs then22:34
clarkbwaynr: it should log everything it did for that project when the change came in22:34
mtreinishmordred: yeah, that was the idea, I had a wip patch up now to do a cron delete on old rows on the append only tables22:35
waynri'm watching them, running in non-daemonized mode22:35
mtreinishfungi: heh, well last time there was enough space, but trove didn't configure mysql to use all of it correctly22:35
mtreinishbut yeah, we'd likely hit that again22:35
clarkbwaynr: so you should have a log that says "created ref blahblahblah22:36
mordredkk22:36
clarkbwaynr: do you see anything like that?22:36
waynrnope22:36
clarkb"2015-05-04 22:35:41,006 DEBUG zuul.Repo: CreateZuulRef master/Z755990147e834176a684021c571cd261 at 80c53f437fab2a5d0974b18dce720ef3a654d652" is an example from one of our mergers22:37
*** baoli has quit IRC22:37
SlickNikmtreinish: That sounds like a trove bug.  Any more information on it?22:38
waynryeah i see where that log message gets emitted in the code, there must be something about my setup that prevents it from trying?22:38
SlickNik(sorry jumping in, and don't have all of the context — reading scrollback now)22:39
clarkbwaynr: ya it may be bailing before that for some reason? do you get lines for resetting repo, updating repo, checking out $REF?22:39
clarkbwaynr: those are the steps it does before it does a merge, OH! it may be trying to talk to gerrit there?22:39
clarkbmaybe?22:39
waynrclarkb: oh it looks like it is only trying to run the "merge:update" job, not "merge:merge"22:40
waynrso it never reaches the merge codepath22:40
clarkbwaynr: ah ya, updates are there if you don't need a zuul ref and just need the latest code in place22:40
waynrwhat determines that?22:41
clarkbwaynr: merge is called when you have a thing that needs merging. Basic difference between the two, is you do an update for things like "test code that has already merged" eg ref updates or tags being made and you do merger:merge when you have unmerged code that is being proposed and needing to be tested. Eg new patchset or worflow +122:41
waynrinteresting22:42
fungiwaynr: no errors logged by the zuul-merger about being unable to clone from review.openstack.org? (or whatever gerrit it's pointed at? or is this with some github mojo already?)22:42
waynrfungi: the zuul-merger is definitely cloneing correctly from github, it seems like that url got passed from the EventTrigger object I created during webhook handling22:43
fungiwaynr: okay, so you can see where it has managed to clone the repos on disk at least22:43
waynryeah22:43
waynrand it sorta makes sense that only merge:update is being run22:43
*** bknudson has quit IRC22:44
mtreinishSlickNik: oh, yeah IIRC it was the temp storage for mysql wasn't on the volume used for the general storage, so when I did an alter table to add a column it ran out of space22:44
*** cody-somerville has quit IRC22:44
mtreinishSlickNik: so we had to do this: https://review.openstack.org/#/c/164838/22:45
mtreinishSlickNik: I can dig up the irc logs from when we tried doing this22:45
mtreinishSlickNik: the details are in http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2015-03-16.log somewhere22:46
*** shashankhegde has joined #openstack-infra22:46
mtreinishI think we pinged you at one point, so maybe grep for SlickNik :)22:46
SlickNikmtreinish: Thanks much — that helps!22:48
jheskethMorning22:48
* clarkb remembers the pbr thing22:50
lifelessclarkb: pbr thing?22:50
clarkbdims: dhellmann mordred is there a good reason for having all of oslo-core be able to tag pbr releases?22:50
lifelessoh yeah22:51
clarkblifeless: ya, remembering the release trouble we had and that the acl was funny because ^22:51
lifelessI spotted that and WTF'd22:51
*** mattfarina has joined #openstack-infra22:51
mordredonly because of Oslo association22:51
lifelessclarkb: I'm in pbr-release now, thanks to dims but we should sort the other thing out22:51
lifelessall of oslo can't release all of oslo22:51
*** mattfarina has quit IRC22:51
lifelessbut all of oslo can release pbr22:51
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add migration to convert append only tables to MyISAM  https://review.openstack.org/17996422:52
mtreinishmordred: ^^^ I doubt that will work as is...22:52
mtreinishoh and it'll definitely fail the mysql migration tests (because they check for innodb :) )22:53
clarkbright having oslo-release be able to release pbr is fine, my question is why oslo-core too22:53
lifeless-> the list, better for tz's since dhellmann is already off for the night22:54
*** ZZelle_ has quit IRC22:55
clarkbjeblair: btw, I don't see anything in gear that would make RESET_ABILITIES nuke a running job22:55
lifelessI'll send a mail22:55
fungivague recollection is that originally oslo-core was a very small group handling a handful of libraries. fast forward a couple release cycles and some of the acls didn't get updated to match that early state22:55
clarkbjeblair: so I think that is placing the blame back on the geraman plugin implementation. I am likely going to take a stab at rewriting the work() loop so that we can have state transitions all in one place22:56
fungier, s/early/later/22:56
clarkbjeblair: at least then this should be easier to reason about22:56
*** Krinkle is now known as Krinkle|detached22:56
*** armax has quit IRC22:57
clarkbunless22:57
clarkbzaro: is there any chance that the gearman java lib is maintaining some state too that we don't have control over?22:57
clarkbzaro: or is it purely just for input and output and dispatching the handleevent method?22:57
waynr\22:58
*** _nadya_ has joined #openstack-infra22:58
clarkbmordred: have you been able to look at my comments on https://review.openstack.org/#/c/175891/24 ? my only real concern there with fixing it later is you intended to make a release22:59
*** prad has quit IRC23:00
*** Sukhdev has quit IRC23:00
*** _nadya_ has quit IRC23:02
*** zz_jgrimm has quit IRC23:03
*** zz_ja has quit IRC23:03
*** sdake has joined #openstack-infra23:04
*** zz_ja has joined #openstack-infra23:06
*** zz_jgrimm has joined #openstack-infra23:06
*** camunoz has quit IRC23:07
*** rlandy has quit IRC23:07
*** sdake_ has quit IRC23:08
mordredI have, I agree with them and will fix when I take off23:08
openstackgerritMerged openstack-infra/shade: Switch tasks vs put on a boolean config flag  https://review.openstack.org/17888623:09
openstackgerritRamy Asselin proposed openstack-infra/puppet-openstackci: Create a log server like http://logs.openstack.org  https://review.openstack.org/16742523:10
asselin__nibalizer, you around? I'm still confused23:11
asselin__::jenkins class doesn't have an ssh_key...?23:11
asselin__did you mean ::jenkinsuser?23:12
*** shashankhegde has quit IRC23:12
*** maishsk has quit IRC23:13
*** ajo has quit IRC23:13
*** cody-somerville has joined #openstack-infra23:14
asselin__anyone see this error on icehouse? 2015-05-04 23:12:39.096 | 'keystonemiddleware' is not a global requirement but it should be,something went wrong23:14
fungiasselin__: i do know that there was no keystonemiddleware in icehouse23:15
fungiasselin__: it was split from keystoneclient during the juno development cycle23:15
*** maishsk has joined #openstack-infra23:15
asselin__fungi, ok....nevermind...my error.....thanks23:16
*** whoops has quit IRC23:16
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add migration to convert append only tables to MyISAM  https://review.openstack.org/17996423:17
lifelessDaviey: oh hai :)23:19
nibalizerasselin__: did it fail?23:21
nibalizerlet me look23:21
nibalizersorry I'm donig like 100 things today23:21
asselin__nibalizer, ok....np...me2. it's running now, just trying to make sence of it23:22
*** shashankhegde has joined #openstack-infra23:22
nibalizerso using what I meant was to use if ! defined(Class['::jenkins::jenkinsuser']) { instead of if ! defined(Class['jenkins::jenkinsuser']) {23:23
nibalizerthats all23:23
openstackgerritJoshua Harlow proposed openstack/requirements: Bump futures up to 3.0 or greater  https://review.openstack.org/17996723:24
asselin__nibalizer, ok! that makes more sense23:24
nibalizerya should be a super easy/nonbreaking change23:24
openstackgerritDavide Guerri proposed openstack-infra/shade: Add port resource methods  https://review.openstack.org/17996823:26
openstackgerritJoshua Harlow proposed openstack/requirements: Bump futures up to 3.0 or greater  https://review.openstack.org/17996723:27
*** zz_jgrimm has quit IRC23:27
openstackgerritJoshua Harlow proposed openstack/requirements: Bump futures up to 3.0 or greater  https://review.openstack.org/17996723:27
*** samueldmq has joined #openstack-infra23:27
*** maurosr has quit IRC23:28
*** dangers is now known as dangers_away23:28
*** zz_ja has quit IRC23:28
*** maurosr has joined #openstack-infra23:29
*** zz_jgrimm has joined #openstack-infra23:29
*** zz_ja has joined #openstack-infra23:29
*** chlong has joined #openstack-infra23:30
openstackgerritRamy Asselin proposed openstack-infra/puppet-openstackci: Create a log server like http://logs.openstack.org  https://review.openstack.org/16742523:30
asselin__nibalizer, ^^ compare patch 9 & 1223:31
nibalizerasselin__: kk23:32
nibalizerasselin__: precisely!23:32
*** dannywilson has quit IRC23:32
asselin__nibalizer, I also understand you comment now. so LGTM23:33
clarkbzaro: jeblair ok, I think I may have finally figured this out! gearman plugin calls registerFunctions() which clears the function map in the instance, handleEvent's addNewJob() has a check for the function being in the functions map, if it doesn't come back with a valid function it sends a work fail then never sends a new send grab job from what I can tell23:34
clarkbzaro: jeblair I think this is the problem, its also completely unlogged. I will try taking a stab at fixing this23:34
clarkbzaro: jeblair: I think we can sanely fix this lock free if we put the events on a thread safe queue then have work() pull them off one by one and process them23:35
clarkbbut that way we won't race the functionMap updates and won't need to handle them separately23:35
*** stevemar has quit IRC23:37
*** hemna is now known as hemnafk23:37
*** claudiub has quit IRC23:37
*** viglesias has quit IRC23:37
*** bswartz has joined #openstack-infra23:38
*** viglesias has joined #openstack-infra23:38
*** gokrokve has quit IRC23:39
clarkbor I suppose we can just clear out the grab job sent flag and have it try again later23:39
clarkbbut I would prefer to not send the work fail at all23:39
*** ajmiller has quit IRC23:41
openstackgerritMonty Taylor proposed openstack-infra/shade: Add inventory command to shade  https://review.openstack.org/17589123:41
mordredclarkb: ok- I think I got your things there ^^23:42
*** dims_ has joined #openstack-infra23:42
mordredand redid one thing - put a parameter around the exclusion in the get_hosts call ... the logic there is carry over from ansible, but non-ansible may want different things23:42
*** dims has quit IRC23:44
clarkbmordred: I like that better, it more clearly indicates if you have the host or not23:49
*** cody-somerville has quit IRC23:49
clarkbmordred: rather than just giving an ambiguous {} which may mean unconnected or does not exist23:49
mordred\o/23:49
*** jtriley has joined #openstack-infra23:49
mordredclarkb: look - code review leading to better code23:49
*** signed8bit is now known as signed8bit_ZZZzz23:50
jeblairclarkb: ah good -- that explains the lack of stuck zuul jobs23:50
*** freerunner has joined #openstack-infra23:51
*** ashleighfarnham has quit IRC23:51
*** freerunner has quit IRC23:52
*** freerunner has joined #openstack-infra23:52
clarkbjeblair: zaro: I think my plan is to collapse the work() method's loop a bit (I am pretty sure that we can do a single driveSessionIO which should make that easier to reason about, then move handleSessionEvent into the worker thread so that functionMap updates are serialized, and finally use the "add noop event to kick off get job requests" everywhere since we already do that in one place and this23:54
clarkbwill make that state change consistent23:54
lifelessnew testtools release just cut23:56
lifelessits the first using pbr23:56
lifelessso - it might be a disaster.23:56
*** SumitNaiksatam has quit IRC23:56
*** sdake_ has joined #openstack-infra23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!