Tuesday, 2016-11-22

*** mhickey has quit IRC00:02
*** s-shiono has joined #openstack-infra00:02
*** rbrndt has quit IRC00:03
*** wolverineav has joined #openstack-infra00:03
*** wolverineav has quit IRC00:08
*** armax has quit IRC00:08
openstackgerritJoshua Hesketh proposed openstack-infra/project-config: Allow nodepool-release to push merge commits  https://review.openstack.org/40045200:10
*** jhesketh has left #openstack-infra00:10
*** jhesketh has joined #openstack-infra00:10
jheskethjeblair: ^00:10
openstackgerritMerged openstack-infra/irc-meetings: Add meeting for policy  https://review.openstack.org/39850000:13
openstackgerritMerged openstack-infra/irc-meetings: Add agenda link to Zuul meeting  https://review.openstack.org/39979600:16
*** onovy has quit IRC00:16
*** peterlisak has quit IRC00:18
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Merge branch 'master' into workingv3  https://review.openstack.org/38947000:18
openstackgerritMerged openstack-infra/irc-meetings: Change time for the FWaaS team meeting.  https://review.openstack.org/39379300:21
*** armax has joined #openstack-infra00:22
*** EricGonczer_ has joined #openstack-infra00:22
*** mmandel has joined #openstack-infra00:25
*** jgriffith has quit IRC00:25
*** jgriffith has joined #openstack-infra00:26
*** gmann has quit IRC00:28
*** gmann has joined #openstack-infra00:28
*** mmandel has quit IRC00:29
openstackgerritMerged openstack-infra/project-config: Allow nodepool-release to push merge commits  https://review.openstack.org/40045200:29
*** armax has quit IRC00:30
*** jgriffith has quit IRC00:30
*** rbuzatu has joined #openstack-infra00:32
*** wolverineav has joined #openstack-infra00:32
*** lcastell has quit IRC00:32
*** jgriffith has joined #openstack-infra00:33
*** armax has joined #openstack-infra00:36
*** mtan_____ has joined #openstack-infra00:36
*** rbuzatu has quit IRC00:36
*** jgriffith has quit IRC00:36
*** wolverineav has quit IRC00:37
*** mtanino has quit IRC00:37
*** dimtruck is now known as zz_dimtruck00:39
*** lcastell has joined #openstack-infra00:39
*** hongbin has quit IRC00:43
*** mriedem has joined #openstack-infra00:45
*** mdrabe has quit IRC00:45
openstackgerritAdam Gandelman proposed openstack-infra/zuul: Re-enable test_success_pattern as test_success_url  https://review.openstack.org/40045500:46
*** peterlisak has joined #openstack-infra00:50
*** onovy has joined #openstack-infra00:50
openstackgerritJames E. Blair proposed openstack-infra/zuul: Correct logic problem with job trees  https://review.openstack.org/40045600:53
*** thorst has joined #openstack-infra00:54
*** EricGonczer_ has quit IRC00:55
zigofungi: When they started os-api-ref, I've noticed the non-free files, and wrote about it on the list. Then I've been told that it would never be used but for publishing the docs on openstack.org. Later on, everyone included the lib, and now we really do have a bug. :P00:56
* zigo continues reading the backlog.00:56
jeblairpabelanger, mordred, jhesketh, ianw: "topic:nodepool-zuulv3" patches to puppet-nodepool lgtm.  i'm about to eod, but if someone who can keep an eye wants to approve them before i wake up tomorrow, that would be swell.00:58
*** thorst has quit IRC00:59
jeblairthen next step is to make a change to create 'nodepool-builder01.o.o' (!)01:00
*** Julien-zte has joined #openstack-infra01:00
ianwjeblair: ok, looks like they shouldn't modify the status quo...01:01
jeblairianw: yeah, that's the intent01:01
ianwjeblair: if you want to approve them, i will keep an eye on nodepool to make sure it's still running01:01
jheskethjeblair: yep, I've +w'd them01:02
jheskethianw: sorry, went off to read them before I saw your comments01:02
jheskethbut I'm around to watch too01:02
*** Julien-zte has quit IRC01:02
zigofungi: Believe me, by-hand managing merge conflict as the upstream changes for non-free files is really what we want to do for Debian.01:03
*** Julien-zte has joined #openstack-infra01:04
openstackgerritMerged openstack-infra/puppet-nodepool: include ::diskimage_builder in builder.pp too  https://review.openstack.org/35648401:06
*** Julien-zte has quit IRC01:08
*** Julien-zte has joined #openstack-infra01:08
*** armax has quit IRC01:08
*** Julien-zte has quit IRC01:08
*** Julien-zte has joined #openstack-infra01:09
*** armax has joined #openstack-infra01:09
*** Julien-zte has quit IRC01:09
*** Julien-zte has joined #openstack-infra01:10
openstackgerritMerged openstack-infra/puppet-nodepool: Add option not to install nodepool-builder from init.pp  https://review.openstack.org/39974201:10
*** pvaneck has quit IRC01:11
*** Swami has quit IRC01:12
ianwhttps://review.openstack.org/400311 makes me wish i was teaching operating systems again.  some nice "why do you think this happens" questions to analyse :)01:12
*** tuanluong has joined #openstack-infra01:15
*** Julien-zte has quit IRC01:16
*** newmember has quit IRC01:17
*** Julien-zte has joined #openstack-infra01:17
*** EricGonczer_ has joined #openstack-infra01:19
*** rfolco has joined #openstack-infra01:21
*** Julien-zte has quit IRC01:21
*** Julien-z_ has joined #openstack-infra01:22
*** Sukhdev has quit IRC01:23
openstackgerritMerged openstack-infra/shade: Add docstring for create_image_snapshot  https://review.openstack.org/40044601:26
*** EricGonczer_ has quit IRC01:28
*** dave-mccowan has joined #openstack-infra01:30
*** yanyanhu has joined #openstack-infra01:32
*** zhurong has joined #openstack-infra01:41
*** Apoorva has quit IRC01:41
*** zz_dimtruck is now known as dimtruck01:41
openstackgerritBob Ball proposed openstack-infra/project-config: Use custom group to release os-xenapi  https://review.openstack.org/40015301:42
fungizigo: yep, it sounds un-fun and the sourceless font and js inclusions sound like things we should solve together as a community. i'm personally disappointed that as an upstream we have non-dfsg-free files in git repos for official openstack deliverables01:43
openstackgerritMichael Johnson proposed openstack-infra/project-config: Consolidate neutron-lbaas API tests  https://review.openstack.org/38624501:49
*** jgriffith has joined #openstack-infra01:49
sc68calianw: hmmmmmmmmmmmm :)01:50
mriedemwow we might need to think about skipping some tests http://status.openstack.org//elastic-recheck/index.html#164211101:50
mriedemsmcginnis: has anyone from cinder been looking into ^?01:51
mriedemthat's by far the top gate faliure01:51
mriedem*failure01:51
*** mtanino has joined #openstack-infra01:56
*** mtan_____ has quit IRC01:57
*** wolverineav has joined #openstack-infra02:01
openstackgerritRui Zang proposed openstack-infra/project-config: Set DEVSTACK_GATE_REMOVE_STACK_SUDO=0 for networking-odl fullstack  https://review.openstack.org/40028102:01
*** mtanino has quit IRC02:04
*** wolverineav has quit IRC02:06
*** dave-mccowan has quit IRC02:07
*** jogo has quit IRC02:07
*** gongysh has joined #openstack-infra02:08
*** hongbin has joined #openstack-infra02:09
*** sdake has joined #openstack-infra02:10
*** wolverineav has joined #openstack-infra02:11
*** jgriffith has quit IRC02:12
*** sdake_ has quit IRC02:13
*** derekjhyang has joined #openstack-infra02:13
*** armax has quit IRC02:14
mriedemthis is the workaround for now https://review.openstack.org/#/c/400465/02:15
mriedemit's the thinly provisioned lvm volumes that's causing the random snapshot failure02:15
mriedemNov 15 22:06:36 ubuntu-xenial-osic-cloud1-s3700-5383810 kernel: device-mapper: table: 252:4: thin: Unable to activate thin device while pool is suspended02:15
mriedemNov 15 22:06:36 ubuntu-xenial-osic-cloud1-s3700-5383810 kernel: device-mapper: ioctl: error adding target to table02:15
mriedemdo we have any devstack cores around? https://review.openstack.org/#/c/400465/02:16
*** wolverineav has quit IRC02:16
mriedemmtreinish: ^02:16
mriedem200+ failures in the check/gate in the last 10 days due to that02:16
mriedemsc68cal: ^02:17
*** tiswanso has joined #openstack-infra02:18
sc68calmriedem: ok let's rip it back out then02:18
*** tiswanso has quit IRC02:18
mriedemat first i thought it was an lvm2 package update in xenial but we're using the latest version which is from april02:18
mriedemso it's not that02:18
mriedemsyslog told the tale02:18
*** tiswanso has joined #openstack-infra02:18
*** tdasilva has quit IRC02:19
sc68calmriedem: +2'd02:20
*** armax has joined #openstack-infra02:20
mriedemthanks02:20
*** armax has quit IRC02:21
*** wolverineav has joined #openstack-infra02:21
*** tdasilva has joined #openstack-infra02:21
*** wolverineav has quit IRC02:25
*** hrubi has quit IRC02:35
*** tqtran has quit IRC02:36
pabelangerjeblair: ianw: mordred: jhesketh: We still have a little more work to do before we can launch nodepool-builder01.o.o (nb01.o.o?) need to create a common.pp file for shared bits02:36
*** Alagar has joined #openstack-infra02:37
*** hrubi has joined #openstack-infra02:39
openstackgerritJeremy Stanley proposed openstack-infra/system-config: Correct signing key export instructions  https://review.openstack.org/40046902:40
*** s-shiono_ has joined #openstack-infra02:41
*** s-shiono has quit IRC02:45
openstackgerritMichael Johnson proposed openstack-infra/project-config: Consolidate neutron-lbaas API tests  https://review.openstack.org/38624502:45
*** hrubi has quit IRC02:48
*** yamamoto_ has joined #openstack-infra02:48
*** tjones has quit IRC02:49
*** wolverineav has joined #openstack-infra02:50
*** hrubi has joined #openstack-infra02:50
openstackgerritTuan Luong-Anh proposed openstack-infra/gear: Replace assertEquals with assertEqual  https://review.openstack.org/40047202:51
*** Jeffrey4l has joined #openstack-infra02:53
*** wolverineav has quit IRC02:55
fungiadarazs: sorry for the delay. i've pushed the two branches from https://github.com/adarazs/tripleo-quickstart-extras into the openstack/tripleo-quickstart-extras repo in review.openstack.org, and see them matching on our git farm now: https://git.openstack.org/cgit/openstack/tripleo-quickstart-extras02:55
*** yamamoto_ has quit IRC02:57
*** yamamoto_ has joined #openstack-infra03:00
*** kaisers has quit IRC03:01
openstackgerritJeffrey Zhang proposed openstack-infra/project-config: Install docker-engine by using builder in kolla and kolla-ansible  https://review.openstack.org/40047403:04
*** zhurong has quit IRC03:05
*** zhurong has joined #openstack-infra03:06
cloudnullevenings03:06
*** sdake has quit IRC03:07
*** rfolco has quit IRC03:08
*** wolverineav has joined #openstack-infra03:10
*** tnovacik has joined #openstack-infra03:12
AlagarIam trying to integrate openstack with xen03:12
Alagari have installed openstack using devstack script top of xen hypervisor, in this open stack as a virtual machine.03:12
Alagarwhen i create instance in openstack,   the instance should create in xen hypervisor.  but its not happening.03:12
AlagarSome one could you please guide me please03:12
*** kaisers has joined #openstack-infra03:14
*** tnovacik has quit IRC03:14
*** jogo has joined #openstack-infra03:15
*** jogo has quit IRC03:15
*** jogo has joined #openstack-infra03:15
*** wolverineav has quit IRC03:15
*** jogo has quit IRC03:20
*** wolverineav has joined #openstack-infra03:20
*** knangia has quit IRC03:23
*** wolverineav has quit IRC03:25
EmilienMianw: http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/elements/stackviz/install.d/89-prepare-stackviz03:26
EmilienMif the npm is not found, script won't fail.03:27
ianwah, well, yeah03:27
ianwno wonder nobody has looked at it :)03:27
EmilienMianw: i'm failing to reproduce tonight but if nobody takes it, i'll try later this week or so03:27
EmilienMit sounds like a packaging thing in centos03:27
EmilienMI could deploy npm on my fedora03:27
ianwEmilienM: yeah, not sure ... we've got at least one problem with grub and fedora builds atm which i'm looking at, and possibly another with centos & service enablement.03:29
timothyb89EmilienM, ianw: it might be related to this workaround: http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/elements/stackviz/pkg-map#n503:29
*** wolverineav has joined #openstack-infra03:30
*** vikrant has joined #openstack-infra03:31
*** yamahata has quit IRC03:32
fungiAlagar: you may have misunderstood the topic of this channel. it's not for discussion of running an infrastructure that uses openstack, it's for running the infrastructure that enables the development of openstack (code review system, continuous integration and testing toolchain, mailing lists, wiki server, irc bots, et cetera)03:34
*** isviridov_away has quit IRC03:34
*** wolverineav has quit IRC03:34
fungiAlagar: you're probably looking for #openstack (general discussion about running and using openstack) or #openstack-dev (discussion of developing openstack)03:35
*** snarwade has joined #openstack-infra03:35
*** jgriffith has joined #openstack-infra03:35
jeblairi just typed 'rmkdir'.  i'm not sure what i expected my shell to do.03:36
*** jgriffith has quit IRC03:36
fungithat should be a no-op03:37
clarkbor maybe it deletes recursively to give you an empty dir?03:37
clarkbjust like new!03:37
fungirmkdir(1) atomically create a directory and then remove it again03:38
fungi[implementation: symlink to /bin/true]03:38
*** armax has joined #openstack-infra03:39
jeblairfungi: but if you reverse the order of the arguments, it does the opposite.03:39
funginow my brain hurts03:39
fungiperhaps that will still work if you run the system clock in reverse03:39
*** wolverineav has joined #openstack-infra03:40
jeblairfungi: is that why your workstation is up on blocks?03:40
* fungi sighs at excellent 80s film reference03:41
*** mriedem has quit IRC03:41
* clarkb feels a woosh go by03:42
fungibueller?03:42
jeblairclarkb: http://www.imdb.com/title/tt0091042/03:42
clarkbwas the car on cinder blocks?03:43
jeblairit might have been jacks?03:43
fungione of those03:43
fungiuntil it wasn't, suddenly03:43
clarkbeven then I wouldn't have gotten it :)03:43
*** mmandel has joined #openstack-infra03:43
jeblairthen it was on trees, iirc03:44
fungimore or less, yes03:44
*** wolverineav has quit IRC03:44
*** hichihara has joined #openstack-infra03:45
* fungi calls it a night and does things which involve more pie and less computer03:45
clarkbI don't have pie03:45
jeblairclarkb: :(03:45
fungii didn't until a few minutes ago when i moved it to a cooling rack03:46
clarkbI did find my second bottle of hawaiian moonshine though03:46
fungithat sounds at least as good03:46
clarkbso now I need tropical fruit drink makings for thursday03:46
*** psachin has joined #openstack-infra03:47
*** mmandel has quit IRC03:47
openstackgerritChaoyi Huang proposed openstack-infra/project-config: Update jobs in Tricircle project  https://review.openstack.org/40049303:49
*** wolverineav has joined #openstack-infra03:49
*** hongbin has quit IRC03:51
*** tjones has joined #openstack-infra03:52
*** Odd_Bloke has quit IRC03:53
*** gongysh has quit IRC03:54
*** wolverineav has quit IRC03:54
*** isviridov_away has joined #openstack-infra03:54
*** hongbin has joined #openstack-infra03:55
*** Odd_Bloke has joined #openstack-infra03:56
*** wolverineav has joined #openstack-infra03:59
*** esikache1 has joined #openstack-infra04:04
*** wolverineav has quit IRC04:04
*** thorst has joined #openstack-infra04:06
*** links has joined #openstack-infra04:08
*** esikache1 has quit IRC04:08
*** rbuzatu has joined #openstack-infra04:08
*** wolverineav has joined #openstack-infra04:09
*** gildub has joined #openstack-infra04:11
*** sdake has joined #openstack-infra04:13
*** rbuzatu has quit IRC04:13
*** hongbin has quit IRC04:13
*** wolverineav has quit IRC04:14
*** sdake_ has joined #openstack-infra04:14
*** yamamoto_ has quit IRC04:17
*** sdake has quit IRC04:18
*** tiswanso has quit IRC04:18
*** wolverineav has joined #openstack-infra04:19
*** ijw has quit IRC04:20
*** wolverineav has quit IRC04:23
*** adriant has quit IRC04:24
*** gongysh has joined #openstack-infra04:24
*** yamamoto_ has joined #openstack-infra04:25
*** yamamoto_ has quit IRC04:27
*** wolverineav has joined #openstack-infra04:29
*** inc0 has quit IRC04:29
*** kong has quit IRC04:30
*** alexey_weyl has joined #openstack-infra04:31
alexey_weylHi,04:31
*** yamamoto_ has joined #openstack-infra04:32
alexey_weylCan you please approve this change: https://review.openstack.org/#/c/400243/404:33
*** Alagar has quit IRC04:33
*** wolverineav has quit IRC04:33
*** rlandy has quit IRC04:33
*** Alagar has joined #openstack-infra04:33
*** tqtran has joined #openstack-infra04:34
*** yamamoto_ has quit IRC04:34
*** alexey_weyl has quit IRC04:34
*** flwang1 has quit IRC04:37
*** wolverineav has joined #openstack-infra04:39
*** tlian has quit IRC04:40
openstackgerritMerged openstack-infra/project-config: Change node to xenial.  https://review.openstack.org/40024304:40
*** wolverineav has quit IRC04:43
*** drifterza has quit IRC04:44
openstackgerritTuan Luong-Anh proposed openstack/diskimage-builder: Replace six.iteritems() with .items()  https://review.openstack.org/40051404:48
*** wolverineav has joined #openstack-infra04:48
*** wolverineav has quit IRC04:53
*** Sukhdev has joined #openstack-infra04:57
*** wolverineav has joined #openstack-infra04:58
*** drifterza has joined #openstack-infra05:01
*** yamamoto_ has joined #openstack-infra05:02
*** wolverineav has quit IRC05:03
*** yamamoto_ has quit IRC05:03
*** Alagar has quit IRC05:04
*** Alagar has joined #openstack-infra05:06
*** thorst has quit IRC05:08
*** yamamoto_ has joined #openstack-infra05:09
openstackgerritArmando Migliaccio proposed openstack-infra/project-config: Revise ACLs for networking-[onos|l2gw] projects  https://review.openstack.org/39979805:18
*** yamamoto_ has quit IRC05:21
*** sandanar has joined #openstack-infra05:24
*** sdake_ has quit IRC05:24
*** sdake has joined #openstack-infra05:26
*** wolverineav has joined #openstack-infra05:28
*** sdake has quit IRC05:30
*** dkehn__ has joined #openstack-infra05:31
*** wolverineav has quit IRC05:32
openstackgerritHoolio Wobbits proposed openstack-infra/shade: Added documentation for delete_image()  https://review.openstack.org/40051905:33
Jeffrey4li got POST_FAILURE, see the jobs is killed when timeout, so the log is not collected.  is there any way to collect console.log when timeout?   https://review.openstack.org/#/c/400488/105:33
*** dkehn_ has quit IRC05:33
*** dkehn_ has joined #openstack-infra05:37
*** dkehn__ has quit IRC05:38
*** hurgleburgler has quit IRC05:39
*** winggundamth has quit IRC05:40
*** winggundamth has joined #openstack-infra05:41
*** mmandel has joined #openstack-infra05:44
openstackgerritHoolio Wobbits proposed openstack-infra/shade: Added documentation for delete_image()  https://review.openstack.org/40051905:44
openstackgerritHoolio Wobbits proposed openstack-infra/shade: Added documentation for delete_image()  https://review.openstack.org/40051905:46
*** mmandel has quit IRC05:49
openstackgerritHoolio Wobbits proposed openstack-infra/shade: Added documentation for delete_image()  https://review.openstack.org/40051905:51
*** ad_rien_ has joined #openstack-infra05:52
*** ad_rien_ has quit IRC05:52
*** ad_rien_ has joined #openstack-infra05:52
hoolioah yes, look at all that.  soz guys05:53
*** rbuzatu has joined #openstack-infra05:57
*** senk has joined #openstack-infra05:59
*** Julien-zte has joined #openstack-infra05:59
*** Julien-z_ has quit IRC06:00
*** thorst has joined #openstack-infra06:00
*** rbuzatu has quit IRC06:01
*** ad_rien_ has joined #openstack-infra06:02
*** yamamoto_ has joined #openstack-infra06:02
*** claudiub has joined #openstack-infra06:03
*** esikache1 has joined #openstack-infra06:04
*** ad_rien_ has quit IRC06:06
*** drifterza has quit IRC06:07
*** thorst has quit IRC06:08
*** esikache1 has quit IRC06:09
*** jaosorior has joined #openstack-infra06:17
fricklerJeffrey4l: I think disabling v6 on your testing node is a bad idea, lots of them only have v6 as external connectivity these days. if you remove external connectivity, there's no chance to get anything back from that node06:20
*** abregman has joined #openstack-infra06:23
Jeffrey4lfrickler, got. i will  try to enable ipv6 and check whether it will collect the logs.06:25
*** chandankumar has joined #openstack-infra06:31
*** pgadiya has joined #openstack-infra06:32
clarkbfrickler: Jeffrey4l correct, disabling ipv6 will break your tests06:33
clarkbalso fallocate will be slightly faster than dd for the swapfile but can only be used reliably on ext fses06:34
Jeffrey4lroger. thanks.06:36
*** yamahata has joined #openstack-infra06:39
openstackgerritTuan Luong-Anh proposed openstack/diskimage-builder: Replace six.iteritems() with .items()  https://review.openstack.org/40051406:43
openstackgerritRenat Akhmerov proposed openstack-infra/project-config: Add mistral gate for mysql unit tests  https://review.openstack.org/40053506:45
openstackgerritJoshua Hesketh proposed openstack-infra/nodepool: Merge branch 'master' into feature/zuulv3  https://review.openstack.org/40053606:46
*** jaosorior has quit IRC06:49
*** jaosorior has joined #openstack-infra06:49
*** oanson has joined #openstack-infra06:51
*** senk has quit IRC06:52
*** thorst has joined #openstack-infra06:56
*** wolverineav has joined #openstack-infra06:56
*** fandi has joined #openstack-infra07:00
*** esikache1 has joined #openstack-infra07:00
*** NBziouech has quit IRC07:01
*** wolverineav has quit IRC07:01
*** flepied has joined #openstack-infra07:02
openstackgerritliusheng proposed openstack-infra/project-config: Enable Nimble tempest plugin test  https://review.openstack.org/40053907:03
*** rfolco has joined #openstack-infra07:04
*** thorst has quit IRC07:04
*** yuval has joined #openstack-infra07:04
*** rfolco has quit IRC07:04
openstackgerritMerged openstack-infra/project-config: n8g-bgpvpn and n8g-bagpipe fixes for grafana jobs  https://review.openstack.org/39848507:04
*** nherciu has joined #openstack-infra07:06
*** wolverineav has joined #openstack-infra07:06
*** tjones has quit IRC07:10
*** wolverineav has quit IRC07:11
*** hashar has joined #openstack-infra07:12
*** rbuzatu_ has joined #openstack-infra07:13
openstackgerritIan Wienand proposed openstack/diskimage-builder: Catch errors in DIB_INIT_SYSTEM export  https://review.openstack.org/40054207:14
openstackgerritRenat Akhmerov proposed openstack-infra/project-config: Add mistral gate for mysql unit tests  https://review.openstack.org/40053507:20
openstackgerritIan Wienand proposed openstack-infra/puppet-diskimage_builder: Revert "Add a use_git flag"  https://review.openstack.org/40054507:23
*** drifterza has joined #openstack-infra07:28
AJaegerianw: could you review https://review.openstack.org/399105, https://review.openstack.org/399079, and https://review.openstack.org/399667 for project-config, please?07:28
*** senk has joined #openstack-infra07:30
*** jtomasek has joined #openstack-infra07:32
*** sdake has joined #openstack-infra07:33
*** tovin07 has quit IRC07:33
*** sdake_ has joined #openstack-infra07:34
ianwAJaeger: i'm not sure way if we want to do that, you don't just call it as part of the tox command?07:34
ianwmaybe we've dropped sudo by that point i guess07:35
*** ad_rien_ has joined #openstack-infra07:35
*** ad_rien_ has quit IRC07:35
*** tovin07_ has joined #openstack-infra07:36
AJaegerianw: it's before the revoke-sudo call, so should work at this place07:37
*** sdake has quit IRC07:38
ianwAJaeger: what i mean is, that the primary reason for *not* just doing this as part of the tox command is that we have root at this point?07:38
AJaegerWe had some discussion last week about this - with fungi and jeblair . But if you have concerns.07:38
*** chandankumar has quit IRC07:38
ianwnot really, just trying to understand when you'd put something in there, and when not07:38
AJaegerThe point is we can remove the unneeded db setup - and similar setup for searchlight07:39
AJaegerLet me get a quote from fungi on this one ;)07:39
*** chandankumar has joined #openstack-infra07:39
AJaegerianw: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-11-17.log.html#t2016-11-17T15:09:3607:41
openstackgerritliusheng proposed openstack-infra/project-config: Enable Nimble tempest plugin test  https://review.openstack.org/40053907:41
AJaegerSo, basically, the goal is to get rid of the specific db setup and move that to the repos.07:41
AJaegerFor example, we have a nodepool bindep.txt addition that adds postgres and mysql - but the project only needs mysql. So, if we move this over, we can setup only postgres and don't need to setup mysql - and have it properly documented what is needed.07:42
*** jtomasek has quit IRC07:44
*** jtomasek_ has joined #openstack-infra07:44
ianwAJaeger: ok, my thought was more "why can't tox just run this" but I think, as i say, i've realised that we've dropped root by the time we run tox, so it can't do this setup07:44
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/40056607:44
AJaegerianw: YEs, exactly. We cannot run this as part of tox without root.07:45
*** senk has quit IRC07:48
*** daemontool has joined #openstack-infra07:48
*** saibarspeis has joined #openstack-infra07:51
*** fandi__ has joined #openstack-infra07:51
*** thorst has joined #openstack-infra07:51
*** ralonsoh has joined #openstack-infra07:51
openstackgerritAndreas Jaeger proposed openstack-infra/nodepool: Add tools/test-setup.sh  https://review.openstack.org/39907907:52
AJaegerianw: better? ^07:52
*** armax has quit IRC07:52
*** senk has joined #openstack-infra07:53
*** fandi has quit IRC07:55
*** pcaruana has joined #openstack-infra07:56
openstackgerritAndreas Jaeger proposed openstack-infra/nodepool: Add tools/test-setup.sh  https://review.openstack.org/39917707:56
*** matrohon has joined #openstack-infra07:57
*** berendt has joined #openstack-infra07:57
*** thorst has quit IRC07:58
*** esikache1 has quit IRC07:59
*** esikachev has joined #openstack-infra07:59
openstackgerritMerged openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/40056608:01
*** Sukhdev has quit IRC08:06
*** pgadiya has quit IRC08:09
*** gouthamr has joined #openstack-infra08:12
openstackgerritMasayuki Igawa proposed openstack/gertty: [WIP] Make size column configurable a graph  https://review.openstack.org/38095008:14
*** rcernin_ has joined #openstack-infra08:14
*** flepied has quit IRC08:16
*** florianf has joined #openstack-infra08:19
*** arxcruz has joined #openstack-infra08:20
BobBallJust asking because I don't see any discussion on it - are people aware that the gate is blocked?08:26
BobBalloh... maybe it's not08:27
*** ssbarnea has joined #openstack-infra08:29
*** ad_rien_ has joined #openstack-infra08:29
*** dingyichen has quit IRC08:31
*** Hal has joined #openstack-infra08:35
*** Hal is now known as Guest9823708:35
*** mmandel has joined #openstack-infra08:35
*** daemontool has quit IRC08:37
*** tqtran has quit IRC08:38
*** mmandel has quit IRC08:40
*** rbuzatu_ has quit IRC08:40
*** amoralej|off is now known as amoralej08:40
*** thorst has joined #openstack-infra08:40
*** jpena|off is now known as jpena08:41
openstackgerritSpyros Trigazis proposed openstack-infra/project-config: [magnum] Update fedora-atomic build job to 24  https://review.openstack.org/40059808:43
*** hogepodge has quit IRC08:45
*** nmagnezi has joined #openstack-infra08:45
*** thorst has quit IRC08:48
*** s-shiono_ has quit IRC08:52
*** sdake_ is now known as sdake08:52
*** ad_rien_ has quit IRC08:55
*** ad_rien_ has joined #openstack-infra08:56
openstackgerritliusheng proposed openstack-infra/project-config: Enable Nimble tempest plugin test  https://review.openstack.org/40053908:56
*** jpich has joined #openstack-infra08:57
*** sdake has quit IRC08:57
*** rossella__ has joined #openstack-infra08:57
*** flepied has joined #openstack-infra08:58
*** pgadiya has joined #openstack-infra09:00
*** zzzeek has quit IRC09:00
*** zzzeek has joined #openstack-infra09:01
*** martinkopec has joined #openstack-infra09:04
*** panda|zZ is now known as panda09:04
*** gongysh has quit IRC09:05
openstackgerritMichal Dulko proposed openstack-infra/project-config: Set Cinder multinode grenade on master to xenial  https://review.openstack.org/39449609:06
openstackgerritPascal Mazon proposed openstack-infra/project-config: Add 6WIND Virtual Accelerator Charm  https://review.openstack.org/40062109:07
*** NBziouech has joined #openstack-infra09:09
*** pgadiya has quit IRC09:11
openstackgerritMichal Dulko proposed openstack-infra/project-config: Set Cinder multinode grenade on master to xenial  https://review.openstack.org/39449609:14
*** isaacb has joined #openstack-infra09:14
*** dmanchad has quit IRC09:15
*** tpsilva has joined #openstack-infra09:16
*** dmanchad has joined #openstack-infra09:16
*** dtantsur|afk is now known as dtantsur09:18
*** mhickey has joined #openstack-infra09:21
openstackgerritSpyros Trigazis proposed openstack-infra/project-config: [magnum] Update fedora-atomic build job to 24  https://review.openstack.org/40059809:22
*** gildub has quit IRC09:22
*** mmoster_ has joined #openstack-infra09:25
*** pgadiya has joined #openstack-infra09:26
*** Julien-zte has quit IRC09:27
*** rcarrillocruz has quit IRC09:29
*** lucas-afk is now known as lucasagomes09:31
*** rossella__ has quit IRC09:31
*** rossella__ has joined #openstack-infra09:31
*** mmoster_ has quit IRC09:32
*** ricky1 has joined #openstack-infra09:33
*** ricky1 is now known as rcarrillocruz09:33
openstackgerritNoam Angel proposed openstack/diskimage-builder: add option to configure cloud-init to allow password authentication  https://review.openstack.org/39176509:34
*** rbuzatu has joined #openstack-infra09:34
*** tqtran has joined #openstack-infra09:35
*** derekh has joined #openstack-infra09:36
*** pblaho has quit IRC09:38
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Check the status of the stack deployed instead of the return code  https://review.openstack.org/40064409:40
*** tqtran has quit IRC09:40
openstackgerritMichal Dulko proposed openstack-infra/project-config: Set Cinder multinode grenade on master to xenial  https://review.openstack.org/39449609:40
*** asettle has joined #openstack-infra09:40
*** asettle has quit IRC09:41
*** asettle__ has joined #openstack-infra09:41
*** asettle__ is now known as asettle09:42
*** dizquierdo has joined #openstack-infra09:43
*** ad_rien_ has quit IRC09:43
*** rbuzatu has quit IRC09:44
*** rbuzatu has joined #openstack-infra09:45
*** ccamacho has joined #openstack-infra09:45
*** ad_rien_ has joined #openstack-infra09:46
*** electrofelix has joined #openstack-infra09:47
*** tovin07_ has quit IRC09:48
*** thorst has joined #openstack-infra09:50
*** ihrachys has joined #openstack-infra09:50
*** tuanluong has quit IRC09:56
openstackgerritJakub Libosvar proposed openstack-infra/project-config: grafana: In Neutron dashboard split Xenial and Trusty graphs  https://review.openstack.org/40065409:56
*** pblaho has joined #openstack-infra09:56
*** thorst has quit IRC09:58
openstackgerritMichal Dulko proposed openstack-infra/project-config: Add cinder job with zeromq and multibackend  https://review.openstack.org/40065810:01
*** baoli has quit IRC10:02
*** yanyanhu has quit IRC10:03
*** zhurong has quit IRC10:03
*** sambetts|afk is now known as sambetts10:05
*** esikachev has quit IRC10:05
openstackgerritErlon R. Cruz proposed openstack-infra/devstack-gate: [DO NOT MERGE] Testing gate job  https://review.openstack.org/39659210:07
openstackgerritPascal Mazon proposed openstack-infra/project-config: Add 6WIND Virtual Accelerator Charm  https://review.openstack.org/40062110:08
openstackgerritMarkos Chandras proposed openstack/diskimage-builder: elements: Add new openssh-server element  https://review.openstack.org/38917110:08
openstackgerritPascal Mazon proposed openstack-infra/project-config: Add 6WIND Virtual Accelerator Charm  https://review.openstack.org/40062110:10
*** kzaitsev_mb has joined #openstack-infra10:11
*** berendt has quit IRC10:16
openstackgerritVitaly Gridnev proposed openstack-infra/project-config: [sahara] scenario jobs: jobs modifications  https://review.openstack.org/39962310:19
*** thiagolib has joined #openstack-infra10:22
wznoinskhi infra, would someone have a moment to weigh in on https://review.openstack.org/#/c/394476/ ?10:24
*** fandi__ has quit IRC10:26
*** rkukura_ has joined #openstack-infra10:26
openstackgerritPascal Mazon proposed openstack-infra/project-config: Add 6WIND Virtual Accelerator Charm  https://review.openstack.org/40062110:26
*** rkukura has quit IRC10:30
*** rkukura_ is now known as rkukura10:30
*** e0ne has joined #openstack-infra10:30
*** tosky has joined #openstack-infra10:32
*** ldnunes has joined #openstack-infra10:32
*** yolanda has quit IRC10:35
openstackgerritRenat Akhmerov proposed openstack-infra/project-config: Add mistral gate for mysql unit tests  https://review.openstack.org/40053510:35
*** yolanda has joined #openstack-infra10:38
openstackgerritJulia Varlamova proposed openstack-infra/devstack-gate: Add MULTI_KEYSTONE flag and new roles  https://review.openstack.org/39489510:39
*** ad_rien_ has quit IRC10:42
*** mmandel has joined #openstack-infra10:44
*** ad_rien_ has joined #openstack-infra10:44
openstackgerritJulia Varlamova proposed openstack-infra/devstack-gate: Add MULTI_KEYSTONE flag and new roles  https://review.openstack.org/39489510:45
*** jtomasek_ is now known as jtomasek10:45
*** thorst has joined #openstack-infra10:46
*** mmandel has quit IRC10:48
*** gouthamr has quit IRC10:49
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Output additional info when exceptions occur  https://review.openstack.org/30973510:49
*** jlibosva has joined #openstack-infra10:49
*** ad_rien_ has quit IRC10:50
*** gnuoy has quit IRC10:51
*** ad_rien_ has joined #openstack-infra10:51
*** gnuoy has joined #openstack-infra10:52
*** saibarspeis has quit IRC10:52
*** thorst has quit IRC10:53
jlibosvaAJaeger: hi, do you know a person that could help us with http://logs.openstack.org/41/396641/5/check/gate-grenade-dsvm-neutron-dvr-multinode-ubuntu-trusty/f723242/logs/grenade.sh.txt.gz#_2016-11-22_10_05_23_404 ?10:53
jlibosvaor is there already any issue tracker we could follow?10:53
jlibosvaI only reported bug on Neutron but it sounds more like an infra issue10:53
*** dizquierdo has quit IRC10:54
openstackgerritPascal Mazon proposed openstack-infra/project-config: Add 6WIND Virtual Accelerator Charm  https://review.openstack.org/40062110:54
AJaegerjlibosva: We had some problems the last days with ssh set up in our images. Perhaps pabelanger has an idea later on this. Otherwise no idea10:54
jlibosva:-/10:55
jlibosvait's specific to grenade multinode setups10:55
*** shardy has joined #openstack-infra10:55
AJaegerjlibosva: The #openstack-qa channel is always an option for these...10:56
jlibosvaAJaeger: ok, I'll try to complain there :) Thanks10:56
AJaegerjlibosva: otherwise, please come back once the US and Canada is awake10:57
jlibosvaAJaeger: ack10:57
openstackgerritMichal Dulko proposed openstack-infra/project-config: Set Cinder multinode grenade on master to xenial  https://review.openstack.org/39449610:58
*** ad_rien_ has quit IRC10:59
*** jascott1 has quit IRC10:59
*** ad_rien_ has joined #openstack-infra11:02
ihrachysAJaeger: do we have a way to bump a patch in merge gate?11:11
ihrachysAJaeger: https://review.openstack.org/400183 will take another 6h or so before landing11:11
ihrachyswould be cool to get it at the top11:11
ihrachysand I believe it is the reason for the long queue11:12
*** yamahata has quit IRC11:14
*** psilvad has joined #openstack-infra11:16
*** ccamacho has left #openstack-infra11:17
*** ccamacho has quit IRC11:17
AJaegerihrachys: an admin can do that - but none is around right now. fungi, pabelanger, once you wake up, could you help ihrachys, please? ^11:18
*** ccamacho has joined #openstack-infra11:20
*** psilvad has quit IRC11:20
*** saibarspeis has joined #openstack-infra11:20
*** psilvad has joined #openstack-infra11:21
*** martinkopec has quit IRC11:22
*** martinkopec has joined #openstack-infra11:22
*** ccamacho has quit IRC11:26
*** ccamacho has joined #openstack-infra11:31
*** Goneri has quit IRC11:38
*** mmoster_ has joined #openstack-infra11:41
*** dizquierdo has joined #openstack-infra11:41
*** thorst has joined #openstack-infra11:43
*** rtheis has joined #openstack-infra11:45
*** rcarrillocruz has quit IRC11:46
*** rcarrillocruz has joined #openstack-infra11:46
*** thorst has quit IRC11:48
*** mmoster_ has quit IRC11:48
*** nicolasbock has joined #openstack-infra11:49
*** jkilpatr has quit IRC12:03
*** jkilpatr has joined #openstack-infra12:03
*** kzaitsev_mb has quit IRC12:07
*** wolverineav has joined #openstack-infra12:16
*** tesseract has joined #openstack-infra12:19
*** tesseract is now known as Guest378712:19
openstackgerritJulia Varlamova proposed openstack-infra/devstack-gate: Add MULTI_KEYSTONE flag and new roles  https://review.openstack.org/39489512:21
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Full quickstart gate run on OVB  https://review.openstack.org/38109412:23
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: DONT MERGE: Test quickstart experimental job  https://review.openstack.org/39760912:23
*** kzaitsev_mb has joined #openstack-infra12:27
*** ccamacho is now known as ccamacho|lunch12:28
*** ad_rien_ has quit IRC12:29
*** gouthamr has joined #openstack-infra12:30
*** nexusz99 has joined #openstack-infra12:30
*** nexusz99 has quit IRC12:31
*** ad_rien_ has joined #openstack-infra12:32
openstackgerritAlexandra Settle proposed openstack-infra/project-config: Add Install Guide Jobs to Designate  https://review.openstack.org/34167112:33
*** wolverineav has quit IRC12:34
*** dtantsur is now known as dtantsur|brb12:35
*** thorst has joined #openstack-infra12:38
*** wolverineav has joined #openstack-infra12:38
amrithAJaeger, g'afternoon.12:40
openstackgerritRob Cresswell proposed openstack-infra/project-config: Remove duplicated Horizon test runs  https://review.openstack.org/40008412:41
*** wolverineav has quit IRC12:43
*** rhallisey has joined #openstack-infra12:43
*** wolverineav has joined #openstack-infra12:44
*** thorst has quit IRC12:45
robcresswellAJaeger: Any suggestions on a sensible way to modify a test command for a single branch? Referring to https://review.openstack.org/#/c/399868/ I was just going to add a new job called "selenium-legacy" (or similar) with the old command for newton/mitaka, and then just use the current job for ocata and future.12:45
openstackgerritMerged openstack-infra/project-config: Use custom group to release os-xenapi  https://review.openstack.org/40015312:46
openstackgerritMerged openstack-infra/project-config: [sahara] scenario jobs: jobs modifications  https://review.openstack.org/39962312:47
openstackgerritMerged openstack-infra/project-config: Add cinder job with zeromq and multibackend  https://review.openstack.org/40065812:47
ihrachysfungi: pabelanger: another gate fix we may need to bump in gate is https://review.openstack.org/#/c/400465/ the bug just bumped the patch blocking bad heatclient out of gate12:47
openstackgerritMerged openstack-infra/project-config: make python3.5 voting for oslo.middleware  https://review.openstack.org/40043812:48
openstackgerritMerged openstack-infra/project-config: Make oslo.messaging amqp1 driver non-voting  https://review.openstack.org/40043012:48
*** wolverineav has quit IRC12:49
openstackgerritMerged openstack-infra/project-config: add non-voting functional tests job for networking-odl  https://review.openstack.org/39447612:49
openstackgerritMerged openstack-infra/project-config: Promote Monasca tempest tests on ubuntu-xenial  https://review.openstack.org/39904412:49
*** mmoster_ has joined #openstack-infra12:50
*** dprince has joined #openstack-infra12:53
openstackgerritRob Cresswell proposed openstack-infra/project-config: Update Horizons selenium-headless test  https://review.openstack.org/39986812:57
*** abregman is now known as abregman|mtg12:57
wznoinskyolanda, thanks12:58
openstackgerritZhenguo Niu proposed openstack-infra/project-config: Enable Nimble tempest plugin test  https://review.openstack.org/40053912:58
*** saibarspeis has quit IRC12:58
*** claudiub|2 has joined #openstack-infra12:58
*** yolanda has quit IRC12:58
*** lucasagomes is now known as lucas-hungry12:59
*** sdake has joined #openstack-infra12:59
*** pgadiya has quit IRC13:00
*** claudiub has quit IRC13:01
*** abregman|mtg has quit IRC13:02
*** zhurong has joined #openstack-infra13:02
*** wolverineav has joined #openstack-infra13:03
*** dave-mccowan has joined #openstack-infra13:04
*** sdake_ has joined #openstack-infra13:04
*** bhunter71 has joined #openstack-infra13:05
*** amoralej is now known as amoralej|lunch13:06
*** sdake has quit IRC13:07
*** wolverineav has quit IRC13:07
*** trown|afk is now known as trown13:07
openstackgerritSagi Shnaidman proposed openstack-infra/devstack-gate: Find and delete broken symlinks in logs  https://review.openstack.org/40074213:08
bhunter71what OS is nodepool currently?  Has any prod host moved to xenial?   Its there still some sort of puppetboard up, or another way to see this data?  thanks!!13:08
*** denisra_ has joined #openstack-infra13:08
*** dizquierdo has quit IRC13:10
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config: neutron-fwaas: Consume gate hook for tempest plugin jobs  https://review.openstack.org/40074513:10
*** Julien-zte has joined #openstack-infra13:11
*** wolverineav has joined #openstack-infra13:12
*** denisra has quit IRC13:12
*** dizquierdo has joined #openstack-infra13:12
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: DONT REVIEW: Test timeout  https://review.openstack.org/39341513:13
*** wolverineav has quit IRC13:16
*** wolverineav has joined #openstack-infra13:17
openstackgerritRodrigo Duarte proposed openstack-infra/project-config: Enable keystone-saml2-federation service  https://review.openstack.org/40075013:22
openstackgerritmathieu bultel proposed openstack-infra/project-config: Add mitaka and newton UC and full upgrade jobs  https://review.openstack.org/40031713:23
*** wolverineav has quit IRC13:23
*** ggnel_t has joined #openstack-infra13:26
*** esikachev has joined #openstack-infra13:27
*** tlian has joined #openstack-infra13:28
*** ociuhandu has quit IRC13:30
*** jcoufal has joined #openstack-infra13:30
*** zul has quit IRC13:30
*** rossella__ has quit IRC13:30
*** tojuvone has joined #openstack-infra13:31
*** rossella__ has joined #openstack-infra13:31
*** yamamoto_ has quit IRC13:31
*** rlandy has joined #openstack-infra13:32
*** gordc has joined #openstack-infra13:32
*** jeckersb is now known as jeckersb_gone13:33
*** zul has joined #openstack-infra13:33
njohnstonHello infra!  I have a change to an IRC meeting that has merged - https://review.openstack.org/393793 - but the eavesdrop meeting entry has not changed - http://eavesdrop.openstack.org/#Firewall_as_a_Service_(FWaaS)_Team_Meeting - nor has the .ics file13:34
*** fguillot has joined #openstack-infra13:35
*** mdrabe has joined #openstack-infra13:35
*** thorst has joined #openstack-infra13:36
*** chlong has joined #openstack-infra13:36
*** tqtran has joined #openstack-infra13:36
*** adarazs is now known as adarazs_lunch13:38
*** wolverineav has joined #openstack-infra13:39
*** tqtran has quit IRC13:41
*** thorst has quit IRC13:42
openstackgerritMerged openstack-infra/shade: Allow server to be snapshot to be name, id or dict  https://review.openstack.org/40044713:47
*** wolverineav has quit IRC13:47
*** dizquierdo has quit IRC13:50
*** wolverineav has joined #openstack-infra13:51
*** jpena is now known as jpena|lunch13:51
pabelangermorning13:52
pabelangerwill be online intermittently this morning, attending OpenStackDayca in Montreal13:53
*** jgriffith has joined #openstack-infra13:53
*** thethorongil has joined #openstack-infra13:55
*** thorst has joined #openstack-infra13:55
*** ad_rien_1 has joined #openstack-infra13:55
*** ad_rien_ has quit IRC13:56
*** lucas-hungry is now known as lucasagomes13:56
*** wolverineav has quit IRC13:56
*** hurgleburgler has joined #openstack-infra13:57
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: TEST: DONT RECHECK: periodic jobs  https://review.openstack.org/35921513:58
*** gongysh has joined #openstack-infra13:58
openstackgerritMerged openstack-infra/shade: Add an e to the word therefore  https://review.openstack.org/40045013:59
*** ramishra has quit IRC13:59
*** ad_rien_1 has quit IRC13:59
*** ad_rien_ has joined #openstack-infra14:00
*** wolverineav has joined #openstack-infra14:00
*** ricardoas has quit IRC14:01
*** ramishra has joined #openstack-infra14:01
*** ricardoas has joined #openstack-infra14:03
*** sree has joined #openstack-infra14:03
*** ccamacho|lunch is now known as ccamacho14:03
*** adarazs_lunch is now known as adarazs14:03
*** jeckersb_gone is now known as jeckersb14:04
*** wolverineav has quit IRC14:06
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Full quickstart gate run on OVB  https://review.openstack.org/38109414:07
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: DONT MERGE: Test quickstart experimental job  https://review.openstack.org/39760914:07
*** hurgleburgler has quit IRC14:07
openstackgerritTung Doan proposed openstack-infra/project-config: Tacker changes to xenial node  https://review.openstack.org/40078014:09
*** thorst has quit IRC14:09
*** Goneri has joined #openstack-infra14:10
*** zul has quit IRC14:10
*** thorst has joined #openstack-infra14:10
*** wolverineav has joined #openstack-infra14:11
*** baoli has joined #openstack-infra14:12
*** abregman has joined #openstack-infra14:12
openstackgerritSorin Sbârnea proposed openstack-infra/jenkins-job-builder: Removed some deprecation warnings and enabled py35 on tox.  https://review.openstack.org/39999414:12
*** matt-borland has joined #openstack-infra14:13
*** daemontool has joined #openstack-infra14:13
*** sree has quit IRC14:13
*** zul has joined #openstack-infra14:14
openstackgerritRob Cresswell proposed openstack-infra/project-config: Update Horizons selenium-headless test  https://review.openstack.org/39986814:14
*** alex_xu has quit IRC14:14
*** jordanP has joined #openstack-infra14:14
fungiihrachys: AJaeger: taking a look at that requirements fix now14:16
*** yamamoto has joined #openstack-infra14:16
*** wolverineav has quit IRC14:16
*** yolanda has joined #openstack-infra14:17
openstackgerritPaul Belanger proposed openstack-infra/nodepool: Re-enable test_hold test  https://review.openstack.org/39975814:17
openstackgerritPaul Belanger proposed openstack-infra/nodepool: Enable test_delete / test_delete_now tests  https://review.openstack.org/39973414:17
fungiihrachys: any feel for whether the devstack revert or the requirements cap should go before the other? i can push them both into the gate and up to the front since they're approved already14:18
*** mriedem has joined #openstack-infra14:18
*** alex_xu has joined #openstack-infra14:18
*** amoralej|lunch is now known as amoralej14:18
fungilooks like devstack change is probably better to put first as it's impacting more than just rally-based jobs14:19
openstackgerritVitaly Gridnev proposed openstack-infra/project-config: fix regex issue for job  https://review.openstack.org/40078414:19
*** thorst has quit IRC14:19
*** nmagnezi has quit IRC14:20
*** zhurong has quit IRC14:21
fungiihrachys: 400465,1 (devstack) is now in the gate at the front and 400183,2 (requirements) is immediately behind it14:22
ihrachysfungi: I think devstack then reqs since reqs patch failed because of devstack14:23
ihrachysfungi: cool, thanks a lot14:23
*** ad_rien_ has quit IRC14:23
fungiboth have workers assigned for all jobs now, eta if neither fails is ~1hr14:23
*** sree has joined #openstack-infra14:25
*** thorst has joined #openstack-infra14:26
*** thorst has quit IRC14:27
*** ad_rien_ has joined #openstack-infra14:28
*** thorst has joined #openstack-infra14:28
*** daemontool has quit IRC14:28
*** wolverineav has joined #openstack-infra14:29
fungi#status log rebooted ask.openstack.org for a kernel update14:29
openstackstatusfungi: finished logging14:29
*** tiswanso has joined #openstack-infra14:29
*** vikrant has quit IRC14:29
*** gouthamr has quit IRC14:30
*** jeckersb is now known as jeckersb_gone14:31
*** ad_rien_ has quit IRC14:32
*** florianf has quit IRC14:32
*** gouthamr has joined #openstack-infra14:32
*** thorst has quit IRC14:33
*** ad_rien_ has joined #openstack-infra14:33
*** Julien-zte has quit IRC14:34
*** jordanP has quit IRC14:35
*** florianf has joined #openstack-infra14:35
*** thethorongil has quit IRC14:35
*** ad_rien_1 has joined #openstack-infra14:35
*** Julien-zte has joined #openstack-infra14:36
*** Julien-zte has quit IRC14:36
*** yolanda has quit IRC14:36
*** Julien-zte has joined #openstack-infra14:36
*** Julien-zte has quit IRC14:37
*** wolverineav has quit IRC14:37
*** jeckersb_gone is now known as jeckersb14:37
*** Julien-zte has joined #openstack-infra14:37
*** Julien-zte has quit IRC14:37
*** ad_rien_ has quit IRC14:37
*** Julien-zte has joined #openstack-infra14:38
*** Julien-zte has quit IRC14:38
*** Julien-zte has joined #openstack-infra14:39
*** Julien-zte has quit IRC14:39
*** Julien-zte has joined #openstack-infra14:39
*** Julien-zte has quit IRC14:40
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add query for libvirt crash corrupt memory bug 1643911  https://review.openstack.org/40079814:40
openstackbug 1643911 in OpenStack Compute (nova) "libvirt randomly crashes on xenial nodes with "*** Error in `/usr/sbin/libvirtd': malloc(): memory corruption:"" [Medium,Confirmed] https://launchpad.net/bugs/164391114:40
*** daemontool has joined #openstack-infra14:41
*** denisra_ is now known as denisra14:41
openstackgerritThomas Herve proposed openstack-infra/project-config: Add a amqp1 job for Heat  https://review.openstack.org/40079914:41
*** hongbin has joined #openstack-infra14:42
*** yolanda has joined #openstack-infra14:43
*** jamesdenton has joined #openstack-infra14:45
*** nadya has joined #openstack-infra14:47
rcarrillocruzhey folks, i won't attend today's infra meeting. My wife won't be around by the time and I'll be taking care of my son for dinner/bath/playing/moarplaying/spongebobswatching/etc14:48
mordredrcarrillocruz: yay spongebob14:49
*** jaosorior has quit IRC14:49
*** jaosorior has joined #openstack-infra14:50
jeblairrcarrillocruz: hey, i took a quick look at the d-g changes -- do you think you can switch it to use each new play as you add it?14:51
*** jaosorior has quit IRC14:51
*** jaosorior has joined #openstack-infra14:51
rcarrillocruzyou mean one play per role invocation, then have an upper playbook that calls them?14:51
mtreinishis anyone else having trouble with the lp auth for gerrit?14:52
*** wolverineav has joined #openstack-infra14:52
fungii lp auth'd to ask.o.o a few minutes ago successfully, but am afraid to log out of review.o.o now :/14:52
rcarrillocruzjeblair: ^14:52
jeblairrcarrillocruz: some of the roles happen one after another, so those can share a playbook14:53
mtreinishfungi: hmm, I'm seeing: http://i.imgur.com/OeNkytr.png14:53
*** vincentll has joined #openstack-infra14:53
*** kashyap has joined #openstack-infra14:53
jeblairrcarrillocruz: (and we might start out with more playbooks than we need, and combine them over time)14:53
*** mmandel has joined #openstack-infra14:53
jeblairrcarrillocruz: but mostly, i just wondered if we can go ahead and start using the roles as we add them (and remove the code they are replacing)14:53
fungimtreinish: that's usually what we get when login.launchpad.net is offline/broken14:53
kashyapclarkb: Hiya, when you get a moment, would be possible to run libvirtd under valgrind in the Gate?14:53
jeblairmtreinish, fungi: lp auth to gerrit wfm14:54
openstackgerritMerged openstack-infra/elastic-recheck: Add query for libvirt crash corrupt memory bug 1643911  https://review.openstack.org/40079814:54
kashyapTrying to debug a memory corruption issue w/ a libvirt dev, they're asking if it's possible to run it under valgrind14:54
*** vincentll has quit IRC14:54
openstackbug 1643911 in OpenStack Compute (nova) "libvirt randomly crashes on xenial nodes with "*** Error in `/usr/sbin/libvirtd': malloc(): memory corruption:"" [Medium,Confirmed] https://launchpad.net/bugs/164391114:54
rcarrillocruzjeblair: i have a change that is 'setup_host' that is a playbook that calls the other changes that define the roles : https://review.openstack.org/#/c/391519/3/playbooks/setup_host.yaml . You mean something like that ?14:54
kashyap^ Heh, it's the above one14:54
stevemarsitting here with mtreinish wondering what the heck he has to log into14:54
jeblairrcarrillocruz: ah, i didn't see that one14:54
rcarrillocruzyes, it's wip'd14:55
cloudnullmornings14:55
rcarrillocruzthat's probably why14:55
mtreinishfungi, jeblair: maybe it's something with the network here at the openstack day ca venue14:55
mtreinishstevemar hit the same thing when he tried it14:55
fungimtreinish: oh, exciting14:55
stevemar(with an incognito session)14:55
mtreinishpabelanger: ^^^ are you able to log in?14:55
fungimtreinish: my guess is that you're all behind a common overload nat and tripped a sanity check on max http requests from one address to lp sso14:56
*** isaacb has quit IRC14:56
*** wolverineav has quit IRC14:56
jeblairrcarrillocruz: what i want to see is the roles actually running in the jobs14:56
fungiespecially given the spam problems we see from accounts coming from there, i would not be surprised if they're pretty draconian about it14:56
fungimtreinish: can probably request whitelisting of the nat address in #launchpad14:57
jeblairrcarrillocruz: if we can do it one at a time, that would make those changes easier to use14:57
jeblairrcarrillocruz: er, easier to review14:57
mtreinishfungi: meh, it's a good excuse to not do tc reviews for the meeting I can't make today :)14:58
mtreinishfungi: or I can just login into my machines at home and use gertty14:58
fungimtreinish: gertty should still work fine, though i'm sure that's not the answer you want to hear ;)14:58
jeblairyeah, i was going to do mtreinish a favor and *not* mention gertty, for once ;)14:58
*** szaher has quit IRC14:58
fungiindeed, gertty should even work fine from the conf net since it doesn't depend on openid14:59
rcarrillocruzso, let's say the 'fix_etc_hosts', would adding an actual playbook calling the 'fix_etc_hosts' role to the change do what you ask?14:59
mtreinishsigh, yeah it's still working :)14:59
jeblairrcarrillocruz: yes, and ideally, remove the old bash code14:59
rcarrillocruzthen in the upper setup_host, instead of having 'roles' invocations, we would include playbooks that tehmselves invoke roles14:59
mtreinishbut it's been a while, so the sync is slow14:59
rcarrillocruzjeblair: ++ sure, i'll amend14:59
jeblairrcarrillocruz: cool, thanks!15:00
*** bin_ has joined #openstack-infra15:01
openstackgerritMerged openstack-infra/nodepool: Add tools/test-setup.sh  https://review.openstack.org/39907915:02
*** szaher has joined #openstack-infra15:02
*** jlibosva has quit IRC15:03
*** dnikishov has joined #openstack-infra15:03
openstackgerrityolanda.robla proposed openstack/diskimage-builder: Dummy change to test feature/v2 branch  https://review.openstack.org/40081015:03
*** jpena|lunch is now known as jpena15:03
*** wolverineav has joined #openstack-infra15:04
*** hichihara has quit IRC15:04
dnikishovhello folks, I've got a couple of changes, which have +2 CR and +2 W, however, they haven't been picked up by zuul's gate jobs15:04
dnikishovis there anything I can do about that?15:04
fungidnikishov: do they have dependencies that aren't in the gate/merged yet?15:04
*** claudiub has joined #openstack-infra15:05
fungidnikishov: or did you set workflow +1 before there were any code review +2 votes, then added cr +2 afterward?15:05
dnikishovno, they don't seem to have these15:05
dnikishovthe changes I need help with are: https://review.openstack.org/368738 https://review.openstack.org/36878015:05
fungidnikishov: a link to one of the changes in question would help15:05
fungidnikishov: another cause could be that zuul isn't configured to run any jobs for those changes15:05
*** Julien-zte has joined #openstack-infra15:06
*** claudiub|2 has quit IRC15:07
*** rbrndt has joined #openstack-infra15:08
*** dtantsur|brb is now known as dtantsur15:10
*** hongbin has quit IRC15:10
*** yolanda has quit IRC15:12
*** wolverineav has quit IRC15:13
kashyapclarkb: Okay, disregard my question above, as Gate jobs would take ages to complete when running libvirtd under valgrind.  (Would only be useful if we can come up with a reproducer) CC: mriedem15:13
mordredrcarrillocruz, jeblair: yah - I agree. it'll make it easier for me to review that the ansible is doing the right thing if I see the shell it's replacing :)15:15
*** thorst has joined #openstack-infra15:17
dnikishovfungi: it seems like workflow +1 happened after the first CR +215:17
fungidnikishov: 368780 depends on a merged change that's marked as "not current" so i wonder if gerrit considers it non-mergeable15:18
*** ad_rien_1 has quit IRC15:19
openstackgerritVitaly Gridnev proposed openstack-infra/project-config: finish transitioning scenario jobs to xenial  https://review.openstack.org/40081415:19
openstackgerritVitaly Gridnev proposed openstack-infra/project-config: [sahara] finish transitioning scenario jobs to xenial  https://review.openstack.org/40081415:20
*** wolverineav has joined #openstack-infra15:20
*** tjones has joined #openstack-infra15:21
*** thorst has quit IRC15:21
*** thorst has joined #openstack-infra15:22
openstackgerritVitaly Gridnev proposed openstack-infra/project-config: [sahara] finish transitioning scenario jobs to xenial  https://review.openstack.org/40081415:22
*** sree has quit IRC15:22
dnikishovfungi: right, thanks15:22
fungidnikishov: yeah, it says its parent is d55b9c9810adf5c0825f3347a9e553d21f744dbc which was patchset 18 of 395042, while patchset 19 is what ended up merging15:22
dnikishovstill have no idea how that happened, but it seems to be ok now15:22
*** sree has joined #openstack-infra15:22
*** yamahata has joined #openstack-infra15:22
*** ad_rien_ has joined #openstack-infra15:23
fungidnikishov: so when patchset 19 of 395042 was pushed to gerrit, nobody rebased 368780 or any of the subsequent children15:23
fungithey were all depending on an older patchset of 395042 which isn't what merged15:23
fungiand gerrit gets picky about that if it has an explicitly tracked dependency on changes which were stacked together at some point15:24
fungizuul will happily run check pipeline tests on it still because it just goes ahead and merges with the branch tip, but when it comes time to enqueue in the gate it asks whether gerrit would be willing to merge that change, and refrains from enqueuing it if not15:25
*** wolverineav has quit IRC15:26
*** saibarspeis has joined #openstack-infra15:26
*** sree has quit IRC15:27
*** thorst has quit IRC15:27
*** tosky has quit IRC15:27
*** yamamoto has quit IRC15:27
*** tosky has joined #openstack-infra15:28
*** ologvinova_ has joined #openstack-infra15:29
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: logging: extract journal to compressed text file, before gathering  https://review.openstack.org/40081715:30
openstackgerritMark Hamzy proposed openstack-infra/project-config: All Python35 issues should be fixed for MoltenIron  https://review.openstack.org/40082015:32
*** dizquierdo has joined #openstack-infra15:35
openstackgerritSorin Sbârnea proposed openstack-infra/jenkins-job-builder: Removed some deprecation warnings and enabled py35 on tox.  https://review.openstack.org/39999415:36
*** azefe has joined #openstack-infra15:36
*** Nrk9t1x has joined #openstack-infra15:37
*** wolverineav has joined #openstack-infra15:37
openstackgerritOleksii Zamiatin proposed openstack-infra/project-config: Fix different zmq configurations for functional tests  https://review.openstack.org/40082315:38
*** NBziouech has quit IRC15:39
*** ccamacho has left #openstack-infra15:40
*** yamamoto has joined #openstack-infra15:41
*** wolverineav has quit IRC15:42
*** xyang1 has joined #openstack-infra15:43
*** chandankumar has quit IRC15:43
*** Alagar has quit IRC15:44
*** Alagar has joined #openstack-infra15:45
*** jkilpatr has quit IRC15:45
*** jkilpatr has joined #openstack-infra15:46
*** daemontool has quit IRC15:46
*** gongysh has quit IRC15:47
*** wolverineav has joined #openstack-infra15:49
*** yolanda has joined #openstack-infra15:52
*** wolverineav has quit IRC15:54
azefeDoes anyone have an experience pushing tag to an openstack project? My project looks like it is configured to do so, but the git command fails "access denied or repository not exported" / "failed to push some refs to"15:54
*** jaosorior has quit IRC15:55
*** dave-mccowan has quit IRC15:55
fungiazefe: http://docs.openstack.org/infra/manual/creators.html#creation-of-tags15:56
openstackgerritJakub Libosvar proposed openstack-infra/project-config: grafana: In Neutron dashboard split Xenial and Trusty graphs  https://review.openstack.org/40065415:56
fungiazefe: also http://docs.openstack.org/infra/manual/drivers.html#tagging-a-release15:57
clarkbif I had to guess you need to sign the tag15:57
fungiyeah, it's one of that or the acl doesn't have pushsignedtag granted to a group in which you're a member15:57
fungiwithout knowing the project name, it's hard to say15:58
*** yamamoto has quit IRC15:59
azefethanks fungi & clarkb , the project (synergy) has pushsignedtag, but you guessed correctly, I didn't sign the tag15:59
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool: Re-enable alien-image-list command and tests  https://review.openstack.org/40083616:00
*** berendt has joined #openstack-infra16:00
*** ihrachys has quit IRC16:01
*** ihrachys has joined #openstack-infra16:01
*** yuval has quit IRC16:02
openstackgerritSorin Sbârnea proposed openstack-infra/jenkins-job-builder: Removed some deprecation warnings and enabled py35 on tox.  https://review.openstack.org/39999416:02
*** ociuhandu has joined #openstack-infra16:02
*** EricGonczer_ has joined #openstack-infra16:03
*** links has quit IRC16:04
*** yolanda has quit IRC16:04
*** inc0 has joined #openstack-infra16:05
*** rcernin_ has quit IRC16:06
*** EricGonczer_ has quit IRC16:06
*** EricGonczer_ has joined #openstack-infra16:07
*** senk has quit IRC16:10
*** wolverineav has joined #openstack-infra16:10
*** dave-mccowan has joined #openstack-infra16:10
*** jlibosva has joined #openstack-infra16:11
*** psachin has quit IRC16:13
openstackgerritOmar Rivera proposed openstack-infra/project-config: Add fuel-plugin-nitrous project  https://review.openstack.org/40042916:14
*** azefe has quit IRC16:14
*** wolverineav has quit IRC16:15
*** Julien-zte has quit IRC16:15
*** Julien-zte has joined #openstack-infra16:16
*** Julien-zte has quit IRC16:16
*** Julien-zte has joined #openstack-infra16:16
*** Guest3787 has quit IRC16:16
*** Julien-zte has quit IRC16:17
*** ociuhandu has quit IRC16:17
*** oanson has quit IRC16:17
*** Julien-zte has joined #openstack-infra16:17
*** Julien-zte has quit IRC16:17
*** Julien-zte has joined #openstack-infra16:18
*** Julien-zte has quit IRC16:18
*** Julien-zte has joined #openstack-infra16:19
*** ologvinova_ has quit IRC16:19
*** ociuhandu has joined #openstack-infra16:19
*** Julien-zte has quit IRC16:19
*** yamamoto has joined #openstack-infra16:19
*** armax has joined #openstack-infra16:19
*** Julien-zte has joined #openstack-infra16:20
*** Julien-zte has quit IRC16:20
clarkbsshnaidm: looks like http://logs.openstack.org/15/393415/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/1236f50/ timed out and logs were collected16:20
clarkbsshnaidm: I notice you have a second change though to remove broken symlinks?16:21
*** Julien-zte has joined #openstack-infra16:21
*** Julien-zte has quit IRC16:21
sshnaidmclarkb, yep, while fixing one problem, discovered another :)16:21
clarkbsshnaidm: we need both changes to make ^ work?16:21
*** Julien-zte has joined #openstack-infra16:21
*** Julien-zte has quit IRC16:21
sshnaidmclarkb, no, only first, the second is for this job i.e.: http://logs.openstack.org/15/393415/9/check/gate-tripleo-ci-centos-7-nonha-multinode/709b186/_zuul_ansible/ansible_log.txt16:22
*** martinkopec has quit IRC16:23
clarkbah ok, will review both shortly16:23
AJaegerrobcresswell: Commented on 399868, approach is fine but needs a few fixes16:23
robcresswellGotcha, thanks AJaeger16:23
*** cody-somerville has joined #openstack-infra16:23
*** cody-somerville has quit IRC16:23
*** cody-somerville has joined #openstack-infra16:23
andreykurilinhi folks! can anyone suggest how to see real-time logs for running job with ipv6 address?16:24
clarkbfungi: jeblair any chance I can have a quick rereview on https://review.openstack.org/#/c/399789/, will try booting that server today finally16:24
clarkbandreykurilin: you'll need an ipv6 address. hurricane electric provides tunnels for free. Rackspace and dreamhost and vexxhost cloud instances come with them too. And finally some ISPs do actually give customers ipv6 out of the box16:25
clarkbandreykurilin: there is work in progress with zuulv3 to proxy those to a webapp thing using websockets but its not implemented yet as far as I know16:25
*** sandanar has quit IRC16:26
mordrednope. not yet16:26
andreykurilinclarkb: got it. thanks!16:26
fungiandreykurilin: fwiw, even though my home isp provides some ipv6 connectivity, it's terrible so i still end up using a free https://tunnelbroker.net/ ipv6-over-gre tunnel from my home firewall to hurricane electric16:27
*** kzaitsev_mb has quit IRC16:28
bkerocomcast is the same way for me :(16:28
sshnaidmclarkb, I ran a job with two these patches, https://review.openstack.org/#/c/393415/  but seems like second didn't work well if it was cloned.. need to check16:28
*** claudiub|2 has joined #openstack-infra16:28
sshnaidmclarkb, but first one works perfectly16:28
*** ociuhandu has quit IRC16:29
andreykurilinfungi: thanks!16:29
clarkbsshnaidm: look at http://logs.openstack.org/15/393415/12/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/ce4351e/_zuul_ansible/ansible_log.txt it seems like the symlink one is working fine too?16:29
*** ociuhandu has joined #openstack-infra16:29
*** yolanda has joined #openstack-infra16:29
clarkbsshnaidm: where do you see problems with it16:29
*** hongbin has joined #openstack-infra16:30
sshnaidmclarkb, in this one http://logs.openstack.org/15/393415/12/check/gate-tripleo-ci-centos-7-nonha-multinode/6aad182/16:30
sshnaidmclarkb, multinode job there16:30
*** Alagar has quit IRC16:30
*** Alagar has joined #openstack-infra16:31
*** yamamoto has quit IRC16:31
*** claudiub has quit IRC16:31
*** yamamoto has joined #openstack-infra16:32
*** kzaitsev_mb has joined #openstack-infra16:32
*** abregman has quit IRC16:33
*** abregman has joined #openstack-infra16:33
*** csomerville has joined #openstack-infra16:34
openstackgerritKen Giusti proposed openstack-infra/project-config: Enable devstack plugins oslo.messaging drivers  https://review.openstack.org/40085616:35
clarkbsshnaidm: we should run cleanup_host on both nodes first, then we synchronize subnode to primary node, then we synchronize primary node logs to log server16:35
clarkbsshnaidm: so its odd that we would get a symlink there16:36
fungijeblair: interesting zuul behavior. i haven't looked at the logs yet but i promoted 400465,1 followed by 400183,2 in the gate and when 400465,1 merged successfully zuul restarted testing on 400183,2 behind it16:37
*** claudiub|2 has quit IRC16:37
sshnaidmclarkb, as I saw we use rsync --archive, I think it'll collect symlinks too16:37
sshnaidmclarkb, these are logs from primary node16:37
*** cody-somerville has quit IRC16:37
clarkbsshnaidm: yes, but if you look at the broken symlink path its a thing that originated on the subnode16:38
sshnaidmclarkb, actually _almost_ all tripleo run on one node16:38
clarkbsshnaidm: and the process is copy from subnodes to primary nodes after running cleanup_host everywhere, then the zuul ansible rsync runs and fails16:38
sshnaidmclarkb, oh, you're right16:38
sshnaidmclarkb, completely missed "subnode-2" there16:39
clarkbsshnaidm: so somehow that symlink is surviving the cleanup_host run on the subnode, then its being copied to the primary node, then failing to copy as part of the zuul playbook16:39
openstackgerritMark Hamzy proposed openstack-infra/project-config: All Python35 issues should be fixed for MoltenIron  https://review.openstack.org/40082016:40
*** EricGonczer_ has quit IRC16:42
*** yolanda has quit IRC16:42
*** yamamoto_ has joined #openstack-infra16:43
*** drifterza has quit IRC16:46
*** dizquierdo has quit IRC16:46
*** yamamoto has quit IRC16:46
*** wolverineav has joined #openstack-infra16:47
amrithfungi, clarkb ping ...16:47
clarkbpong16:47
amrithre: switching to xenial for newton16:47
amrithhiya clarkb16:47
amrithdid you get a chance to see my email on this subject, wanted to close the loop with y'all on this16:48
clarkbI did, I think you have misdirected your comments. The TC says we support these things, infra team tries to enable that (by getting images that work and helping projects move) I have no real say in where we run the tests16:48
*** jascott1 has joined #openstack-infra16:48
clarkbI am happy to help you get running on xenial, but I can't say "its totally ok to keep using 4 year old software"16:49
*** jascott1 has quit IRC16:49
*** senk has joined #openstack-infra16:49
*** matrohon has quit IRC16:49
clarkbI think greghaynes left some comments on how to make the dib situation better too?16:49
*** jascott1 has joined #openstack-infra16:49
amriththe dib situation is not the issue16:49
amriththe issue is that we are proposing to make (or are being told to make) a change to a shipping, stable branch16:50
openstackgerritMerged openstack-infra/system-config: Allow for many translate-dev servers  https://review.openstack.org/39978916:50
amriththat doesn't really meet the criteria for stable branches16:50
pabelangermtreinish: likely late, but I have no auth issues16:50
*** yamamoto_ has quit IRC16:50
clarkbyes you were asked to make that change months before the stable branch was cut though16:50
amriththe dib issues are merely a matter of code.16:50
amrithsorry clarkb, that's water under a bridge now; did I miss it on the ML, maybe yes, or maybe it was not on the ML16:51
clarkbhowever I don't think this is that big a deal16:51
amrithI don't believe that "you were asked to do it"16:51
clarkbjust build trusty guest images on xenial16:51
amriththat's not the configuration that people use the software in16:51
clarkbsince aiui your control plane is happy on xenial already16:51
amriththey use it on trusty16:51
clarkbamrith: yes I am saying use trusty...16:51
amrithon the control plane16:51
*** yolanda has joined #openstack-infra16:51
*** panda is now known as panda|bbl16:52
clarkbamrith: openstack ansible at least says their deployments are using xenial16:52
amrithpeople deploying trove today use trove with trusty on control plane and guest16:52
clarkb(they were one of the first groups to switch to xenial way back in like may)16:52
amrithand they don't test trove16:52
amriththey run tempest tests; period16:52
amrithnot the full ci or any guests16:52
amrithor clusters16:52
clarkbok, I am just going off of the data that is available to me16:52
fungiamrith: "that's not the configuration people run" is a matter to take up with the tc... but also if you're not supporting it they're not going to be running it. catch-2216:53
*** sshnaidm is now known as sshnaidm|mtg16:53
*** inc0 has quit IRC16:53
amrithfungi, we are going to in ocata16:53
amrithand this is the whole issue that we're trying to deal with in the 'release goals' mechanism so we don't have the 'but you should have done this months ago' situation.16:53
*** mhickey has quit IRC16:53
fungiamrith: and i replied to your e-mail just now. i don't think infra will be able to support your stable/newton branch effectively. you'll be on your own trying to keep devstack and other stuff working when it's not tested/gated16:54
amrithoh well16:54
fungiwe could keep providing you trusty nodes to run jobs on, but we can't force the qa team to support that in devstack16:54
clarkband we area laready having trouble doing os16:55
*** pmalik has joined #openstack-infra16:55
clarkbthere is at least one change that breaks compat with mysql 5.5 in trusty16:55
*** wolverineav has quit IRC16:55
clarkband I know nova is wanting to bump min libvirt versions16:55
*** inc0 has joined #openstack-infra16:55
clarkbwe have to move forward at some point. Which is why I think the TC made that decision in the first place16:55
*** vgnbkr has joined #openstack-infra16:55
*** aliadil has joined #openstack-infra16:56
fungiamrith: basically, if openstack is one big integrated ecosystem, then everyone needs to be able to support the same distros on the same branches16:56
mtreinishpabelanger: well, now it's working for me too16:56
mtreinishpabelanger: so you must have fixed it :)16:56
*** jordanP has joined #openstack-infra16:56
*** nherciu_ has joined #openstack-infra16:57
*** hashar is now known as hasharAway16:57
clarkbamrith: I don't think we can really say what your guest image runs, thats like an appliance. But openstack awnts to run on newer software and that means newer control plane16:57
*** nherciu has quit IRC16:57
amrithsounds good fungi I've just released a newton release now; we'll work on getting ocata to xenial. newton when we're done with ocata, time permitting16:58
amrithand clarkb we can say that all we want, the reality is that people deploying this stuff don't treat it as an appliance16:58
amriththey stipulate the os that must run on it; that's our reality16:58
clarkbamrith: right and we say what we will test on...16:58
amrithsounds good; i think that's a good compromise16:59
amrithI'll let stable maint know that in the foreseeable future newton stable branch may break and we'll fix it16:59
clarkb"as an upstream this is what we think you should be doing, etc etc" via the test platforms, requirements, constraints and bindep16:59
*** berendt has quit IRC16:59
*** wolverineav has joined #openstack-infra16:59
clarkbif we let large enterprises dictate our test platform we would still be testing on centos616:59
fungiamrith: going back to the service vm image builds topic, what i proposed to dougshelley66 was to see whether the trove functional test jobs on stable/newton could run on xenial by using pre-built images that were created on trusty17:00
amrithfungi, that doesn't really represent how we'd be telling people to deploy; for example constraint changes (or new things on pypi).17:00
amrithit is for this exact reason that we moved away from cached images17:00
amrithwe used to do that17:00
amrithstuff images on tarballs.o.o17:01
amrithand that became a nightmare;17:01
amriththe CI would pass but locally run tests would fail, deployments would fail17:01
amrithall because the prebuilt images were out of date17:01
dougshelley66fungi I was thinking about our discussion yesterday and it would result in exactly what amrith just indicated17:01
amrithalso the pre built images have trove code17:01
amrithon them17:01
fungii mean as an alternative to not being able to run those jobs on xenial at all17:01
amrithso, what we would effectively be testing was some unicorn17:02
amrithfungi, in the short term, we'll focus on master/ocata17:02
amrithonce we're done with that, we'll have to figure out what to do with stable17:02
*** EricGonczer_ has joined #openstack-infra17:02
amrithbecause stable is complicated by the trove-integration switch which also happened17:02
amrithl, m and n are trove-integration based17:02
amrithand so we have to figure out what we're doing there.17:02
*** Alagar has quit IRC17:02
clarkbamrith: like I siad, my suggestion would be to run the trusty guest image if that is important, the control plane being on xenial vs trusty should be less important to trove (because you make use of neutron and nova and such that care more17:03
amrithand we may end up just doing a no-op for now because no matter what node you give us, we run the guest on trusty anyway17:03
amrithand clarkb, that is how the misleading trove-integration xenial jobs are doing things today17:03
fungiamrith: fair enough. just don't be surprised if you cease being able to run integration and functional test jobs on stable/newton when other repos you depend on cease being able to work there17:03
amrithfungi, understood.17:03
clarkbamrith: misleading?17:03
amrithclarkb, yes; blindly :)17:03
dougshelley66clarkb just for my own understanding. You are saying it is ok for us to now state that the stable/mitake control plane will no longer be tested on trusty even though that's what was supported when it released?17:04
dougshelley66s/mitake/mitaka17:04
amrithno17:04
amrithI think they are only saying newton and master17:04
*** mvandijk has joined #openstack-infra17:04
clarkbdougshelley66: newton, and no the intent was xenial was the supported platform, we switched all unitesting, doc building, linting and the "core" integration tests to xenial in newton17:04
openstackgerritKen Giusti proposed openstack-infra/project-config: Enable devstack plugins oslo.messaging drivers  https://review.openstack.org/40085617:04
amrithmitaka will be trusty, I believe.17:04
dougshelley66ok thanks for the clarification. So the stable branch for mitaka will still be tested on trusty?17:05
clarkbyes17:05
dougshelley66ok understood17:05
amrithso clarkb what's misleading you ask ...17:06
amrithsee http://logs.openstack.org/98/400398/1/check/gate-trove-legacy-functional-dsvm-mysql-newton/ddca412/17:06
amriththis is a job that runs now for newton17:06
amrithit is running on trusty17:06
clarkbyes17:06
fungiright, the stable/mitaka branches of devstack, nova, et cetera are and will continue to be tested on trusty until eol. the challenge is mixing testing some integrated projects on trusty and some on xenial for the stable/newton branch. it stands a very good chance of breaking down quickly17:07
*** ad_rien_ has quit IRC17:08
clarkbamrith: my point is the things that intersect with major differences in the kernel, ovs, etc are what care. Your control planedoes not do this as far as I know. Nova and neutron and magnum et al do and they have already switched17:08
*** ad_rien_ has joined #openstack-infra17:08
clarkband they largely switched before the newton release17:08
fungithe way we avoided that for the precise to trusty transition was just to switch everyone and tell them they had to fix their broken jobs so they could go back to merging changes. our mistake with the trusty to xenial transition was to give projects the option of doing it on their own timeframes17:08
amrithclarkb, I think we have an agreement here ...17:09
amrithwe will fix the stuff17:09
amrithwe realize that in the intervening period, something may break17:09
amrithwe have some ideas on how to get past those things.17:09
amrithbut the priority will be Ocata/Xenial17:09
*** wolverineav has quit IRC17:09
amrithI don't want either stable or infra to be surprised when some newton stable jobs for trove start to fail on Dec 6th or whenever17:09
amrithwe'll try to get it done before that17:10
amrithbut it will be tough; I think.17:10
amrithand fungi I think we agree, the next time something like this comes along, we have the cross-project-release-goals mechanism which we can use.17:10
*** nadya has quit IRC17:11
*** kashyap has left #openstack-infra17:11
*** tqtran has joined #openstack-infra17:11
clarkbin the past we just hard switched everyone, said sorry, and forced you to fix it at that point17:11
clarkbthis time around I think we tried to be more diplomatic which allowed many projects to ignore the problem17:11
clarkbhaving a cross project goal mechanisn will be a good way to straddle both options there and have something that should produce the outcome we want17:12
clarkbwhile not making things too terrible for the projects17:12
*** rbuzatu has quit IRC17:12
openstackgerritRodrigo Duarte proposed openstack-infra/project-config: Enable keystone-saml2-federation service  https://review.openstack.org/40075017:13
amrithclarkb, agreed17:13
*** sshnaidm|mtg is now known as sshnaidm17:13
openstackgerritOmar Rivera proposed openstack-infra/project-config: Add fuel-plugin-nitrous project  https://review.openstack.org/40042917:14
fungiyep, i concur, should be better in the future17:14
*** knangia has joined #openstack-infra17:15
*** tqtran has quit IRC17:16
*** wolverineav has joined #openstack-infra17:16
*** newmember has joined #openstack-infra17:17
*** asettle has quit IRC17:18
*** dtantsur is now known as dtantsur|afk17:18
*** sdake_ has quit IRC17:18
*** chem has quit IRC17:19
*** chem has joined #openstack-infra17:19
*** weshay is now known as weshay_brb17:20
*** wolverineav has quit IRC17:20
*** wolverineav has joined #openstack-infra17:21
*** inc0 has quit IRC17:22
*** gomarivera has joined #openstack-infra17:22
*** mmandel has quit IRC17:22
*** sdake has joined #openstack-infra17:24
*** lucasagomes is now known as lucas-afk17:25
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Full quickstart gate run on OVB  https://review.openstack.org/38109417:25
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: DONT MERGE: Test quickstart experimental job  https://review.openstack.org/39760917:25
*** wolverineav has quit IRC17:26
*** wolverineav has joined #openstack-infra17:28
gomariveraquick question - I'm fairly new contributor - this is my first review opened on openstack-infra - https://review.openstack.org/#/c/400429/ - should I add the repository group to my review? - what's the best way to notify the approvers group?17:28
*** weshay_brb is now known as weshay17:28
*** dprince has quit IRC17:29
*** tqtran has joined #openstack-infra17:30
*** dprince has joined #openstack-infra17:30
sshnaidmclarkb, I think this symlink was not-broken on subnode and pointing to some file there (abs. path), but when it was copied to primary node it started to point to non-existing file and turned to be broken. But I'm not sure why it was copied and not file itself..17:30
*** rossella__ has quit IRC17:30
clarkbsshnaidm: we use the ansible synchronize module, maybe we need to add a flag there to follow symlinks too17:31
*** hurgleburgler has joined #openstack-infra17:31
clarkbsshnaidm: look in the -wrap.sh script at the end to see the synchronize call17:31
*** rossella__ has joined #openstack-infra17:31
clarkbgomarivera: typically reviewers get to things as they are able. Pinging for reviewers here also happens a bit though not sure how effective it is. I know I personally can't keep up with gerrit email so I ignore when I am added as a reviewer directly17:32
*** wolverineav has quit IRC17:32
openstackgerritJames E. Blair proposed openstack-infra/system-config: Add a cron job to release AFS volumes  https://review.openstack.org/40088717:33
jeblairmordred, clarkb, fungi, pabelanger, AJaeger: ^17:33
sshnaidmclarkb, you're right, need to add "copy-links=yes" to ansible params, will do this17:33
*** rbuzatu has joined #openstack-infra17:33
clarkbsshnaidm: thanks17:34
openstackgerritJames E. Blair proposed openstack-infra/system-config: Add a cron job to release AFS volumes  https://review.openstack.org/40088717:34
fungijeblair: i guess this was to avoid having a slew of redundant vos release jobs for that volume piling up in the post pipeline and choking the release node?17:35
mordredjeblair: does the specific node trump the regex node in puppet?17:35
*** fguillot has quit IRC17:36
gomarivera@clarkb: thank you for the tips17:36
fungiyeah, i've always wondered if that's a first-match situation, or if resources get merged17:36
*** rbuzatu has quit IRC17:38
*** fguillot has joined #openstack-infra17:38
*** Alagar has joined #openstack-infra17:39
clarkbsshnaidm: looks like its copy_links in the ansible docs, not sure if - and _ are treated as equivalent there17:39
*** yolanda has quit IRC17:40
*** gomarivera has quit IRC17:40
*** nherciu_ has quit IRC17:41
*** inc0 has joined #openstack-infra17:42
*** wolverineav has joined #openstack-infra17:43
*** rbuzatu has joined #openstack-infra17:43
*** mmandel has joined #openstack-infra17:44
jeblairfungi: yes; we can probably revisit after zuulv317:44
jeblairmordred: i hope? :)17:44
*** rbuzatu has quit IRC17:48
*** flepied has quit IRC17:48
*** jlibosva has quit IRC17:50
*** wolverineav has quit IRC17:50
*** ldnunes has quit IRC17:51
*** ldnunes has joined #openstack-infra17:52
*** nherciu has joined #openstack-infra17:54
AJaegerjeblair, fungi: I tried to document the tools/test-setup work we discussed on https://etherpad.openstack.org/p/su4PMQ9kN9 - want to send that to openstack-infra mailing list as heads up and for discussion. Could you quickly review, please?17:55
*** raildo has quit IRC17:57
*** wolverineav has joined #openstack-infra17:58
*** dprince has quit IRC17:58
*** weshay is now known as weshay1-117:59
*** dprince has joined #openstack-infra17:59
*** abregman is now known as abregman|afk17:59
*** raildo has joined #openstack-infra17:59
*** derekh has quit IRC18:01
*** EricGonczer_ has quit IRC18:02
*** EricGonczer_ has joined #openstack-infra18:02
*** baoli has quit IRC18:02
*** ralonsoh has quit IRC18:03
*** baoli has joined #openstack-infra18:03
*** Guest98237 has quit IRC18:04
*** EricGonczer_ has quit IRC18:05
*** nt has quit IRC18:06
*** EricGonczer_ has joined #openstack-infra18:06
*** nadya has joined #openstack-infra18:07
*** nt has joined #openstack-infra18:07
*** wolverineav has quit IRC18:07
*** kzaitsev_mb has quit IRC18:07
*** wolverineav has joined #openstack-infra18:08
*** hyakuhei has quit IRC18:08
*** Alagar has quit IRC18:11
openstackgerritVitaly Gridnev proposed openstack-infra/project-config: fix regex issue for job  https://review.openstack.org/40078418:13
*** wolverineav has quit IRC18:13
*** Alagar has joined #openstack-infra18:14
*** hyakuhei has joined #openstack-infra18:15
*** psilvad has quit IRC18:15
*** sshnaidm is now known as sshnaidm|away18:16
*** maishsk has joined #openstack-infra18:18
*** maishsk has quit IRC18:18
*** esikachev has quit IRC18:19
*** ihrachys has quit IRC18:19
*** wolverineav has joined #openstack-infra18:19
*** senk has quit IRC18:19
clarkbsshnaidm|away: I went ahead and left a -1 on the devstack-gate change so we don't forget what the fix is there for multinode18:20
*** yolanda has joined #openstack-infra18:20
*** weshay1-1 is now known as weshay18:20
*** senk has joined #openstack-infra18:21
*** e0ne has quit IRC18:22
*** ad_rien_ has quit IRC18:22
*** yolanda has quit IRC18:23
*** yolanda has joined #openstack-infra18:23
*** wolverineav has quit IRC18:24
*** yamahata has quit IRC18:24
openstackgerritAnusha Unnam proposed openstack-infra/project-config: Do not merge: Test patch to check post_hook script  https://review.openstack.org/40092618:24
*** shardy has quit IRC18:27
*** saibarspeis has quit IRC18:27
*** saibarspeis has joined #openstack-infra18:27
*** hyakuhei has quit IRC18:28
*** hyakuhei has joined #openstack-infra18:28
*** hyakuhei has quit IRC18:28
*** hyakuhei has joined #openstack-infra18:28
*** oanson has joined #openstack-infra18:28
AJaegerjeblair: any feedback on the etherpad - or ok to send to openstack-infra? Or do you propose another way of sharing?18:29
*** rbuzatu has joined #openstack-infra18:29
*** flwang1 has joined #openstack-infra18:30
*** nadya has quit IRC18:30
clarkbAJaeger: it lgtm. Might be worth thinkingabout how that may fit in with swift's xfs stuff but I don't think it needs to go in the email or be used for that today18:31
jeblairclarkb: agreed18:31
*** wolverineav has joined #openstack-infra18:31
jeblairAJaeger: seems good -- i'd ask for fungi's review18:31
openstackgerritJimmy McCrory proposed openstack-infra/system-config: Refactor mariadb mirror into 2 repos  https://review.openstack.org/35420318:32
*** florianf has quit IRC18:32
fungiyep, it's on my radar just trying to get meeting-day stuff behind me18:32
AJaegerthanks, clarkb and jeblair. Will check the swift setup and add a paragraph.18:32
fungii'll look quickly so i don't hold you up18:32
notmynameis that proposed tools/test-setup.sh in the project repo or in an infra repo?18:32
*** jpena is now known as jpena|off18:32
*** asettle has joined #openstack-infra18:32
clarkbnotmyname: you would put the tools/test-setup.sh in your repo (swift) then the python unittest jobs would automatically run it if it is present18:33
clarkbfungi: ianw: is there any gerrit project renaming prep work I can help with this morning?18:33
*** rbuzatu has quit IRC18:34
fungiclarkb: it's only one project, so i figured we were just going to wing it in the window18:34
clarkbfungi: ok18:34
*** yolanda has quit IRC18:34
*** thorst has joined #openstack-infra18:34
notmynameAJaeger: want me to ask in here or on the etherpad?18:35
fungiclarkb: though a reread of the channel log from our last rename maintenance might be in order. i think i may have identified bugs/oversights in the playbook that i never got around to producing patches for18:35
*** tosky has quit IRC18:35
AJaegernotmyname: feel free to add a comment on the etherpad - but hearing you, I'll update...18:35
AJaegerjeblair, clarkb I added "The xfs TMPDIR setup of swift [2] could been done in general this way as well but that change needs to set TMPDIR for the unittests, just pass information from the set up builder to the tox builder. This is currently not possible."18:35
clarkbAJaeger: thanks18:35
*** wolverineav has quit IRC18:36
*** wolverineav has joined #openstack-infra18:36
*** flepied has joined #openstack-infra18:36
notmynameAJaeger: I see you said setting the TMPDIR isn't possible. but the reason I did that is because it avoided coordination of magic strings across different repos. swift could still use mkstemp() and if TMPDIR is set or not, it works. however, if the directory is set in the same repo (in test-setup.sh in swift), then the tests in swift can use that too (right?)18:37
notmynamepoint is, if swift is doing the setup in swift (instead of in an infra repo), then it gets much easier to set it in one place and Just Work18:37
*** wolverin_ has joined #openstack-infra18:37
notmynameI think18:37
*** thorst has quit IRC18:39
*** Nrk9t1x has quit IRC18:39
AJaegernotmyname: yes, you could do it this way: create directoy in tools/test-setup.sh and in your tox.ini file set TMPDIR to the directoy you used in tools/test-setup.sh.18:39
clarkbhttp://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-09-02.log.html#t2016-09-02T18:02:03 is where the logs start for the last project rename18:39
clarkbrereading through that now18:39
funginotmyname: yeah, if you used some other mechanism to communicate what TMPDIR should be used, then you could leverage a generalized solution like this18:39
AJaegerfungi: or hardcode it18:40
clarkbfungi: https://review.openstack.org/#/c/365067/ looks to be the set of docs we were operating with18:40
fungiAJaeger: yep, i considered hard-coded strings one way to "communicate" it18:40
*** thorst has joined #openstack-infra18:40
AJaegerok18:40
fungiAJaeger: in general the message lgtm. i made a few edits/additions for clarity18:41
openstackgerritAndy McCrae proposed openstack-infra/project-config: Skip ubuntu-trusty jobs for OpenStack-Ansible on Ocata+  https://review.openstack.org/40019718:41
*** wolverineav has quit IRC18:41
notmynameAJaeger: fungi: so that proposed idea seems to be pretty interesting (with my limited understanding). sounds like it would have been a pretty nice thing to have a week or two ago ;-)18:41
AJaegerfungi, thanks!18:41
AJaegernotmyname: yeah ;)18:41
fungiclarkb: oh! hah, so maybe i did propose one and never got around to finishing it18:41
fungilooks like my inline comments on that change are relevant18:42
AJaegerfungi, sent to openstack-infra as FYI?18:42
AJaegerfungi: thanks for your edits!18:42
clarkbfungi: it looks like comments there about slashes and .git things are related to the playbook breaking18:42
clarkbfungi: so ya I think we need thsoe updates if we expect things to work correctly18:42
fungiAJaeger: just sending to -dev is probably fine, but with [all] and [infra] subject tags maybe18:43
*** trown is now known as trown|lunch18:43
AJaegerfungi: ok, will do.18:43
fungiclarkb: have time to propose an updated patchset? otherwise we can just use a locally edited copy18:43
clarkbfungi: ya I can work on that now18:43
fungithanks! at least my playbook comments seem to be pretty straightforward edits18:44
fungiclarkb: and the group renaming piece probably needs to just get ripped out for now18:44
fungiwe can revert to the old manual version using the ssh api with our own credentials or something18:45
*** thorst has quit IRC18:45
*** strigazi is now known as strigazi_AFK18:46
*** Alagar has quit IRC18:46
*** florianf has joined #openstack-infra18:46
clarkbfungi: trying to figure out what these things are is not easy18:47
*** oanson has quit IRC18:47
*** Sukhdev has joined #openstack-infra18:48
fungiclarkb: i think there was additional context around them in the channel log18:48
clarkboh right item is magical ansibleness18:48
fungiand glittery magical ansible unicorns, right18:48
*** Alagar has joined #openstack-infra18:49
AJaegerclarkb: could you later review https://review.openstack.org/#/c/400197/5/zuul/layout.yaml for an ansible xenial change, please?18:50
*** jgriffith has quit IRC18:51
clarkbAJaeger: yes18:51
*** rbuzatu has joined #openstack-infra18:53
*** dprince has quit IRC18:53
*** pvaneck has joined #openstack-infra18:53
clarkbmordred: is remote_user: root paired with sudo: yes something we want?18:54
clarkbmordred: I fail to understand why root would need to sudo18:54
*** yolanda has joined #openstack-infra18:55
fungiclarkb: worth noting, things i left comments on in that patch were things i specifically _had_ to change or the playbook did not work18:55
clarkbfungi: ya just noticing we have remote_user: root and sudo: yes in a lot of places18:56
fungithose weren't just general improvements18:56
*** psilvad has joined #openstack-infra18:56
fungiclarkb: agreed, that strikes me as odd too18:56
clarkbfungi: re the group renaming, that task sshes to review.o.o becomes gerrit2 then sshes to gerrit via the local gerrit user which I think is valid18:56
*** ijw has joined #openstack-infra18:56
clarkbfungi: but going to confirm that18:56
clarkb(I think it failed for you because puppetmaster can't ssh to gerrit2@review.o.o so need to set remote_user: root and become_user: gerrit218:57
*** asettle has quit IRC18:57
*** rbuzatu has quit IRC18:57
*** bin_ has quit IRC18:57
openstackgerritJames E. Blair proposed openstack-infra/puppet-openstackci: Use nodepool::builder for nodepool  https://review.openstack.org/35650318:57
*** asettle has joined #openstack-infra18:58
jeblairclarkb: all the things you are saying make sense to me18:58
fungiclarkb: ahh, right-o18:58
*** yolanda has quit IRC18:59
fungiit's that (weekly infra team meeting) time again! find us in #openstack-meeting for the next hour19:00
*** senk has quit IRC19:00
AJaegerteam, for today's rename I would appreciate a second +2 on https://review.openstack.org/#/c/381332 - I'll remove the WIP now. Also, there's a followup change at https://review.openstack.org/#/c/399993/ for review, please19:00
*** tiswanso has quit IRC19:00
*** tiswanso has joined #openstack-infra19:01
*** jordanP has quit IRC19:01
johnsomtonyb Have they answered your concerns on https://review.openstack.org/#/c/386790?19:01
*** asettle has quit IRC19:02
*** inc0 has quit IRC19:02
*** ijw has quit IRC19:03
*** denisra has quit IRC19:03
*** sdake_ has joined #openstack-infra19:03
*** newmember has quit IRC19:04
*** yamahata has joined #openstack-infra19:05
*** sdake has quit IRC19:06
*** dprince has joined #openstack-infra19:08
*** electrofelix has quit IRC19:09
*** inc0 has joined #openstack-infra19:09
*** rbrndt has quit IRC19:10
openstackgerritMichael Johnson proposed openstack-infra/project-config: Consolidate neutron-lbaas API tests  https://review.openstack.org/38624519:11
*** ijw has joined #openstack-infra19:12
openstackgerritClark Boylan proposed openstack-infra/system-config: Update project rename instructions  https://review.openstack.org/36506719:12
*** dizquierdo has joined #openstack-infra19:14
*** esikachev has joined #openstack-infra19:15
*** psilvad has quit IRC19:15
*** yolanda has joined #openstack-infra19:16
*** denisra has joined #openstack-infra19:18
*** psilvad has joined #openstack-infra19:18
*** inc0 has quit IRC19:19
*** esikachev has quit IRC19:19
*** denisra_ has joined #openstack-infra19:21
*** jtomasek has quit IRC19:21
*** jtomasek has joined #openstack-infra19:22
*** hasharAway is now known as hashar19:22
*** Alagar has quit IRC19:22
*** flwang1 has quit IRC19:24
*** rbuzatu has joined #openstack-infra19:24
*** denisra has quit IRC19:24
*** nadya has joined #openstack-infra19:26
*** e0ne has joined #openstack-infra19:26
*** trown|lunch is now known as trown19:26
*** EricGonczer_ has quit IRC19:27
*** arxcruz has quit IRC19:28
*** matrohon has joined #openstack-infra19:28
*** rbuzatu has quit IRC19:28
*** thorst has joined #openstack-infra19:29
*** denisra_ has quit IRC19:29
*** psilvad has quit IRC19:31
*** yolanda has quit IRC19:32
*** denisra has joined #openstack-infra19:33
*** rossella__ has quit IRC19:33
*** thorst has quit IRC19:33
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: CI test - never merge  https://review.openstack.org/40094419:34
*** denisra has quit IRC19:37
*** psilvad has joined #openstack-infra19:41
*** yolanda has joined #openstack-infra19:42
*** senk has joined #openstack-infra19:43
openstackgerritJimmy McCrory proposed openstack-infra/system-config: Refactor mariadb mirror into 2 repos  https://review.openstack.org/35420319:43
*** nherciu has quit IRC19:44
*** esikachev has joined #openstack-infra19:45
*** rbrndt has joined #openstack-infra19:45
*** amoralej is now known as amoralej|off19:46
*** EricGonczer_ has joined #openstack-infra19:47
*** denisra has joined #openstack-infra19:48
*** psilvad has quit IRC19:53
*** yolanda has quit IRC19:54
*** boden has joined #openstack-infra19:54
*** chlong has quit IRC19:55
*** odyssey4me has quit IRC19:56
*** psilvad has joined #openstack-infra19:56
bodenhi AJaeger sigmavirus: any idea how I can get some eyes on an openstack-dev/hacking patch? https://review.openstack.org/#/c/397940/19:56
*** oanson has joined #openstack-infra19:57
AJaegerboden: I'm not reviewing that repo at all...19:58
bodenAJaeger ack.. sorry I saw your name on some recent commits19:59
*** panda|bbl is now known as panda20:01
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: logging: extract journal to compressed text file, before gathering  https://review.openstack.org/40081720:01
bodendhellmann jd__ ^^20:01
clarkbmordred: eyes on https://review.openstack.org/365067 particularly around the removal of sudo: yes and the use of pause would be great20:01
fungiboden: probably ask in #openstack-qa since it's a qa repo not infra20:01
*** chlong has joined #openstack-infra20:01
clarkbmordred: I know you are TCing20:01
*** baoli has quit IRC20:01
* AJaeger waves good night20:01
bodenfungi: got it. thx20:01
clarkbAJaeger: good night20:01
*** tiswanso has quit IRC20:02
*** tiswanso has joined #openstack-infra20:02
*** odyssey4me has joined #openstack-infra20:02
mordredclarkb: I _believe_ we want become: yes\nbecome_user: gerrit20:02
openstackgerritNate Johnston proposed openstack-infra/project-config: neutron-fwaas: Consume gate hook for tempest plugin jobs  https://review.openstack.org/40074520:02
clarkbmordred: you are right and that is silly20:03
clarkbbut at least the docs document it20:03
*** psilvad has quit IRC20:04
*** vaidy has quit IRC20:04
openstackgerritClark Boylan proposed openstack-infra/system-config: Update project rename instructions  https://review.openstack.org/36506720:04
*** sambetts is now known as sambetts|afk20:05
*** baoli has joined #openstack-infra20:05
*** isviridov_away has quit IRC20:05
*** inc0 has joined #openstack-infra20:06
mgagnepabelanger: I see you encountered this issue in the past: "LaunchNetworkException: Unable to find public IP of server" What was the problem and solution?20:08
*** dave-mccowan has quit IRC20:10
*** rhallisey has quit IRC20:12
*** Goneri has quit IRC20:13
*** ldnunes has quit IRC20:15
mgagnelooks like 'public_v4' is empty in the server object returned by self.manager.createServer or self.manager.waitForServer20:17
*** senk has quit IRC20:17
clarkbmgagne: you might need to explicitly specify a network as public20:19
mgagneit's already done20:19
*** dmanchad has quit IRC20:19
mordredoh. well that's not good then20:19
mgagneusing a config that used to work until I upgraded20:19
mordredmgagne: I'm in a meeting at the moment, but am very much interested in getting to the bottom of your problem20:19
mgagnemaybe there is a regression in the old config syntax (before clouds.yaml)20:20
*** jcoufal has quit IRC20:20
mgagneyea, now we are out of slaves =)20:20
*** jpena|off has quit IRC20:20
mgagneI guess I could downgrade and see where it goes20:20
*** jschwarz has quit IRC20:20
*** chem has quit IRC20:21
*** chem has joined #openstack-infra20:21
mordredmgagne: if you have a sec, could you paste a copy of your clouds.yaml (scrubbed of auth info of course) and then also run a script like http://paste.openstack.org/show/590139/ and capture its log output?20:23
*** rbuzatu has joined #openstack-infra20:23
mgagnemordred: I don't have a clouds.yaml =)20:23
openstackgerritThomas Herve proposed openstack-infra/project-config: Add a amqp1 job for Heat  https://review.openstack.org/40079920:23
mordredmgagne: oh. weird. ok - then can you just paste your nodepool.yaml snippet?20:24
mgagnemordred: that's why I suspect a regression in the compat code or in combination with shade20:24
mordredthat shows the network config?20:24
mgagnesure20:24
mordredyah. we may have missed a case for sure. :(20:24
Shrewsmgagne: what shade version did you upgrade from?20:25
*** ajo has quit IRC20:25
Shrewsand to?20:25
*** ajo has joined #openstack-infra20:25
mgagnelooks like shade didn't get upgraded20:25
mgagnecurrently at shade==1.13.120:26
mgagnewas running nodepool==0.2.1.dev51 before that20:26
*** adriant has joined #openstack-infra20:26
mgagnewhich is 1fa3ee77791f7c5dd4faa38a2a8012662f6e88a820:26
jd__boden: reviewed, looks legit :)20:26
Shrewsoh, a nodepool upgrade. i see20:26
*** jschwarz has joined #openstack-infra20:26
openstackgerritThomas Morin proposed openstack-infra/project-config: Grafana fixes for n8g-(bagpipe|bgpvpn)  https://review.openstack.org/40095320:27
mgagnemordred: provider config: http://paste.openstack.org/show/590141/20:27
bodenjd__ thanks for taking the time20:27
mordredmgagne: thanks!20:27
openstackgerritMerged openstack-infra/project-config: Skip ubuntu-trusty jobs for OpenStack-Ansible on Ocata+  https://review.openstack.org/40019720:27
mgagneShrews: yea, maybe it's related to 0.3.X20:27
mgagneI don't mind updating to use clouds.yaml but docs shows that old config should still work so I got lazy ¯\_(ツ)_/¯20:28
mordred:)20:28
mordredmgagne: I agree - it is surprising to me that it stopped working and I apologize for that20:28
*** rbuzatu has quit IRC20:28
mgagneno problem, hopefully my coworker wajdi wasn't performing a live demo during his talk on CI at the openstack canada day =)20:29
mordredhahahah20:29
mgagne"now we just have to wait for Jenkins to run the job... maybe wait a little more..."20:30
*** jpena|off has joined #openstack-infra20:30
*** florianf has quit IRC20:30
*** dmanchad has joined #openstack-infra20:32
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool: Re-enable test: test_list_nodes  https://review.openstack.org/40095520:32
*** mhickey has joined #openstack-infra20:34
*** pradk has quit IRC20:35
mgagnemordred: the object/data returned by shade has the IP in private_v4, not public_v4.20:36
mordredmgagne: yah. looking at things, you were on a VERY old nodepool20:36
mgagneI think it was the first iteration of the public: true flag20:36
mordredyah- I believe between now and then we gave up on allowing setting that in the nodepool config because it got VERY weird20:37
*** flwang1 has joined #openstack-infra20:37
mgagneshould I just upgrade to clouds.yaml and call it a day?20:37
mordredyes please20:37
mordredand I'll fix the docs20:37
mgagnealright, thanks!20:37
mordredmgagne: sorry for the trouble :(20:38
mgagneno problem, with responsive support, it's much better than cursing alone against a screen =)20:38
mordred\o/ - also, I can attest that the clouds.yaml way works with internap well ... given that we use it :)20:39
*** raildo has quit IRC20:39
mgagneyea, we don't consume our public cloud, private cloud with similar setup (ip is directly reachable, no FIP)20:40
*** abregman|afk has quit IRC20:40
openstackgerritJulius Roberts proposed openstack-infra/shade: Added documentation for delete_image()  https://review.openstack.org/40051920:41
mordred++20:42
hooliow00t! :)20:43
pabelangermgagne: Not the best help today, mingling with your co-workers at openstackdayCA :)20:44
* sc68cal claps as the gate delay goes from 5hrs to 1hr20:44
* mordred hands sc68cal a beer20:44
mgagnepabelanger: hehe, no problem. I reverted to older nodepool and will use clouds.yaml as suggested by mordred20:44
pabelangerI seen that, what was the issue of delay today?20:44
clarkbmordred: the nodepool option is supposed to work iirc, and wr use it20:45
mordredclarkb: I see no code in nodepool that does anything with it20:45
clarkbwe use it in osic20:46
jeblairplacebo option?20:46
clarkbhttps://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/nodepool.yaml#n75320:46
clarkbmay be20:46
mordredclarkb: it's vestigal20:46
mordredwe have the clouds.yaml options20:46
mordredb38290421483c19428910c31fe4c918229c14d86 removed the place where we set routes_externally=True20:47
clarkbok so we need to clean up our config then20:47
mordredyah20:49
mordredand the nodepool docs that mention that option20:49
*** jeckersb is now known as jeckersb_gone20:51
*** pradk has joined #openstack-infra20:52
*** inc0 has quit IRC20:52
*** oanson has quit IRC20:53
*** oanson has joined #openstack-infra20:54
*** mhickey has quit IRC20:55
openstackgerritMerged openstack-infra/shade: Try to return working IP if we get more than one  https://review.openstack.org/39014420:55
*** rtheis has quit IRC20:57
fungiokay, i guess it's time to start doing gerritstuffs?21:00
*** newmember has joined #openstack-infra21:01
openstackgerritPaul Belanger proposed openstack-infra/nodepool: Re-enable TestWebApp tests  https://review.openstack.org/39971621:01
openstackgerritPaul Belanger proposed openstack-infra/nodepool: Remove waitForBuiltImages() / JobTracker() from nodepool.py  https://review.openstack.org/39972721:01
openstackgerritPaul Belanger proposed openstack-infra/nodepool: Add test to validate when a node build is disabled  https://review.openstack.org/39964221:01
*** inc0 has joined #openstack-infra21:01
fungi%status alert Gerrit is offline until 21:30 UTC for scheduled maintenance: http://lists.openstack.org/pipermail/openstack-dev/2016-November/107379.html21:01
*** kzaitsev_mb has joined #openstack-infra21:01
fungiianw: clarkb: ^21:01
fungi?21:01
clarkb++21:02
ianwok, now what :)21:02
*** dmanchad has quit IRC21:02
fungiwe need to stop and dump zuul queues unless we think we can make this outage really brief21:02
fungii'll go ahead and stop ansible now21:02
clarkbI think we should follow the doc21:03
clarkbwill help keep us honest21:03
*** dmanchad has joined #openstack-infra21:03
fungiyes, agreed21:03
openstackgerritPaul Belanger proposed openstack-infra/nodepool: Re-enable test_hold test  https://review.openstack.org/39975821:04
ianwwe need to merge 365067?21:04
clarkbhttp://docs-draft.openstack.org/67/365067/3/check/gate-system-config-docs-ubuntu-xenial/19b394e//doc/build/html/gerrit.html#renaming-a-project21:04
fungithe rename change gets approved/merged after we bring gerrit back online21:04
clarkbfungi: thats the playbook/docs update change21:05
*** dprince has quit IRC21:05
fungioh, that. i would check it out and run a local copy21:05
fungithere may still be more edits we discover it needs21:05
*** saibarspeis has quit IRC21:05
ianwplaybook updtae i mean21:05
clarkbya then we can update if we have more issues. Also my changes to the playbook are not tested ...21:05
*** Goneri has joined #openstack-infra21:05
fungiclearly the previous state of that playbook wasn't tested either, which is why i started that change in the first place21:05
fungipuppet is disabled globally for the moment and isn't running anywhere21:06
*** hoolio has left #openstack-infra21:06
fungii'll go ahead with the status alert21:06
fungi#status alert Gerrit is offline until 21:30 UTC for scheduled maintenance: http://lists.openstack.org/pipermail/openstack-dev/2016-November/107379.html21:06
openstackstatusfungi: sending alert21:06
*** hoolio has joined #openstack-infra21:06
fungiwho wants to export zuul queues?21:07
clarkbI can do that21:07
clarkbunless ianw wants to do it for learning21:07
ianwi'm ok with that bit :)21:07
*** weshay_ has joined #openstack-infra21:08
clarkbianw: so I should go ahead and do it?21:08
fungijust keeping track: #1 and #3 are done. i can work on #221:08
ianwyep21:08
clarkbI am doing #4 now21:08
ianwfungi / clarkb: shared screen on puppetmaster?21:08
*** baoli has quit IRC21:08
-openstackstatus- NOTICE: Gerrit is offline until 21:30 UTC for scheduled maintenance: http://lists.openstack.org/pipermail/openstack-dev/2016-November/107379.html21:08
*** ChanServ changes topic to "Gerrit is offline until 21:30 UTC for scheduled maintenance: http://lists.openstack.org/pipermail/openstack-dev/2016-November/107379.html"21:09
fungiianw: sure, but start screen as root so it's easier21:09
fungiunless we shouldn't run the playbook as root21:09
ianwi've started one as root21:09
clarkbfungi: ianw queues saved. Ready for me to stop zuul now?21:09
clarkbhello?21:10
fungiclarkb: yes, go for it21:10
fungiianw: looks like you're working on #2 so i'll leave you to that21:10
clarkbok done21:10
*** jamesdenton has quit IRC21:11
ianwdo we need core & release in the groups?  i guess so21:11
fungiyep21:11
fungican find them looking in the change's acl21:12
openstackstatusfungi: finished sending alert21:12
fungilooks right to me21:13
*** jgriffith has joined #openstack-infra21:13
clarkboh sorry I connected to an old screen of mine (for zanata dev server) will hop on root one now21:13
pabelangerclarkb: ianw: fungi: are we restarting zuul today?21:14
ianwpabelanger: rename21:14
fungiyou have a checkout of 365067 to use?21:14
clarkbpabelanger: yes its already stopped21:14
ianwnot yet21:14
clarkbpabelanger: its part of our project rename steps21:14
pabelangerright, thanks21:14
*** Apoorva has joined #openstack-infra21:14
pabelangerjust confirming, as it might fix our allocation calculations (if we think that was the issue yesterday)21:15
clarkbianw: fungi why did that have to merge?21:15
*** esikachev has quit IRC21:15
fungiianw: git fetch https://git.openstack.org/openstack-infra/project-config refs/changes/32/381332/9 && git checkout FETCH_HEAD21:15
fungiclarkb: i don't think it did have to merge21:15
ianwi think cause i pulled21:16
clarkbah yes I never pull and keep master pristine21:16
fungimeh, as long as that one file is corrected i don't think we care about the rest of the repo state anyway21:16
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool: [WIP] Re-enable test: test_image_upload_fail  https://review.openstack.org/40097021:16
clarkbfungi: ianw also after running ansible zaro's comments on https://review.openstack.org/#/c/365067/1/doc/source/gerrit.rst tell us how to do the indexing21:16
fungiright21:17
ianwok ... should we try it?21:17
*** jgriffith has quit IRC21:17
fungiskimming it one last time21:18
fungiwe pulled out the line to start the offline index so i'm good with it21:18
clarkbthe pause has me skeptical21:19
fungianything else we can work around if something goes sideways21:19
clarkbnot sure if that is valid yaml actually21:19
clarkbyay ansible21:19
clarkbI can test that really quickly if we want21:19
clarkbor if it fails then we deal with it then21:19
ianwERROR! 'prompt' is not a valid attribute for a Task21:20
*** jgriffith has joined #openstack-infra21:20
clarkbya dangit ansible21:21
clarkbianw: change it to pause: prompt="string here"21:21
*** srobert has joined #openstack-infra21:22
fungiahh, yeah the problem was the prompt was added in a list rather than as a straight dict key21:22
EmilienMgerrit 50321:22
fungiEmilienM: see /topic21:22
EmilienMfungi: oops my bad21:23
*** rbuzatu has joined #openstack-infra21:23
jheskethMorning21:23
clarkbianw: fungi for this step I think you can just do ssh -p29418 review.o.o gerrit ls-projects locally21:23
*** jpich has quit IRC21:23
clarkband once that works do ^21:23
clarkber do the thing in screen21:24
fungiagreed21:24
*** oanson has quit IRC21:24
fungilooks like it's there now21:25
clarkbya working for me21:25
ianwyep, me too, let's go on21:25
fungi`ossh -p 29418 review gerrit ls-projects|grep /valence$`21:25
fungino errors!21:26
clarkbianw: fungi did you use enter or ^C c?21:26
ianwenter21:26
clarkbcool21:26
* fungi used nothing, merely spectator21:26
clarkbdocs weren't super clear on which should work or if both21:26
ianwhuh, well that was easy21:26
clarkbindeed. now we do the indexing21:26
fungi`ossh -p 29418 review gerrit ls-groups|grep valence`21:27
fungishows what we want21:27
ianw[DEPRECATION WARNING]: Using bare variables is deprecated. Update your playbooks so that the environment value uses the full variable syntax ('{{repos}}').21:27
fungiso it definitely did proceed21:27
clarkbfungi: what about storyboard?21:27
ianwmight be good to do that in the change for consistency21:27
clarkbansible claims that didn't change anything guessing tehy never used storyboard so its fine21:27
fungiclarkb: can skip sb since this is not an sb-using project21:27
clarkbfungi: gotcha21:27
clarkbianw: ++21:27
clarkbI can work on a new patchset for thatn ow21:28
fungishould be able to start zuul and start online reindex now, presumably21:28
clarkbif you all want to handle the reindexing21:28
clarkboh ya zuul I can start zuul first21:28
clarkbready to start zuul when you are21:28
ianwapprove 381332?21:28
clarkbianw: need zuul up first21:28
ianwfungi: is online reindex in the doc?21:29
clarkbalso need to reorder apporving and merging the rename change to happen before we reenable puppet cron21:29
*** dizquierdo has quit IRC21:29
*** jgriffith has quit IRC21:29
clarkbianw: its zaro's commetn on https://review.openstack.org/#/c/365067/1/doc/source/gerrit.rst21:29
fungii'm working on reindexing now unless someone else already has it21:29
clarkbI haven't21:29
clarkbfungi: ianw should I start zuul now?21:29
fungigo for it21:29
ianwyep21:30
fungirunning account reindex first this time around21:30
ianwso anyone can do that?  or you ahve to be an admin?21:30
clarkbianw: I think you must be admin21:30
fungianyone in the administrators group21:31
fungiwhich you should be in21:31
*** jascott1 has quit IRC21:31
clarkbzuul looks "stuck"21:31
clarkbits not logging anything and the first gate change to be enqueued is blocking21:32
*** jascott1 has joined #openstack-infra21:32
clarkboh there it goes21:32
clarkbhrm looks like it may have blocked on connecting to gearman?21:32
ianwyeah, last time i started it, it was very quiet on startup too21:32
clarkbonce that happened everything scrolled by21:32
ianwbut then got going21:32
clarkbenqueue still blocking though21:33
clarkbah there it goes21:33
clarkbjeblair: ^ any idea what is going on there?21:33
ianwi don't think it logs a "i'm starting now message", which maybe it should21:33
fungizaro: i did `gerrit index start --force accounts` and it's queued a reindex of changes, so presumably the last argument is just arbitrary?21:33
clarkbjeblair: zuul debug logs from 21:10-21:33 roughly show the sequence21:33
*** jascott1 has quit IRC21:34
*** jascott1 has joined #openstack-infra21:34
*** inc0 has quit IRC21:34
*** Jeffrey4l has quit IRC21:34
clarkbzuul is done21:35
clarkbremember don't reenable puppet until after the rename change ahs merged21:35
*** Jeffrey4l has joined #openstack-infra21:35
clarkbI will change that ordering in the next patchset too21:35
fungigate changes are getting nodes assigned, so nodepool seems to be happy21:35
fungioh, though maybe nodepool itself will get confused because gearman went away for a while?21:35
fungiseems like in the recent past we've had to restart nodepoold after any zuul scheduler restart21:36
fungianyway, gerrit and zuul are running, so probably safe to end the status alert?21:36
fungiwe can work through the rest of this transparent to the users21:37
ianwseems ok to me21:37
fungichange reindexing is down to <700 repos already21:37
jeblairoh hi21:37
clarkbfungi: I think it may leak some nodes21:37
clarkbfungi: but they should eventually get cleaned up21:38
fungiclarkb: ianw: objections to ending the status alert now?21:38
clarkbfungi: not really, we need to github too21:38
ianwnope21:38
fungiyeah, we still have steps to do, but users can go back to using gerrit again21:38
fungi#status ok21:38
openstackstatusfungi: sending ok21:38
*** jascott1 has quit IRC21:38
ianwnot sure who is scrolling but there's a screenlog.0 as well taht should have the ansible run21:39
fungii'll rename the repo on github unless someone else is already doing that21:39
clarkbianw: that was me, reviewing it as I update the playbook :)21:39
clarkbianw: all done now though21:39
*** ChanServ changes topic to "Discussion of OpenStack Developer and Community Infrastructure | docs http://docs.openstack.org/infra/ | bugs https://storyboard.openstack.org/ | source https://git.openstack.org/cgit/openstack-infra/ | channel logs http://eavesdrop.openstack.org/irclogs/%23openstack-infra/"21:40
jeblairclarkb: probably gearman server swamped with job registrations21:40
*** jkilpatr has quit IRC21:40
clarkbjeblair: ah21:40
fungihttps://github.com/openstack/valence is done now21:40
*** fernnest has joined #openstack-infra21:41
fungii guess i'll go ahead and approve 38133221:41
openstackstatusfungi: finished sending ok21:41
openstackgerritClark Boylan proposed openstack-infra/system-config: Update project rename instructions  https://review.openstack.org/36506721:42
*** sdake_ has quit IRC21:42
clarkbthat should be a more complete playbook now. Also has todos for the reindexing21:42
*** jascott1 has joined #openstack-infra21:42
fungithanks!21:42
clarkbfungi: do the reindex commands not block?21:42
clarkbfungi: and we want them to run in sequence only after the previous has completed?21:42
*** rlandy is now known as rlandy|brb21:43
fungiclarkb: yes and no. the reindex command returns after queuing all the reindex tasks, but gerrit maintains a lock around it so you can't run it again until it finishes21:43
fungihowever see my comment to zaro a few minutes ago about index start --force accounts queuing tasks to index changes21:43
fungii have a feeling something still not quite 100% on our syntax there21:44
clarkbfungi: I guess I don't parse that21:44
*** jgriffith has joined #openstack-infra21:44
clarkbyou told gerrit to index account but it indexed changes?21:44
ianwjust running kids to school ... back in a few21:44
openstackgerritNeil Jerram proposed openstack-infra/project-config: Run {pipeline}-tempest-dsvm-networking-calico on Xenial  https://review.openstack.org/40097521:44
*** e0ne has quit IRC21:45
*** tpsilva has quit IRC21:45
clarkbfungi: thinking about how to ansiblize this. The first command is easy21:45
clarkbbut wondering what the requirements are for running the second one and if we need to do logic things there21:45
fungiclarkb: i don't exactly know what i told gerrit to do. from zaro's instructions i assumed i was telling it to index accounts and then to index changes (i wanted to do them in that order because accounts should finish faster) but it seems to have decided to reindex everything when i asked it to reindex accounts, so whatever21:45
clarkbgotcha21:46
clarkbso more testing needed21:46
clarkbfungi: ianw if there is no objection I am going to boot a new translate-dev01 now21:47
openstackgerritMerged openstack-infra/project-config: Rename rsc to valence  https://review.openstack.org/38133221:47
fungiclarkb: looking at `gerrit index start --help` it appears it expected `index start accounts --force` so likely took omission of the index name as a signal to index everything21:47
clarkb(I don't want to juggle two things if you think we are not done with rename)21:47
clarkbfungi: can we do the easy mode and just tell it to reindex everything instead?21:48
fungithough the --help output doesn't indicate that the index name is optional21:48
*** hashar has quit IRC21:49
fungiclarkb: apparently we can, though the api help doesn't match the observed behavior21:49
*** rbuzatu_ has joined #openstack-infra21:49
fungiso this is worth getting zaro to confirm later21:49
*** gouthamr has quit IRC21:49
*** baoli has joined #openstack-infra21:50
fungilooking at http://docs-draft.openstack.org/67/365067/3/check/gate-system-config-docs-ubuntu-xenial/19b394e/doc/build/html/gerrit.html#renaming-a-project we're done up through #8 now. any objections to me reenabling the ansible run_all cron on puppetmaster.o.o now?21:50
fungialmost down to 250 repos still pending reindex21:51
clarkbhrm puppet is still running on translate-dev.openstack.org even though I thought I had disabled it21:51
clarkbmordred: ^ can you tell me what I did wrong there?21:51
mordredclarkb: did you put translate-dev.o.o in emergency?21:52
*** jkilpatr has joined #openstack-infra21:52
clarkbmordred: I put its uuid in emergency21:52
*** rbuzatu has quit IRC21:53
clarkbworkign to double check the uuid now21:53
mordredclarkb: I believe that's only a thing if there are more than one of them - although you now make me realize we could do that better21:53
*** rbuzatu_ has quit IRC21:53
mordredclarkb: yah - there is no host named afd4a8d9-98a7-4a21-a827-33106abeeb8a in the inventory21:54
*** kgiusti has quit IRC21:54
fungiyeah, being able to put the uuid in the emergency list before booting a replacement would have been nice21:54
clarkbok so I have to use the proper name here21:54
mordredclarkb: there is a host named instance-afd4a8d9-98a7-4a21-a827-33106abeeb8a - but the hostname to put in the emergency file in this case is translate-dev.openstack.org21:54
*** ssbarnea has quit IRC21:54
mordredlooking in /var/cache/ansible-inventory/ansible-inventory.cache to see what's defined is what I just did21:54
fungiclarkb: though worth noting, since you've decided it was worth replacing it with a differently-named host anyway, you no longer need to use the uuid21:54
fungiyou can just disable it by name instead21:55
clarkbfungi: yup21:55
fungiokay, i'm going to go ahead and reenable ansible/puppet cron before the 22:00 pulse starts21:56
*** matrohon has quit IRC21:56
clarkbkk21:57
clarkbI have updated the mergency file to use translate-dev.openstack.org and have stopped services again (puppet restarted them)21:57
clarkbgoing to boot translate-dev01.openstack.org now21:57
*** yolanda has joined #openstack-infra21:57
fungii was worried it might21:57
clarkbfungi: the problem with your mangle symlinks suggestion is that upstart doesn't have an easy way to do that iirc21:58
clarkbsince it doesn't honor runlevls properly21:58
fungiright, and you can't just stick "exit 0" at the top of the unit file either21:58
clarkbiirc the official way to do it is to add code to the upstart init scripts that say this one is disabled21:58
fungiansible has just now kicked off22:00
clarkbderp that moment you realize you forgot to set the flavor to xenial because it used a trusty image22:00
clarkbs/flavor/image22:00
*** wolverin_ has quit IRC22:01
*** wolverineav has joined #openstack-infra22:01
*** trown is now known as trown|outtypewww22:02
*** jgriffith_ has joined #openstack-infra22:03
*** inc0 has joined #openstack-infra22:03
*** asettle has joined #openstack-infra22:03
clarkbmordred: any idea what is up with http://paste.openstack.org/show/590146/ ?22:03
*** wolverineav has quit IRC22:03
*** wolverineav has joined #openstack-infra22:04
clarkbthat seems to have resulted in not copying the group hieradata file that has the mysql info and made puppet fail22:04
mordredclarkb: likely one of the entries in /etc/ansible/groups.txt not matching anyhting?22:05
*** wolverineav has quit IRC22:05
mordredclarkb: lemme run it real quick ...22:05
*** masuberu has joined #openstack-infra22:05
clarkbmordred: oh ya thats it, I forgot we have to double account this stuff22:05
clarkbcan we stop doing that?22:05
*** ssbarnea has joined #openstack-infra22:06
mordredclarkb: longer conversation. short version - I'd love to - longer version - we should talk about the mechanics22:06
*** jgriffith has quit IRC22:06
clarkbor maybe use symlinks and just fqdn everything22:07
mordredlet's work through it tomorrow when we can both think with full brains - I'm just about to AFK22:07
*** asettle has quit IRC22:08
clarkbok22:08
*** masber has quit IRC22:08
*** rlandy|brb is now known as rlandy22:08
clarkbmordred: can you review the fix patch I will be pushing in a sec?22:09
openstackgerritClark Boylan proposed openstack-infra/system-config: Do the other half of translate-dev grouping  https://review.openstack.org/40098222:10
clarkbmordred: ^there22:10
clarkbpabelanger: fungi once ^ is in I will give this another go22:12
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: Transition to quickstart: make toci_gate_test.sh a symlink to toci_gate_orig.sh.  https://review.openstack.org/39645122:12
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: DO NOT MERGE Transition to quickstart: simulate transition  https://review.openstack.org/39925722:12
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: Transition to quickstart: add toci_gate_oooq.sh and jobs configurations  https://review.openstack.org/39925622:12
clarkbin the meantime I need to step away and pick up a turkey22:13
clarkbwill be back around in an hour or so to hopefully actually boot this instance22:13
*** jgriffith_ is now known as jgriffith22:13
*** ssbarnea has quit IRC22:14
pabelangergobble gobble22:14
*** ssbarnea has joined #openstack-infra22:14
*** ssbarnea has quit IRC22:14
*** tiswanso has quit IRC22:14
openstackgerritAdam Gandelman proposed openstack-infra/zuul: Re-enable test_success_pattern as test_success_url  https://review.openstack.org/40045522:14
*** ssbarnea has joined #openstack-infra22:15
*** nicolasbock has quit IRC22:18
*** nadya has quit IRC22:19
*** matt-borland has quit IRC22:21
openstackgerritAdam Gandelman proposed openstack-infra/zuul: Re-enable test_success_pattern as test_success_url  https://review.openstack.org/40045522:22
openstackgerritMerged openstack-infra/project-config: fix regex issue for job  https://review.openstack.org/40078422:23
*** sdake has joined #openstack-infra22:23
*** sree has joined #openstack-infra22:23
openstackgerritAdam Gandelman proposed openstack-infra/zuul: Re-enable test_success_pattern as test_success_url  https://review.openstack.org/40045522:23
openstackgerritPaul Belanger proposed openstack-infra/puppet-openstackci: Use nodepool::builder for nodepool  https://review.openstack.org/35650322:27
*** pradk has quit IRC22:28
*** sree has quit IRC22:28
*** Apoorva has quit IRC22:28
*** Apoorva has joined #openstack-infra22:28
*** Apoorva_ has joined #openstack-infra22:29
*** john-davidge has joined #openstack-infra22:31
*** Apoorva has quit IRC22:33
*** Apoorva has joined #openstack-infra22:35
*** srobert has quit IRC22:35
*** Apoorva has quit IRC22:36
*** Apoorva__ has joined #openstack-infra22:36
*** Apoorva__ has quit IRC22:37
*** Apoorva_ has quit IRC22:37
*** Apoorva has joined #openstack-infra22:37
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: DO NOT MERGE Transition to quickstart: simulate transition  https://review.openstack.org/39925722:38
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: Transition to quickstart: add toci_gate_oooq.sh and jobs configurations  https://review.openstack.org/39925622:38
*** Apoorva has quit IRC22:39
*** Apoorva has joined #openstack-infra22:40
*** ssbarnea has quit IRC22:40
*** Apoorva has quit IRC22:42
*** Apoorva_ has joined #openstack-infra22:42
*** john-davidge has quit IRC22:43
inc0hey, fungi mordred say, I'll give you shellscript that will install and start docker registry, it's really simple22:43
inc0can you help me with puppetry please?22:43
*** Apoorva has joined #openstack-infra22:44
inc0or I can prep ansible if it's easier22:44
inc0for you22:44
*** Apoorva__ has joined #openstack-infra22:45
*** sdake has quit IRC22:46
*** sdake has joined #openstack-infra22:46
*** Apoorva_ has quit IRC22:47
openstackgerritPaul Belanger proposed openstack-infra/puppet-openstackci: Use nodepool::builder for nodepool  https://review.openstack.org/35650322:48
*** Apoorva has quit IRC22:49
*** Apoorva__ has quit IRC22:49
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: DO NOT MERGE Transition to quickstart: simulate transition  https://review.openstack.org/39925722:49
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: Transition to quickstart: add toci_gate_oooq.sh and jobs configurations  https://review.openstack.org/39925622:49
*** chlong has quit IRC22:52
jeblairpabelanger: i was thinking -- do we really need a common file at this point?  can we do 'nodepool' (with the builder disabled) on the nodepool host, and then 'nodepool' and 'nodepool_builder' on the builder host?22:53
jeblairpabelanger: (then, when we make the zk launchers, we can do a nodepool_launcher class, and what we have left in the nodepool class will be the common stuff at that point)22:54
*** bknudson has left #openstack-infra22:55
*** yolanda has quit IRC22:58
openstackgerritJames E. Blair proposed openstack-infra/puppet-openstackci: Use nodepool::builder for nodepool  https://review.openstack.org/40099323:00
jeblairpabelanger: consider that as an alternative to 35650323:00
*** xyang1 has quit IRC23:00
*** jklare has quit IRC23:01
*** mriedem has quit IRC23:02
*** jklare has joined #openstack-infra23:03
*** panda is now known as panda|Zz23:03
*** rbuzatu has joined #openstack-infra23:05
*** boden has quit IRC23:06
*** mmandel has quit IRC23:07
fungiinc0: i forget who was looking into it, but docker-registry is already packaged in ubuntu. don't need a container at all23:07
inc0yeah23:08
inc0it's literally apt-get install -> config file we prepare -> htpasswd file we prepare -> service docker-registry start23:08
inc0that will give us all - auth, mirroring23:08
inc0super easy23:08
openstackgerritJames E. Blair proposed openstack-infra/puppet-openstackci: Add a nodepool_builder manifest  https://review.openstack.org/40099823:09
jeblairpabelanger: ^23:09
inc0just, my puppetry had seen better days23:09
inc0so mirroring should be as simple as rsync of directories23:09
*** rbuzatu has quit IRC23:09
inc0or afs mirroring23:09
inc0or nfs23:10
inc0or swift;)23:10
inc0fungi, we were looking at apache auth, but that's not even required23:11
*** rbuzatu has joined #openstack-infra23:11
openstackgerritMerged openstack-infra/shade: Added documentation for delete_image()  https://review.openstack.org/40051923:12
openstackgerritJames E. Blair proposed openstack-infra/system-config: Add nodepool builder nodes  https://review.openstack.org/40099923:12
*** knangia has quit IRC23:13
*** baoli has quit IRC23:18
openstackgerritJames E. Blair proposed openstack-infra/system-config: Add docker server  https://review.openstack.org/40100323:18
jeblairinc0: ^ can you take that and modify it?23:19
openstackgerritMerged openstack-infra/system-config: Do the other half of translate-dev grouping  https://review.openstack.org/40098223:19
inc0jeblair, sure, I don't really need docker itself, but will take a look23:20
jeblairinc0: yes, i used the wrong docker words.  :)23:20
jeblairinc0: but that's a puppet skeleton for "install something along with a config file"23:20
inc0no problem:) how do I run it locally so I can test if it works?23:21
jeblairinc0: there are some instructions here: http://docs.openstack.org/infra/system-config/sysadmin.html#making-a-change-in-puppet23:21
inc0thanks, I'll play around23:21
inc0<< spraying "Make Ansible Great Again" on sign and going out to protest >>23:22
*** jamielennox is now known as jamielennox|away23:24
*** jamielennox|away is now known as jamielennox23:24
*** wolverineav has joined #openstack-infra23:24
jeblairinc0: i think some of us would rather run ansible at this point, but we could use some help migrating our puppet; until we know we can actually accomplish that, we don't want to run 2 systems.23:26
* clarkb is back now, just waiting on puppet to update puppetmaster to include newer groups.txt23:26
*** wolverineav has quit IRC23:26
inc0jeblair, understandable23:26
*** asettle has joined #openstack-infra23:27
inc0ansible -m shell -a "puppet apply ..." for now?;)23:27
*** gordc has quit IRC23:27
* inc0 bursts into evil laugh23:27
inc0laughter, laught...english is hard23:27
clarkbinc0: its actually ansible -m puppet "things"23:27
clarkber -a "things"23:27
inc0there is ansible module for puppet?23:28
clarkbyes23:28
inc0ok, I'm not nearly as evil as I thought23:28
*** rbrndt has quit IRC23:28
*** asettle has quit IRC23:30
*** asettle has joined #openstack-infra23:31
jeblairwe wrote it23:31
jeblairand it's how we run puppet23:31
jeblairinc0: you are in the company of great evil23:32
*** pradk has joined #openstack-infra23:33
inc0duh...and still you're not allowing me to write ansible23:34
inc0yes, evil indeed23:34
clarkbjeblair: the more I use ansible the more I am scared things like service management and package management will just annoy the crap out of us. Which is sad beacuse those are some of the most basic things you want it to do. (most recently with the zuul launcher restarts ansible claimed they were all restarted but about half of them didn't actually start)23:34
fungifor great justice^H^H^H^H^H^H^Hevil!23:34
clarkbof course we could just attempt to fix those bugs, then you go read the service module in ansible and run away crying (yes I have done this)23:34
inc0well ansible won't check status of stuff23:34
inc0it's glorified shellscript23:34
fungiyeah, ansible seems like a great way to deploy stuff, and a less terrific way to do configuration management23:35
clarkbinc0: the service module is supposed to though23:35
clarkbinc0: and fail if it doesn't23:35
clarkbbut its a giant mess of spaghetti so good luck understanding why it doesn't in some cases23:35
jeblairclarkb: yes.  to be fair, we've had our problems with puppet services.23:35
clarkbjeblair: indeed23:35
clarkbjeblair: the wrost part is a lot of the problems overlap too :(23:35
*** asettle has quit IRC23:35
jeblair:(23:35
clarkb(puppet too suffers from cranky service management at times)23:36
clarkbthey both suffer different pacakge management grumps too23:37
clarkband in some cases the asme grumps23:37
clarkbeg why isn't bindep a thing built into config management yet?23:37
clarkb(translate-dev01 is building again nwo)23:37
*** watanabe_isao has joined #openstack-infra23:38
clarkbmordred: I still get that error I pasted, but this time it copied the group hiera file too23:39
*** asettle has joined #openstack-infra23:39
clarkband puppet is runnign so I don't think that issue is fatal23:39
*** baoli has joined #openstack-infra23:40
*** baoli has quit IRC23:40
*** dingyichen has joined #openstack-infra23:40
openstackgerritSagi Shnaidman proposed openstack-infra/devstack-gate: Find and delete broken symlinks in logs  https://review.openstack.org/40074223:41
openstackgerritMichal Jastrzebski (inc0) proposed openstack-infra/system-config: Add docker server  https://review.openstack.org/40100323:41
inc0jeblair, ^23:41
inc0I believe we should add ensure service docker-registry started23:42
inc0and that's about it23:42
*** asettle has quit IRC23:42
fungigerrit online reindexing completed already. that was far faster than anticipated23:42
inc0and maybe move contents of users file to hiera23:42
fungias long as it's safe for puppet to automatically start the docker registry service if it crashes for some reason, i guess that's fine. in general i'm not thrilled about mechanisms that helpfully restart failing services for you23:44
clarkbhttp://paste.openstack.org/show/590149/ looks like we need more systemdification on the wildfly install23:44
inc0fungi, well it has to be restarted after config files lands23:44
fungiinc0: sure, and for that the service resource in puppet can either be subscribed to the conffile, or the file resource can be set to notify the service23:45
*** watanabe_isao has quit IRC23:45
fungiat least that's how we usually have it working23:45
*** wolverineav has joined #openstack-infra23:45
inc0fungi, well, you know better;) but docker registry is really simple service to run as it turns out23:46
clarkblooks like we will need a patch at https://github.com/biemond/biemond-wildfly/blob/master/manifests/service.pp#L35-L46 will respond to the i18n team to see if they are able to take that on23:47
clarkbpleia2: ^ have you patched that in the past?23:47
pleia2clarkb: nope23:48
*** gildub has joined #openstack-infra23:49
fungiinc0: sure, but ensure=>started won't actually take care of restarting after config updates. you need a subscribe or notify relationship to handle that regardless23:49
fungiwhich was my point23:49
inc0understood23:49
*** wolverineav has quit IRC23:49
*** dingyichen has quit IRC23:50
*** dchen has joined #openstack-infra23:50
*** dingyichen has joined #openstack-infra23:51
*** hongbin has quit IRC23:51
zarofungi: i remember we ran into the same thing on last rename.  we ran force reindex on accounts but it ran for both accounts and changes.23:53
openstackgerritMerged openstack-infra/system-config: Update project rename instructions  https://review.openstack.org/36506723:53
zarofungi, clarkb which i think is a bug but probably fixed in newer version of gerrit23:54
fungizaro: the output of `gerrit index start --help` suggests that it's very picky about putting "accounts" or "changes" _before_ "--force" for some reason23:54
fungiso it could just be we have the argument order incorrect23:54
zarohmm, interesting. i'll have to check that23:54
fungii didn't retry it with the order indicated by its context help because by the time i saw that it was already running anyway23:55
*** hurgleburgler has quit IRC23:56
clarkbpleia2: pabelanger fungi ok update sent to list and ian and aeng directly23:56
clarkbI doubt I am going to be able to update upstream this week, may have time next week. I have a bunch of stuff to do tomorrow in prep for thursday23:57
clarkbmordred: I put a few more things on https://etherpad.openstack.org/p/openstack-user-api-improvements for identity and horizon23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!