| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Delete query 1258682 https://review.openstack.org/80424 | 00:00 |
|---|---|---|
| *** maxbit_ has joined #openstack-infra | 00:01 | |
| *** maxbit has joined #openstack-infra | 00:01 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Add a footprint for bug 1283522 https://review.openstack.org/80286 | 00:01 |
| *** yassine has quit IRC | 00:01 | |
| *** sarob has quit IRC | 00:02 | |
| *** mrodden has quit IRC | 00:03 | |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Added support for CI Game plugin https://review.openstack.org/79244 | 00:04 |
| openstackgerrit | A change was merged to openstack-infra/config: Enable Tempest for Murano dvsm job https://review.openstack.org/79164 | 00:05 |
| openstackgerrit | Spencer Krum proposed a change to openstack-infra/config: Add PuppetBoard to openstack infra https://review.openstack.org/77928 | 00:07 |
| jogo | so neutron has a bunch of fixes in the pipeline that should help gate | 00:07 |
| nibalizer | clarkb: i may even have it now | 00:07 |
| jogo | was thinking maybe we can bump them to the top of the queue | 00:07 |
| jogo | arosen2 and dansmith are working on them | 00:07 |
| fungi | jogo: the one failing its unit tests and the other one which depends on it? | 00:08 |
| jogo | fungi: yup and they have a patch for the unit test bug too | 00:08 |
| jogo | oh wait that didn't work | 00:08 |
| clarkb | jogo: https://review.openstack.org/#/c/79794/ bug me about that when I am not about to drive to portland and I can approve it | 00:08 |
| clarkb | fungi: :P | 00:08 |
| jogo | clarkb: ack | 00:09 |
| mattoliverau | jeblair: So I created a RAX centos box, changed resolv.conf, chattr +i /etc/resov.conf, made the server into an image. Created a new server from the image and all worked as expected no networking errors and my change to resolv.conf was consistent. | 00:09 |
| jogo | arosen2: the unit test failure is a race right? | 00:09 |
| jogo | fungi: so the catch with those two patches is that nova needs them before we can push a patch to actually fix the issue | 00:10 |
| arosen2 | jogo: i think so. | 00:10 |
| jogo | so this wouldn't result in an immediate fix, but it would help expedite the solution | 00:10 |
| * clarkb afks to make that trip | 00:11 | |
| jogo | fungi: ^ | 00:11 |
| jeblair | mattoliverau: cool, thanks | 00:11 |
| jogo | fungi: thoughts on a bump on this one? | 00:11 |
| * arosen2 Looking at the test too | 00:11 | |
| jogo | these two* | 00:11 |
| fungi | jogo: i guess they just got sniped out by another upload? | 00:11 |
| *** SumitNaiksatam has quit IRC | 00:11 | |
| fungi | they vanished from the gate moments ago | 00:12 |
| jogo | fungi: https://review.openstack.org/#/c/75253/ | 00:12 |
| jogo | other way around | 00:12 |
| jogo | everything ahead flushed out | 00:12 |
| fungi | oic, they had already finished all their testing | 00:13 |
| *** dolphm has quit IRC | 00:14 | |
| *** emagana has quit IRC | 00:15 | |
| *** dolphm has joined #openstack-infra | 00:15 | |
| *** dkranz has quit IRC | 00:16 | |
| *** katyafervent has quit IRC | 00:16 | |
| *** dkranz has joined #openstack-infra | 00:16 | |
| nibalizer | clarkb: want to look at https://review.openstack.org/#/c/77928/ when you get a chance? | 00:18 |
| nibalizer | fungi: maybe you to if you get a chance | 00:18 |
| *** maxbit has quit IRC | 00:18 | |
| *** katyafervent_ has joined #openstack-infra | 00:18 | |
| *** maxbit_ has quit IRC | 00:18 | |
| *** maxbit has joined #openstack-infra | 00:18 | |
| arosen2 | jogo: markmcclain is going to approve this: https://review.openstack.org/#/c/80426/2 not sure if we can bump it ahead at all | 00:18 |
| jogo | arosen2: if you pray (make a logical and sound arguement) to the infra gods perhaps you can | 00:18 |
| * fungi hears your pleas | 00:18 | |
| arosen2 | jogo: or i guess an under the table cash bribe | 00:19 |
| fungi | i also take credit | 00:19 |
| jogo | unlike regular gods, the infra gods are honest | 00:20 |
| jogo | and reasonable | 00:20 |
| arosen2 | :) | 00:20 |
| * fungi is irregular. see, says right here on my jacket | 00:20 | |
| *** pcrews has quit IRC | 00:25 | |
| *** maurosr has quit IRC | 00:25 | |
| *** zehicle_at_dell has joined #openstack-infra | 00:25 | |
| *** zehicle_at_dell has quit IRC | 00:27 | |
| jogo | haha | 00:27 |
| jogo | anyway just finished my beer so time to go outside and enjoy the great weather | 00:27 |
| jogo | arosen2: good luck with all the neutron patches | 00:27 |
| arosen2 | jogo: enjoy! | 00:27 |
| markmcclain | fungi: so it would make sense to promote https://review.openstack.org/#/c/80426/2 if there is another gate reset | 00:27 |
| markmcclain | this test has been randomly failing depending on the actual test ordering | 00:27 |
| fungi | markmcclain: looks like it's still in check right now, but appears safe enough to bump it straight into the gate at the very least | 00:27 |
| *** pcrews_ has joined #openstack-infra | 00:27 | |
| *** maurosr- has joined #openstack-infra | 00:27 | |
| *** maurosr- is now known as maurosr | 00:27 | |
| *** zehicle has joined #openstack-infra | 00:27 | |
| *** dstanek_afk has joined #openstack-infra | 00:27 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Use unbound https://review.openstack.org/80092 | 00:27 |
| markmcclain | fungi: bumping it to the gate would be nice | 00:27 |
| fungi | markmcclain: i've popped it into the gate | 00:27 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Delete query 1258682 https://review.openstack.org/80424 | 00:27 |
| markmcclain | fungi: thanks | 00:27 |
| jeblair | mattoliverau: attempt to add immutable to puppet ^ (because i think we actually will need the same functionality on our long-lived centos servers -- either that, or we would need to update one of the injected files in /etc) | 00:27 |
| fungi | markmcclain: if i see a good opportunity crop up, i'll send it up to the front | 00:27 |
| markmcclain | awesome | 00:27 |
| fungi | but there's only 5 changes ahead of it currently, so if we hold our teeth right it just might go straight in | 00:27 |
| *** dstanek has quit IRC | 00:28 | |
| mordred | nibalizer: on https://review.openstack.org/#/c/77928/ - shouldn't line 2 of the vhost.erb be an fqdn - or $::fqdn ? | 00:29 |
| *** matsuhashi has joined #openstack-infra | 00:31 | |
| *** jooools has joined #openstack-infra | 00:33 | |
| *** ryanpetrello has quit IRC | 00:33 | |
| *** mgagne has quit IRC | 00:35 | |
| nibalizer | mordred: thats fine with me, i'll do that | 00:36 |
| *** thedodd has joined #openstack-infra | 00:36 | |
| *** zehicle has quit IRC | 00:37 | |
| mattoliverau | jeblair: nice, made a comment, but looks good to me. | 00:44 |
| *** thuc has quit IRC | 00:45 | |
| dims_ | jamespage, ping | 00:45 |
| *** thuc has joined #openstack-infra | 00:45 | |
| *** mrodden has joined #openstack-infra | 00:47 | |
| *** milki has quit IRC | 00:48 | |
| *** openstack has joined #openstack-infra | 00:53 | |
| *** dims_ has joined #openstack-infra | 00:53 | |
| *** bogdando has joined #openstack-infra | 00:54 | |
| *** dstufft has joined #openstack-infra | 00:54 | |
| *** rfolco has joined #openstack-infra | 00:54 | |
| *** dstufft has quit IRC | 00:55 | |
| *** dkranz has joined #openstack-infra | 00:55 | |
| *** dstufft has joined #openstack-infra | 00:55 | |
| *** nati_uen_ has quit IRC | 00:55 | |
| *** dstufft has quit IRC | 00:55 | |
| *** dstufft has joined #openstack-infra | 00:55 | |
| *** jasondot_ has quit IRC | 00:55 | |
| *** openstack has joined #openstack-infra | 00:57 | |
| *** openstack has quit IRC | 00:57 | |
| *** openstack has joined #openstack-infra | 01:00 | |
| *** afazekas has quit IRC | 01:01 | |
| *** dstufft has quit IRC | 01:01 | |
| *** hdd_ has joined #openstack-infra | 01:02 | |
| *** jgrimm has quit IRC | 01:02 | |
| *** blamar_ has joined #openstack-infra | 01:02 | |
| *** dstufft has joined #openstack-infra | 01:02 | |
| *** SpamapS has quit IRC | 01:04 | |
| *** blamar has quit IRC | 01:06 | |
| *** dstufft has quit IRC | 01:07 | |
| *** dolphm has quit IRC | 01:07 | |
| *** rpodolyaka1 has joined #openstack-infra | 01:07 | |
| *** dstufft has joined #openstack-infra | 01:07 | |
| *** Daisy_ has joined #openstack-infra | 01:07 | |
| *** sarob has quit IRC | 01:08 | |
| *** matsuhashi has quit IRC | 01:08 | |
| *** SpamapS_ has joined #openstack-infra | 01:08 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Added paging to list endpoints https://review.openstack.org/79757 | 01:08 |
| *** rpodolyaka has quit IRC | 01:08 | |
| *** dolphm_ has joined #openstack-infra | 01:08 | |
| *** thedodd has quit IRC | 01:08 | |
| *** dolphm_ is now known as dolphm | 01:08 | |
| *** blamar_ is now known as blamar | 01:08 | |
| *** dstufft has quit IRC | 01:08 | |
| *** dstufft has joined #openstack-infra | 01:08 | |
| *** amcrn has quit IRC | 01:08 | |
| *** afazekas has joined #openstack-infra | 01:08 | |
| *** julim has quit IRC | 01:12 | |
| *** matsuhashi has joined #openstack-infra | 01:12 | |
| *** rpodolyaka1 has quit IRC | 01:14 | |
| *** FallenPegasus has joined #openstack-infra | 01:14 | |
| *** Daisy_ has quit IRC | 01:14 | |
| *** openstack has joined #openstack-infra | 01:15 | |
| -dickson.freenode.net- [freenode-info] channel flooding and no channel staff around to help? Please check with freenode support: http://freenode.net/faq.shtml#gettinghelp | 01:15 | |
| *** unicell has joined #openstack-infra | 01:15 | |
| *** openstack has joined #openstack-infra | 01:19 | |
| *** nosnos has joined #openstack-infra | 01:19 | |
| *** Daisy__ has quit IRC | 01:24 | |
| *** openstack has quit IRC | 01:24 | |
| *** openstack has joined #openstack-infra | 01:26 | |
| *** openstack has joined #openstack-infra | 15:09 | |
| jeblair | wow its log is not rotated. | 15:10 |
| jeblair | i'm not sure what happened to its connection | 15:11 |
| fungi | somehow i missed the part message for it in my client log, but the channel logs suggest it went missing shorty after 2014-03-14T01:26:00 | 15:11 |
| fungi | shortly | 15:12 |
| mordred | morning all | 15:12 |
| *** rcleere has joined #openstack-infra | 15:12 | |
| devananda | morning! | 15:12 |
| jeblair | it was disconnecting and reconnecting a lot for the previous hour or so from then | 15:12 |
| devananda | mordred: hey look! https://review.openstack.org/#/c/80575/ | 15:12 |
| jeblair | and there's some weird async connection stuff going on | 15:13 |
| *** miqui has quit IRC | 15:13 | |
| mordred | devananda: oooo | 15:13 |
| devananda | it's like we almost have integration tests :) | 15:13 |
| jeblair | INFO 2014-03-14T01:28:31 supybot Join to #openstack-trove on FreeNode synced in 193.02 seconds. | 15:14 |
| jeblair | WARNING 2014-03-14T01:28:37 supybot Error message from FreeNode: Ping sent at 2014-03-14T01:26:36 not replied to. | 15:14 |
| jeblair | INFO 2014-03-14T01:28:37 supybot Reconnecting to FreeNode. | 15:14 |
| jeblair | and then that repeats until the present time | 15:14 |
| mordred | devananda: I don't even want to know what this is: sudo add-apt-repository cloud-archive:havana do I? | 15:14 |
| *** yamahata has quit IRC | 15:14 | |
| jeblair | note how long it took to join trive | 15:14 |
| jeblair | trove | 15:14 |
| *** yamahata has joined #openstack-infra | 15:14 | |
| devananda | mordred: ask clark. tldr - we need ovs from not-precise | 15:14 |
| *** adalbas has quit IRC | 15:15 | |
| jeblair | i'm guessing the irc server or our network was having problems | 15:15 |
| mordred | devananda: yeah - I figured it was probably something like that ... | 15:15 |
| *** rpodolyaka has joined #openstack-infra | 15:15 | |
| devananda | mordred: the ssh thing is fun, too. i was just explaining that to jeblair :) | 15:15 |
| devananda | bbiafm | 15:16 |
| *** lcostantino has joined #openstack-infra | 15:18 | |
| *** dprince has joined #openstack-infra | 15:18 | |
| *** HenryG has quit IRC | 15:20 | |
| *** lcheng has joined #openstack-infra | 15:21 | |
| *** emagana has joined #openstack-infra | 15:22 | |
| *** wenlock has joined #openstack-infra | 15:22 | |
| *** sarob has joined #openstack-infra | 15:22 | |
| sdague | jeblair: is there a reason not to send https://review.openstack.org/#/c/80575/ through now? I'd like to get those experimental runs on the devstack patch to make sure a few things in it will work on gate nodes | 15:22 |
| *** sdake has joined #openstack-infra | 15:24 | |
| *** smarcet has quit IRC | 15:24 | |
| jeblair | sdague, devananda so i'm a little puzzled why the ssh and apt repo stuff is in the job def and not devstack (or the ironic plugin for devstack, or whatever) | 15:25 |
| openstackgerrit | A change was merged to openstack-infra/config: run link-logs builder first https://review.openstack.org/79794 | 15:26 |
| fungi | i suppose that stuff could go in a pre-hook before ovs and similar things get pulled in by devstack itself? | 15:26 |
| jeblair | fungi: oh, is the reason because devstack would install the wrong ovs? | 15:26 |
| fungi | that's what i assumed, but maybe i was misinterpreting | 15:27 |
| *** sarob has quit IRC | 15:27 | |
| jeblair | sdague: is this job to prove this is ready for things landing in devstack? if so, maybe this can move later on? | 15:27 |
| sdague | jeblair: yeh, this is to evaluate the devstack patch | 15:27 |
| fungi | once 79794 propagates i'll manually update jjb on all the jenkins masters, since the list of jobs touched is so large i expect the puppet exec on them to time out | 15:27 |
| jeblair | ok. i think that in long run, those things should not be in the job defn | 15:28 |
| sdague | and I agree, this should not be in there. Honestly, given how all the timelines are lining up I expect that before this is voting we'll just hard require 14.04 for ironic anyway | 15:28 |
| jeblair | so once things settle out, we should move them | 15:28 |
| sdague | agreed | 15:28 |
| jeblair | (aprvd btw) | 15:28 |
| openstackgerrit | A change was merged to openstack-infra/config: Add experimental virtual-ironic test https://review.openstack.org/80575 | 15:28 |
| sdague | 14.04 has ovs 2.0, which should dramatically make everything using ovs work better | 15:28 |
| sdague | which means I expect as soon as the gate can handle it we'll just hard require that ovs layer for anything using neutron | 15:29 |
| Ajaeger | fungi, jeblair: Thanks for reviewing and approving 80515! | 15:29 |
| sdague | jeblair: thanks for the +A | 15:30 |
| devananda | sdague: sounds good. how soon should we start testing all the ironic things with 14.04? | 15:30 |
| devananda | jeblair: fungi: thanks for the help with that patch! | 15:30 |
| sdague | devananda: once there is an image that can be run in infra | 15:30 |
| sdague | which means basically once the cloud providers start pushing them up | 15:31 |
| devananda | sdague: gotcha | 15:31 |
| fungi | yeah, doing that now would require starting to use glance so we could upload a beta image | 15:31 |
| openstackgerrit | A change was merged to openstack-infra/config: Fix manuals translation jobs https://review.openstack.org/80515 | 15:31 |
| *** zehicle_at_dell has joined #openstack-infra | 15:31 | |
| fungi | i just checked rax and hpcloud and neither have them published in their catalog | 15:31 |
| sdague | rax just announced glance, is that actually running ? | 15:31 |
| sdague | not that I think it's actually worth spending that time in this part of the cycle | 15:32 |
| sdague | mostly curious | 15:32 |
| devananda | fungi: about half an hour or so before that virtual-ironic job can be run by anything, ya? | 15:32 |
| fungi | devananda: yeah, i'm going to be forcing job updates everywhere here in just a bit, and after that it should be runnable | 15:33 |
| devananda | fungi: great. I'm going to head into downtown in a bit, back in an hour or two to check on things | 15:33 |
| *** zehicle has quit IRC | 15:34 | |
| *** HenryG has joined #openstack-infra | 15:35 | |
| *** smarcet has joined #openstack-infra | 15:38 | |
| *** miqui has joined #openstack-infra | 15:39 | |
| *** marun has quit IRC | 15:39 | |
| *** Ryan_Lane has joined #openstack-infra | 15:41 | |
| *** rpodolyaka has quit IRC | 15:42 | |
| *** talluri_ has quit IRC | 15:42 | |
| *** talluri has joined #openstack-infra | 15:43 | |
| *** CaptTofu has quit IRC | 15:44 | |
| *** CaptTofu has joined #openstack-infra | 15:44 | |
| *** thuc has joined #openstack-infra | 15:44 | |
| *** dcramer_ has joined #openstack-infra | 15:46 | |
| *** yaguang has quit IRC | 15:46 | |
| *** talluri has quit IRC | 15:47 | |
| mordred | sdague: both hp and rax have glance available - but we'll need to re-tool nodepool a bit to make use of it. also, hp 1.0 doesn't have it - so we still can't REALLY use it until we've moved off of 1.0 | 15:47 |
| *** drumkilla is now known as rustlebee | 15:47 | |
| *** thuc_ has quit IRC | 15:48 | |
| sdague | mordred: cool, sounds like early juno fodder | 15:48 |
| mordred | yup | 15:48 |
| mordred | I'm looking forward to it - and looking forward to not being dependent on our providers for image content at all | 15:48 |
| *** thuc has quit IRC | 15:48 | |
| *** CaptTofu has quit IRC | 15:48 | |
| sdague | what... you don't like reverse engineering the differences between their nodes? | 15:48 |
| *** adalbas has joined #openstack-infra | 15:48 | |
| *** NobodyCam is now known as BlartVersenwald | 15:49 | |
| sdague | to figure out why things work one place and not the other | 15:49 |
| *** pdmars_ has joined #openstack-infra | 15:49 | |
| *** gyee has joined #openstack-infra | 15:49 | |
| jeblair | i'm curious how that interacts with rax doing file injection for network config | 15:49 |
| sdague | jeblair: yeh... my guess is poorly :) | 15:49 |
| sdague | open glance is kind of a loaded gun | 15:49 |
| *** zns has quit IRC | 15:49 | |
| sdague | plenty of opportunity to shoot yourself in the foot | 15:50 |
| fungi | mordred: while we *can* (and almost certainly should) improve nodepool to start using glance, we can make use of the service asap by just uploading the distro-provided cloud images and telling nodepool we want it to use those as its base images, right? | 15:50 |
| mordred | fungi: yes | 15:50 |
| mordred | fungi: good point | 15:50 |
| sdague | just a thought, for whenever anyone starts actually doing this | 15:50 |
| mordred | and in fact, we can do that in only one provider | 15:50 |
| *** markmcclain has quit IRC | 15:50 | |
| fungi | mordred: i mean, it's what we've done in the tripleo-ci cloud already | 15:50 |
| mordred | fungi wins | 15:50 |
| mordred | yeah | 15:50 |
| *** pdmars has quit IRC | 15:50 | |
| fungi | we already *do* that in one provider ;) | 15:51 |
| sdague | we should make sure that whatever process ends up prepping and uploading images, also makes them generally available for people | 15:51 |
| mordred | sdague: I'm guessing we'll need to make the images be able to handle metadata and injectino | 15:51 |
| sdague | so that we can get them to replicate with the gate image locally | 15:51 |
| mordred | sdague: ++ | 15:51 |
| *** VINOD__ has joined #openstack-infra | 15:51 | |
| mordred | I believe it will make local gate replication easier once we | 15:52 |
| mordred | get there | 15:52 |
| sdague | because I'd still like to get us to a place where gate replication locally with vagrant (or something else) is super simple | 15:52 |
| mordred | I whole heartedly agree | 15:52 |
| *** pblaho has quit IRC | 15:52 | |
| jeblair | yeah, i think that's been our plan for like the 2 years we've been yelling^Wasking for glance :) | 15:52 |
| *** mrodden has quit IRC | 15:53 | |
| sdague | my goofy multinode localy devstack vagrant setup has actually been super useful to me of late, so I'm all on board building this tooling | 15:53 |
| *** zns has joined #openstack-infra | 15:55 | |
| *** adalbas has quit IRC | 15:56 | |
| jeblair | does someone have a link to the rax announcement? | 15:57 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Fixed DEVSTACK_GATE_TEMPEST_REGEX. https://review.openstack.org/80359 | 15:57 |
| sdague | it was on the twitters, let me go find it | 15:59 |
| *** esker has joined #openstack-infra | 15:59 | |
| fungi | because that is apparently the new and highly professional ways to make service update announcements | 16:00 |
| fungi | s/ways/way/ | 16:00 |
| sdague | you know, now that I think of it, I think mikal was the RT on that, and it was actually a link into pyrax on github, so maybe not as official as I thought | 16:02 |
| jeblair | https://twitter.com/mikal/status/443885710840328193 | 16:02 |
| *** pdmars has joined #openstack-infra | 16:02 | |
| *** lcostantino has quit IRC | 16:03 | |
| *** pdmars_ has quit IRC | 16:03 | |
| *** lcostantino has joined #openstack-infra | 16:03 | |
| sdague | yeh, so I actually have no idea what that means for using upstream tools | 16:03 |
| jeblair | wowie glanceclient pulls in a lot of stuff that nova doesn't | 16:04 |
| fungi | jeblair: any objection to https://review.openstack.org/80345 ? without that (or something like it), nodepool precise image updates are broken in rax regions for the past 48 hours or so | 16:04 |
| clarkb | morning | 16:04 |
| jeblair | in ubuntu, installing glanceclient gets you a glance user. | 16:04 |
| sdague | jeblair: um.... really? | 16:04 |
| *** mrodden has joined #openstack-infra | 16:04 | |
| sdague | that's crazy pants | 16:04 |
| *** hogepodge_ has joined #openstack-infra | 16:04 | |
| sdague | why would you do that on the client and not the server | 16:05 |
| *** gokrokve_ has quit IRC | 16:05 | |
| clarkb | sdague: need to keep that client sandboxed in its own user | 16:06 |
| jeblair | so precise glanceclient bombs with Details: [Errno 113] EHOSTUNREACH | 16:06 |
| sdague | why? | 16:06 |
| jeblair | i'm trying to install python-glanceclient from pypi now | 16:07 |
| sdague | jeblair: precise original? | 16:07 |
| sdague | or precise + cloud archive | 16:07 |
| jeblair | sdague: precise orig | 16:07 |
| sdague | so, essex level code | 16:07 |
| *** hogepodge has quit IRC | 16:07 | |
| *** hogepodge_ is now known as hogepodge | 16:07 | |
| sdague | not entirely surprised that would be borked | 16:07 |
| *** nicedice has joined #openstack-infra | 16:07 | |
| jeblair | sdague: not surprising, yes. but also not good. | 16:08 |
| sdague | yeh | 16:08 |
| sdague | look, I yell at people about backwards incompat as much as anyone :) | 16:08 |
| clarkb | btw gerrit >=2.6 has a major bug in it to be fixed in 2.8.3. So we should keep that in mind as part of our testing | 16:08 |
| clarkb | zaro: ^ | 16:08 |
| jeblair | Request returned failure status. | 16:08 |
| jeblair | 404 Not Found | 16:08 |
| fungi | i'm guessing old glanceclient defaulted to trying to reach a different port number | 16:09 |
| jeblair | sorry, that's new glanceclient ^ | 16:09 |
| fungi | right, just surmising as to the reason for the difference in errors | 16:09 |
| *** adalbas has joined #openstack-infra | 16:09 | |
| jeblair | ah | 16:10 |
| fungi | sounds like this one is actually hitting an api endpoint, which is telling it that what it wasn't isn't there | 16:10 |
| fungi | er, that what it wants isn't there | 16:10 |
| fungi | some firewalls may send back icmp host-unreach responses instead of port-unreach or tcp rst in cases where a particular port is blocked | 16:11 |
| fungi | (or icmp admin-prohibited or just not respond at all or... i hate poorly-configured firewalls) | 16:12 |
| openstackgerrit | A change was merged to openstack-infra/config: Restrict nodepool memory only when possible https://review.openstack.org/80345 | 16:12 |
| clarkb | fungi: they love you back | 16:12 |
| *** johnthetubaguy has quit IRC | 16:13 | |
| fungi | once that ^ gets to nodepool.o.o i'll start some updates of precise images in rax | 16:13 |
| *** chandan_kumar has quit IRC | 16:14 | |
| *** Ryan_Lane has quit IRC | 16:15 | |
| *** esker has quit IRC | 16:17 | |
| *** jnoller has joined #openstack-infra | 16:17 | |
| clarkb | jeblair: fungi: zaro mordred SergeyLukjanov and anyone else. I think we have been avoiding testing 2.8 point releases so far to keep things simple but we really want to avoid this gerrit bug. basically gerrit will think stuff is merged without actually merging it if you have a string of dependent changes | 16:17 |
| clarkb | its teh sort of thing that we probably would have found in the first week of using new gerrit had we been using new gerrit | 16:18 |
| fungi | clarkb: yeah, that does sound... not too great | 16:18 |
| *** hogepodge has quit IRC | 16:18 | |
| clarkb | it will merge the first change, report the otehrs as merged without merging in git :) | 16:18 |
| *** thuc has joined #openstack-infra | 16:18 | |
| clarkb | so I think we should go to 2.8.3 when it is available on review-dev | 16:18 |
| fungi | we would have found that in the first day in production, if not in the first hour in production | 16:19 |
| *** thuc has quit IRC | 16:19 | |
| *** thuc has joined #openstack-infra | 16:19 | |
| *** comstud is now known as bearhands | 16:19 | |
| *** thuc has quit IRC | 16:20 | |
| *** rdwrer is now known as marktraceur | 16:20 | |
| *** marktraceur is now known as rdwrer | 16:20 | |
| *** jlibosva has quit IRC | 16:20 | |
| *** dansmith is now known as damnsmith | 16:20 | |
| *** thuc has joined #openstack-infra | 16:21 | |
| *** hogepodge has joined #openstack-infra | 16:21 | |
| jeblair | jnoller: do you know if glance support needs to be enabled for accounts on rax? | 16:22 |
| jeblair | jnoller: i just tried "glance image-list" and got "404 Not Found" | 16:23 |
| *** jgallard has quit IRC | 16:23 | |
| *** sarob has joined #openstack-infra | 16:23 | |
| *** caleb_ has quit IRC | 16:24 | |
| *** nati_ueno has quit IRC | 16:24 | |
| *** rpodolyaka has joined #openstack-infra | 16:25 | |
| *** vkozhukalov_ has joined #openstack-infra | 16:26 | |
| jnoller | jeblair: It should be "just on" | 16:27 |
| *** sarob has quit IRC | 16:27 | |
| jnoller | jeblair: can you PM me your account number so I can check with the product team? | 16:28 |
| openstackgerrit | A change was merged to openstack-infra/config: Pass branch-override on jobs missing it https://review.openstack.org/80568 | 16:29 |
| *** zns_ has joined #openstack-infra | 16:29 | |
| *** zns has quit IRC | 16:30 | |
| *** gokrokve has joined #openstack-infra | 16:30 | |
| clarkb | fungi: https://review.openstack.org/#/c/80111/ what sort of verification went on for that? should we go ahead and flip the switch today? | 16:31 |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Adding creator_id field assignment https://review.openstack.org/80555 | 16:32 |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Fix load_superuser to work with new oslo https://review.openstack.org/80541 | 16:32 |
| clarkb | fungi: (leaning on your judgement here as I believe you tested it) | 16:33 |
| clarkb | or checked it | 16:33 |
| fungi | clarkb: it's currently working on review.o.o in dry-run mode, logging the first contribution from each new contributor. it's been checked and confirmed that it's really only firing once per contributor too, on the first patchset of their first change, so at worse we'll expose some code path hidden by --dry-run when we merge that | 16:33 |
| fungi | clarkb: but i'll keep an eye on the logs at that point anyway | 16:33 |
| zaro | clarkb: yeah i just saw that gerrit email. | 16:34 |
| clarkb | fungi: cool I will approve ti then | 16:34 |
| zaro | fungi: the gerrit-2.8.3 should be available soon, do you want to merge the changes with what's in openstack/2.8 branch when it becomes available? | 16:35 |
| clarkb | zaro: or create a 2.8.3 branch | 16:35 |
| clarkb | that might be easier depending on the deltas involved | 16:35 |
| zaro | ++ | 16:35 |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Fix migration tests https://review.openstack.org/80236 | 16:36 |
| fungi | yeah, for other updates like that we basically just re-propose a patch series rebased on the new branch/tag and pencil-whip an approval on it if tests pass | 16:36 |
| *** talluri has joined #openstack-infra | 16:36 | |
| *** jooools has quit IRC | 16:37 | |
| *** openstackstatus has joined #openstack-infra | 16:38 | |
| jeblair | fungi, clarkb: statusbot was running but offline with nothing in debug logs; i restarted it in screen | 16:38 |
| clarkb | jeblair: rgr | 16:39 |
| fungi | exciting | 16:41 |
| *** blamar has quit IRC | 16:41 | |
| clarkb | jeblair: reviewed the unbound change. one small thing inline | 16:42 |
| *** blamar has joined #openstack-infra | 16:42 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: only display events in irc on gate issues https://review.openstack.org/80621 | 16:42 |
| *** esker has joined #openstack-infra | 16:42 | |
| *** esker has quit IRC | 16:43 | |
| jeblair | sdague: you don't want to see known errors in check? | 16:44 |
| sdague | jeblair: no | 16:44 |
| sdague | because we get things like this: | 16:44 |
| sdague | openstack/tempest change: https://review.openstack.org/50337 failed check-grenade-dsvm, check-tempest-dsvm-full, check-tempest-dsvm-neutron, check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-tempest-dsvm-cells-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-ironic-nv, check-tempest-dsvm-ironic-postgres-nv, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm- | 16:44 |
| sdague | sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1253896 | 16:44 |
| sdague | and people are completely tuning out the bot at this point | 16:45 |
| *** adalbas has quit IRC | 16:46 | |
| jeblair | sdague: is the signature for that bug inaccurate? | 16:48 |
| sdague | the signature is fine, it matches one of those failed tests somewhere | 16:48 |
| *** Ryan_Lane has joined #openstack-infra | 16:48 | |
| sdague | however, the signatures really are only full accurate on otherwise working code | 16:48 |
| *** esker has joined #openstack-infra | 16:49 | |
| openstackgerrit | A change was merged to openstack-infra/config: Enable Welcome Message to post review comments https://review.openstack.org/80111 | 16:49 |
| jeblair | sdague: *nod* | 16:49 |
| sdague | if people throw random fuzz at things | 16:49 |
| sdague | you'll get hits some times as well | 16:49 |
| jeblair | sdague: also, is the test list very useful in irc? maybe omitting it, and thereby reducing the wall of text may help with attention... | 16:49 |
| sdague | jeblair: that's a good point as well | 16:49 |
| *** BobBall is now known as BobBall_Away | 16:50 | |
| sdague | honestly the thing I'm mostly interested in is a failed gate job that is unclassified | 16:50 |
| jeblair | (or alternately, limiting it to 1, so it's either "failed FOO or failed multiple tests" | 16:50 |
| *** jgrimm has quit IRC | 16:50 | |
| sdague | yeh, there are a few fixes here that would be also useful. dropping the check queue was already about 50% done so I just did that one | 16:51 |
| *** rpodolyaka has quit IRC | 16:51 | |
| fungi | perhaps if it were slightly smarter, not fire at all if more than one job reported the same failure pattern on the same change/patchset | 16:53 |
| clarkb | mordred: https://review.openstack.org/#/c/79537/ has a comment | 16:53 |
| *** dims has joined #openstack-infra | 16:54 | |
| * jeblair quietly removes his +2 and hopes no one notices it was ever there | 16:54 | |
| clarkb | jeblair: >_> | 16:54 |
| *** matrohon has quit IRC | 16:55 | |
| clarkb | I should actually update my review to have it check in the comprehension /me does that | 16:56 |
| *** rpodolyaka has joined #openstack-infra | 16:58 | |
| *** hogepodge has quit IRC | 16:58 | |
| openstackgerrit | David Kranz proposed a change to openstack-infra/config: Add periodic tempest non-isolated job with neutron https://review.openstack.org/80632 | 16:58 |
| *** VINOD__ has quit IRC | 16:59 | |
| *** esker has quit IRC | 16:59 | |
| *** dims has quit IRC | 17:00 | |
| *** markmcclain has joined #openstack-infra | 17:00 | |
| *** markmcclain has quit IRC | 17:00 | |
| *** markmcclain has joined #openstack-infra | 17:01 | |
| *** nati_ueno has joined #openstack-infra | 17:01 | |
| clarkb | jeblair: if it helps I had written my own version of that change too before I realized mordred wrote oen so was familar with the code | 17:01 |
| *** hogepodge has joined #openstack-infra | 17:02 | |
| *** Ryan_Lane has quit IRC | 17:02 | |
| * fungi is heading out to grab some sushi, back in an hour or so | 17:02 | |
| *** pdmars has quit IRC | 17:02 | |
| *** harlowja_away is now known as harlowja | 17:02 | |
| fungi | jjb updates on all the jenkins masters finished btw | 17:02 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: only display events in irc on gate issues https://review.openstack.org/80621 | 17:05 |
| *** Ryan_Lane has joined #openstack-infra | 17:06 | |
| mordred | jeblair: it helps if you don't tell people you're quietly removing things :) | 17:06 |
| mordred | clarkb: thanks - I will definitely move the code - I think I'd like to keep it as an if though - putting it in the comprehension seems slightly harder to read | 17:06 |
| clarkb | mordred: do you know why the split on ',' exists? | 17:07 |
| clarkb | mordred: the if won't handle the split on ',' if you have a list of images delimited by ',' and vmdk shows up in the non last element | 17:07 |
| clarkb | mordred: you could do the comprehension then do a filter on it | 17:07 |
| mordred | ah. gotcha. excellent point | 17:08 |
| clarkb | a second step filte rwould be readable I think | 17:08 |
| clarkb | you could even do the filter outside the loop before returning | 17:08 |
| openstackgerrit | A change was merged to openstack-infra/gerritbot: Support SSL https://review.openstack.org/79049 | 17:09 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Skip downloading giant VMware images https://review.openstack.org/79537 | 17:09 |
| mordred | clarkb: nah - I think the filter in the comprehension is fine if there's an operational reason for it | 17:10 |
| damnsmith | would it be a welcome change for the progress bar of jobs to change color once we learn that the job is failing? | 17:10 |
| damnsmith | I get that via dash.py and it's really helpful to know that something is failing early | 17:10 |
| *** hogepodge has quit IRC | 17:11 | |
| jeblair | damnsmith: how does it know it's failing? | 17:11 |
| damnsmith | jeblair: don't we know? or do we only know once it has failed? | 17:11 |
| damnsmith | I thought we got early warning now.. | 17:11 |
| damnsmith | I don't see the jobs broken out in dash, so maybe I'm confusing failed jobs with failing jobs | 17:12 |
| jeblair | damnsmith: we have the theoretical possibility of determining that by having something read the subunit stream from testr, but nothing has been written yet to do that | 17:12 |
| damnsmith | jeblair: ohh, I thought it had | 17:12 |
| jeblair | damnsmith: hopefully we will get that, because it means that zuul could make even earlier decisions about changes... | 17:12 |
| *** pbelanyi_ has quit IRC | 17:12 | |
| mordred | ++ | 17:12 |
| damnsmith | jeblair: right, I thought it was now.. okay.. | 17:13 |
| jeblair | damnsmith: but i think at this point that might end up landing in the post-jenkins job runner... | 17:13 |
| *** samalba has left #openstack-infra | 17:13 | |
| damnsmith | jeblair: you trampled my hopes and dreams :( | 17:13 |
| jeblair | damnsmith: because i'm thinking it might be a bit too much dev for something we decided we want to replace | 17:13 |
| jeblair | damnsmith: yeah, sounds like dash is looking at overall completion as soon as a single job for the change fails | 17:14 |
| damnsmith | okay, I really thought this was already done | 17:14 |
| damnsmith | yeah | 17:14 |
| jeblair | damnsmith: (equivalent to the red/green dot) | 17:14 |
| damnsmith | yeah | 17:14 |
| damnsmith | too much magic to keep in my head, I guess | 17:14 |
| *** skraynev is now known as skraynev_afk | 17:14 | |
| clarkb | jeblair: https://review.openstack.org/#/c/80152/ is very topical | 17:15 |
| devananda | command to trigger an experimental run is "check experimental", yes? | 17:15 |
| clarkb | devananda: yes | 17:15 |
| devananda | ty | 17:15 |
| *** SumitNaiksatam has joined #openstack-infra | 17:16 | |
| *** smarcet has left #openstack-infra | 17:16 | |
| mordred | clarkb: remember back when we exceeded 60k reviews and thought that was alot? | 17:17 |
| *** afazekas has quit IRC | 17:18 | |
| jeblair | we should plot review # over time | 17:19 |
| jeblair | | | 17:19 |
| jeblair | _/ | 17:19 |
| jeblair | oh there it is ^ :) | 17:19 |
| *** hashar has quit IRC | 17:20 | |
| clarkb | jeblair++ | 17:20 |
| *** zns_ has quit IRC | 17:21 | |
| *** morganfainberg_Z is now known as morganfainberg | 17:21 | |
| *** e0ne has quit IRC | 17:22 | |
| jeblair | hrm | 17:24 |
| jeblair | is it possible that selinux is not enforcing on rax, but is on hpcloud? | 17:24 |
| pleia2 | https://review.openstack.org/#/c/74196/ looks to be stuck | 17:25 |
| *** wenlock has quit IRC | 17:27 | |
| jeblair | pleia2: thx reapproved | 17:27 |
| jeblair | selinux is disabled on the rax node i booted, and the errors certainly seem to support in being enabled on the hpcloud ones | 17:27 |
| pleia2 | jeblair: https://review.openstack.org/#/c/74167/ & https://review.openstack.org/#/c/69461/ too | 17:27 |
| *** jp_at_hp1 has joined #openstack-infra | 17:28 | |
| *** zns has joined #openstack-infra | 17:28 | |
| *** jp_at_hp has quit IRC | 17:28 | |
| jeblair | i approved the 1st one, and recheck-no-bugged the second one since i didn't actually review it. | 17:30 |
| *** dprince has quit IRC | 17:31 | |
| bknudson | on this https://review.openstack.org/#/c/80137/ | 17:31 |
| pleia2 | thanks | 17:31 |
| bknudson | e-r says check-tempest-dsvm-neutron-full: https://bugs.launchpad.net/bugs/1248757 | 17:32 |
| bknudson | but check-grenade-dsvm-partial-ncpu FAILURE in 22m 04s (non-voting) | 17:32 |
| clarkb | bknudson: the bug signature isn't specific enough and matched the successful test | 17:32 |
| jeblair | rustlebee: https://review.openstack.org/#/q/status:open+project:openstack-infra/config+branch:master+topic:new-project,n,z | 17:36 |
| jeblair | rustlebee: i think you had a nova-specs change, but it may have the wrong topic, and it looks like jenkins is -1 on nova-docker | 17:37 |
| clarkb | oh right new project friday /me reviews those | 17:37 |
| *** mriedem has quit IRC | 17:37 | |
| *** mriedem has joined #openstack-infra | 17:39 | |
| openstackgerrit | Brant Knudson proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1291677 https://review.openstack.org/80639 | 17:40 |
| pleia2 | nibalizer: when you're about, would be nice to have your thought's on clarkb's comment here so we can nudge this along (I think it would be great to get this going): https://review.openstack.org/#/c/71739/1 | 17:40 |
| *** dims has joined #openstack-infra | 17:40 | |
| *** amcrn has joined #openstack-infra | 17:40 | |
| *** khyati has joined #openstack-infra | 17:42 | |
| zaro | fungi: fungi gerrit-2.8.3 has been released. could you please update with one of our branches? | 17:42 |
| *** wenlock has joined #openstack-infra | 17:43 | |
| zaro | clarkb: i've commented https://review.openstack.org/#/c/60893 | 17:43 |
| *** esker has joined #openstack-infra | 17:43 | |
| clarkb | jeblair: https://review.openstack.org/#/c/78681/ is another | 17:44 |
| clarkb | zaro: thank you | 17:44 |
| *** rpodolyaka has quit IRC | 17:45 | |
| jeblair | clarkb: i intend to hold the line on the new-project topic | 17:45 |
| *** hogepodge has joined #openstack-infra | 17:45 | |
| clarkb | jeblair: ok, I will leave a comment on any I see without the topic to add the topic | 17:45 |
| openstackgerrit | A change was merged to openstack-infra/gitdm: Add header to output files and allow date ranges https://review.openstack.org/69461 | 17:46 |
| *** CaptTofu has joined #openstack-infra | 17:46 | |
| clarkb | jeblair: https://review.openstack.org/#/c/78939/1 I think that would be good to get in | 17:46 |
| clarkb | but I also think gettin larger consensus is good because it changes a big piece of the automated infra | 17:47 |
| jeblair | pleia2: just a reminder, i would like to have puppetboard up and running before we merge any puppet refactor changes | 17:47 |
| *** thuc has quit IRC | 17:48 | |
| *** jpich has quit IRC | 17:48 | |
| pleia2 | jeblair: ah, ok | 17:48 |
| *** thuc has joined #openstack-infra | 17:48 | |
| *** rpodolyaka has joined #openstack-infra | 17:48 | |
| *** homeless_ has quit IRC | 17:48 | |
| jeblair | (i want the folks working on the puppet refactor to be able to spot and fix any problems themselves without infra-root being a bottleneck) | 17:49 |
| pleia2 | yeah, that makes sense | 17:49 |
| clarkb | jeblair: https://review.openstack.org/#/c/79363/ is the nova specs change. Iam leaving a commetn about new-project topic | 17:50 |
| *** zns has quit IRC | 17:51 | |
| *** zns has joined #openstack-infra | 17:52 | |
| openstackgerrit | Dan Smith proposed a change to openstack-infra/devstack-gate: DO NOT MERGE - Test larger largerops https://review.openstack.org/80643 | 17:52 |
| *** vponomaryov has left #openstack-infra | 17:52 | |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/config: Create nova-specs git repo https://review.openstack.org/79363 | 17:52 |
| *** thuc has quit IRC | 17:52 | |
| devananda | sdague: http://logs.openstack.org/48/70348/24/experimental/check-tempest-dsvm-virtual-ironic-nv/194612c/ | 17:52 |
| rustlebee | jeblair: i don't know why the topic isn't right. it's "new-project" locally. | 17:52 |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/config: Create nova-specs git repo https://review.openstack.org/79363 | 17:53 |
| rustlebee | nm, fixed with manual git push instead of git review .. | 17:53 |
| jeblair | rustlebee: "git review -t new-project" should do it on a new patchset | 17:54 |
| devananda | i'm reading it now | 17:54 |
| rustlebee | jeblair: ah, cool | 17:54 |
| rustlebee | jeblair: i went with git push gerrit HEAD:refs/publish/master/new-project, heh | 17:54 |
| devananda | hrmm. didn't configure nova to use ironic ... | 17:54 |
| clarkb | rustlebee: beat me to it | 17:55 |
| rustlebee | clarkb: thanks for telling me what i did wrong on the nova-docker one | 17:55 |
| rustlebee | i was stumped | 17:56 |
| *** sarob has joined #openstack-infra | 17:56 | |
| *** thuc has joined #openstack-infra | 17:56 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: update docs and All-Project configuration for gerrit 2.8 https://review.openstack.org/70818 | 17:56 |
| *** derekh has quit IRC | 17:58 | |
| clarkb | breakfast has arrived /me afks | 17:58 |
| jeblair | clarkb, sdague: i'm not keen on the commit message in https://review.openstack.org/#/c/78939/ | 17:59 |
| jeblair | clarkb, sdague: i think it has some inaccuracies and seems to have been written in reaction to an event that, as it turns out, is unrelated to the change | 18:00 |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/config: Create a nova-docker repo https://review.openstack.org/79900 | 18:00 |
| devananda | fungi: have a minute to help me figure out why the experimental run of virtual-ironic looks like it didn't test any of the code in that devstack patch? | 18:00 |
| *** Ryan_Lane has quit IRC | 18:01 | |
| jeblair | in practice, it is not a huge problem (it doesn't happen "very often"). we have a tool to more or less instantly recover from such a situation.... | 18:01 |
| *** khyati has quit IRC | 18:01 | |
| jeblair | documented here: https://review.openstack.org/#/c/79794/ | 18:02 |
| *** sarob has joined #openstack-infra | 18:02 | |
| sdague | jeblair: that's fair, colorfulness happens after debugging for 3 - 4hrs before anyone on infra root is up :) | 18:02 |
| *** dcramer_ has quit IRC | 18:03 | |
| *** amcrn has quit IRC | 18:03 | |
| devananda | fungi: the console log https://jenkins02.openstack.org/job/check-tempest-dsvm-virtual-ironic-nv/2/consoleText shows the right exports, but http://logs.openstack.org/48/70348/24/experimental/check-tempest-dsvm-virtual-ironic-nv/194612c/logs/devstacklog.txt.gz doesn't seem to indicate they were used | 18:04 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: offset nodepool update by 12 hrs https://review.openstack.org/78939 | 18:05 |
| devananda | ooh. instead of exporting, do all these https://review.openstack.org/#/c/80575/4/modules/openstack_project/files/jenkins_job_builder/config/devstack-gate.yaml need to be put in localrc? | 18:05 |
| *** marun has joined #openstack-infra | 18:05 | |
| devananda | sdague: ^ ? | 18:06 |
| *** homeless has joined #openstack-infra | 18:06 | |
| *** zns has quit IRC | 18:06 | |
| sdague | devananda: we don't automatically pass the environment all the way through | 18:06 |
| devananda | ahh | 18:06 |
| sdague | see devstack-gate-wrap | 18:06 |
| *** zns has joined #openstack-infra | 18:08 | |
| *** adalbas has joined #openstack-infra | 18:08 | |
| devananda | sdague: hm. so what's the right way to pass these in? | 18:08 |
| *** yolanda has quit IRC | 18:09 | |
| *** amcrn has joined #openstack-infra | 18:10 | |
| sdague | thinking... | 18:11 |
| devananda | sdague: i see a lot of similar things in devstack-vm-gate.sh | 18:11 |
| *** e0ne has joined #openstack-infra | 18:11 | |
| sdague | right, the real question is how much non default configuration this needs | 18:11 |
| sdague | it feels like the way it should be handled is if VIRT_DRIVER=virtual_ironic it would set up all that behind the scenes | 18:12 |
| devananda | sdague: all those exports which we landed in 80575 | 18:12 |
| devananda | sdague: hmm | 18:13 |
| devananda | so _VM_COUNT and _VM_SPECS_* can be defaulted and then dropped from here | 18:13 |
| devananda | yea, i think all of them could be defaulted | 18:14 |
| devananda | except IRONIC_BAREMETAL_BASIC_OPS | 18:14 |
| devananda | that enables a big chunk of code specific to this environment, which isn't necessary for just starting the ironic api/conductor services | 18:14 |
| *** jp_at_hp1 has quit IRC | 18:14 | |
| sdague | I think so. That would be at least consistent in that from a devstack user perspective, if yuo wanted to use it for ironic development, set VIRT_DRIVER=virtual_ironic, and you get a working setup | 18:14 |
| sdague | sure | 18:14 |
| *** hogepodge has quit IRC | 18:15 | |
| devananda | if you want to test ironic itself, set VIRT_DRIVER. if you want a local ovs bridge and some fake VMs, also set IRONIC_BAREMETAL_BASIC_OPS | 18:15 |
| devananda | cool. i'll refactor that and fix up the infra config and d-g code | 18:15 |
| sdague | so IRONIC_BAREMETAL_BASIC_OPS would then get handled like all the other export = 1 thing in d-g | 18:15 |
| devananda | ya | 18:15 |
| *** zns has quit IRC | 18:15 | |
| sdague | yeh, I think that will work | 18:15 |
| devananda | :) | 18:15 |
| *** sweston has joined #openstack-infra | 18:16 | |
| *** mgagne has joined #openstack-infra | 18:17 | |
| *** mgagne has quit IRC | 18:17 | |
| *** mgagne has joined #openstack-infra | 18:17 | |
| devananda | sdague: worth having two flags in d-g? DEVSTACK_GATE_IRONIC => just enable the service. DEVSTACK_GATE_VIRT_DRIVER => mutate the environment to use the nova.virt.ironic driver | 18:17 |
| devananda | ? | 18:17 |
| *** zns has joined #openstack-infra | 18:17 | |
| sdague | sure, I think that's fine | 18:18 |
| devananda | i think that also makes it explicit -- the ironic service is separate from the nova virt driver (which it is) | 18:18 |
| sdague | yep | 18:18 |
| *** zns has quit IRC | 18:22 | |
| *** zns has joined #openstack-infra | 18:22 | |
| openstackgerrit | Alex Gaynor proposed a change to openstack-infra/config: Gate python-glanceclient on PyPy https://review.openstack.org/76072 | 18:23 |
| openstackgerrit | Devananda van der Veen proposed a change to openstack-infra/config: Fix the virtual-ironic job's export list https://review.openstack.org/80652 | 18:24 |
| openstackgerrit | Devananda van der Veen proposed a change to openstack-infra/devstack-gate: Configure localrc for virtual-ironic tests https://review.openstack.org/80653 | 18:24 |
| openstackgerrit | Dan Smith proposed a change to openstack-infra/devstack-gate: DO NOT MERGE - Test larger largerops https://review.openstack.org/80643 | 18:27 |
| *** jergerber has joined #openstack-infra | 18:29 | |
| *** rwsu has quit IRC | 18:30 | |
| *** ryanpetrello has quit IRC | 18:30 | |
| *** jgrimm has joined #openstack-infra | 18:31 | |
| fungi | zaro: openstack-infra/gerrit:upstream/stable-2.8 hasn't picked up the new commit yet, but it should when we rerun manage-projects here in a bit. after that i can branch openstack/2.8.3 from it for you | 18:32 |
| *** hogepodge has joined #openstack-infra | 18:32 | |
| openstackgerrit | A change was merged to openstack-infra/config: Skip downloading giant VMware images https://review.openstack.org/79537 | 18:36 |
| fungi | devananda: keep in mind that if you *do* decide to try and modify localrc prior to running devstack-vm-gate-wrap.sh, we'll probably need a feature like the post_setup hook sdague mentions in his comment on https://review.openstack.org/80566 | 18:37 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: update gerrit project configs to use new Workflow label https://review.openstack.org/79107 | 18:38 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: update trigger to match new workflow label https://review.openstack.org/79103 | 18:38 |
| *** salv-orlando has joined #openstack-infra | 18:38 | |
| *** thomasbiege has joined #openstack-infra | 18:39 | |
| *** rpodolyaka has quit IRC | 18:39 | |
| adam_g | devananda, heads up https://bugs.launchpad.net/bugs/1248025 will likely bite you when things end up running on rackspace. | 18:42 |
| adam_g | gonna see if there is a manual workaround in the meantime while the update to cloud archive package makes its way thru | 18:42 |
| *** dprince has joined #openstack-infra | 18:43 | |
| *** rpodolyaka has joined #openstack-infra | 18:47 | |
| *** dizquierdo has quit IRC | 18:48 | |
| devananda | adam_g: things are running on rackspace ... | 18:50 |
| devananda | fungi: not sure I follow your comment yet. are you suggesting I change 80653 ? | 18:51 |
| fungi | devananda: no, i was catching up on scrollback and you asked about altering localrc from the job definition... i was just going to point out that there are currently changes in flight to make that possible, but it's not doable yet | 18:52 |
| devananda | fungi: oh. gotcha. so instead, i did the above two patches, which is more in line with what ya'll do today | 18:52 |
| *** Ryan_Lane has joined #openstack-infra | 18:53 | |
| fungi | devananda: makes sense, though i haven't had time to look closely yet | 18:53 |
| *** caleb_ has joined #openstack-infra | 18:56 | |
| sdague | fungi: while I think we could do this this way, if that hook was available, I don't think it requires the hook the same way that the 3rd party folks need it | 18:57 |
| fungi | sdague: agreed, the current solution seems better for a project already in incubarion and targeting integration | 18:57 |
| devananda | adam_g: that paste is interesting | 18:58 |
| *** thuc has quit IRC | 18:59 | |
| devananda | fungi: thoughts on http://paste.ubuntu.com/7091752/ ? | 18:59 |
| adam_g | the bug has more details, obviously :) | 18:59 |
| openstackgerrit | Brant Knudson proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1291677 https://review.openstack.org/80639 | 18:59 |
| devananda | adam_g has pointed out that virtual-ironic tests will fail on rax nodes unless we ^ | 18:59 |
| devananda | bug -- https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1248025 | 18:59 |
| *** thuc has joined #openstack-infra | 18:59 | |
| fungi | devananda: yeah, i was looking at the bug report briefly. it looks like something you could do as part of your job or in a pre hook | 19:01 |
| *** esker has quit IRC | 19:01 | |
| *** salv-orlando has quit IRC | 19:02 | |
| fungi | devananda: though i wonder if providing our own vm images will mitigate that | 19:02 |
| *** tstevenson has joined #openstack-infra | 19:02 | |
| adam_g | it would | 19:03 |
| adam_g | assuming that RAX-specific stuff is not installed in them | 19:03 |
| *** jogo is now known as flashgordon | 19:03 | |
| clarkb | that bug affects precise? | 19:03 |
| clarkb | we dont do saucy | 19:04 |
| *** thuc has quit IRC | 19:04 | |
| devananda | clarkb: we're pulling in libvirt from cloud-archives for this test | 19:04 |
| clarkb | oh dont do that | 19:05 |
| clarkb | uca libvirt is currently broken | 19:05 |
| clarkb | (being fixed though \o/) | 19:05 |
| devananda | we need to... | 19:05 |
| adam_g | clarkb, broken in what regard? | 19:05 |
| adam_g | well,other than not starting | 19:05 |
| adam_g | :P | 19:05 |
| clarkb | adam_g aiui its related to removing the global lock in newer libvirt | 19:06 |
| fungi | adam_g: dims is testing it... we had issues under devstack/tempest and also running nova unit tests with it | 19:06 |
| clarkb | and it deadlocks. danpb fixed upstream and uca will include the fix(es) soonish | 19:06 |
| *** thuc has joined #openstack-infra | 19:07 | |
| devananda | clarkb: so uca will be fixed soonish. so pulling it from there is the right thing to do soonish. so we're doign the right thing, then | 19:07 |
| fungi | devananda: anyway, that's what i meant with my apparently vague reference on your etherpad. once it's been sussed out, i expect we'll have it enabled everywhere and then that part of your job will cease being necessary | 19:07 |
| adam_g | clarkb, do you have a bug? i was going to see if i can expedite bug #1248025 getting pushed out, could make sure they both go out together | 19:07 |
| *** tstevenson has quit IRC | 19:08 | |
| devananda | fungi: that will be great. do yhou think we can proceed with this prior to that fix being pushed out? | 19:08 |
| fungi | devananda: i think it's fair to add a non-voting job to see if it's working for your use | 19:09 |
| devananda | fungi: my goal today is to get 70348 landed in devstack, so ironic developers can start working on tempest tests. | 19:09 |
| *** che-arne has joined #openstack-infra | 19:09 | |
| fungi | adam_g: https://launchpad.net/bugs/1228977 was one of the two we had open as uca blockers i think | 19:09 |
| devananda | fungi: great :) | 19:09 |
| fungi | looking for the other one now | 19:09 |
| clarkb | adam_g I dont have the bug handy it is filed against ubuntu libvirt and nova iirc | 19:10 |
| zaro | fungi: a few of my changes have already merged onto openstack/2.8 branch. should i cherry pick those changes to openstack/2.8.3 and go thru gerrit review again? | 19:10 |
| *** marun has quit IRC | 19:11 | |
| fungi | zaro: yes, we'd be starting fresh from upstream on 2.8.3, so just leave comments on the ones which were already approved once and if they're passing tests i'm happy to approve them without a second +2 | 19:11 |
| jeblair | clarkb, fungi: i just noticed in my dns tests that hpcloud-az2 builds seem to have puppet timing out a lot... | 19:12 |
| *** hogepodge has quit IRC | 19:12 | |
| jeblair | clarkb, fungi: in the middle of the traceback is this: INFO:nodepool.image.build.hpcloud-az2.bare-centos6: from /usr/lib/ruby/site_ruby/1.8/facter/ec2.rb:5:in `metadata' | 19:12 |
| fungi | jeblair: that's no good. just az2? | 19:12 |
| jeblair | clarkb, fungi: i'm thinking that the same metadata problem that's causing problems on first boot is also hitting puppet | 19:12 |
| clarkb | huh thats facter getting metadata facts | 19:12 |
| clarkb | jeblair agree | 19:12 |
| clarkb | :/ | 19:13 |
| fungi | oh, ick | 19:13 |
| *** pdmars has joined #openstack-infra | 19:13 | |
| * jeblair lunches | 19:13 | |
| *** ryanpetrello has joined #openstack-infra | 19:13 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Replaced first_name and last_name with full_name. https://review.openstack.org/79720 | 19:13 |
| *** rpodolyaka has quit IRC | 19:14 | |
| fungi | adam_g: looking back at my notes i guess it's been mainly 1228977 we've been chasing on the libvirt/uca side of things. the other i had noted was a bug in nova exposed by newer libvirt, already addressed now | 19:15 |
| adam_g | fungi, cool. looks like 1228977 has an update ready for testing for yesterday | 19:16 |
| *** caleb_ has quit IRC | 19:16 | |
| fungi | adam_g: yep, they've seen renewed vigor on it this week so i think we're close to a resolution | 19:16 |
| *** rpodolyaka has joined #openstack-infra | 19:17 | |
| fungi | it's just been a loooong time coming, or else we'd probably have been using uca for close to a year now | 19:17 |
| flashgordon | clarkb: what is involved in getting http://logs.openstack.org/20/62820/4/gate/gate-tempest-dsvm-full/6a9f94a/logs/horizon_error.txt.gz into ES | 19:17 |
| flashgordon | or is already there | 19:17 |
| clarkb | timestamps | 19:17 |
| clarkb | iirc that is what is missing | 19:17 |
| fungi | adam_g: while the bug itself doesn't stretch back that far, we had a couple instances of trying to add uca in production which were hastily rolled back because everything fell apart and we didn't have great debugging info, so no bugs were filed for a while | 19:18 |
| openstackgerrit | Devananda van der Veen proposed a change to openstack-infra/config: Fix the virtual-ironic job's export list https://review.openstack.org/80652 | 19:18 |
| dims | devananda, latest status - there are two things i am chasing a 0.9.8+ libvirt that will end up in precise-updates itself. 1.2.2+ libvirt which will be in UCA/icehouse/precise. (both made available courtesy of @jamespage and @hallyn). i've validated the 0.9.8+ change and just waiting for them to push it out to official deb repos. 1.2.2+ is still facing a test failure which @jamespage is chasing | 19:19 |
| jamespage | dims, yeah - not much progress on that today - been chasing down bugs in icehouse3 on 14.04 (as zul is still on the beach) | 19:20 |
| dims | jamespage, thanks! | 19:20 |
| fungi | jamespage: sounds like he chose very well when scheduling his holiday | 19:20 |
| flashgordon | got a missing dependency in stable/havana | 19:21 |
| flashgordon | http://logs.openstack.org/20/62820/4/gate/gate-tempest-dsvm-full/6a9f94a/logs/screen-h-eng.txt.gz | 19:21 |
| flashgordon | http://logs.openstack.org/20/62820/4/gate/gate-tempest-dsvm-full/6a9f94a/logs/horizon_error.txt.gz | 19:21 |
| fungi | flashgordon: i believe horizon logs aren't indexed into logstash yet, though i forget the exact rationale | 19:22 |
| *** CaptTofu has quit IRC | 19:22 | |
| clarkb | no timestamps | 19:22 |
| *** CaptTofu has joined #openstack-infra | 19:22 | |
| fungi | that | 19:22 |
| * flashgordon sees timestamps on http://logs.openstack.org/20/62820/4/gate/gate-tempest-dsvm-full/6a9f94a/logs/horizon_error.txt.gz | 19:22 | |
| dims | jamespage, not sure if you saw my prev comment - "[09:15] <dims_> jamespage, some of the tests don't compile. (libvirt_1.2.1-0ubuntu5~cloud0.dsc). abandoning effort for now. i can build 1.2.2 tag direct from libvirt.org git." | 19:22 |
| *** Ryan_Lane has quit IRC | 19:23 | |
| jamespage | dims, I did - thanks for poking it | 19:23 |
| clarkb | timestamps would be a new development | 19:23 |
| jamespage | dims, I think is specifically todo with the build environment | 19:23 |
| jamespage | dims, however I'm going to get zul to look on Monday | 19:23 |
| jamespage | as penance | 19:23 |
| dims | jamespage, :) | 19:23 |
| fungi | no good vacation goes unpunished | 19:23 |
| clarkb | of course it has its own format yay | 19:23 |
| * clarkb thinks we should make them oslo log if possible | 19:24 | |
| *** rpodolyaka has quit IRC | 19:25 | |
| flashgordon | fungi: it looks like keystone client uses oauthlib but we don't install it in stable/havana oauthlib | 19:25 |
| flashgordon | how is that possible | 19:25 |
| fungi | flashgordon: that looks like maybe someone's confused versions of python-oauth and python-oauthlib? | 19:25 |
| *** rpodolyaka has joined #openstack-infra | 19:26 | |
| flashgordon | fungi: link? | 19:26 |
| flashgordon | sdague: ^ | 19:26 |
| fungi | flashgordon: i'm digging into it | 19:26 |
| flashgordon | fungi: thanks | 19:26 |
| *** CaptTofu has quit IRC | 19:27 | |
| sdague | clarkb: horizon logs are tailing apache | 19:28 |
| flashgordon | fungi: ahh ouahtlib is needed by keystone | 19:28 |
| fungi | flashgordon: yeah, stable/havana of global-requirements only has oauth2 | 19:28 |
| *** yolanda has joined #openstack-infra | 19:28 | |
| flashgordon | and ouath is needed by keystone client | 19:28 |
| sdague | so they will be whatever they are | 19:28 |
| *** lcostantino has quit IRC | 19:28 | |
| sdague | but I would assume that logstash already has a format for that? | 19:28 |
| flashgordon | oauthlib>=0.6 | 19:28 |
| flashgordon | keystone: | 19:28 |
| *** hashar has joined #openstack-infra | 19:28 | |
| clarkb | sdague it does for clr stuff we do it with swift | 19:29 |
| fungi | flashgordon: and master of global-requirements only has oauthlib | 19:29 |
| flashgordon | keystonecleint: oauthlib>=0.6 | 19:29 |
| clarkb | that doesnt look the same. the dates are wrong for instance | 19:29 |
| flashgordon | fungi: be back in a bit have a meeting | 19:29 |
| fungi | flashgordon: k | 19:29 |
| clarkb | I think apache is taking stderr from wsgi and logging that maybe | 19:29 |
| flashgordon | are requirements changes gated on stable? | 19:29 |
| *** yolanda has quit IRC | 19:30 | |
| flashgordon | or at least client changes? | 19:30 |
| sdague | clarkb: good point | 19:30 |
| fungi | flashgordon: clients don't gate on stable currently, but they should | 19:30 |
| fungi | flashgordon: so right now we test client changes against master everything, but we test stable servers with master clients, so asymmetry there | 19:31 |
| fungi | flashgordon: which allows clients to unfortunately break stable jobs right now | 19:31 |
| fungi | flashgordon: it's relatively known that we need clients to also run backward-compat gating jobs, but nobody's had time to add them yet | 19:32 |
| zaro | jeblair: i was wondering if there's gonna be anymore releases to python jenkins? i've seen a few situations where jjb users could use a method to get plugin versions to stay backwards compatible. | 19:32 |
| *** nijaba has quit IRC | 19:33 | |
| sdague | clarkb: so that's apache error log format | 19:33 |
| *** ildikov_ has quit IRC | 19:33 | |
| sdague | which is not chanable in the version of apache in precise (in trusty we could change it) | 19:33 |
| sdague | any idea if logstash already has a baked in template for that? | 19:33 |
| clarkb | it might but I dont remember seeing one. look on the logstash recipes/cookbooks wiki | 19:34 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Replaced first_name and last_name with full_name. https://review.openstack.org/79720 | 19:34 |
| *** zns has quit IRC | 19:36 | |
| *** yolanda has joined #openstack-infra | 19:37 | |
| *** zns has joined #openstack-infra | 19:37 | |
| *** sweston has quit IRC | 19:38 | |
| *** nijaba has joined #openstack-infra | 19:38 | |
| *** nijaba has joined #openstack-infra | 19:38 | |
| *** e0ne has quit IRC | 19:38 | |
| *** rfolco has quit IRC | 19:39 | |
| *** e0ne has joined #openstack-infra | 19:39 | |
| sdague | clarkb: looks like not | 19:39 |
| *** nijaba has quit IRC | 19:40 | |
| *** Ryan_Lane has joined #openstack-infra | 19:41 | |
| *** yolanda has quit IRC | 19:43 | |
| fungi | bwahaha... i just stumbled across mangodb ;) | 19:43 |
| fungi | output = open(os.devnull, 'w') | 19:44 |
| *** e0ne_ has joined #openstack-infra | 19:44 | |
| sdague | heh | 19:44 |
| *** e0ne has quit IRC | 19:45 | |
| dstufft | mangodb is web 3.0 | 19:46 |
| fungi | dstufft: and webSCALE | 19:46 |
| dstufft | also does swiftclient have terrible docs or am I just too stupid to find them | 19:46 |
| *** talluri has quit IRC | 19:46 | |
| fungi | dstufft: what sort of docs are you seeking? man swift? | 19:47 |
| dstufft | I want to use the Python API to do stuff | 19:47 |
| dstufft | I found this http://docs.openstack.org/developer/python-swiftclient/swiftclient.html#module-swiftclient.client :/ | 19:47 |
| *** rpodolyaka has quit IRC | 19:47 | |
| clarkb | fungi: https://pypi.python.org/pypi/fuckitdb/0.1.3 | 19:48 |
| sdague | clarkb: so how does one define a new timestamp format? | 19:49 |
| *** rpodolyaka has joined #openstack-infra | 19:49 | |
| sdague | I'm completely failing with logstash docs | 19:49 |
| clarkb | sdague: logstash comes with a lot of prebuild time/date matchers | 19:49 |
| fungi | dstufft: yeah, the sphinx/autodoc stuff is probably what there is | 19:49 |
| clarkb | sdague: you should be able to use MONTH or DAY etc to match it in grok | 19:49 |
| fungi | dstufft: so what you find at http://docs.openstack.org/developer/python-swiftclient/ | 19:49 |
| dstufft | fungi: :[ | 19:49 |
| clarkb | sdague: then lower in the logstash config we normalize all fo them to one format with a filter | 19:50 |
| jesusaurus | sdague: what do you mean? where are you trying to use the new timestamp? | 19:50 |
| fungi | dstufft: i'm sure nobody would turn down documentation patches ;) | 19:50 |
| dstufft | fungi: well before I do that, I have to actually figure out how to use it :D | 19:50 |
| *** hogepodge has joined #openstack-infra | 19:51 | |
| fungi | dstufft: i do remember fiddling with it a while myself the first time i needed to do something | 19:51 |
| sdague | clarkb: if I add it here - https://github.com/openstack-infra/config/blob/master/modules/openstack_project/templates/logstash/indexer.conf.erb#L99 is that the right place? | 19:51 |
| *** tomhe has joined #openstack-infra | 19:51 | |
| flashgordon | fungi: what needs to be done to add stable jobs to client trunk? | 19:51 |
| sdague | jesusaurus: we need to index this file - http://logs.openstack.org/20/62820/4/gate/gate-tempest-dsvm-full/6a9f94a/logs/horizon_error.txt.gz | 19:51 |
| fungi | dstufft: you might /join #openstack-swift and inquire if the good stuff is hidden behind the counter or something | 19:52 |
| *** nijaba has joined #openstack-infra | 19:52 | |
| devananda | fungi: 80652 and 80653 await your eyes, when ever you get some time. I've already tossed up the changes to the ironic devstack patch, and will re-run the experimental job once these two land. | 19:52 |
| dstufft | fungi: yea good idea | 19:52 |
| dstufft | thanks :) | 19:52 |
| fungi | flashgordon: probably something using branch overrides like i did for the backward-compat devstack-gate jobs | 19:52 |
| devananda | i'm going to step out for ~ an hour or so, lunch and so on | 19:52 |
| *** hashar has quit IRC | 19:53 | |
| fungi | flashgordon: now that the d-g job templates have grown branch-aware flexibility it shouldn't probably be too tough, unless there's something i'm overlooking | 19:53 |
| jesusaurus | sdague: ahh, okay, so you need to add a match to that date filter, but you also need to add an appropriate grok filter for the log lines | 19:53 |
| flashgordon | fungi: well lets sort out the dependency issue first, then I can do the client tests | 19:54 |
| tomhe | Guys, did you hear about the nasty Gerrit bug? Will Zuul ever have two or more pending submits to the same branch? | 19:55 |
| *** talluri has joined #openstack-infra | 19:56 | |
| fungi | flashgordon: i think someone just needs to backport the requirement to openstack/requirements:stable/havana and it should unblock | 19:57 |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Replaced first_name and last_name with full_name. https://review.openstack.org/79720 | 19:57 |
| fungi | flashgordon: ought to be able to test with a requirements patch, unless the requirements integration job on stable/havana has bitrotted too | 19:58 |
| fungi | tomhe: it's entirely possible for zuul to encounter that bug if you're using gerrit 2.6 or later, i expect | 19:58 |
| fungi | tomhe: we haven't seen it, but then again we run gerrit 2.4.4 in production still | 19:59 |
| tomhe | fungi: ok, thanks | 19:59 |
| tomhe | fungi: We're at 2.8.3 now | 19:59 |
| *** sweston has joined #openstack-infra | 20:00 | |
| fungi | tomhe: our dev environment where zaro's been testing our upgrade migration is getting upgraded to 2.8.3 soon too | 20:00 |
| *** talluri has quit IRC | 20:00 | |
| fungi | mainly because of that bug | 20:01 |
| flashgordon | fungi: how does just adding oauthlib to stable reqs fix it? | 20:01 |
| flashgordon | doesn't something need to install it? | 20:01 |
| fungi | flashgordon: won't the client try to install it? | 20:01 |
| openstackgerrit | Donald Stufft proposed a change to openstack-infra/config: Check and gate against Python 2.6 for Barbican https://review.openstack.org/80675 | 20:01 |
| fungi | or maybe i misread the exception | 20:01 |
| tomhe | fungi: Really scary thing. Need to do some consistency checking on our repos. Anyway, thanks fir your input. | 20:02 |
| dstufft | Changes openstack-infra/config get reviewed here? | 20:02 |
| fungi | tomhe: good luck | 20:02 |
| dstufft | changed to* | 20:02 |
| dstufft | changes | 20:03 |
| fungi | dstufft: yep | 20:03 |
| dstufft | I swear I know how to word | 20:03 |
| sdague | so when do the new projects on friday get created? | 20:03 |
| dstufft | https://review.openstack.org/#/c/80675/ please 2 review? <3 | 20:03 |
| fungi | sdague: on friday ;P | 20:03 |
| sdague | fungi: well friday is well on the way to being over here in NY :) | 20:03 |
| fungi | sdague: probably starting in a bit once west-coasters are done lunching | 20:03 |
| clarkb | tomhe: yes it is very possible. so possible that I expect if we upgraded to 2.6 we would have caught this bug on day 1 | 20:03 |
| flashgordon | fungi: keystone client will try | 20:03 |
| *** lcheng has quit IRC | 20:04 | |
| flashgordon | that doesn't cause things to fail if a req cannot be installed? | 20:04 |
| dstufft | fungi: thanks sir! | 20:04 |
| fungi | flashgordon: yeah, but i believe devstack is installing the global-requirements list and ignoring the requirements files in the projects, so we can be sure they work with the global requirements | 20:05 |
| fungi | dstufft: yw. trivial patch so i didn't bother waiting for a second +2 on that one | 20:05 |
| flashgordon | we install all of global reqs? | 20:05 |
| nibalizer | pleia2: using fqdn in the hierarchy does not on its own break anything, i'm not at this point suggesting we get away from site.pp | 20:06 |
| *** sweston has quit IRC | 20:06 | |
| nibalizer | I'm mostly excited about the osfamily and operatingsystem hierarchies | 20:06 |
| pleia2 | nibalizer: ok cool | 20:06 |
| jeblair | ,/ | 20:06 |
| flashgordon | or do we remove a req during the sync global res if its not there | 20:07 |
| nibalizer | basically using hiera for node level data is not something i've ever really liked, i see it as next to site.pp not replacing it | 20:07 |
| jeblair | that's not what i meant to type, but it still conveys the same meaning... | 20:07 |
| nibalizer | but some crazy people do use hiera as an ENC | 20:07 |
| *** sweston has joined #openstack-infra | 20:07 | |
| *** yamahata has quit IRC | 20:07 | |
| jeblair | clarkb, fungi: i'm back from lunch, ready for npf whenever | 20:07 |
| *** yassine has quit IRC | 20:08 | |
| clarkb | I am being yelled at to do lunch myself | 20:08 |
| *** tomhe has quit IRC | 20:08 | |
| clarkb | but don't let me stop you | 20:08 |
| jeblair | oh, i have a centos node in az2! and i _think_ the region-b one is past the tricky part | 20:09 |
| *** rpodolyaka has quit IRC | 20:10 | |
| fungi | i'm still hammering at a couple of the rax precise images to get them to build now that we can again | 20:10 |
| sdague | the infuriating thing, this is basically rfc822 except missing the tz | 20:12 |
| clarkb | sdague: oh then it should work | 20:12 |
| openstackgerrit | Chad Lung proposed a change to openstack-infra/config: Removing non-voting for gate-barbican-devstack-dsvm https://review.openstack.org/80686 | 20:12 |
| sdague | even if the tz is not there? | 20:12 |
| clarkb | sdague: the existing matchter makes tz optional iirc | 20:12 |
| clarkb | and assumes utc if not timezone | 20:12 |
| openstackgerrit | A change was merged to openstack-infra/config: Check and gate against Python 2.6 for Barbican https://review.openstack.org/80675 | 20:12 |
| sdague | oh, so it does | 20:12 |
| sdague | was reading patterns/grok-patterns | 20:13 |
| clarkb | ok really lunch time now. I will probably be afk for most of teh afternoon while I eat and hang out with a friend before they move to san diego | 20:13 |
| fungi | flashgordon: i'm not certain--have to reread the requirements part of the devstack-gate functions list | 20:13 |
| flashgordon | lets see what https://review.openstack.org/80687 does | 20:13 |
| *** rcarrillocruz1 has joined #openstack-infra | 20:13 | |
| fungi | flashgordon: yep, i figure it's worth starting there | 20:14 |
| *** rcarrillocruz has quit IRC | 20:14 | |
| fungi | flashgordon: though we may have trouble getting that through if the reqs-integration job has rotted away for other reasons (it doesn't get run very often), but at least the devstack job results should be enlightening | 20:15 |
| *** rpodolyaka has joined #openstack-infra | 20:15 | |
| *** lcheng has joined #openstack-infra | 20:16 | |
| *** rpodolyaka has quit IRC | 20:16 | |
| *** khyati has joined #openstack-infra | 20:16 | |
| flashgordon | sigh | 20:16 |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Consume full_name rather than first_name and last_name https://review.openstack.org/79723 | 20:17 |
| *** rpodolyaka has joined #openstack-infra | 20:18 | |
| *** alff has joined #openstack-infra | 20:18 | |
| flashgordon | fungi: is it just me or does this delete dependencies not in global reqs: http://git.openstack.org/cgit/openstack/requirements/tree/update.py#n130 | 20:19 |
| *** dcramer_ has joined #openstack-infra | 20:19 | |
| *** talluri has joined #openstack-infra | 20:21 | |
| openstackgerrit | Joe Gordon proposed a change to openstack/requirements: If dependency not in global reqs leave it so install will fail https://review.openstack.org/80690 | 20:21 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: add horizon_error to the indexed logs https://review.openstack.org/80691 | 20:22 |
| flashgordon | fungi sdague: does that sound sane ^ | 20:22 |
| jeblair | flashgordon: maybe i'm late to the conversation, but how does that cause a failure? | 20:23 |
| fungi | flashgordon: why would install fail there? | 20:23 |
| jeblair | fungi: i think it's you and me on new projects; when do you want to get started? | 20:23 |
| *** esker has joined #openstack-infra | 20:23 | |
| fungi | jeblair: i was just finishing reviewing them | 20:24 |
| flashgordon | fungi jeblair: so I am hoping that would mean the file isn't in your pypi mirror and would fail | 20:24 |
| flashgordon | but not sure if it works that way | 20:24 |
| jeblair | flashgordon: ah, we never delete anything from pypi.o.o, so that's not guaranteed | 20:24 |
| jeblair | flashgordon: if something was in g-r then removed, it would still be in the mirror | 20:24 |
| fungi | rustlebee: on 79363 you say creating nova-drivers in gerrit is a prerequisite... any reason? manage-projects is going to create that group for us, them we add you and let you put whoever else you want in there | 20:24 |
| flashgordon | jeblair: in that case how make the script exit out with a 1 and make sure devstack detects that and stops? | 20:25 |
| *** marun has joined #openstack-infra | 20:25 | |
| sdague | fungi: that should be sufficient for what russellb needs | 20:25 |
| jeblair | flashgordon: (the thinking has shifted and pypi.o.o is no longer seen as a strong enforcement mechanism, only a weak one; requirements check jobs, etc are the real enforcement) | 20:25 |
| rustlebee | fungi: ok, then i lied | 20:25 |
| fungi | flashgordon: in the case of branch-specific problems like this, you're talking about expecting the mirror to not hand havana a package which master uses... that's not something we are able to defend against | 20:25 |
| sdague | I think he just wanted to make sure it was a different group than -core, because it should be the smaller drivers group | 20:25 |
| rustlebee | fungi: sorry, misunderstood how it worked | 20:25 |
| rustlebee | right | 20:25 |
| rustlebee | though i'm wondering now if i should make it nova-specs-core | 20:25 |
| *** talluri has quit IRC | 20:25 | |
| flashgordon | fungi: right | 20:26 |
| zaro | jeblair: had a question for you about python jenkins (in scroolback). | 20:26 |
| flashgordon | that makes sense | 20:26 |
| fungi | rustlebee: no worries, just wanted to make sure before we potentially did stuff out of order for reasons of which i'm unaware | 20:26 |
| rustlebee | right | 20:26 |
| rustlebee | i'll remove that from the commit message | 20:26 |
| rustlebee | and change the patch to use nova-specs-core instead, which i think is better | 20:26 |
| fungi | rustlebee: no need, in my opinion | 20:26 |
| fungi | rustlebee: that's fine if you want | 20:26 |
| flashgordon | fungi: so this happened because of the hole with changing deps on clients | 20:26 |
| *** lcheng has quit IRC | 20:26 | |
| fungi | flashgordon: correct | 20:26 |
| rustlebee | fungi: meh, ok, i'll leave it then | 20:26 |
| flashgordon | would it be better just to fix that instead of making the reqs sync fail hard? | 20:27 |
| flashgordon | or do both | 20:27 |
| rustlebee | fungi: is it a big deal to change group names? | 20:27 |
| fungi | flashgordon: about the only sane way we can deal with that currently is to add backward-compat jobs to clients | 20:27 |
| jeblair | zaro: hrm, i've never made a release; i've done very little with it. but we can look into it | 20:27 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Change operations-guide and openstack-manuals jobs to not use maven https://review.openstack.org/77446 | 20:27 |
| jeblair | rustlebee: if you want the alternate group name, let's go ahead and do it | 20:27 |
| rustlebee | ok | 20:27 |
| fungi | rustlebee: it's a patch and a manual action (a click or two and some quick typing) in gerrit. if you want a different name then now is the time to update that | 20:27 |
| rustlebee | just in case nova-drivers in launchpad gets differentiated at some point | 20:27 |
| rustlebee | ok, let me just fix it | 20:27 |
| jeblair | (i'm +1 on that too, because we like to have the names != lp group names to reinforce there is no auto-syncing) | 20:27 |
| rustlebee | cool | 20:28 |
| flashgordon | fungi: agreed that backwards compat jobs to clients is needed -- but should I abandon this current line of thinking for the update.py script to fail as well | 20:28 |
| fungi | yeah, people still get confused since they log in through lp and there are sometimes similarly-named groups | 20:28 |
| *** rwsu has joined #openstack-infra | 20:28 | |
| flashgordon | fungi: in other words should I do 'A and B' or just 'A' | 20:28 |
| sdague | flashgordon: so there is already a check-tempest-dsvm-full-havana that we run on devstack-gate | 20:28 |
| fungi | flashgordon: yeah, we might be able to convince update.py to fail in other ways to help belt and braces this, but that would require some more code there i think | 20:29 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Remove maven from operations-guide and openstack-manuals https://review.openstack.org/77446 | 20:29 |
| sdague | you can probably just slap that on the clients | 20:29 |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/config: Create nova-specs git repo https://review.openstack.org/79363 | 20:29 |
| flashgordon | sdague: thats what I was thinking | 20:29 |
| jesusaurus | sdague: you will need to escape your brackets in your grok filter | 20:29 |
| sdague | jesusaurus: \[ ? | 20:29 |
| jesusaurus | sdague: yep and \] is a good idea too (though closing a spurious ] would be a bug in itself) | 20:30 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: add horizon_error to the indexed logs https://review.openstack.org/80691 | 20:30 |
| *** ArxCruz has quit IRC | 20:30 | |
| sdague | jesusaurus: ok, how about that | 20:30 |
| sdague | and thanks | 20:30 |
| fungi | rustlebee: did you mean to change the group on 79363 in your last patchset? | 20:31 |
| fungi | rustlebee: because if so, it looks like you didn't do so | 20:31 |
| zaro | jeblair: yes, please. I have a pending patch that adds a getPluginInfo method. would be very useful in jjb. thanks. | 20:31 |
| sdague | flashgordon: so for this, please make a client-template | 20:31 |
| fungi | rustlebee: nevermind me | 20:31 |
| jesusaurus | sdague: perfect | 20:31 |
| fungi | rustlebee: i was looking at the old patchset :/ (friday!) | 20:31 |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/config: Note requirement to use 'new-project' https://review.openstack.org/80693 | 20:31 |
| sdague | so when we have to add or remove a check-tempest-dsvm-full- it's one line, and not 15 | 20:32 |
| flashgordon | I see check-tempest-dsvm-full-havana is non-voting | 20:32 |
| jeblair | clarkb: OH! now i know what your comment on my patch meant -- grepping for 10. isn't enough because of the 206.164. thing... | 20:32 |
| *** ArxCruz has joined #openstack-infra | 20:33 | |
| sdague | flashgordon: yeh, in the current scheme that's true | 20:34 |
| *** bookwar has left #openstack-infra | 20:34 | |
| sdague | and realize there is going to be some very interesting implications of gating clients on stable servers | 20:34 |
| sdague | because stable servers bit rot | 20:34 |
| sdague | so clients are going to wedge a lot | 20:34 |
| flashgordon | hmm | 20:34 |
| flashgordon | yeah true | 20:34 |
| sdague | the net effect might be that people rush to make stable work more often | 20:35 |
| fungi | sdague: flashgordon: in previous conversations where this came up, mordred was always keen on pointing out that it would encourage the client devs to pitch in on stable bitrot problems ;) | 20:35 |
| sdague | but it is going to cause some pitch forks | 20:35 |
| fungi | right | 20:35 |
| openstackgerrit | Joe Gordon proposed a change to openstack/requirements: If dependency not in global reqs exit with a 1 https://review.openstack.org/80690 | 20:35 |
| sdague | which I'm totally fine with, if they are coming after you, flashgordon ;) | 20:35 |
| flashgordon | sdague fungi: well lets start with this | 20:36 |
| fungi | but this is where the tc possibly making sure our support policy strongly underscores client support for supported stable servers as important | 20:36 |
| sdague | flashgordon: you still don't have test | 20:36 |
| fungi | could help | 20:36 |
| sdague | I went to great lengths to test the heck out of update.py | 20:36 |
| sdague | because if it ever is wrong | 20:36 |
| sdague | ever | 20:36 |
| sdague | everything stops dead | 20:36 |
| jeblair | solidarity! | 20:36 |
| flashgordon | sdague: I'll add some tests. but is the general idea sound? | 20:36 |
| sdague | flashgordon: I came in late in the conversation, explain to me what you are trying to address | 20:38 |
| *** lcheng has joined #openstack-infra | 20:38 | |
| flashgordon | sdague: yes, so stable/havana broke | 20:38 |
| *** alff has quit IRC | 20:38 | |
| *** Ajaeger has quit IRC | 20:39 | |
| flashgordon | http://logs.openstack.org/35/80435/1/check/check-tempest-dsvm-full/269edcf/logs/screen-h-eng.txt.gz | 20:39 |
| sdague | right, but explain exactly how | 20:39 |
| flashgordon | heat doesn't oauthlib.oauth1 | 20:39 |
| flashgordon | and neuther does horizon | 20:39 |
| fungi | morganfainberg: quick question on the kds new project request (73074)... you've got it in the openstack namespace (being split out from keystone) but under management of existing groups used currently for management of a stackforge project. is this project going to get listed in https://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml#n48 as belonging under the control of the | 20:40 |
| fungi | identity program officially? | 20:40 |
| flashgordon | which has the same error | 20:40 |
| flashgordon | sdague: it looks like oauthlib is a keystoneclient dependency, but it wasn't in the global-reqs for stable/havana | 20:40 |
| flashgordon | so it never got installed | 20:40 |
| morganfainberg | fungi, it should be stackforge i think. | 20:40 |
| flashgordon | so the fix should be: https://review.openstack.org/#/c/80687/ | 20:40 |
| flashgordon | then the follow up is how did this happen | 20:41 |
| flashgordon | and make sure it doesn't any more | 20:41 |
| morganfainberg | fungi, did we (jamielennox and I) forget to fix the namespace? | 20:41 |
| flashgordon | so I figure two parts: | 20:41 |
| *** rhsu has joined #openstack-infra | 20:41 | |
| morganfainberg | fungi, *checks* | 20:41 |
| fungi | morganfainberg: oic, you have stackforge in some places, but openstack on the acl paths | 20:41 |
| flashgordon | if update.py finds a dependency not in global reqs -- which shouldn't happen if we do it right -- then fail hard | 20:41 |
| flashgordon | and secondly gate clients on stable since we use them there | 20:41 |
| morganfainberg | fungi, aha, so we have some mismatch | 20:41 |
| mordred | fungi: aroo? | 20:41 |
| openstackgerrit | Victor Seva proposed a change to openstack-infra/jenkins-job-builder: use {obj:key} as a way to pass an object and not only strings to templates https://review.openstack.org/73990 | 20:41 |
| fungi | mordred: aroo | 20:42 |
| mordred | sdague: I actually think that the stable servers will stop bitrotting, not because of more people - but because we'll more frequently run jobs against them that people will care about | 20:42 |
| sdague | mordred: sure | 20:42 |
| fungi | morganfainberg: i see other typos too. updating the review in a moment | 20:42 |
| morganfainberg | fungi, yeah, barbican is meant to take over the project | 20:42 |
| morganfainberg | fungi, if it needs to have it's own acls, thats fine | 20:42 |
| morganfainberg | fungi, oh and i see the issue | 20:43 |
| fungi | morganfainberg: it does have its own acls, they just have the wrong file paths | 20:43 |
| morganfainberg | fungi, ok. thanks | 20:43 |
| morganfainberg | fungi, ++ eyah was missed when we updated from openstack to stackforbge | 20:43 |
| fungi | morganfainberg: but i've got other comments going in real quick too if you're about to follow up with another patchset | 20:43 |
| sdague | flashgordon: ok, sure, that's probably the right fix | 20:43 |
| sdague | that will explode early which is good | 20:44 |
| flashgordon | sdague: fail early and fail hard | 20:44 |
| flashgordon | and often ;) | 20:44 |
| sdague | we should bring those stable gate jobs back | 20:44 |
| sdague | or create them based on the d-g template | 20:44 |
| *** zehicle has joined #openstack-infra | 20:44 | |
| *** zehicle_at_dell has quit IRC | 20:44 | |
| sdague | though the whole thing kind of remains all kinds of fubar because python packaging is so excellent | 20:45 |
| sdague | because it's going to trigger a lot of desire to change stable global-requirements | 20:45 |
| sdague | mordred: have you thought through where this crazy train takes us? | 20:46 |
| fungi | morganfainberg: okay, updated | 20:46 |
| sdague | because I'm a little concerned it's not a great place | 20:46 |
| morganfainberg | fungi, thnkx much :) | 20:46 |
| sdague | flashgordon: actually.... | 20:47 |
| sdague | I'm far more inclined that this is the wrong fix | 20:47 |
| sdague | and your first change was better | 20:47 |
| *** sarob has quit IRC | 20:47 | |
| flashgordon | sdague: ? | 20:47 |
| *** julim has quit IRC | 20:47 | |
| mordred | sdague: what do you mean? | 20:47 |
| sdague | because global-requirements was brought in largely to handle the crazy wedges of incompatible packages between components | 20:48 |
| flashgordon | the make update error out | 20:48 |
| sdague | however, if we're talking about a few adds by clients | 20:48 |
| fungi | jeblair: okay, i think we can probably dispense with the etherpad since we've only got four patches (counting morganfainberg's on the assumption he's got a patch on the way correcting it) | 20:48 |
| *** sarob has joined #openstack-infra | 20:48 | |
| *** rpodolyaka has quit IRC | 20:48 | |
| sdague | I think it's better to just let those slide | 20:48 |
| sdague | because we are enforcing they are in g-r in master | 20:48 |
| fungi | jeblair: are you aware of any new jeepyb patches we needed to test with this round? | 20:48 |
| *** jnoller has quit IRC | 20:48 | |
| *** lcheng has quit IRC | 20:48 | |
| mordred | sdague: I say that we get WAY stricter with client deps | 20:48 |
| mordred | sdague: because our client libs are used in our stable servers | 20:48 |
| jeblair | mordred: ^ can you address fungi's jeepyb question? | 20:48 |
| mordred | jeblair: looking | 20:49 |
| clarkb | fungi I dont think there are any | 20:49 |
| sdague | mordred: so what we're saying is you can't add a client lib dependency unless you update g-r in master, n-1, and n-2 ? | 20:49 |
| sdague | or bump a minimum version | 20:49 |
| clarkb | I think jeepyb is happy now. the cross host orchestration is what hurts us now | 20:49 |
| mordred | sdague: yes | 20:49 |
| sdague | ok | 20:49 |
| sdague | I think that's got to be an ML thread at least | 20:50 |
| mordred | clarkb, jeblair: I was just talking to jesusaurus about the cross-host orchestration - or at least part of it | 20:50 |
| sdague | because that's going to be a big hammer | 20:50 |
| mordred | sdague: sure. I could be VERY wrong about that, btw | 20:50 |
| sdague | I'm not sure I disagree with the approach | 20:50 |
| mordred | but for client libs, I think we should probably e pretty able to not use the latest bleeding edge stuff | 20:50 |
| sdague | however it is a very substantial shift from current | 20:50 |
| sdague | so I don't think something that can be done by fiat, at minimum needs to be at the project meeting | 20:51 |
| jeblair | fungi: sounds like we're gtg. | 20:51 |
| jeblair | fungi: what's the procedure? | 20:51 |
| *** lcheng has joined #openstack-infra | 20:51 | |
| mordred | sdague: ++ | 20:51 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/config: Test stable branch on client changes https://review.openstack.org/80698 | 20:51 |
| flashgordon | sdague mordred: I will let you guys sort out if we want this or not | 20:52 |
| flashgordon | but that should do it | 20:52 |
| *** sarob has quit IRC | 20:52 | |
| flashgordon | as long as we at least get https://review.openstack.org/#/c/80690/ working I am happty | 20:52 |
| fungi | jeblair: at this point, i think merge 'em all and let zuul sort 'em out, then once they're on gerrit (assuming we've got manage-projects disabled there from a puppet perspective), run manage-projects manually and make sure the groups/acls were created in gerrit and try cloning them from all the git servers (and retrigger replication for them if not, but if we wait a but before running that shouldn't | 20:52 |
| fungi | happen) | 20:52 |
| fungi | i'm double-checking the puppet hypothesis now | 20:53 |
| *** vkozhukalov_ has quit IRC | 20:53 | |
| jeblair | fungi: which is supposed to happen first? create-cgit-repos or manage-projects? | 20:53 |
| *** hashar has joined #openstack-infra | 20:53 | |
| fungi | jeblair: if create-cgit-repos happens first then we're good | 20:53 |
| fungi | jeblair: if manage-projects happens first then we don't get replicated to some of the farm | 20:54 |
| jeblair | fungi: ok, so merge, wait for cgit to run, then run mp. | 20:54 |
| *** mark0n has joined #openstack-infra | 20:54 | |
| fungi | also if puppet gets stuck on one of the git servers then create-cgit-repos never runs, so we end up with a problem, but the last time that happened i found we still had github remotes for /opt/jeepyb on a few of them, so corrected that | 20:55 |
| fungi | in theory not likely any longer | 20:55 |
| fungi | also, clarkb took m-p back out of puppet last friday in aa84680 so we should be fine there | 20:56 |
| clarkb | yup and your plan sounds good | 20:56 |
| *** lcheng has quit IRC | 20:57 | |
| *** esker has quit IRC | 20:57 | |
| *** lcheng has joined #openstack-infra | 20:57 | |
| sdague | flashgordon: so I think we're back to that approach, it needs tests, also an ML thread | 20:57 |
| sdague | flashgordon: as you found the issue, you want to start that thread? | 20:57 |
| fungi | 79900 and 79900 need a second +2 ideally, and i think morganfainberg has a patchset on the way for 73074 | 20:58 |
| *** mbacchi has quit IRC | 20:59 | |
| flashgordon | sdague: so many threads | 20:59 |
| flashgordon | I'll start one big thread for both | 20:59 |
| flashgordon | explaining what happend and the fix | 20:59 |
| sdague | hey, at least this won't be a vim comments thread | 20:59 |
| fungi | sdague: we can make any thread into a vim comments thread if you really want | 21:00 |
| flashgordon | sdague: that one resulted in hours of entertainment for many folks | 21:00 |
| *** jp_at_hp has joined #openstack-infra | 21:00 | |
| *** caleb_ has joined #openstack-infra | 21:01 | |
| jeblair | fungi: 3 changes in the queue. | 21:01 |
| fungi | jeblair: saw, thanks | 21:01 |
| *** salv-orlando has joined #openstack-infra | 21:02 | |
| *** salv-orlando has quit IRC | 21:02 | |
| openstackgerrit | A change was merged to openstack-infra/config: Create nova-specs git repo https://review.openstack.org/79363 | 21:02 |
| jeblair | fungi: https://review.openstack.org/#/c/79900/ conflicts | 21:02 |
| fungi | i'll rebase it quick | 21:02 |
| *** salv-orlando has joined #openstack-infra | 21:02 | |
| *** thomasem has quit IRC | 21:04 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Create a nova-docker repo https://review.openstack.org/79900 | 21:05 |
| fungi | rebased ^ | 21:05 |
| fungi | it conflicted on layout.yaml because the nova-specs addition was within its context margin | 21:05 |
| *** mfer has quit IRC | 21:06 | |
| *** pdmars has quit IRC | 21:07 | |
| *** caleb_ has quit IRC | 21:07 | |
| fungi | looks like we probably have about a 15 minute wait on check nodes for those two changes | 21:07 |
| *** ildikov_ has joined #openstack-infra | 21:08 | |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Ruby metrics plugin support added https://review.openstack.org/73528 | 21:08 |
| fungi | though the graph suggests it may be catching up faster than i expect | 21:08 |
| jeblair | zaro: pushed jjb 0.7.0 | 21:08 |
| *** david-lyle has quit IRC | 21:09 | |
| *** sdake_ has quit IRC | 21:10 | |
| jeblair | fungi: once we get the sudo stuff through (which is like 1 or 2 more changes), we're probably pretty close to combining bare-precise and devstack-precise into one node type | 21:10 |
| *** sdake_ has joined #openstack-infra | 21:10 | |
| *** sdake_ has quit IRC | 21:10 | |
| *** sdake_ has joined #openstack-infra | 21:10 | |
| jeblair | fungi: which should make nodepool a little more responsive | 21:10 |
| fungi | jeblair: that will be swell | 21:10 |
| lifeless | mordred: so - https://review.openstack.org/#/c/80457/ ? | 21:10 |
| fungi | jeblair: maybe we can collapse py3k-precise into them too | 21:10 |
| jeblair | fungi: oh could we? that can coexist? | 21:11 |
| fungi | though that may require a little more bootstrapping in the jobs, so maybe not | 21:11 |
| fungi | jeblair: yeah, forget i suggested it ;) nontrivial for now still | 21:12 |
| jeblair | fungi: is it a matter of deb installation? if we cache it, that shouldn't take long... | 21:12 |
| jeblair | fungi: if there's a mess of puppet stuff, that may be a bit further off. | 21:12 |
| *** aysyd has quit IRC | 21:12 | |
| sdague | has there been any thinking about doing the unit tests inside a container in a node, so the node doesn't have to be single use? or is that just way too far down the inception hole. | 21:12 |
| openstackgerrit | Martin Konrad proposed a change to openstack-infra/jenkins-job-builder: SCM module: Add support for multiple Git remotes. https://review.openstack.org/80084 | 21:12 |
| fungi | jeblair: i think we might be able... but for right now it depends on a different puppet class | 21:12 |
| jeblair | sdague: we used to not have single use nodes, if you recall :) | 21:13 |
| sdague | jeblair: yes | 21:13 |
| fungi | jeblair: also, i think with the advent of trusty it will become (maybe mostly) a no-op change | 21:13 |
| sdague | but they had the disadvantage of getting cruddy | 21:13 |
| fungi | sdague: they also had the disadvantage of jenkins | 21:13 |
| jeblair | sdague: one of the reasons we moved to using them so quickly is to avoid a jenkins problem; when we get rid of jenkins that will no longer be necessary... | 21:14 |
| sdague | ok | 21:14 |
| sdague | just one of those random friday thoughts | 21:14 |
| jeblair | sdague: but another reason was to increase flexibility -- | 21:14 |
| sdague | sure | 21:14 |
| jeblair | sdague: so stackforge projects can install their mongodb on rails or whatever. | 21:14 |
| sdague | right, though containers would permit that as well | 21:15 |
| fungi | sdague: however, at least some of our jobs currently don't like containers (i believe things testing iscsi for example) | 21:15 |
| fungi | so we'd potentially need a mix | 21:16 |
| sdague | I guess we just need to actually get containers in openstack integrated at the right level, re: not as virt drivers | 21:16 |
| *** mriedem has quit IRC | 21:16 | |
| sdague | fungi: does that expose in unit tests? | 21:16 |
| sdague | and if so.... yuck | 21:16 |
| fungi | sdague: which is why i said we'd still need a mix. i don't think the unittests are an issue there though | 21:16 |
| *** caleb_ has joined #openstack-infra | 21:17 | |
| sdague | yes, definitely | 21:17 |
| fungi | the idea is not without merit, but i think we have other things which need to happen first before we can consider it more strongly | 21:17 |
| sdague | still would need a mix | 21:17 |
| clarkb | containers arent secure | 21:17 |
| clarkb | you cant allow root in them and reuse | 21:17 |
| clarkb | at least not with precise/centos kernels | 21:17 |
| *** thedodd has quit IRC | 21:17 | |
| clarkb | trusty may be better | 21:18 |
| sdague | clarkb: right, this would need at least the trusty kernel, iirc a bunch got fixed by then | 21:18 |
| *** esker has joined #openstack-infra | 21:18 | |
| fungi | clarkb: true. if we're concerned about jobs leaking root privs to other containers or the host vm, then that's a good reason to avoid them until trusty as well | 21:18 |
| fungi | and centos7 | 21:18 |
| sdague | though you could also take an educated risk and kill them after some number of uses anyway | 21:18 |
| *** rcleere has quit IRC | 21:18 | |
| fungi | which i guess is due out soon? (well, rhel7 anyway) | 21:18 |
| jeblair | there's actually a stub idea in nodepool for a "reset" script that would clean up a node after use instead of destroy | 21:19 |
| jeblair | so the thought has definitely occured | 21:19 |
| jeblair | but yeah, as long as we have jenkins, it's going to be problematic | 21:19 |
| sdague | sure | 21:19 |
| jeblair | so post-jenkins+post-trusty maybe so | 21:19 |
| sdague | cool | 21:19 |
| fungi | and yeah, something like a max reuse metric for them (akin to apache recycling worker threads) | 21:19 |
| jeblair | post-precise rather | 21:19 |
| flashgordon | WAA http://logs.openstack.org/87/80687/1/check/check-requirements-integration-dsvm/565dc91/console.html | 21:20 |
| fungi | flashgordon: right, i thought we might have bitrotted there | 21:20 |
| fungi | since that job only gets run on the blue moon when someone updates havana global reqs | 21:21 |
| *** mkoderer has quit IRC | 21:21 | |
| *** openstackgerrit has quit IRC | 21:21 | |
| sdague | flashgordon: https://review.openstack.org/#/c/80027/ this should run clean now? | 21:21 |
| sdague | I just hit recheck on it | 21:21 |
| *** openstackgerrit has joined #openstack-infra | 21:21 | |
| sdague | I would like to see the partial job actually pass | 21:21 |
| *** talluri has joined #openstack-infra | 21:22 | |
| flashgordon | sdague: partial-job needs https://review.openstack.org/#/c/80435/ | 21:22 |
| flashgordon | which needshttps://review.openstack.org/#/c/80687/ | 21:22 |
| flashgordon | https://review.openstack.org/#/c/80687/ | 21:23 |
| flashgordon | so yeah | 21:23 |
| openstackgerrit | A change was merged to openstack-infra/config: Create qa-specs git repo https://review.openstack.org/80371 | 21:23 |
| sdague | flashgordon: oh, so that's all borked in the head | 21:23 |
| sdague | woot! | 21:23 |
| flashgordon | sdague: exactly | 21:23 |
| sdague | well I'm leaving for beers in 30 minutes | 21:24 |
| fungi | looks like when installing trove the integration job hits "No local packages or download links found for pbr>=0.5.16,<0.6" | 21:24 |
| fungi | is trove expected to be operable on havana? | 21:24 |
| fungi | or should it be ripped out of the projects.txt in the openstack/requirements:stable/havana branch? | 21:25 |
| fungi | flashgordon: ^ | 21:25 |
| flashgordon | fungi: not sure, sdague ^ | 21:25 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/jenkins-job-builder: Migrate to pbr https://review.openstack.org/76200 | 21:26 |
| mtreinish | sdague: cool, we've got a qa specs repo now | 21:26 |
| sdague | yep | 21:26 |
| *** talluri has quit IRC | 21:26 | |
| fungi | mtreinish: well, soon anyway. the change merged but we haven't run the creation step manually yet | 21:27 |
| openstackgerrit | A change was merged to openstack-infra/config: Create a nova-docker repo https://review.openstack.org/79900 | 21:27 |
| sdague | fungi: trove should be ripped out of stable/havana | 21:27 |
| sdague | even in master it's not doing anything in the integrated jobs | 21:27 |
| fungi | mtreinish: then i think you probably have to do that in the same patch | 21:27 |
| flashgordon | so now my patch needs that patch too heh | 21:27 |
| fungi | er, flashgordon not mtreinish | 21:28 |
| sdague | so requirements sync in stable/havana for trove is pretty pointless | 21:28 |
| fungi | all you electrons look alike to me | 21:28 |
| SlickNik | fungi: We're not supporting trove on stable/havana, so I think ripping it out of stable/havana is the right call. | 21:28 |
| jeblair | morganfainberg: are you going to update the kite patch? | 21:28 |
| sdague | SlickNik: to be honest, it's not actually ripping it out of stable/havana | 21:28 |
| openstackgerrit | Joe Gordon proposed a change to openstack/requirements: If dependency not in global reqs exit with a 1 https://review.openstack.org/80690 | 21:28 |
| sdague | it's just not going to be in the sync job | 21:28 |
| fungi | SlickNik: okay. it wound up in the projects.txt for requirements stable/havana branch, so that makes some sense that it's been overlooked there | 21:28 |
| morganfainberg | jeblair, in a moment i will | 21:29 |
| morganfainberg | jeblair, sorry i'm fighting with some non-upsteam stuff | 21:29 |
| morganfainberg | jeblair, was distracted | 21:29 |
| sdague | SlickNik: so ... when you all going to get some tempest tests? :) I noticed a very distinct lack of anything when looking at jobs the other day | 21:29 |
| fungi | morganfainberg: if there's no rush on it, we can just catch it next friday | 21:29 |
| morganfainberg | jeblair, i'll get that updated in the next ~5min | 21:29 |
| SlickNik | sdague: I just merged some basic tempest tests in a week ago. | 21:30 |
| sdague | they running in the gate? | 21:30 |
| *** dstanek is now known as dstanek_afk | 21:30 | |
| SlickNik | sdague: Still need to make the experimental (non-voting) job in the gate. | 21:30 |
| SlickNik | sdage: nope; hope to do that shortly. | 21:31 |
| SlickNik | sdague* | 21:31 |
| sdague | SlickNik: why aren't they just added to this job - https://jenkins04.openstack.org/job/check-tempest-dsvm-trove-full/127/ | 21:31 |
| flashgordon | sdague: can you push up the fix for trove in stable/havana | 21:31 |
| sdague | which also should probably be voting | 21:31 |
| SlickNik | sdague: That _is_ the job they're added to. | 21:31 |
| sdague | ok | 21:31 |
| * flashgordon drafts an email | 21:32 | |
| *** zehicle_at_dell has joined #openstack-infra | 21:32 | |
| *** openstackgerrit has quit IRC | 21:32 | |
| SlickNik | Just need to submit a patch to infra to make it voting. | 21:32 |
| SlickNik | It's been running stable for a while now. | 21:32 |
| *** openstackgerrit has joined #openstack-infra | 21:32 | |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/config: Add the kite (KDS) project and kiteclient project to gerrit/jenkins https://review.openstack.org/73074 | 21:34 |
| fungi | morganfainberg: thanks--reviewing | 21:34 |
| SlickNik | sdague: Also working on adding more tests than the initial set that we have in there. So expect more patches to tempest from us. :) | 21:34 |
| *** zehicle has quit IRC | 21:34 | |
| openstackgerrit | Salvatore Orlando proposed a change to openstack-infra/elastic-recheck: Add a fingeprint for bug 1283522 https://review.openstack.org/80286 | 21:34 |
| *** dims has quit IRC | 21:34 | |
| morganfainberg | fungi, i think i got those right | 21:34 |
| sdague | SlickNik: ok, cool :) | 21:34 |
| morganfainberg | fungi, and i am working on registering the project | 21:34 |
| morganfainberg | in pypi | 21:35 |
| *** lcheng has quit IRC | 21:35 | |
| fungi | morganfainberg: great--that was more of a heads up that when it comes time to release, your upload job will fail without it existing on pypi. also, someone might decide to register it who isn't you | 21:35 |
| sdague | so the reality is we should probably just put the trove tests into the main batch | 21:35 |
| morganfainberg | nod | 21:36 |
| hashar | you guys dont have enough VM to run the tests during peak hours :-D | 21:36 |
| fungi | hashar: are you offering us more vms? thanks@ | 21:36 |
| fungi | ;) | 21:36 |
| hashar | does each job consume a vm ? | 21:36 |
| fungi | currently, yes | 21:36 |
| *** vkozhukalov_ has joined #openstack-infra | 21:37 | |
| sdague | flashgordon: https://review.openstack.org/#/c/80702/ | 21:37 |
| sdague | also, damn you for having a nick with a different starting character for casual friday | 21:38 |
| clarkb | ha | 21:39 |
| hashar | fungi time for some fundraising ? :-] | 21:39 |
| fungi | hashar: we need fancy banners like wikipedia uses! | 21:39 |
| *** ryanpetrello has quit IRC | 21:39 | |
| hashar | and Jenkins Job Builder is migrating to pbr!!!!!! https://review.openstack.org/#/c/76200/ :-D (thank you all) | 21:40 |
| hashar | fungi: yeah that is very effective. Comes from a few years of trial and errors. Wikimedia even leash its fundraising rounds to avoid raising too much money. | 21:41 |
| hashar | (as I understood it) | 21:41 |
| jeblair | we'll be able to use some more nodes when we move to hpcloud 1.1... | 21:41 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Migrate to pbr https://review.openstack.org/76200 | 21:42 |
| jeblair | and once we do that, we'll be in a better position to ask for more nodes | 21:42 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/devstack-gate: trove is an integrated project add it by default https://review.openstack.org/80706 | 21:42 |
| sdague | SlickNik: so that should add it to the main job | 21:42 |
| sdague | also, has the advantage of being self testing | 21:42 |
| *** zhiyan is now known as zhiyan_ | 21:42 | |
| morganfainberg | fungi, also added openstackci as index maintainer role | 21:43 |
| *** jeckersb is now known as jeckersb_gone | 21:43 | |
| fungi | morganfainberg: checking it out | 21:44 |
| SlickNik | sdague: Thanks! | 21:44 |
| jeblair | fungi: specs repos are on all git servers, so i think things are working on all of them. just waiting for kite to show up there. | 21:45 |
| fungi | morganfainberg: pypi registration lgtm | 21:45 |
| fungi | jeblair: thanks! | 21:45 |
| *** lcheng has joined #openstack-infra | 21:46 | |
| *** rpodolyaka has joined #openstack-infra | 21:47 | |
| flashgordon | sdague mordred fungi: email draft https://etherpad.openstack.org/p/sillyclients | 21:47 |
| fungi | silly clients, reqs are for servers! | 21:48 |
| *** jp_at_hp has quit IRC | 21:48 | |
| clarkb | fungi ++ | 21:48 |
| sdague | fungi: so I should just be able to git clone git://git.openstack.org/openstack/qa-specs , then add a .gitreview, and git review -s ? | 21:49 |
| sdague | because the last part is failing on me | 21:49 |
| jeblair | sdague: not yet | 21:49 |
| sdague | ok | 21:49 |
| jeblair | sdague: hehe, friday isn't over yet! :) | 21:49 |
| sdague | it is for me :) | 21:49 |
| sdague | beer is calling | 21:49 |
| sdague | ok, I'll play tomorrow morning | 21:49 |
| fungi | utc friday has 2.2 hours remaining | 21:49 |
| flashgordon | any comments on th email before I send it out? | 21:49 |
| jeblair | sdague: probably be working in about 30 mins | 21:49 |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Fixed mobile footer padding. https://review.openstack.org/80410 | 21:50 |
| fungi | flashgordon: lgtm | 21:50 |
| sdague | night all | 21:50 |
| fungi | night sdague! | 21:50 |
| flashgordon | have a good weekend sdague | 21:50 |
| jeblair | sdague: also, there should already be a .gitreview in the repo once everything is done here | 21:50 |
| jeblair | sdague: cheers! | 21:50 |
| flashgordon | fungi: who do I need to poke to get a reviews on https://review.openstack.org/#/c/80702/ | 21:51 |
| flashgordon | sdague's trove patch | 21:51 |
| fungi | flashgordon: i'm fine, but approving depends on whether it ends up needing to get squashed with yours | 21:52 |
| flashgordon | fungi: oh right sigh | 21:52 |
| fungi | flashgordon: also, it's stable, so you need stable branch manglers | 21:53 |
| jeblair | that also has the side effect of switching off the mirror for trove and ending requirements sync proposals for trove on havana | 21:53 |
| *** dims has joined #openstack-infra | 21:53 | |
| fungi | jeblair: i believe SlickNik and hub_cap have both previously acknowledged that the stable/havana branch of trove is unmaintained | 21:54 |
| jeblair | both of which are probably ok | 21:54 |
| jeblair | fungi: speaking of which, when is grizzly eol? | 21:54 |
| openstackgerrit | A change was merged to openstack-dev/pbr: Fix up some docstrings. https://review.openstack.org/79855 | 21:54 |
| openstackgerrit | A change was merged to openstack-infra/config: Add the kite (KDS) project and kiteclient project to gerrit/jenkins https://review.openstack.org/73074 | 21:54 |
| *** dkliban has quit IRC | 21:55 | |
| *** changbl has quit IRC | 21:55 | |
| *** bada_ has joined #openstack-infra | 21:55 | |
| fungi | jeblair: last thursday, but we had a bit of a mismatch between when we agreed to eol it at the last summit (i-3 date) and when the grizzly stable point release schedule said the last point release was going to be, si i just need to get apevec/adam_g/ttx to agree before i pull the trigger | 21:55 |
| *** bada__ has quit IRC | 21:56 | |
| *** changbl has joined #openstack-infra | 21:56 | |
| *** dkliban has joined #openstack-infra | 21:56 | |
| mordred | fungi: I'm quite happy about it going away myself | 21:57 |
| jeblair | fungi: i can't wait. you know how i like deleting branches! | 21:57 |
| jeblair | _so much_ cruft gets cleaned up | 21:57 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Task list now shows associated project https://review.openstack.org/80417 | 21:57 |
| fungi | jeblair: mordred: https://wiki.openstack.org/wiki/StableBranchRelease#Planned_stable.2Fgrizzly_releases says "2013.1.5 Mar 20 2014" so worst case, a week out | 21:57 |
| *** medieval1 has quit IRC | 21:58 | |
| adam_g | fungi, apevec has frozen the grizzly branches for new merges and sent out an email preparing for a final release on the 20th | 21:58 |
| fungi | though the vmt has basically stopped considering backports needed for vulnerabilities on it because it was supposed to be eol last week | 21:58 |
| fungi | adam_g: great | 21:58 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Roll up node stats https://review.openstack.org/76064 | 21:59 |
| *** medieval1 has joined #openstack-infra | 21:59 | |
| *** rcarrillocruz has joined #openstack-infra | 22:00 | |
| openstackgerrit | Dan Smith proposed a change to openstack-infra/devstack-gate: DO NOT MERGE - Test larger largerops https://review.openstack.org/80643 | 22:00 |
| *** medieval1_ has joined #openstack-infra | 22:01 | |
| *** rcarrillocruz1 has quit IRC | 22:02 | |
| fungi | so, in the next 10 minutes the git servers should (in theory) have the kite project repos, then i'll fire up the engine and watch it spin | 22:02 |
| openstackgerrit | A change was merged to openstack-dev/pbr: Return the real class in VersionInfo __repr__. https://review.openstack.org/79856 | 22:02 |
| jeblair | fungi: can we screen that? | 22:02 |
| *** adalbas has quit IRC | 22:04 | |
| openstackgerrit | A change was merged to openstack-infra/zuul: Fix ZUUL_COMMIT in certain conditions https://review.openstack.org/77953 | 22:04 |
| *** medieval1 has quit IRC | 22:04 | |
| fungi | jeblair: definitely | 22:04 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/jenkins-job-builder: Support the Post Build Script plugin https://review.openstack.org/77628 | 22:05 |
| fungi | jeblair: do we want to just run it for the specific projects being added, or do a full (unqualified) run? also, verbose or debug output? | 22:06 |
| jeblair | fungi: specific + debug |tee log ? | 22:06 |
| fungi | actually i think we want to do a full run anyway, so that remote tracking updates will pick up zaro's desired upstreamgerrit commits | 22:06 |
| jeblair | fungi: maybe do full + verbose after the specific? | 22:06 |
| fungi | sounds good | 22:07 |
| jeblair | fungi: or do you want to exercise project creation in the midst of a full run? | 22:07 |
| jeblair | fungi: (i'm joined) | 22:07 |
| fungi | jeblair: nah, i'm fine doing the full run separate so we can debug more easily if we hit a snag on project creation for now | 22:08 |
| fungi | we've done it successfully both ways prior so i'm not too concerned about exercising that | 22:08 |
| *** zehicle has joined #openstack-infra | 22:10 | |
| *** zehicle_at_dell has quit IRC | 22:10 | |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Hide "Create Project" for non-superusers https://review.openstack.org/79939 | 22:13 |
| *** esker has quit IRC | 22:14 | |
| zaro | hashar: will you go thru pending changes and let people know they need to rebase and move their plugin entried to setup.cfg? | 22:14 |
| hashar | zaro: yeah I am looking at them slowly one by one | 22:14 |
| *** esker has joined #openstack-infra | 22:15 | |
| zaro | hashar: thx. | 22:15 |
| dstufft | people sure do get angry about gerrit requiring you to squash your commits | 22:15 |
| fungi | dstufft: who gets angry? what commits? | 22:17 |
| hashar | zaro: though I am afraid folks would miss review comments when I rebase | 22:17 |
| jeblair | dstufft: what people? | 22:17 |
| clarkb | but it doesnt | 22:17 |
| fungi | dstufft: gerrit requires you to edit and resubmit your commits to refine them before they're approved, but not quite the same as squashing them | 22:17 |
| clarkb | it requires you to have good commits | 22:17 |
| jeblair | krotscheck: where on the list is 'adding comments' to storyboard? also, is there a place i can look to answer that myself? | 22:17 |
| fungi | dstufft: that's like getting angry because the lkml wouldn't let you just contribute a thread of additions to a patch | 22:17 |
| krotscheck | jeblair: We're logging everything here: http://storyboard.openstack.org/#!/project/456/stories | 22:18 |
| dstufft | fungi: jeblair I've seen like 4 or 5 people in random different venues complain about Openstack's Gerrit requiring them to either get each individual commit reviewed or squash everything down to one commit, instead of just reviewing the final diff like Github does :) | 22:18 |
| dstufft | (Note, I'm not one of those people) | 22:18 |
| dstufft | I don't actually care either way | 22:18 |
| fungi | dstufft: they're angry that gerrit's workflow is not github's workflow. i think they also wouldn't enjoy submitting kernel patches then | 22:19 |
| jeblair | dstufft: yeah, we're fans of bisectability here, and the idea that each commit should be a good one, that is well thought out and accomplishes what it describes. | 22:19 |
| fungi | dstufft: personally i find the github fork/pull workflow horrible | 22:19 |
| fungi | but people get stockholm syndrome from such experiences | 22:20 |
| jeblair | dstufft: i find the 'keep plastering over commits until you get something that works' workflow distasteful. | 22:20 |
| dstufft | Hilarously both sides of the coin say it's for better bisectablity :D | 22:20 |
| dstufft | I like the github PR thing better, but that's probably because I edit my history to be logical changes and not just whatever I happened to have to save at the time | 22:21 |
| jeblair | dstufft: yeah, so don't squash beyond bisectability... | 22:21 |
| dstufft | Although I do think it would be hilarous to use the github style thing to get developers to run malcious code | 22:21 |
| fungi | not only hilarious but easy | 22:22 |
| jeblair | dstufft: if you have 10 logical changes to do something, push all 10 commits to gerrit | 22:22 |
| *** yjiang5 has quit IRC | 22:22 | |
| jeblair | dstufft: (i'm using the royal you here, i realize i'm not talking directly to you) | 22:22 |
| fungi | and gerrit will even notice that they are dependent and will build up a tree of reviews from them | 22:23 |
| jeblair | dstufft: gerrit actually deals fairly well with patch series like that | 22:23 |
| fungi | l | 22:23 |
| fungi | l | 22:23 |
| dstufft | jeblair: I would do that, except I feel like a jerk for getting people to review each and every change like that :] | 22:23 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/jenkins-job-builder: Add workspace scm https://review.openstack.org/79345 | 22:23 |
| fungi | ignore my sudden extra l's. i think my keyboard overfloweth | 22:23 |
| *** yjiang5 has joined #openstack-infra | 22:24 | |
| *** talluri has joined #openstack-infra | 22:24 | |
| fungi | dstufft: i would much rather review individual (good) commits rather than one giant code dump | 22:24 |
| *** mark0n has quit IRC | 22:24 | |
| fungi | i suspect that the people who mostly prefer iterative commits followed by one monolithic merge are probably people who don't spend 20+ hours a week reviewing code | 22:24 |
| *** MarkAtwood has joined #openstack-infra | 22:26 | |
| *** rpodolyaka has quit IRC | 22:26 | |
| dstufft | I have an irrational hate for the commits which are like... | 22:26 |
| dstufft | "fix typo" | 22:26 |
| dstufft | "whitespace" | 22:26 |
| dstufft | "some other crap" | 22:26 |
| marun | fungi: +1 | 22:26 |
| jeblair | fungi: doing what to code? | 22:26 |
| *** sarob has joined #openstack-infra | 22:26 | |
| fungi | dstufft: those are easy approvals | 22:26 |
| dstufft | fungi: No I mean in github when you have a string of like 20 commits with those kinds interleaved between them :) | 22:27 |
| dstufft | in a PR | 22:27 |
| jeblair | dstufft: yeah, i think that's the worst (and natural) outcome of that model | 22:27 |
| fungi | oh, right. those should not be interspersed in unrelated patch series | 22:27 |
| *** talluri has quit IRC | 22:27 | |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/jenkins-job-builder: Added support for Exclusion plugin https://review.openstack.org/77940 | 22:29 |
| marun | is anyone willing to help me debug the neutron functional job? It needs to run tox in a devstack-configured environment but the braindead way I wrote it has a permission problem | 22:29 |
| flashgordon | fungi: looks like the trove patch needs to be squashed into mine | 22:29 |
| marun | https://review.openstack.org/#/c/66967/1/modules/openstack_project/files/jenkins_job_builder/config/neutron-functional.yaml | 22:29 |
| fungi | Mar 14 22:00:40 git01 puppet-agent[1635]: (/Stage[main]/Openstack_project::Git_backend/Exec[create_cgitrepos]) Failed to call refresh: create-cgitrepos returned 1 instead of one of [0] at /opt/config/production/modules/openstack_project/manifests/git_backend.pp:99 | 22:29 |
| fungi | flashgordon: yeah, i suspected they would have to | 22:29 |
| hashar | zaro: I have rebased all pending JJB changes that had a new entry point :] | 22:30 |
| jeblair | krotscheck: i don't see adding comments on that list; i think that was something we discussed in bru; was all that copied over? | 22:30 |
| fungi | jeblair: trying it manually--i probably approved a typo | 22:30 |
| *** zehicle_at_dell has joined #openstack-infra | 22:30 | |
| *** zehicle has quit IRC | 22:30 | |
| jeblair | marun: you may need to sudo to the 'stack' user to unstack | 22:31 |
| marun | jeblair: the unstack isn't failing | 22:31 |
| jeblair | marun: and possibly run the test | 22:31 |
| marun | jeblair: the tox invocation is | 22:31 |
| krotscheck | jeblair: Comments were specifically not part of the MVP, no. While discussed, we did make the choice to drop it. | 22:31 |
| marun | jeblair: should it be necessary to run tox with sudo? | 22:32 |
| jeblair | krotscheck: when did we make that choice? | 22:32 |
| krotscheck | jeblair: In brussles? On friday. | 22:32 |
| *** ArxCruz has quit IRC | 22:32 | |
| jeblair | marun: i'm guessing it can't write to the checkout | 22:32 |
| *** reed has joined #openstack-infra | 22:32 | |
| marun | jeblair: that's correct. | 22:32 |
| jeblair | krotscheck: was i in the restroom at the time? i can't imagine using it without comments, and even the POC had comments | 22:33 |
| reed | clarkb, fungi, jeblair: can someone push this in https://review.openstack.org/#/c/80351/ please? it's a simple mlist, should not take too long | 22:33 |
| jeblair | krotscheck: i'm pretty sure i went on at length about the necessity of comments | 22:33 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Correct syntax error in review.projects.yaml https://review.openstack.org/80711 | 22:35 |
| fungi | reed: lgtm | 22:35 |
| fungi | jeblair: that ^ was the delay | 22:35 |
| krotscheck | jeblair: we can certainly explore the priority on commenting. | 22:36 |
| marun | jeblair: should tox be run with sudo though? | 22:36 |
| krotscheck | jeblair: more features would get done faster if, say, there was more than one person consistently working on it. | 22:37 |
| jeblair | marun: it probably needs to be run as the stack user since stack owns the checkout | 22:37 |
| marun | jeblair: ah, so 'sudo -u stack'? | 22:38 |
| *** mgagne has quit IRC | 22:38 | |
| jeblair | krotscheck: i'm not complaining about the speed of implementation; i'm just looking for where in the project roadmap i can find a feature we agreed on | 22:39 |
| jeblair | krotscheck: (and i'll repeat what i said in brussels so you might recall how important it is: there was a bug in the POC that prevented adding comments from working, and that _alone_ is the reason we were not using it yet) | 22:40 |
| krotscheck | jeblair: Hoenstly, I've been adding things to the task list as I remember them. At the moment, I'm the only one who's dogfooding storyboard, what with most of the other people who were in brussles having gotten sucked into HP stuff. | 22:40 |
| jeblair | i remembered correctly; it's in the etherpad for mvp user stories | 22:41 |
| *** caleb_ has quit IRC | 22:41 | |
| jeblair | krotscheck: i'll add a story for it | 22:41 |
| krotscheck | jeblair: So if you'd like to add a feature, add it. | 22:41 |
| *** mrodden has quit IRC | 22:41 | |
| reed | where is the dogfood? | 22:42 |
| * reed lazytypist | 22:42 | |
| *** marun has quit IRC | 22:42 | |
| jeblair | krotscheck: i just want to be perfectly clear: are you saying "if i want comments, i shoud add them?" | 22:42 |
| reed | where are you dogfooding storyboard? | 22:42 |
| krotscheck | jeblair: No - add a story | 22:42 |
| jeblair | reed: storyboard.openstack.org | 22:42 |
| devananda | fungi: think you'll have time to circle back to the virtual-ironic changes today? | 22:42 |
| jeblair | krotscheck: ok, will do. | 22:42 |
| flashgordon | so good news bad news about the requirements update | 22:42 |
| krotscheck | jeblair: That's what it's there for. | 22:42 |
| flashgordon | good news it apperas to work, bad news we have missing deps | 22:42 |
| *** lcheng has quit IRC | 22:42 | |
| flashgordon | http://logs.openstack.org/90/80690/3/check/check-requirements-integration-dsvm/0fa1374/console.html | 22:42 |
| krotscheck | jeblair: Having said that, if you _want_ to help out, I would be more than happy to have you. | 22:43 |
| krotscheck | The few times you helped me debug things were quite helpful. | 22:43 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added pagination to webclient. https://review.openstack.org/80712 | 22:43 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Task list now shows associated project https://review.openstack.org/80417 | 22:43 |
| *** hashar has quit IRC | 22:44 | |
| devananda | or jeblair ? i've got 2 small patches up so we can start doign experimental checks | 22:44 |
| jeblair | krotscheck: i'm trying to get to that point. :) | 22:44 |
| fungi | devananda: yep, just a sec i'll take a peek | 22:44 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added pagination to webclient. https://review.openstack.org/80712 | 22:44 |
| devananda | fungi: awesome, thanks | 22:44 |
| jeblair | krotscheck: i put (MVP) in the description since we lack priority | 22:44 |
| krotscheck | jeblair: Fair warning - the project dropdown still fails ahrd. | 22:45 |
| krotscheck | jeblair: It's much easier to use the create story button from the actual project. | 22:45 |
| jeblair | krotscheck: that's what i did, so i didn't notice the problem | 22:45 |
| jeblair | reed: http://storyboard.openstack.org/#!/project/456/stories are storyboard's own stories | 22:46 |
| jeblair | apparently we're about to break the 250 project mark | 22:47 |
| jeblair | in a few minutes | 22:47 |
| openstackgerrit | A change was merged to openstack-infra/config: Added list for Content team https://review.openstack.org/80351 | 22:48 |
| jeblair | reed: ^ | 22:48 |
| reed | wonderful, thanks | 22:48 |
| *** zns has quit IRC | 22:49 | |
| *** rpodolyaka has joined #openstack-infra | 22:49 | |
| jeblair | fungi: haha -- my dns image testing finally managed to get through the critical section, only to be stymied by trying to clone the projects that aren't in gerrit yet | 22:51 |
| fungi | jeblair: yeah, this apparently impacts third-party test rigs using nodepool too | 22:52 |
| fungi | ArxCruz noticed last week or the week before | 22:52 |
| jeblair | fungi: well, if they are using our setup scripts | 22:52 |
| fungi | exactly | 22:52 |
| openstackgerrit | Alex Gaynor proposed a change to openstack-infra/config: Simplify gate jobs for python-openstacksdk and add docs https://review.openstack.org/80713 | 22:52 |
| *** esker has quit IRC | 22:53 | |
| *** Ryan_Lane has quit IRC | 22:53 | |
| openstackgerrit | A change was merged to openstack-infra/config: Correct syntax error in review.projects.yaml https://review.openstack.org/80711 | 22:55 |
| *** zns has joined #openstack-infra | 22:55 | |
| openstackgerrit | A change was merged to openstack-infra/config: enable doc generation for pycadf https://review.openstack.org/75998 | 22:55 |
| openstackgerrit | Joe Gordon proposed a change to openstack/requirements: Add missing requirements https://review.openstack.org/80714 | 22:56 |
| openstackgerrit | Joe Gordon proposed a change to openstack/requirements: If dependency not in global reqs exit with a 1 https://review.openstack.org/80690 | 22:56 |
| flashgordon | this is going be a fun game of detect the missing dependency ^ | 22:56 |
| *** reed has quit IRC | 22:56 | |
| fungi | flashgordon: playing catch-up always is. i suspect the hole in devstack was hiding these from us for a while | 22:57 |
| damnsmith | flashgordon: https://review.openstack.org/#/c/80643/ | 23:00 |
| damnsmith | flashgordon: is there any better way to test that than rechecking a bunch? that ends up running a lot of tests and taking lots of resources for just two actual results that we care about | 23:00 |
| flashgordon | damnsmith: yeah add an silent queue job | 23:01 |
| jeblair | ++ | 23:01 |
| jeblair | or experimental | 23:01 |
| damnsmith | can you link me to an example? | 23:01 |
| flashgordon | damnsmith: but we should hold off on this until juno | 23:01 |
| flashgordon | as we have plenty of other things to deal with | 23:01 |
| *** Ryan_Lane has joined #openstack-infra | 23:01 | |
| *** wenlock has quit IRC | 23:01 | |
| flashgordon | I05e3d0e9f3a75afb62ff192938a623fbbc11c7f5 | 23:02 |
| *** mrodden has joined #openstack-infra | 23:02 | |
| damnsmith | flashgordon: well, we don't have to merge it any time soon, but I want to gather the data to show that we're better off now | 23:02 |
| flashgordon | ^^ | 23:02 |
| flashgordon | shuold point you in the right direction | 23:02 |
| damnsmith | yep, thanks | 23:02 |
| *** che-arne has quit IRC | 23:03 | |
| flashgordon | dumb bash foo/devstack question | 23:03 |
| damnsmith | flashgordon: er, does that let me change the d-g code? that's what I need to do... | 23:04 |
| flashgordon | if update.py returns a 1 in devstack today will that cause devstack to stop | 23:04 |
| fungi | flashgordon: it's run in devstack, within the errexit, so i believe so | 23:05 |
| *** miqui has quit IRC | 23:05 | |
| *** miqui has joined #openstack-infra | 23:05 | |
| fungi | flashgordon: you could always submit a wip patch which adds exit 1 at the end and just see what happens | 23:05 |
| flashgordon | damnsmith: yeah you will need to change d-g code as well | 23:06 |
| flashgordon | perhaps so DEVSTACK_GATE_TEMPEST_LARGE_OPS=NUMBER_OF_VMS | 23:06 |
| flashgordon | or something | 23:06 |
| fungi | jeblair: okay, all 5 git servers have the kite repos now, so we can procede to run m-p i think | 23:06 |
| flashgordon | fungi: good idea | 23:06 |
| fungi | proceed | 23:07 |
| jeblair | fungi: ++ | 23:07 |
| *** mrodden has quit IRC | 23:07 | |
| fungi | jeblair: nevermind... ha! | 23:07 |
| fungi | review.o.o hasn't pulled that change yet ;) | 23:07 |
| fungi | doing | 23:07 |
| *** khyati has quit IRC | 23:07 | |
| fungi | so that i don't have to wait for it | 23:08 |
| *** wenlock has joined #openstack-infra | 23:08 | |
| jeblair | fungi: you can do that without stopping puppet i believe; it will error out with a helpful msg if its already running | 23:08 |
| openstackgerrit | Joe Gordon proposed a change to openstack/requirements: TESTING, DO NOT MERGE https://review.openstack.org/80716 | 23:08 |
| flashgordon | fungi: ^ that should confirm | 23:08 |
| jeblair | fungi: 1 sec | 23:08 |
| flashgordon | thanks for the idea | 23:08 |
| fungi | jeblair: okay | 23:09 |
| fungi | jeblair: yeah, the gerrit module is in need of cleanup there | 23:09 |
| jeblair | fungi: all good | 23:09 |
| fungi | ready then? | 23:09 |
| jeblair | yes | 23:09 |
| *** maxbit has joined #openstack-infra | 23:10 | |
| *** mrodden has joined #openstack-infra | 23:10 | |
| krotscheck | Incidentally, jeblair, that entire exchange we just had left a nasty taste in my mouth. | 23:10 |
| markmcclain | question about strange gerrit issue: https://review.openstack.org/#/c/50337/ | 23:11 |
| markmcclain | when I click review I can't score. Is that as sign there are later drafts? | 23:11 |
| jeblair | krotscheck: sorry. perhaps we need to get better at communicating with each other in irc. i don't think it needed to be that way. | 23:12 |
| krotscheck | jeblair: That's fair, and apologies from my end as well. | 23:12 |
| krotscheck | jeblair: Don't get me wrong - I get the need for comments. | 23:13 |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Task list now shows associated project https://review.openstack.org/80417 | 23:13 |
| fungi | added rustlebee to nova-specs-core | 23:13 |
| krotscheck | jeblair: But blargh getting defensive on a text-based medium. | 23:13 |
| jeblair | krotscheck: so we're on the same page; i'm not badgering you at all, i have no issues with the speed of implementation or your work or anything. i think it's excellent all around actually. | 23:14 |
| fungi | looks like we eventually need members added to nova-docker-core, barbican-ptl and barbican-milestone | 23:15 |
| *** Ryan_Lane has quit IRC | 23:15 | |
| fungi | rustlebee: morganfainberg: ^ who should be the initial members of those | 23:15 |
| fungi | ? | 23:15 |
| morganfainberg | fungi, mordred ^ | 23:16 |
| mordred | morganfainberg: sup/ | 23:17 |
| jeblair | fungi: i see kite repos with content on all 5 git mirrors | 23:18 |
| morganfainberg | fungi, pinged me instead of you a couple lines up | 23:18 |
| morganfainberg | i'm sure | 23:18 |
| morganfainberg | fungi, unless you really meant me? | 23:18 |
| mordred | fungi: oy. how would I know? | 23:18 |
| * morganfainberg is confused | 23:18 | |
| jeblair | morganfainberg: nope you. :) we made your kite project and want to know who should be in the barbican-ptl and barbican-milestone groups | 23:19 |
| morganfainberg | jeblair | 23:19 |
| morganfainberg | oh | 23:19 |
| morganfainberg | uhm | 23:19 |
| morganfainberg | uhhhhhhhh | 23:19 |
| morganfainberg | you mean barbican didn't already have that stuff? | 23:19 |
| jeblair | morganfainberg: apparently not. we don't really keep track of that for stackforge projects. | 23:19 |
| jeblair | self-service and all | 23:19 |
| morganfainberg | jeblair, heh | 23:19 |
| morganfainberg | hmm. | 23:20 |
| morganfainberg | well barbican-ptl should be the barbican ptl ;) | 23:20 |
| morganfainberg | and... | 23:20 |
| morganfainberg | let me see about what milestone is supposed to be | 23:20 |
| *** e0ne has joined #openstack-infra | 23:21 | |
| jeblair | fungi: yay, that happened in time for the next run of my test to succeed. i now have 3 centos images in 3 providers | 23:22 |
| fungi | morganfainberg: no worries--since you drafted a change mentioning those groups, i thought you might know who to put in the empty ones. we *can* wait for them to just eventually ask us | 23:22 |
| fungi | jeblair: perfect | 23:22 |
| morganfainberg | fungi, i think we should just wait on those. | 23:22 |
| *** e0ne_ has quit IRC | 23:22 | |
| morganfainberg | fungi, originally we were using the keystone structure but i think some stuff doesn't translate 1:1 for barbican | 23:22 |
| openstackgerrit | Dan Smith proposed a change to openstack-infra/devstack-gate: Allow configurable large-ops tests https://review.openstack.org/80643 | 23:22 |
| fungi | jeblair: also all the projects seemed to get the right stuff created on the gerrit side, so i'll fire up the full run now (which should hopefully just be updating remote tracking branches) | 23:23 |
| morganfainberg | fungi, since this falls under barbican now, when the review was updated i think that stuff was left in assuming barbican was the same structure wise | 23:23 |
| fungi | morganfainberg: fair enough! it's not urgent | 23:23 |
| morganfainberg | fungi, i'll dig up the infor and ping you guys as we get a bit further with this stuff :) | 23:23 |
| *** talluri has joined #openstack-infra | 23:23 | |
| flashgordon | so trove is still being troublesome http://logs.openstack.org/87/80687/2/check/check-requirements-integration-dsvm/cc4c318/console.html | 23:24 |
| *** che-arne has joined #openstack-infra | 23:25 | |
| *** sweston has quit IRC | 23:27 | |
| *** flaper87 is now known as flaper87|afk | 23:27 | |
| *** gokrokve has quit IRC | 23:28 | |
| openstackgerrit | Victor Seva proposed a change to openstack-infra/jenkins-job-builder: use {obj:key} as a way to pass an object and not only strings to templates https://review.openstack.org/73990 | 23:28 |
| *** talluri has quit IRC | 23:28 | |
| flashgordon | it looks like the job isn't using stable branches | 23:29 |
| flashgordon | http://logs.openstack.org/87/80687/2/check/check-requirements-integration-dsvm/cc4c318/console.html#_2014-03-14_22_51_04_905 | 23:29 |
| *** e0ne has quit IRC | 23:30 | |
| *** e0ne has joined #openstack-infra | 23:30 | |
| * flashgordon wonders where the bottom of this rabbit hole is | 23:31 | |
| *** zehicle_at_dell has quit IRC | 23:32 | |
| *** rpodolyaka has quit IRC | 23:32 | |
| *** zehicle_at_dell has joined #openstack-infra | 23:32 | |
| *** openstackstatus has quit IRC | 23:36 | |
| *** e0ne has quit IRC | 23:37 | |
| *** jergerber has quit IRC | 23:38 | |
| *** jdurgin has quit IRC | 23:38 | |
| *** rwsu has quit IRC | 23:38 | |
| *** unicell has quit IRC | 23:38 | |
| *** miqui has quit IRC | 23:38 | |
| *** jeckersb_gone has quit IRC | 23:38 | |
| *** dstufft has quit IRC | 23:38 | |
| *** openstackgerrit has quit IRC | 23:39 | |
| *** rossella_s has quit IRC | 23:39 | |
| *** changbl has quit IRC | 23:42 | |
| *** miqui has joined #openstack-infra | 23:42 | |
| *** nati_ueno has quit IRC | 23:42 | |
| *** HenryG has quit IRC | 23:42 | |
| *** thuc has quit IRC | 23:43 | |
| *** sdake has quit IRC | 23:43 | |
| *** mwagner_lap has quit IRC | 23:45 | |
| *** julienvey has quit IRC | 23:45 | |
| *** zehicle_at_dell has quit IRC | 23:45 | |
| *** wenlock has quit IRC | 23:45 | |
| *** sarob has quit IRC | 23:45 | |
| *** yjiang5 has quit IRC | 23:46 | |
| *** rcarrillocruz has quit IRC | 23:46 | |
| *** jepoy has quit IRC | 23:46 | |
| *** yongli has quit IRC | 23:46 | |
| *** SpamapS has quit IRC | 23:46 | |
| *** dkranz has quit IRC | 23:46 | |
| *** bogdando has quit IRC | 23:46 | |
| *** cody-somerville has quit IRC | 23:46 | |
| *** dangers_` has quit IRC | 23:46 | |
| *** milki has quit IRC | 23:46 | |
| *** maurosr has quit IRC | 23:46 | |
| *** katyafervent_ has quit IRC | 23:46 | |
| *** KurtMartin has quit IRC | 23:46 | |
| *** Hefeweizen has quit IRC | 23:46 | |
| *** krotscheck has quit IRC | 23:46 | |
| *** sdague has quit IRC | 23:46 | |
| *** dragondm has quit IRC | 23:46 | |
| *** rainya has quit IRC | 23:46 | |
| *** che-arne has quit IRC | 23:46 | |
| *** zns has quit IRC | 23:46 | |
| *** medieval1_ has quit IRC | 23:46 | |
| *** Alex_Gaynor has quit IRC | 23:46 | |
| *** dolphm has quit IRC | 23:46 | |
| *** dstanek_afk has quit IRC | 23:46 | |
| *** spredzy has quit IRC | 23:46 | |
| *** mkerrin1 has quit IRC | 23:46 | |
| *** tristanC has quit IRC | 23:46 | |
| *** cody-somerville- has quit IRC | 23:46 | |
| *** exz has quit IRC | 23:46 | |
| *** tian has quit IRC | 23:46 | |
| *** weiwei has quit IRC | 23:46 | |
| *** cyeoh has quit IRC | 23:46 | |
| *** JoshNang has quit IRC | 23:46 | |
| *** mtreinish has quit IRC | 23:46 | |
| *** Daviey has quit IRC | 23:46 | |
| *** sc68cal has quit IRC | 23:46 | |
| *** hughsaunders has quit IRC | 23:46 | |
| *** mrodden has quit IRC | 23:46 | |
| *** maxbit has quit IRC | 23:46 | |
| *** bada_ has quit IRC | 23:46 | |
| *** vkozhukalov_ has quit IRC | 23:46 | |
| *** nicedice has quit IRC | 23:46 | |
| *** harlowja has quit IRC | 23:46 | |
| *** bknudson has quit IRC | 23:46 | |
| *** chmouel has quit IRC | 23:46 | |
| *** EntropyWorks has quit IRC | 23:46 | |
| *** damnsmith has quit IRC | 23:46 | |
| *** ianw has quit IRC | 23:46 | |
| *** moted has quit IRC | 23:46 | |
| *** antonym has quit IRC | 23:46 | |
| *** mrda has quit IRC | 23:46 | |
| *** loquacities has quit IRC | 23:46 | |
| *** nibalizer has quit IRC | 23:46 | |
| *** asadoughi has quit IRC | 23:46 | |
| *** shortstop has quit IRC | 23:46 | |
| *** weshay has quit IRC | 23:46 | |
| *** fungi has quit IRC | 23:46 | |
| *** JayF has quit IRC | 23:46 | |
| *** simonmcc has quit IRC | 23:46 | |
| *** EmilienM has quit IRC | 23:46 | |
| *** locke105 has quit IRC | 23:46 | |
| *** AaronGr has quit IRC | 23:46 | |
| *** psedlak has quit IRC | 23:46 | |
| *** Adri2000 has quit IRC | 23:46 | |
| *** pabelanger has quit IRC | 23:46 | |
| *** odyi has quit IRC | 23:46 | |
| *** jd__ has quit IRC | 23:46 | |
| *** vhoward- has quit IRC | 23:46 | |
| *** mancdaz has quit IRC | 23:46 | |
| *** gondoi has quit IRC | 23:46 | |
| *** zhiyan_ has quit IRC | 23:46 | |
| *** salv-orlando has quit IRC | 23:46 | |
| *** dprince has quit IRC | 23:46 | |
| *** jgrimm has quit IRC | 23:46 | |
| *** homeless has quit IRC | 23:46 | |
| *** blamar has quit IRC | 23:46 | |
| *** pcrews_ has quit IRC | 23:46 | |
| *** wchrisj has quit IRC | 23:46 | |
| *** dripton has quit IRC | 23:46 | |
| *** esmute has quit IRC | 23:46 | |
| *** ilyashakhat has quit IRC | 23:46 | |
| *** vipul has quit IRC | 23:46 | |
| *** saschpe has quit IRC | 23:46 | |
| *** sbadia has quit IRC | 23:46 | |
| *** dkehn has quit IRC | 23:46 | |
| *** dhellmann has quit IRC | 23:46 | |
| *** dachary has quit IRC | 23:46 | |
| *** saper has quit IRC | 23:46 | |
| *** morgabra has quit IRC | 23:46 | |
| *** gaelL_ has quit IRC | 23:46 | |
| *** Shrews has quit IRC | 23:46 | |
| *** clarkb has quit IRC | 23:46 | |
| *** kevinbenton has quit IRC | 23:46 | |
| *** dcramer_ has quit IRC | 23:46 | |
| *** amcrn has quit IRC | 23:46 | |
| *** rhsu has quit IRC | 23:46 | |
| *** SumitNaiksatam has quit IRC | 23:46 | |
| *** markwash has quit IRC | 23:46 | |
| *** alexpilotti has quit IRC | 23:46 | |
| *** sahumada has quit IRC | 23:46 | |
| *** sandywalsh_ has quit IRC | 23:46 | |
| *** mikal has quit IRC | 23:46 | |
| *** gmoro has quit IRC | 23:46 | |
| *** juice has quit IRC | 23:46 | |
| *** davidlenwell has quit IRC | 23:46 | |
| *** mattoliverau has quit IRC | 23:46 | |
| *** jaypipes has quit IRC | 23:46 | |
| *** kashyap has quit IRC | 23:46 | |
| *** yamahata_ has quit IRC | 23:46 | |
| *** jamielennox|away has quit IRC | 23:46 | |
| *** flaper87|afk has quit IRC | 23:46 | |
| *** bradm has quit IRC | 23:46 | |
| *** koolhead17 has quit IRC | 23:46 | |
| *** cp16net has quit IRC | 23:46 | |
| *** nijaba has quit IRC | 23:46 | |
| *** hogepodge has quit IRC | 23:46 | |
| *** gyee has quit IRC | 23:46 | |
| *** emagana has quit IRC | 23:46 | |
| *** atiwari has quit IRC | 23:46 | |
| *** gothicmindfood has quit IRC | 23:46 | |
| *** enikanorov_ has quit IRC | 23:46 | |
| *** mugsie has quit IRC | 23:46 | |
| *** ProfFalken has quit IRC | 23:46 | |
| *** Mithrandir has quit IRC | 23:46 | |
| *** wayneeseguin has quit IRC | 23:46 | |
| *** skraynev_afk has quit IRC | 23:46 | |
| *** dkehn_ has quit IRC | 23:46 | |
| *** pfallenop has quit IRC | 23:46 | |
| *** fbo has quit IRC | 23:46 | |
| *** msuriar_ has quit IRC | 23:46 | |
| *** jeblair has quit IRC | 23:46 | |
| *** jpeeler has quit IRC | 23:46 | |
| *** jeremyb has quit IRC | 23:46 | |
| *** soren has quit IRC | 23:46 | |
| *** gema has quit IRC | 23:46 | |
| *** alaski has quit IRC | 23:46 | |
| *** NikitaKonovalov has quit IRC | 23:46 | |
| *** michchap has quit IRC | 23:46 | |
| *** westmaas has quit IRC | 23:46 | |
| *** DuncanT- has quit IRC | 23:46 | |
| *** SergeyLukjanov has quit IRC | 23:46 | |
| *** ruhe has quit IRC | 23:46 | |
| *** DinaBelova has quit IRC | 23:46 | |
| *** ogelbukh has quit IRC | 23:46 | |
| *** notmyname has quit IRC | 23:46 | |
| *** pleia2 has quit IRC | 23:46 | |
| *** bearhands has quit IRC | 23:46 | |
| *** Vivek has quit IRC | 23:46 | |
| *** niska has quit IRC | 23:46 | |
| *** rahmu has quit IRC | 23:46 | |
| *** plomakin_ has quit IRC | 23:46 | |
| *** rdwrer has quit IRC | 23:46 | |
| *** dosaboy has quit IRC | 23:46 | |
| *** vladan has quit IRC | 23:46 | |
| *** spiffxp_ has quit IRC | 23:46 | |
| *** jcooley has quit IRC | 23:46 | |
| *** belliott has quit IRC | 23:46 | |
| *** adarazs_ has quit IRC | 23:46 | |
| *** flashgordon has quit IRC | 23:46 | |
| *** guitarzan has quit IRC | 23:46 | |
| *** mburned has quit IRC | 23:46 | |
| *** aude has quit IRC | 23:46 | |
| *** Steap has quit IRC | 23:46 | |
| *** gilliard has quit IRC | 23:46 | |
| *** lifeless has quit IRC | 23:46 | |
| *** mika has quit IRC | 23:46 | |
| *** jesusaurus has quit IRC | 23:46 | |
| *** primeministerp has quit IRC | 23:46 | |
| *** gpocentek has quit IRC | 23:46 | |
| *** mordred has quit IRC | 23:46 | |
| *** dtroyer has quit IRC | 23:46 | |
| *** sileht has quit IRC | 23:46 | |
| *** zigo has quit IRC | 23:46 | |
| *** tteggel has quit IRC | 23:46 | |
| *** jgriffith has quit IRC | 23:46 | |
| *** StevenK has quit IRC | 23:46 | |
| *** jraim has quit IRC | 23:46 | |
| *** GheRivero has quit IRC | 23:46 | |
| *** vishy has quit IRC | 23:46 | |
| *** isviridov has quit IRC | 23:46 | |
| *** openstack has joined #openstack-infra | 23:47 | |
| -dickson.freenode.net- [freenode-info] channel trolls and no channel staff around to help? please check with freenode support: http://freenode.net/faq.shtml#gettinghelp | 23:47 | |
| *** openstackstatus has joined #openstack-infra | 23:47 | |
| *** dstufft has quit IRC | 23:47 | |
| *** dstufft has joined #openstack-infra | 23:47 | |
| *** zns has joined #openstack-infra | 23:49 | |
| *** mkerrin has joined #openstack-infra | 23:51 | |
| *** Alex_Gaynor has joined #openstack-infra | 23:53 | |
| *** che-arne has joined #openstack-infra | 23:53 | |
| *** dstufft has quit IRC | 23:53 | |
| *** dstanek_afk has joined #openstack-infra | 23:54 | |
| *** bada_ has joined #openstack-infra | 23:54 | |
| *** cody-somerville- has joined #openstack-infra | 23:56 | |
| *** dstufft has joined #openstack-infra | 23:57 | |
| *** cyeoh has joined #openstack-infra | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!